Target keyword-based JSON data dynamic addition and management method
Through syntactic dependency analysis and semantic identifier mapping, the problems of semantic inconsistency and lack of adaptability in JSON data management are solved, precise semantic positioning and dynamic appending are achieved, the accuracy and efficiency of JSON data operations are improved, and the maintainability and adaptability of data are enhanced.
Patent Information
- Application Number
- CN202511241072.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-02
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2045-09-02
AI Technical Summary
Existing technologies lack a deep understanding of keyword semantic levels in JSON data management and are unable to effectively capture the semantic associations between keywords, resulting in semantic inconsistency when appending data. They also make it difficult to adaptively adjust to achieve precise positioning and intelligent organization, and are unable to accurately identify explicit and implicit reference relationships between JSON data nodes, affecting data consistency and efficient access.
Through grammatical dependency analysis, the target keywords are converted into feature vectors, a keyword affinity matrix is generated, adaptive classification vector clustering is performed, a mapping relationship between semantic identifiers and JSON data nodes is established, a dependency topology graph and data blocks are constructed, and accurate semantic positioning and dynamic appending are achieved.
It improves the accuracy and efficiency of JSON data operations, reduces the error rate of operations, implements modular management, enhances the maintainability and scalability of data, and improves the ability to understand user intentions, adaptability and intelligence.
Smart Images

Figure CN120744117A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a method for dynamically appending and managing JSON data based on target keywords. Background Art
[0002] With the rapid development of internet technology, the JSON data format has become the mainstream format for data exchange and storage due to its lightweight, easy-to-read, and cross-language features. In large-scale application systems, JSON data structures often need to be dynamically appended and managed based on business needs. Effective semantic recognition and dynamic management of JSON data are particularly important in scenarios such as processing complex data relationships, big data analysis, and knowledge graph construction.
[0003] Traditional JSON data management typically relies on simple key-value pair matching and path navigation. This approach lacks semantic understanding when dealing with complex JSON structures, making it difficult to identify implicit data relationships. While some research has been conducted on keyword-based JSON data management techniques, significant shortcomings remain.
[0004] Existing technologies still have some defects in the dynamic appending and management of keyword-based JSON data. Existing technologies lack a deep understanding of the semantic hierarchy of keywords and cannot effectively capture the semantic associations between keywords, which easily leads to semantic inconsistency problems when appending data; traditional JSON data management methods usually rely on predefined data structure patterns. When faced with dynamically changing business needs, they lack adaptive adjustment capabilities and find it difficult to achieve accurate positioning and intelligent organization of data; existing technologies have failed to establish an effective data dependency analysis mechanism and cannot accurately identify the explicit and implicit reference relationships between JSON data nodes, thereby affecting data consistency maintenance and efficient access, especially in the processing of JSON data with complex nested structures. Summary of the Invention
[0005] The embodiment of the present invention provides a method for dynamically appending and managing JSON data based on target keywords, which can solve the problems in the prior art.
[0006] A first aspect of an embodiment of the present invention provides a method for dynamically appending and managing JSON data based on a target keyword, comprising:
[0007] Receive target keywords and JSON data;
[0008] The target keywords are converted into feature vectors by using grammatical dependency analysis, and a keyword affinity matrix is generated by cosine similarity calculation. Based on the keyword affinity matrix, an adaptive classification vector clustering operation is performed to obtain keyword cluster families.
[0009] Assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table;
[0010] Locating the target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block based on the association strength, and assigning a globally unique block identifier to the data block;
[0011] Retrieve candidate semantic identifiers in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information;
[0012] The semantic addressing table and the dependency topology map are updated according to the block change information.
[0013] In an optional embodiment, the target keyword is converted into a feature vector using grammatical dependency analysis, a keyword affinity matrix is generated by cosine similarity calculation, and an adaptive classification vector clustering operation is performed based on the keyword affinity matrix to obtain keyword cluster families including:
[0014] Segmenting the target keyword to obtain a word sequence, using part-of-speech tagging to obtain the part-of-speech features of each word in the word sequence, constructing a grammatical dependency tree based on the part-of-speech features and position information, extracting syntactic relationship features from the grammatical dependency tree, and combining the part-of-speech features and syntactic relationship features to generate a semantic feature matrix;
[0015] Performing dimensionality reduction on the semantic feature matrix based on orthogonal transformation, selecting feature dimensions in sequence according to variance contribution rates, and combining corresponding eigenvalues to form a target feature vector;
[0016] Calculating the cosine distance between any two target feature vectors to obtain a similarity value, constructing an initial similarity matrix based on the similarity value, and normalizing the element values in the initial similarity matrix to determine an affinity matrix;
[0017] Calculating the mean and variance of each row of elements in the affinity matrix, setting a dynamic threshold based on the mean and variance, and setting element values smaller than the dynamic threshold to zero to obtain an optimized affinity matrix;
[0018] The Laplace matrix of the optimized affinity matrix is calculated, the eigenvector of the Laplace matrix is extracted to construct a feature space, and an adaptive classification vector clustering operation is performed in the feature space to obtain a keyword cluster family.
[0019] In an optional embodiment, performing an adaptive classification vector clustering operation in the feature space to obtain a keyword cluster family includes:
[0020] Mapping keywords in the feature space into sample points;
[0021] Establish a search radius for each sample point, count the number of samples within the search radius to obtain a local density value; calculate the Euclidean distance between any two sample points, and for each target sample point, select the sample point with the smallest Euclidean distance from all sample points whose local density value is greater than that of the target sample point, determine the comparison sample point, and record the Euclidean distance between the target sample point and the comparison sample point as the distance value;
[0022] Calculating the product of the local density value and the distance value to obtain a density index, sorting the density indexes in descending order, setting a density index threshold according to the numerical distribution of the density index, and using sample points greater than the density index threshold as initial clustering points;
[0023] Construct a classifier, using the initial cluster points as the classification vector of the classifier; input sample points one by one, calculate the similarity between the sample points and each classification vector, and select the classification vector with the highest similarity as the matching vector;
[0024] Based on the position of the current input sample point and the matching vector, adjusting the position of the matching vector according to a preset distance ratio, and adjusting other classification vectors adjacent to the matching vector;
[0025] Repeat the process until the classification vector becomes stable and the sample points associated with the same classification vector are grouped into one category to obtain a keyword cluster family.
[0026] In an optional embodiment, assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table includes:
[0027] Perform word frequency statistics on the keywords in the keyword cluster, extract the part-of-speech features and syntactic dependencies of the keywords to construct semantic vectors, calculate the semantic topic categories of the keyword cluster based on the semantic vectors, and assign semantic identifiers to the keyword clusters in combination with preset hierarchical semantic encoding rules;
[0028] Parse the hierarchical structure of JSON data to obtain JSON nodes, extract the key name and data type of the JSON nodes, determine the parent-child relationship and sibling relationship between the JSON nodes, construct a tree-structured adjacency matrix of the JSON nodes, and calculate the relationship distance between the JSON nodes based on the tree-structured adjacency matrix;
[0029] Calculate the text similarity between the semantic identifier and the key name information of the JSON node, construct a semantic context vector based on the relationship distance between the JSON nodes, and perform weighted fusion of the text similarity and the semantic context vector to obtain a semantic relevance matrix;
[0030] Setting a mapping threshold to filter the semantic relevance matrix, eliminating redundant mapping relationships based on transitive closure, handling conflicts between one-to-many mapping and many-to-one mapping, and determining a mapping relationship between a semantic identifier and a JSON node;
[0031] A semantic identification index and a node path index are constructed based on the mapping relationship to form a bidirectional mapping structure, and a semantic addressing table is generated according to the bidirectional mapping structure.
[0032] In an optional embodiment, locating a target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block according to the association strength, and assigning a globally unique block identifier to the data block includes:
[0033] Locating the target node according to the semantic addressing table, extracting the reference relationship of the target node, including explicit reference relationship and implicit reference relationship, wherein the key-value correspondence relationship in the target node is extracted to determine the foreign key reference and ID reference as the explicit reference relationship, and calculating the co-occurrence frequency of the node attribute value and the similarity of the naming rule to determine the implicit reference relationship;
[0034] Using the target node as a graph node and the reference relationship as a connection edge to construct a dependency topology graph, calculating the transitive closure matrix of the dependency topology graph to identify a node subset with circular references, and optimizing the dependency topology graph using a minimum spanning tree algorithm;
[0035] Calculate the shortest path length between nodes in the dependency topology graph to obtain a distance correlation factor, count the citation frequency to obtain a citation correlation factor, calculate the data type matching degree and the intersection of the numerical interval to obtain a feature correlation factor, calculate the correlation coefficient of the node data update time series to obtain an update correlation factor, and perform weighted summation of the distance correlation factor, the citation correlation factor, the feature correlation factor, and the update correlation factor to obtain the correlation strength between the nodes;
[0036] Divide the node set into candidate blocks according to a preset association strength threshold, calculate the ratio of the average degree of nodes in the candidate blocks to the maximum degree to obtain a clustering coefficient, merge and optimize the candidate blocks according to the clustering coefficient, calculate the number of shared nodes between the candidate blocks and assign the shared nodes to the block with the largest clustering coefficient to obtain a data block;
[0037] A globally unique block identifier is generated according to the association strength, clustering coefficient and reference relationship of the data blocks.
[0038] In an optional embodiment, based on the target keyword, a candidate semantic identifier is retrieved in the semantic addressing table, a target data block and a write hierarchical position are determined, the JSON data to be appended is written into the target data block, and the node association relationship in the dependency topology graph is updated. Recording block change information includes:
[0039] A set of candidate semantic identifiers is retrieved from the semantic addressing table according to the target keyword, the temporal activity and citation heat of each semantic identifier in the candidate semantic identifier set are calculated, and a weighted calculation is performed to obtain a priority score of the semantic identifier, and the semantic identifier with the highest priority score is selected to determine the target data block;
[0040] Performing structural analysis on the JSON data to be appended to construct an attribute association tree, calculating the inheritance depth and association width between nodes based on the attribute association tree, determining the hierarchical position of the data write based on the inheritance depth and association width, and writing the JSON data to be appended into the target data block;
[0041] A bidirectional reference index is constructed for the newly added node in the dependency topology graph, and the circular reference path between nodes is identified based on the bidirectional reference index. The association relationship with the weakest reference strength in the circular reference path is marked as a soft connection. The soft connection only retains the one-way reference between nodes and is automatically disassociated after the expiration timestamp expires. The closed loop path in the circular reference path is blocked by the soft connection, the node association relationship in the dependency topology graph is updated, and the block change information is recorded.
[0042] A second aspect of an embodiment of the present invention provides a target keyword-based JSON data dynamic appending and management system, including:
[0043] The first unit is used to receive target keywords and JSON data;
[0044] The second unit is used to convert the target keyword into a feature vector by using grammatical dependency analysis, generate a keyword affinity matrix by calculating cosine similarity, and perform an adaptive classification vector clustering operation based on the keyword affinity matrix to obtain a keyword cluster family;
[0045] The third unit is used to assign semantic identifiers to keyword clusters, establish a mapping relationship between the semantic identifiers and nodes in the JSON data, and generate a semantic addressing table;
[0046] A fourth unit is configured to locate a target node in the JSON data according to the semantic addressing table, extract explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculate the association strength between nodes, construct a data block based on the association strength, and assign a globally unique block identifier to the data block;
[0047] The fifth unit is used to retrieve a candidate semantic identifier in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information;
[0048] The sixth unit is configured to update the semantic addressing table and the dependency topology map according to the block change information.
[0049] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:
[0050] processor;
[0051] a memory for storing processor-executable instructions;
[0052] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0053] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.
[0054] In an embodiment of the present invention, by introducing syntactic dependency analysis and semantic identifier mapping, accurate semantic positioning and dynamic appending of JSON data are achieved, which greatly improves the accuracy and efficiency of data operations, significantly reduces the operational error rate caused by complex data structure, and can intelligently identify the correlation between data and organize it reasonably; a dependency topology map and data blocks are constructed to enable modular management of JSON data, solve the problem of complex structure and difficult maintenance of large-scale JSON data in traditional methods, improve the maintainability and scalability of data, and at the same time achieve traceability of data operations through the block change recording mechanism; adaptive classification vector clustering and keyword affinity matrix realize semantic management of JSON data, effectively solve the problem of inaccurate keyword matching in traditional methods, improve the ability to understand user intentions, make data appending more in line with actual business needs, and greatly enhance adaptability and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 This is a flow chart of a method for dynamically appending and managing JSON data based on target keywords according to an embodiment of the present invention;
[0056] Figure 2 The logical flow chart of keyword clustering. DETAILED DESCRIPTION
[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.
[0058] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0059] Figure 1 Schematic diagram of the process of dynamically appending and managing JSON data based on target keywords according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0060] Receive target keywords and JSON data;
[0061] The target keywords are converted into feature vectors by using grammatical dependency analysis, and a keyword affinity matrix is generated by cosine similarity calculation. Based on the keyword affinity matrix, an adaptive classification vector clustering operation is performed to obtain keyword cluster families.
[0062] Assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table;
[0063] Locating the target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block based on the association strength, and assigning a globally unique block identifier to the data block;
[0064] Retrieve candidate semantic identifiers in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information;
[0065] The semantic addressing table and the dependency topology map are updated according to the block change information.
[0066] In an optional embodiment, the target keyword is converted into a feature vector using grammatical dependency analysis, a keyword affinity matrix is generated by cosine similarity calculation, and an adaptive classification vector clustering operation is performed based on the keyword affinity matrix to obtain keyword cluster families including:
[0067] Segmenting the target keyword to obtain a word sequence, using part-of-speech tagging to obtain the part-of-speech features of each word in the word sequence, constructing a grammatical dependency tree based on the part-of-speech features and position information, extracting syntactic relationship features from the grammatical dependency tree, and combining the part-of-speech features and syntactic relationship features to generate a semantic feature matrix;
[0068] Performing dimensionality reduction on the semantic feature matrix based on orthogonal transformation, selecting feature dimensions in sequence according to variance contribution rates, and combining corresponding eigenvalues to form a target feature vector;
[0069] Calculating the cosine distance between any two target feature vectors to obtain a similarity value, constructing an initial similarity matrix based on the similarity value, and normalizing the element values in the initial similarity matrix to determine an affinity matrix;
[0070] Calculating the mean and variance of each row of elements in the affinity matrix, setting a dynamic threshold based on the mean and variance, and setting element values smaller than the dynamic threshold to zero to obtain an optimized affinity matrix;
[0071] The Laplace matrix of the optimized affinity matrix is calculated, the eigenvector of the Laplace matrix is extracted to construct a feature space, and an adaptive classification vector clustering operation is performed in the feature space to obtain a keyword cluster family.
[0072] In one specific implementation, a set of target keywords is received as input, such as keywords related to technology fields, such as "smartphone screen," "mobile phone display technology," and "high-definition display screen." Each target keyword is then segmented into word sequences using a word segmentation tool (e.g., a conditional random field-based word segmentation algorithm). For example, for "smartphone screen," the segmentation result is ["smart," "mobile phone," "screen"].
[0073] After tokenization, the word sequence is tagged using a hidden Markov model-based part-of-speech tagging method to assign a part-of-speech label to each word. In the example above, the tagging result is ["smart / adj", "mobile phone / n", "screen / n"], indicating that "smart" is an adjective and "mobile phone" and "screen" are nouns.
[0074] Based on the part-of-speech tagging results and the word's position in the sequence, a grammatical dependency tree is constructed. A transfer-based dependency analysis algorithm is used to identify dependencies between words. In this example, "smart" modifies "mobile phone," forming an attributive-to-central relationship (ATT). "Mobile phone" and "screen" form an attributive-to-central relationship (ATT), with "screen" being the core word (HED).
[0075] Extract syntactic relationship features from the grammatical dependency tree, including the type and direction of dependency relationships between words. In this example, the extracted syntactic relationship features include "smartphone → phone: ATT" and "phone → screen: ATT." Combine the part-of-speech features and syntactic relationship features to generate a semantic feature matrix. For the example keyword, the resulting feature matrix contains three rows (corresponding to three words), each row containing a part-of-speech feature (one-hot encoding) and a syntactic relationship feature (one-hot encoding of the dependency type).
[0076] To reduce feature dimensionality and extract key information, the semantic feature matrix undergoes orthogonal transformation and dimensionality reduction. Specifically, principal component analysis (PCA) is used. The covariance matrix of the feature matrix is calculated, and its eigenvalues and eigenvectors are determined. For example, for a 10×10 feature matrix, 10 eigenvalues and their corresponding eigenvectors may be obtained.
[0077] Sort the eigenvalues by size and calculate the variance contribution of each eigenvalue. Assuming the variance contributions of the first three eigenvalues are 45%, 30%, and 15%, respectively, and the cumulative contribution reaches 90%, the first three feature dimensions are selected. The corresponding three eigenvectors are combined to form the target eigenvector. For the "smartphone screen" example, this might result in a three-dimensional eigenvector [0.75, 0.42, -0.18].
[0078] For all target keywords, we generate their feature vectors and calculate the cosine similarity between any two feature vectors. Cosine similarity is calculated by taking the cosine of the angle between two vectors. Cosine similarity ranges from -1 to 1, with larger values indicating greater similarity between the two vectors. For example, the cosine similarity between the feature vectors for "smartphone screen" and "mobile phone display technology" is 0.82, indicating a high semantic correlation between the two keywords.
[0079] Construct an initial similarity matrix S based on the calculated similarity values. For N keywords, S is an N×N matrix, where S[i,j] represents the similarity between the i-th and j-th keywords. Normalize the elements in matrix S by dividing each element by the maximum value in the matrix to obtain the normalized affinity matrix A.
[0080] To optimize the affinity matrix, calculate the mean μ and variance σ of each row of the affinity matrix A. For example, the first row has a mean μ = 0.45 and a variance σ = 0.08. A dynamic threshold is set based on the mean and variance. The threshold value is calculated as τ = μ - α × σ, where α is a tuning parameter with a value of 1.5. For the first row in this example, the dynamic threshold τ = 0.45 - 1.5 × 0.08 = 0.33.
[0081] The element values in the affinity matrix A that are less than the corresponding dynamic threshold are set to 0 to obtain the optimized affinity matrix A'. This step can effectively filter out weak correlations, retain strong correlations, and improve clustering accuracy.
[0082] Calculate the degree matrix D of the optimized affinity matrix A'. D is a diagonal matrix, where the diagonal elements are the sums of the corresponding row elements of A'. Based on the degree matrix D and the affinity matrix A', calculate the Laplacian matrix L = D - A'. Extract the eigenvectors of L, sort them in ascending order of eigenvalue, and select the first K eigenvectors to construct the feature space, where K is the expected number of clusters.
[0083] In the constructed feature space, the K-means clustering algorithm is applied to adaptively cluster keywords into classification vectors. In its implementation, K cluster centers are randomly initialized, and the assignment and update steps are iteratively performed until convergence. The assignment step assigns each keyword to the nearest cluster center; the update step recalculates the center point of each cluster.
[0084] After clustering, the keywords in the example can be divided into three clusters: {"smartphone screen", "mobile phone display technology", "high-definition display"}, {"artificial intelligence algorithm", "machine learning model"}, and {"wireless charging technology", "fast charging solution"}. These clusters represent different technical fields or topics.
[0085] In an optional embodiment, performing an adaptive classification vector clustering operation in the feature space to obtain a keyword cluster family includes:
[0086] Mapping keywords in the feature space into sample points;
[0087] Establish a search radius for each sample point, count the number of samples within the search radius to obtain a local density value; calculate the Euclidean distance between any two sample points, and for each target sample point, select the sample point with the smallest Euclidean distance from all sample points whose local density value is greater than that of the target sample point, determine the comparison sample point, and record the Euclidean distance between the target sample point and the comparison sample point as the distance value;
[0088] Calculating the product of the local density value and the distance value to obtain a density index, sorting the density indexes in descending order, setting a density index threshold according to the numerical distribution of the density index, and using sample points greater than the density index threshold as initial clustering points;
[0089] Construct a classifier, using the initial cluster points as the classification vector of the classifier; input sample points one by one, calculate the similarity between the sample points and each classification vector, and select the classification vector with the highest similarity as the matching vector;
[0090] Based on the position of the current input sample point and the matching vector, adjusting the position of the matching vector according to a preset distance ratio, and adjusting other classification vectors adjacent to the matching vector;
[0091] Repeat the process until the classification vector becomes stable and the sample points associated with the same classification vector are grouped into one category to obtain a keyword cluster family.
[0092] In one specific embodiment, each keyword can be represented as a vector with a specific feature dimension in the feature space. To effectively cluster keywords, these keywords need to be mapped to sample points in the feature space. Specifically, a word vector model can be used to convert keywords into vector representations of a fixed dimension, such as a 300-dimensional vector space, with each keyword corresponding to a sample point in the feature space.
[0093] After the sample points are mapped, a search radius is established for each sample point to determine its local density value. The search radius can be set based on the data distribution, for example, 20% of the average distance between all sample points in the feature space. For each sample point, the number of other sample points that fall within its search radius is counted, and this number is the local density value of the sample point. For example, if the search radius of sample point P1 is 0.5 and there are 7 other sample points within this radius, the local density value of P1 is 7.
[0094] Then the Euclidean distance between any two sample points is calculated. The Euclidean distance calculation is based on the square root of the sum of the squares of the coordinate differences of the sample points in each dimension in the feature space. For each target sample point in the feature space, it is necessary to find the sample point with the smallest Euclidean distance to the target sample point among all the sample points whose local density value is greater than that of the target sample point as the comparison sample point. The Euclidean distance between the target sample point and the comparison sample point is recorded as the distance value. For example, the local density value of sample point P2 is 5. Among all the sample points with a local density value greater than 5, the Euclidean distance between sample point P3 and P2 is the smallest, which is 0.8. Then the distance value of P2 is 0.8.
[0095] Based on the local density value and the distance value, the density index of each sample point is calculated, which is the product of the local density value and the distance value. The density index reflects the potential of the sample point as a cluster center. All sample points are sorted in descending order according to the density index, and the numerical distribution of the density index is analyzed. Usually, the density index of the true cluster center point will be significantly higher than that of other points. By observing the distribution curve of the density index, an appropriate density index threshold can be set. For example, if the density indexes of the first five sample points after density index sorting are 25, 23, 22, 12, and 11, respectively, and the density indexes of subsequent sample points are all below 10 and are evenly distributed, the density index threshold can be set to 20, and the first three sample points can be selected as the initial cluster points.
[0096] When building the classifier, these initial cluster points are used as the classification vectors. A classification vector represents a cluster center. The remaining sample points are input one by one, and the similarity between each sample point and each classification vector is calculated. Similarity can be calculated based on the inverse of the Euclidean distance; the smaller the distance, the higher the similarity. For each input sample point, the classification vector with the highest similarity is selected as its matching vector. For example, if the similarities between sample point P4 and classification vectors V1, V2, and V3 are 0.7, 0.5, and 0.3, respectively, then V1 is the matching vector for P4.
[0097] Adjust the position of the matching vector based on the positional relationship between the current input sample point and the matching vector. The adjustment uses a preset distance ratio, for example, 0.1, which means that the matching vector moves 10% of the distance in the direction of the current sample point. If the distance between the current sample point P5 and its matching vector V2 is 2.0, the new position of V2 will move 0.2 units in the direction of P5. At the same time, other classification vectors adjacent to the matching vector need to be adjusted to maintain the topological relationship between the classification vectors. The adjustment amplitude of adjacent vectors can be set to 50% of the matching vector adjustment amplitude, and the direction is consistent.
[0098] Repeat the above classification and adjustment process until the change in the position of all classification vectors in two consecutive rounds of adjustment is less than a preset threshold (such as 0.01), and the classification vector is considered stable. At this point, all sample points associated with the same classification vector (i.e., using the classification vector as the matching vector) are grouped together to form a keyword cluster.
[0099] To illustrate this with a specific example, consider the keyword set {"computer," "laptop," "tablet," "mobile phone," "camera," "printer," "headphones," and "speakers." The coordinates of these keywords, mapped to the two-dimensional feature space by the word embedding model, are (1.2, 2.3), (1.5, 2.1), (1.8, 2.4), (4.2, 1.1), (4.5, 1.3), (4.7, 1.0), (3.1, 4.2), and (3.4, 4.5). The search radius is set to 0.5, and the local density values of each point are calculated to be 3, 3, 3, 3, 3, 3, 2, and 2, respectively. After distance calculation and density index sorting, ("computer," 1.2, 2.3), ("mobile phone," 4.2, 1.1), and ("headphones," 3.1, 4.2) are selected as the initial cluster points. After multiple rounds of classification vector position adjustment, three keyword clusters are finally formed: {"computer", "laptop", "tablet"}, {"mobile phone", "camera", "printer"} and {"headphones", "speakers"}. These three clusters correspond to different types of electronic devices.
[0100] like Figure 2 As shown, the keyword clustering logic flow chart is displayed.
[0101] In an optional embodiment, assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table includes:
[0102] Perform word frequency statistics on the keywords in the keyword cluster, extract the part-of-speech features and syntactic dependencies of the keywords to construct semantic vectors, calculate the semantic topic categories of the keyword cluster based on the semantic vectors, and assign semantic identifiers to the keyword clusters in combination with preset hierarchical semantic encoding rules;
[0103] Parse the hierarchical structure of JSON data to obtain JSON nodes, extract the key name and data type of the JSON nodes, determine the parent-child relationship and sibling relationship between the JSON nodes, construct a tree-structured adjacency matrix of the JSON nodes, and calculate the relationship distance between the JSON nodes based on the tree-structured adjacency matrix;
[0104] Calculate the text similarity between the semantic identifier and the key name information of the JSON node, construct a semantic context vector based on the relationship distance between the JSON nodes, and perform weighted fusion of the text similarity and the semantic context vector to obtain a semantic relevance matrix;
[0105] Setting a mapping threshold to filter the semantic relevance matrix, eliminating redundant mapping relationships based on transitive closure, handling conflicts between one-to-many mapping and many-to-one mapping, and determining a mapping relationship between a semantic identifier and a JSON node;
[0106] A semantic identification index and a node path index are constructed based on the mapping relationship to form a bidirectional mapping structure, and a semantic addressing table is generated according to the bidirectional mapping structure.
[0107] In a specific embodiment, when performing word frequency statistics on keywords in a keyword cluster family, a sliding window method is used to count the word frequency. For example, for a cluster family containing keywords such as "user information", "personal information", and "account data", it is calculated that the frequency of "user" is 0.35, the frequency of "information" is 0.25, the frequency of "material" is 0.2, the frequency of "account" is 0.15, and the frequency of "data" is 0.05. When extracting part-of-speech features, a word segmenter is used to split the keywords into word units and mark the part of speech, such as "user / noun" and "information / noun". Syntactic dependency relationships are obtained through dependency syntax analysis. For example, "personal / attributive material / central word" means that "personal" modifies "material". The word frequency, part-of-speech features, and syntactic dependency relationships are integrated into a 300-dimensional semantic vector.
[0108] When calculating semantic topic categories, the semantic vectors are matched against a pre-trained topic model. Assuming the preset topic categories include "user management," "order processing," and "payment system," the semantic topic category for this keyword cluster is determined to be "user management" by calculating cosine similarity, with a similarity score of 0.78. The hierarchical semantic encoding rules are based on a three-level structure of "major category, medium category, and minor category." For example, the major category "user management" is coded "U," the minor category "personal information" is coded "UI," and the minor category "basic information" is coded "UIB." Based on this, the semantic identifier "UIB001" is assigned to this keyword cluster.
[0109] When parsing the hierarchical structure of JSON data, a depth-first traversal algorithm is used. For example, JSON data related to user information contains the following structure: {"user":{"basicInfo":{"name":"","age":0,"gender":""},"contactInfo":{"phone":"","email":""}}}. Extract the key names of the nodes, such as "user", "basicInfo", "name", etc., and determine the data type, such as "object", "string", "number", etc. Parent-child relationships are determined by nesting levels, such as "user" is the parent node of "basicInfo"; sibling relationships are determined by nodes at the same level, such as "name", "age", and "gender" are sibling nodes.
[0110] When constructing a tree-structured adjacency matrix, the elements in the matrix represent the connection relationships between nodes. A value of 1 indicates a direct connection, and a value of 0 indicates an indirect connection. For example, in an 8×8 adjacency matrix, the connection relationship between the user node and the basicInfo node is represented by matrix[0][1]=1, indicating that there is a direct connection from user to basicInfo. The adjacency matrix is used to calculate the relationship distance between nodes. For example, the distance from user to name is 2 because the node must pass through the basicInfo node to reach the name node.
[0111] When calculating the textual similarity between semantic identifiers and JSON node keys, the text is preprocessed, including word segmentation and stop word removal. For example, the cosine similarity between the word embedding vectors of the text description "User Basic Information" associated with the semantic identifier "UIB001" and the key of the JSON node "basicInfo" is 0.82. The semantic context vector takes into account the contextual information of the JSON node. For example, the parent node of the "basicInfo" node is "user" and its sibling node is "contactInfo." Relational distances are converted to weights. For example, a distance of 1 to the parent node corresponds to a weight of 0.8, and a distance of 2 to the sibling node corresponds to a weight of 0.5. The weighted fusion of textual similarity and semantic context vectors is calculated as: similarity × 0.7 + context vector score × 0.3, to obtain the final semantic relevance matrix.
[0112] When filtering the semantic relevance matrix using a mapping threshold, the threshold is set to 0.65; mappings below this threshold are filtered out. The specific approach to eliminating redundant mappings based on transitive closure is as follows: if semantic identifier 1 is mapped to node 1, node 1 is the parent of node 2, and semantic identifier 1 is also mapped to node 2, then the mapping from semantic identifier 1 to node 1 is retained, while the mapping from semantic identifier 1 to node 2 is deleted. When processing one-to-many mappings, the semantic relevance of multiple target nodes is compared, and the one with the highest relevance is retained. For example, if the semantic identifier "UIB001" is mapped to both "basicInfo" (with a relevance of 0.82) and "contactInfo" (with a relevance of 0.68), the mapping to "basicInfo" is retained. When processing many-to-one mappings, the hierarchical relationship of semantic identifiers is analyzed, prioritizing the more specific semantic identifiers. For example, if "UIB001" (basic user information) and "UI002" (user information) are both mapped to the "basicInfo" node, the mapping for "UIB001" is retained because it represents a more specific semantic category.
[0113] When building a bidirectional mapping structure, two index tables are created. The semantic identifier index table stores the mapping of semantic identifiers to JSON node paths, such as {"UIB001":" / user / basicInfo"}; the node path index table stores the mapping of JSON node paths to semantic identifiers, such as {" / user / basicInfo":"UIB001"}. The final semantic addressing table contains these two index tables and appends metadata such as creation time and version number. This allows users to quickly locate specific nodes in JSON data using semantic identifiers or search for corresponding semantic identifiers using node paths, thus achieving semantic data access and retrieval.
[0114] Through the above steps, a mapping relationship between the semantic identifiers of keyword cluster families and JSON data nodes was established, and a semantic addressing table that is easy to query and access was generated, providing basic support for subsequent semantic-based data operations.
[0115] In an optional embodiment, locating a target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block according to the association strength, and assigning a globally unique block identifier to the data block includes:
[0116] Locating the target node according to the semantic addressing table, extracting the reference relationship of the target node, including explicit reference relationship and implicit reference relationship, wherein the key-value correspondence relationship in the target node is extracted to determine the foreign key reference and ID reference as the explicit reference relationship, and calculating the co-occurrence frequency of the node attribute value and the similarity of the naming rule to determine the implicit reference relationship;
[0117] Using the target node as a graph node and the reference relationship as a connection edge to construct a dependency topology graph, calculating the transitive closure matrix of the dependency topology graph to identify a node subset with circular references, and optimizing the dependency topology graph using a minimum spanning tree algorithm;
[0118] Calculate the shortest path length between nodes in the dependency topology graph to obtain a distance correlation factor, count the citation frequency to obtain a citation correlation factor, calculate the data type matching degree and the intersection of the numerical interval to obtain a feature correlation factor, calculate the correlation coefficient of the node data update time series to obtain an update correlation factor, and perform weighted summation of the distance correlation factor, the citation correlation factor, the feature correlation factor, and the update correlation factor to obtain the correlation strength between the nodes;
[0119] Divide the node set into candidate blocks according to a preset association strength threshold, calculate the ratio of the average degree of nodes in the candidate blocks to the maximum degree to obtain a clustering coefficient, merge and optimize the candidate blocks according to the clustering coefficient, calculate the number of shared nodes between the candidate blocks and assign the shared nodes to the block with the largest clustering coefficient to obtain a data block;
[0120] A globally unique block identifier is generated according to the association strength, clustering coefficient and reference relationship of the data blocks.
[0121] In one specific embodiment, the target node in the JSON data is located based on a semantic addressing table. The semantic addressing table contains a mapping relationship between field paths and semantic information, for example, "customer.address.city" is mapped to "customer's city." The target node location is determined by traversing the JSON document structure and matching the field paths in the semantic addressing table. For JSON data in an e-commerce system, key business nodes such as "orders," "customers," and "products" can be located.
[0122] When extracting the reference relationship of the target node, it is divided into two categories: explicit reference and implicit reference. Explicit references include foreign key references and ID references. For example, "customer_id: 1001" in the "order" node points to the record with "id: 1001" in the "customers" collection, constituting a foreign key reference relationship; "ref: # / definitions / product" in the "product_info" node points to the "product" definition in the document, constituting an ID reference. Implicit references are determined by calculating the co-occurrence frequency and naming rule similarity of the node attribute values. For example, the "shipping_address" and "billing_address" nodes have similar field structures (street, city, zip), a naming rule similarity of 0.85, and appear together in multiple transaction records with a co-occurrence frequency of 0.92, so they are determined to be implicit reference relationships.
[0123] A dependency topology graph is constructed using target nodes as graph nodes and reference relationships as connecting edges. For example, in an order management system, the "Order" node references the "Customer" node, the "Customer" node references the "Address" node, and the "Order" node references the "Product" node, forming the initial dependency topology graph. The transitive closure matrix of this graph is calculated to identify a subset of nodes with circular references, such as "Department" referencing "Employee" and "Employee" referencing "Department," forming a circular reference. The dependency topology graph is optimized using the Kruskal minimum spanning tree algorithm, retaining the edges with the highest association strength and removing redundant connections. During the optimization process, the weights of edges with circular references are set to lower values to ensure that they are not preferentially selected in the minimum spanning tree.
[0124] Calculating the strength of associations between nodes in a dependency topology graph involves multiple factors. The distance association factor is calculated by calculating the shortest path length between nodes. For example, if the shortest path length from "Order" to "Customer" is 1, the distance association factor is 1.0; if the shortest path length from "Order" to "Address" is 2, the distance association factor is 0.5. The reference association factor is calculated by counting the frequency of references. For example, if "Order" references "Product" an average of 3.5 products per order, the reference association factor is 0.85. The feature association factor calculates the intersection of data type matching and value ranges. For example, if "Product.price" and "OrderItem.unit_price" are both decimal data types and have a 90% overlap in their value ranges, the feature association factor is 0.9. The update association factor calculates the correlation coefficient of the node data update time series. For example, the update time series correlation coefficient of "Inventory" and "OrderItem" nodes is 0.82, indicating a high correlation between their updates. The four factors are weighted and summed according to the weights of 0.3, 0.25, 0.25, and 0.2 to obtain the comprehensive correlation strength between nodes.
[0125] The node set is divided into candidate blocks based on a preset association strength threshold of 0.65. For example, nodes "Order," "OrderItem," and "Shipment" with an association strength greater than 0.65 are assigned to the same candidate block. The clustering coefficient is calculated by calculating the ratio of the average degree of nodes within a candidate block to their maximum degree. For example, if block A has an average degree of 3.2 and a maximum degree of 5, the clustering coefficient is 0.64; block B has a clustering coefficient of 0.58. If the difference in clustering coefficients between two candidate blocks is less than 0.1 and they share nodes, they are merged for optimization. For shared nodes, their connectivity is calculated across each candidate block and assigned to the block with the highest connectivity. For example, if the "PaymentInfo" node is referenced by both the "Order" block and the "Customer" block, and has a connectivity of 4 in the "Order" block and a connectivity of 2 in the "Customer" block, "PaymentInfo" will be assigned to the "Order" block.
[0126] A globally unique block identifier is generated based on the data block's association strength, clustering coefficient, and reference relationships. The identifier format is "prefix-content hash-timestamp," where the prefix indicates the business type, such as "ORD" for an order-related block; the content hash is calculated using the node ID within the block and the association strength; and the timestamp is the block creation time. For example, the identifier for an order-related data block might be "ORD-8F4D2A1B-20230615142538." This identifier uniquely identifies the data block within the distributed system, facilitating subsequent data access, updates, and consistency maintenance.
[0127] Through the above methods, we can achieve precise positioning of nodes in JSON data, extraction of association relationships, construction of data blocks and allocation of globally unique identifiers, providing technical support for efficient management and access of complex data structures.
[0128] In an optional embodiment, based on the target keyword, a candidate semantic identifier is retrieved in the semantic addressing table, a target data block and a write hierarchical position are determined, the JSON data to be appended is written into the target data block, the node association relationship in the dependency topology graph is updated, and the block change information is recorded, including:
[0129] A set of candidate semantic identifiers is retrieved from the semantic addressing table according to the target keyword, the temporal activity and citation heat of each semantic identifier in the candidate semantic identifier set are calculated, and a weighted calculation is performed to obtain a priority score of the semantic identifier, and the semantic identifier with the highest priority score is selected to determine the target data block;
[0130] Performing structural analysis on the JSON data to be appended to construct an attribute association tree, calculating the inheritance depth and association width between nodes based on the attribute association tree, determining the hierarchical position of the data write based on the inheritance depth and association width, and writing the JSON data to be appended into the target data block;
[0131] A bidirectional reference index is constructed for the newly added node in the dependency topology graph, and the circular reference path between nodes is identified based on the bidirectional reference index. The association relationship with the weakest reference strength in the circular reference path is marked as a soft connection. The soft connection only retains the one-way reference between nodes and is automatically disassociated after the expiration timestamp expires. The closed loop path in the circular reference path is blocked by the soft connection, the node association relationship in the dependency topology graph is updated, and the block change information is recorded.
[0132] In one embodiment, each semantic identifier is associated with a specific data block and records temporal activity and citation popularity. Temporal activity indicates how frequently the semantic identifier has been accessed recently, while citation popularity indicates how many times the semantic identifier has been referenced by other nodes.
[0133] When appending JSON data, a target keyword, such as "user preferences," is received. A set of candidate semantic identifiers related to "user preferences" is retrieved from the semantic addressing table, for example, three candidate semantic identifiers, ["user_pref_01," "preference_data_02," and "user_settings_03," are obtained.
[0134] For each candidate semantic identifier, obtain its temporal activity and citation popularity. Assume that the temporal activity of "user_pref_01" is 0.8 and the citation popularity is 0.5; the temporal activity of "preference_data_02" is 0.6 and the citation popularity is 0.9; and the temporal activity of "user_settings_03" is 0.4 and the citation popularity is 0.3. Perform a weighted calculation on the temporal activity and citation popularity, with a temporal activity weight of 0.6 and a citation popularity weight of 0.4. The calculated priority score for "user_pref_01" is 0.68, the priority score for "preference_data_02" is 0.72, and the priority score for "user_settings_03" is 0.36. Therefore, "preference_data_02," with the highest priority score, is selected as the target semantic identifier, and its associated data block is determined as the target data block.
[0135] After obtaining the target data block, perform structural analysis on the JSON data to be appended and build an attribute association tree. Assume that the JSON data to be appended is: {"display": {"theme": "dark", "fontSize": 14}, "notification": {"email": true, "push": {"enabled": true, "quiet_hours": {"start": "22:00", "end": "06:00"}}}}}
[0136] Analyze the JSON structure and construct a property association tree. The root node is the entire JSON object. The first-level child nodes include "display" and "notification." The second-level child nodes include "theme," "fontSize," "email," and "push." Calculate the inheritance depth and association width between nodes. The inheritance depth and association width of "display" and "notification" are 1 and 2, respectively; the inheritance depth and association width of "theme" and "fontSize" are 2 and 2, respectively; and the inheritance depth and association width of "push.quiet_hours.start" are 4 and 1, respectively.
[0137] Based on the calculation results, the hierarchical position of the data is determined. Nodes with an inheritance depth greater than 3 and an association width less than 2, such as the "push.quiet_hours" section, are treated as independent subdocuments. Nodes with an inheritance depth less than or equal to 3 or an association width greater than or equal to 2 are directly inlined into the parent document. This way, the JSON data to be appended is written to the target data block according to the determined hierarchical position.
[0138] After writing the data, a bidirectional reference index is constructed for the newly added node in the dependency graph. The dependency graph is a directed graph, where nodes represent data entities and edges represent reference relationships. Suppose the original dependency graph already contains Node 1 (basic user information) and Node 2 (user permission settings), and a new Node 3 (user preferences) is added. A reference relationship is created for Node 3 pointing to Node 1, and Node 2 also references Node 3, forming a reference chain from Node 1 to Node 2 and back to Node 3.
[0139] Identify potential circular reference paths based on the bidirectional reference index. If node 2 is also referenced by node 1, a circular reference path will form: node 1 ← node 3 ← node 2 ← node 1. Calculate the reference strength of each reference relationship, which is based on factors such as reference frequency and data dependency. Assuming the reference strength of node 1 ← node 2 is 0.8, the reference strength of node 3 ← node 2 is 0.7, and the reference strength of node 2 ← node 1 is 0.5, then node 2 ← node 1 is the weakest reference relationship.
[0140] Mark Node 2 ← Node 1 as a soft link. This soft link only retains a one-way reference and sets an expiration timestamp of 24 hours. By setting Node 2 ← Node 1 as a soft link, the loop in the circular reference path is successfully blocked, avoiding the data dependency infinite loop problem.
[0141] Updates the node relationships in the dependency topology graph and records block change information, including the change timestamp, operation type (append), target data block identifier, and a summary of the written data content. Change information is saved in a dedicated block log for subsequent version tracking and data recovery.
[0142] Through the above implementation, it is possible to efficiently retrieve target data blocks based on semantic identifiers, reasonably arrange hierarchical storage of JSON data, and maintain the dependencies between data nodes, avoid circular reference problems, and achieve efficient data management and access.
[0143] The target keyword-based JSON data dynamic appending and management system according to an embodiment of the present invention includes:
[0144] The first unit is used to receive target keywords and JSON data;
[0145] The second unit is used to convert the target keyword into a feature vector by using grammatical dependency analysis, generate a keyword affinity matrix by calculating cosine similarity, and perform an adaptive classification vector clustering operation based on the keyword affinity matrix to obtain a keyword cluster family;
[0146] The third unit is used to assign semantic identifiers to keyword clusters, establish a mapping relationship between the semantic identifiers and nodes in the JSON data, and generate a semantic addressing table;
[0147] A fourth unit is configured to locate a target node in the JSON data according to the semantic addressing table, extract explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculate the association strength between nodes, construct a data block based on the association strength, and assign a globally unique block identifier to the data block;
[0148] The fifth unit is used to retrieve a candidate semantic identifier in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information;
[0149] The sixth unit is configured to update the semantic addressing table and the dependency topology map according to the block change information.
[0150] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:
[0151] processor;
[0152] a memory for storing processor-executable instructions;
[0153] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0154] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.
[0155] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for dynamically appending and managing JSON data based on target keywords, characterized in that: include: Receive target keywords and JSON data; The target keywords are converted into feature vectors by using grammatical dependency analysis, and a keyword affinity matrix is generated by cosine similarity calculation. Based on the keyword affinity matrix, an adaptive classification vector clustering operation is performed to obtain keyword cluster families. Assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table; Locating the target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block based on the association strength, and assigning a globally unique block identifier to the data block; Retrieve candidate semantic identifiers in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information; The semantic addressing table and the dependency topology map are updated according to the block change information.
2. The method according to claim 1, characterized in that The target keywords are converted into feature vectors using grammatical dependency analysis. The keyword affinity matrix is generated by cosine similarity calculation. Based on the keyword affinity matrix, adaptive classification vector clustering operation is performed to obtain keyword cluster families including: Segmenting the target keyword to obtain a word sequence, using part-of-speech tagging to obtain the part-of-speech features of each word in the word sequence, constructing a grammatical dependency tree based on the part-of-speech features and position information, extracting syntactic relationship features from the grammatical dependency tree, and combining the part-of-speech features and syntactic relationship features to generate a semantic feature matrix; Performing dimensionality reduction on the semantic feature matrix based on orthogonal transformation, selecting feature dimensions in sequence according to variance contribution rates, and combining corresponding eigenvalues to form a target feature vector; Calculating the cosine distance between any two target feature vectors to obtain a similarity value, constructing an initial similarity matrix based on the similarity value, and normalizing the element values in the initial similarity matrix to determine an affinity matrix; Calculating the mean and variance of each row of elements in the affinity matrix, setting a dynamic threshold based on the mean and variance, and setting element values smaller than the dynamic threshold to zero to obtain an optimized affinity matrix; The Laplace matrix of the optimized affinity matrix is calculated, the eigenvector of the Laplace matrix is extracted to construct a feature space, and an adaptive classification vector clustering operation is performed in the feature space to obtain a keyword cluster family.
3. The method according to claim 2, characterized in that The keyword clustering families obtained by performing adaptive classification vector clustering operations in the feature space include: Mapping keywords in the feature space into sample points; Establish a search radius for each sample point, count the number of samples within the search radius to obtain a local density value; calculate the Euclidean distance between any two sample points, and for each target sample point, select the sample point with the smallest Euclidean distance from all sample points whose local density value is greater than that of the target sample point, determine the comparison sample point, and record the Euclidean distance between the target sample point and the comparison sample point as the distance value; Calculating the product of the local density value and the distance value to obtain a density index, sorting the density indexes in descending order, setting a density index threshold according to the numerical distribution of the density index, and using sample points greater than the density index threshold as initial clustering points; Construct a classifier, using the initial cluster points as the classification vector of the classifier; input sample points one by one, calculate the similarity between the sample points and each classification vector, and select the classification vector with the highest similarity as the matching vector; Based on the position of the current input sample point and the matching vector, adjusting the position of the matching vector according to a preset distance ratio, and adjusting other classification vectors adjacent to the matching vector; Repeat the process until the classification vector becomes stable and the sample points associated with the same classification vector are grouped into one category to obtain a keyword cluster family.
4. The method according to claim 1, wherein Assigning semantic identifiers to keyword clusters, establishing a mapping relationship between the semantic identifiers and nodes in JSON data, and generating a semantic addressing table include: Perform word frequency statistics on the keywords in the keyword cluster, extract the part-of-speech features and syntactic dependencies of the keywords to construct semantic vectors, calculate the semantic topic categories of the keyword cluster based on the semantic vectors, and assign semantic identifiers to the keyword clusters in combination with preset hierarchical semantic encoding rules; Parse the hierarchical structure of JSON data to obtain JSON nodes, extract the key name and data type of the JSON nodes, determine the parent-child relationship and sibling relationship between the JSON nodes, construct a tree-structured adjacency matrix of the JSON nodes, and calculate the relationship distance between the JSON nodes based on the tree-structured adjacency matrix; Calculate the text similarity between the semantic identifier and the key name information of the JSON node, construct a semantic context vector based on the relationship distance between the JSON nodes, and perform weighted fusion of the text similarity and the semantic context vector to obtain a semantic relevance matrix; Setting a mapping threshold to filter the semantic relevance matrix, eliminating redundant mapping relationships based on transitive closure, handling conflicts between one-to-many mapping and many-to-one mapping, and determining a mapping relationship between a semantic identifier and a JSON node; A semantic identification index and a node path index are constructed based on the mapping relationship to form a bidirectional mapping structure, and a semantic addressing table is generated according to the bidirectional mapping structure.
5. The method according to claim 1, wherein Locating the target node in the JSON data according to the semantic addressing table, extracting explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculating the association strength between nodes, constructing a data block according to the association strength, and assigning a globally unique block identifier to the data block includes: Locating the target node according to the semantic addressing table, extracting the reference relationship of the target node, including explicit reference relationship and implicit reference relationship, wherein the key-value correspondence relationship in the target node is extracted to determine the foreign key reference and ID reference as the explicit reference relationship, and calculating the co-occurrence frequency of the node attribute value and the similarity of the naming rule to determine the implicit reference relationship; Using the target node as a graph node and the reference relationship as a connection edge to construct a dependency topology graph, calculating the transitive closure matrix of the dependency topology graph to identify a node subset with circular references, and optimizing the dependency topology graph using a minimum spanning tree algorithm; Calculate the shortest path length between nodes in the dependency topology graph to obtain a distance correlation factor, count the citation frequency to obtain a citation correlation factor, calculate the data type matching degree and the intersection of the numerical interval to obtain a feature correlation factor, calculate the correlation coefficient of the node data update time series to obtain an update correlation factor, and perform weighted summation of the distance correlation factor, the citation correlation factor, the feature correlation factor, and the update correlation factor to obtain the correlation strength between the nodes; Divide the node set into candidate blocks according to a preset association strength threshold, calculate the ratio of the average degree of nodes in the candidate blocks to the maximum degree to obtain a clustering coefficient, merge and optimize the candidate blocks according to the clustering coefficient, calculate the number of shared nodes between the candidate blocks and assign the shared nodes to the block with the largest clustering coefficient to obtain a data block; A globally unique block identifier is generated according to the association strength, clustering coefficient and reference relationship of the data blocks.
6. The method according to claim 1, characterized in that Based on the target keyword, a candidate semantic identifier is retrieved in the semantic addressing table, the target data block and the write hierarchical position are determined, the JSON data to be appended is written into the target data block, the node association relationship in the dependency topology graph is updated, and the block change information is recorded, including: A set of candidate semantic identifiers is retrieved from the semantic addressing table according to the target keyword, the temporal activity and citation heat of each semantic identifier in the candidate semantic identifier set are calculated, and a weighted calculation is performed to obtain a priority score of the semantic identifier, and the semantic identifier with the highest priority score is selected to determine the target data block; Performing structural analysis on the JSON data to be appended to construct an attribute association tree, calculating the inheritance depth and association width between nodes based on the attribute association tree, determining the hierarchical position of the data write based on the inheritance depth and association width, and writing the JSON data to be appended into the target data block; A bidirectional reference index is constructed for the newly added node in the dependency topology graph, and the circular reference path between nodes is identified based on the bidirectional reference index. The association relationship with the weakest reference strength in the circular reference path is marked as a soft connection. The soft connection only retains the one-way reference between nodes and is automatically disassociated after the expiration timestamp expires. The closed loop path in the circular reference path is blocked by the soft connection, the node association relationship in the dependency topology graph is updated, and the block change information is recorded.
7. A target keyword-based JSON data dynamic appending and management system for implementing the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to receive target keywords and JSON data; The second unit is used to convert the target keyword into a feature vector by using grammatical dependency analysis, generate a keyword affinity matrix by calculating cosine similarity, and perform an adaptive classification vector clustering operation based on the keyword affinity matrix to obtain a keyword cluster family; The third unit is used to assign semantic identifiers to keyword clusters, establish a mapping relationship between the semantic identifiers and nodes in the JSON data, and generate a semantic addressing table; A fourth unit is configured to locate a target node in the JSON data according to the semantic addressing table, extract explicit reference relationships and implicit reference relationships to construct a dependency topology graph, calculate the association strength between nodes, construct a data block based on the association strength, and assign a globally unique block identifier to the data block; The fifth unit is used to retrieve a candidate semantic identifier in the semantic addressing table based on the target keyword, determine the target data block and the write hierarchical position, write the JSON data to be appended into the target data block, update the node association relationship in the dependency topology graph, and record the block change information; The sixth unit is configured to update the semantic addressing table and the dependency topology map according to the block change information.
8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Data grading and classifying method and device
CN119538118A
Traditional Chinese medicinal material data management system and method for data retrieval
CN120045744A
Sales auxiliary system and method based on AI technology
CN120450656A
Method, apparatus, and system for providing an ai-based business and manufacturer matching platform service
KR102809718B1
Cited By
Automatic discovery method for database mode association for penetrating audit
CN122088658A
User sensitive data desensitization method and system for local database
CN122113172A
A method and system for user sensitive data desensitization for local databases
CN122113172B
