Dictionary-based government-enterprise unstructured data element parsing component implementation method

Abstract

The application provides a dictionary-based government-enterprise unstructured data element analysis component implementation method, relates to the technical field of data processing, and comprises the following steps: performing word segmentation on unstructured data and constructing a feature matrix; establishing a character prefix tree in the vertical direction, constructing an attribute association table in the horizontal direction, setting a cross-reference pointer and an attribute inheritance mechanism; dividing attribute-enhanced entity words into sub-dictionary fragments, distributing the sub-dictionary fragments to multiple element analysis channels according to semantic association strength for parallel processing; and finally, merging analysis results to generate a structured data element set. The application improves the accuracy and efficiency of government-enterprise data element analysis.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for implementing a dictionary-based component for parsing unstructured data elements in government and enterprises. Background Technology

[0002] With the deepening of digital transformation in government and enterprises, the accumulation of a large amount of unstructured data has brought significant challenges to data analysis and utilization. Government and enterprise departments generate various forms of unstructured data in their daily work, including official documents, reports, and meeting minutes. While this data contains rich informational value, its inconsistent formats and loose structures make it difficult to directly use for business analysis and decision support. Traditional data processing methods are primarily designed for structured data and are ill-suited for effectively processing and analyzing unstructured data in the government and enterprise environment. Therefore, there is an urgent need to develop an efficient method for parsing unstructured data elements in government and enterprise systems.

[0003] Existing unstructured data element parsing technologies have significant shortcomings when applied to the government and enterprise sectors. Traditional dictionary-based entity recognition methods typically employ a flat dictionary structure, lacking effective expression of the business attributes and semantic relationships of entity words, resulting in low accuracy when recognizing government and enterprise terminology and specific domain entities. Existing data parsing systems often use a single processing channel, making it difficult to cope with the diverse and cross-departmental data processing needs within the government and enterprise sectors, leading to low processing efficiency and limited scalability. When processing large-scale government and enterprise data, existing technologies lack effective data sharding and parallel processing mechanisms, failing to guarantee the consistency and integrity of results during massive data processing, thus limiting the system's application effectiveness in real-world business scenarios. Summary of the Invention

[0004] This invention provides a dictionary-based method for parsing unstructured data elements of government and enterprises, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a method for implementing a dictionary-based unstructured data element parsing component for government and enterprises, comprising:

[0006] Unstructured data from government and enterprises is segmented into words, and the entity words to be matched in the segmentation results are used to construct a feature matrix.

[0007] For the feature matrix, a character prefix tree is built in the vertical dimension to store the character sequence information of entity words, and an attribute association table is built in the horizontal dimension to record the business attribute information of entity words; a cross-reference pointer is established to associate the character prefix tree with the attribute association table, and an attribute inheritance mechanism is set to store the common business attributes of entity words with the same prefix in the parent node. When a dictionary query is executed, the cross-reference pointer is used for bidirectional retrieval, and the bidirectional retrieval results are used to obtain attribute-enhanced entity words through the attribute inheritance mechanism.

[0008] The attribute-enhanced entity words are divided into multiple sub-dictionary segments according to business domains, and multiple independent element parsing channels are set up. The semantic association strength between the sub-dictionary segments is calculated, and the sub-dictionary segments with semantic association strength exceeding a preset association threshold are assigned to the same element parsing channel. Each element parsing channel performs attribute extraction and value range verification in parallel to obtain element parsing results, and the integrity of the element parsing results is guaranteed by a consistent hashing algorithm.

[0009] The feature parsing results from multiple independent feature parsing channels are merged to generate a structured data feature set, which is then encapsulated as an independent functional component.

[0010] A character prefix tree is built vertically to store the character sequence information of entity words, and an attribute association table is constructed horizontally to record the business attribute information of entity words, including:

[0011] The character sequence is inserted sequentially into the empty prefix tree to establish parent-child relationships between nodes, thus obtaining the basic prefix tree. For each node in the basic prefix tree, the frequency ratio is obtained by calculating the ratio of the frequency of the node's character to the sum of the frequencies of all nodes' characters. The information entropy is obtained by calculating the logarithm of the ratio of the total number of words to the number of words containing the character of that node. The node weight is obtained by multiplying the frequency ratio by the information entropy. Nodes with a weight lower than a preset weight threshold are merged and compressed. The parent-child relationships are recalculated and the node depth is updated for the merged and compressed nodes to obtain the character prefix tree.

[0012] Business attribute information is categorized according to attribute type. For enumeration type attribute values, a mapping dictionary is established to map enumeration values ​​to integer identifiers. For text type attribute values, their feature vectors are calculated and converted to numerical identifiers. For time type attribute values, they are converted to timestamps. A multi-level hash structure is constructed, and a corresponding weight coefficient is set for each level of the hash structure. The converted business attribute information is sequentially processed through each level of the hash structure to obtain multiple hash values. The multiple hash values ​​are multiplied by their corresponding weight coefficients and summed to obtain the hash position of the business attribute information. The business attribute information is then stored in an attribute association table based on the hash position.

[0013] Establish a cross-reference pointer to associate the character prefix tree with the attribute association table, set up an attribute inheritance mechanism, and store the common business attributes of entity words with the same prefix in the parent node. When performing a dictionary query, use the cross-reference pointer to perform bidirectional retrieval, and obtain attribute-enhanced entity words from the bidirectional retrieval results through the attribute inheritance mechanism, including:

[0014] The tree pointer pointing to the character prefix tree node and the table pointer pointing to the attribute association table entry are combined into a cross-reference pointer; the reference weight of the cross-reference pointer is obtained by weighting the tree node depth, the number of associated attributes, and the reference frequency.

[0015] The nodes in the character prefix tree are layered according to the reference weight. In each layer, parent and child node pairs are identified and feature vectors are constructed based on their attribute sets. The attribute similarity is obtained by calculating the cosine similarity between the feature vectors of the parent node and the feature vectors of the child node, and the character similarity is calculated based on the character edit distance between nodes. The weighted sum of the attribute similarity and the character similarity is used as the attribute matching degree. When the attribute matching degree is greater than a preset inheritance threshold, the matching attribute of the parent node and its corresponding reference weight are stored in the parent attribute of the child node, forming a multi-level attribute inheritance chain with weights.

[0016] For nodes obtained by bidirectional retrieval, their attribute enhancement weights are calculated. The attribute enhancement weights are inversely proportional to the inheritance depth of the node in the multi-level attribute inheritance chain and directly proportional to the usage frequency and inheritance confidence of the attribute in the multi-level attribute inheritance chain. Based on the attribute enhancement weights, the attributes on the multi-level attribute inheritance chain are prioritized and ranked. Attributes whose ranking results are higher than a preset priority threshold are merged into the retrieval nodes to generate attribute-enhanced entity words.

[0017] The attribute-enhanced entity words are divided into multiple sub-dictionary segments according to business domains. Multiple independent element parsing channels are set up. The semantic association strength between the sub-dictionary segments is calculated. Sub-dictionary segments with semantic association strength exceeding a preset association threshold are assigned to the same element parsing channel, including:

[0018] A multi-dimensional feature vector is established based on the characteristics of the business domain. The attribute-enhanced entity words are initially divided into multiple basic segments using the multi-dimensional feature vector as the feature benchmark. The segment density is calculated based on the ratio of the total usage frequency of the words in the basic segments to the number of words. The basic segments are then uniformly divided based on the segment density to obtain sub-dictionary segments.

[0019] Feature vectors from the sub-dictionary segments are extracted to construct a high-order tensor. This high-order tensor is then decomposed and dimensionality reduced to obtain a semantic feature matrix. The cosine distance of the semantic feature matrix is ​​calculated to obtain the domain relevance. Simultaneously, the sub-dictionary segments are constructed as a graph structure. Random walk sampling is performed on the graph structure to obtain a node sequence. The Skip-gram algorithm is used to map the node sequence to a low-dimensional vector space to obtain topological features. The structural relevance is calculated based on the Euclidean distance of the topological features. The domain relevance and the structural relevance are then dynamically fused nonlinearly to obtain the semantic relevance strength. Sub-dictionary segments with semantic relevance strength exceeding a preset relevance threshold are grouped into the same cluster.

[0020] Multiple feature parsing channels are established, and an independent parsing thread pool is allocated to each parsing channel. Each parsing thread pool contains multiple parallel processing threads. Sub-dictionary fragments in the same cluster are assigned to the corresponding feature parsing channels, and the parallel processing threads in the parsing thread pools are used to perform feature parsing operations on the sub-dictionary fragments.

[0021] Each feature parsing channel performs attribute extraction and value range verification in parallel to obtain feature parsing results, and the integrity of the feature parsing results is guaranteed by a consistent hashing algorithm, including:

[0022] The dependency strength vector between sub-dictionary segments is calculated based on a preset context dependency relationship template. The co-occurrence probability vector between sub-dictionary segments is calculated based on word frequency statistics. The cosine similarity between the dependency strength vector and the co-occurrence probability vector is calculated to obtain the context relevance.

[0023] Using the sub-dictionary fragments as nodes, connections are established between nodes and the context relevance is used as the connection weight. A semantic network is obtained by iteratively establishing connections between all nodes. A depth-first search algorithm is used to extract association paths in the semantic network, and attribute features are extracted based on the association paths to obtain the attribute to be verified.

[0024] The attribute type consistency score is calculated as a static rule constraint value based on the preset attribute type constraint rules, the attribute dependency integrity score is calculated as a relation constraint value based on the attribute dependency rules, and the attribute temporal continuity score is calculated as a temporal constraint value based on the attribute temporal rules. The static rule constraint value, the relation constraint value, and the temporal constraint value are used to construct a constraint set, and the attribute to be verified that satisfies the constraint set is used as the element parsing result.

[0025] Multiple virtual nodes are evenly distributed on a hash ring. The element parsing result is mapped to the virtual node using a consistent hashing algorithm. The integrity of the element parsing result is verified by calculating and comparing the hash values ​​of the element parsing result on different virtual nodes.

[0026] The feature parsing results from multiple independent feature parsing channels are merged to generate a structured data feature set. This structured data feature set is then encapsulated into independent functional components, including:

[0027] A bidirectional attention mechanism is used to extract features from the feature parsing results to obtain an attention weight matrix. The information entropy is then calculated by performing matrix multiplication on the attention weight matrix and the feature parsing results to obtain the confidence score of the feature parsing channel.

[0028] The confidence score of each element parsing channel is weighted and summed with the corresponding element parsing result to obtain a fusion result containing multiple elements and their associated information. Based on the semantic information of the elements in the fusion result, the elements are subjected to semantic dependency analysis according to a preset attribute template to construct an association network between elements. Based on the association network, the elements are organized according to a hierarchical structure of category-attribute-value to obtain a structured data element set.

[0029] The structured data element set is serialized and encoded to obtain a binary data stream. A component framework is constructed according to a preset component template. The binary data stream is encapsulated in the component framework to obtain a data layer. The binary data stream is parsed and a data operation interface and a function call interface are constructed to obtain an interface layer. The function call interface is encapsulated as a function execution module to obtain a function layer. The interface layer, the function layer and the data layer are packaged into an independent functional component.

[0030] A second aspect of this invention provides a dictionary-based system for parsing unstructured data elements in government and enterprise data, comprising:

[0031] The first unit is used to perform word segmentation on unstructured data from government and enterprises, and to construct a feature matrix from the entity words to be matched in the word segmentation results.

[0032] The second unit is used to build a character prefix tree in the vertical dimension of the feature matrix to store the character sequence information of entity words, and to build an attribute association table in the horizontal dimension to record the business attribute information of entity words; to establish a cross-reference pointer to associate the character prefix tree with the attribute association table, to set an attribute inheritance mechanism, to store the common business attributes of entity words with the same prefix in the parent node, and to use the cross-reference pointer to perform bidirectional retrieval when performing dictionary query, and to obtain attribute-enhanced entity words through the attribute inheritance mechanism from the bidirectional retrieval results;

[0033] The third unit is used to divide the attribute-enhanced entity words into multiple sub-dictionary segments according to the business domain, set up multiple independent element parsing channels, calculate the semantic association strength between the sub-dictionary segments, and assign sub-dictionary segments with semantic association strength exceeding a preset association threshold to the same element parsing channel; each element parsing channel performs attribute extraction and value range verification in parallel to obtain element parsing results, and ensures the integrity of element parsing results through a consistent hashing algorithm;

[0034] The fourth unit is used to merge the feature parsing results of multiple independent feature parsing channels to generate a structured data feature set, and to encapsulate the structured data feature set into an independent functional component.

[0035] A third aspect of the present invention,

[0036] An electronic device is provided, comprising:

[0037] processor;

[0038] Memory used to store processor-executable instructions;

[0039] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0040] Fourth aspect of the embodiments of the present invention,

[0041] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0042] The beneficial effects of this application are as follows:

[0043] By constructing a feature matrix structure of character prefix trees and attribute association tables, and establishing cross-reference pointers and attribute inheritance mechanisms, efficient entity word querying and attribute enhancement are achieved, storage redundancy is reduced, and data retrieval efficiency and matching accuracy are improved.

[0044] By dividing attribute-enhanced entity words into sub-dictionary fragments according to business domains and setting up multiple independent parsing channels, calculating semantic association strength and allocating parsing resources, parallel processing and load balancing are achieved, which significantly improves the throughput and response speed of the system in processing large-scale government and enterprise unstructured data.

[0045] The consistent hashing algorithm is used to ensure the integrity of the parsing results. The parsing results are merged into a set of structured data elements and encapsulated into independent functional components, which improves the accuracy and reusability of data element parsing and facilitates flexible deployment and expansion of applications in different government and enterprise business systems. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the implementation method of the dictionary-based unstructured data element parsing component for government and enterprises according to an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram of the technical process for element analysis and component construction. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0050] Figure 1 This is a flowchart illustrating the implementation method of the dictionary-based unstructured data element parsing component for government and enterprises according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0051] Unstructured data from government and enterprises is segmented into words, and the entity words to be matched in the segmentation results are used to construct a feature matrix.

[0052] For the feature matrix, a character prefix tree is built in the vertical dimension to store the character sequence information of entity words, and an attribute association table is built in the horizontal dimension to record the business attribute information of entity words; a cross-reference pointer is established to associate the character prefix tree with the attribute association table, and an attribute inheritance mechanism is set to store the common business attributes of entity words with the same prefix in the parent node. When a dictionary query is executed, the cross-reference pointer is used for bidirectional retrieval, and the bidirectional retrieval results are used to obtain attribute-enhanced entity words through the attribute inheritance mechanism.

[0053] The attribute-enhanced entity words are divided into multiple sub-dictionary segments according to business domains, and multiple independent element parsing channels are set up. The semantic association strength between the sub-dictionary segments is calculated, and the sub-dictionary segments with semantic association strength exceeding a preset association threshold are assigned to the same element parsing channel. Each element parsing channel performs attribute extraction and value range verification in parallel to obtain element parsing results, and the integrity of the element parsing results is guaranteed by a consistent hashing algorithm.

[0054] The feature parsing results from multiple independent feature parsing channels are merged to generate a structured data feature set, which is then encapsulated as an independent functional component.

[0055] In one optional implementation, a character prefix tree is built in the vertical dimension to store the character sequence information of entity words, and an attribute association table is constructed in the horizontal dimension to record the business attribute information of entity words, including:

[0056] The character sequence is inserted sequentially into the empty prefix tree to establish parent-child relationships between nodes, thus obtaining the basic prefix tree. For each node in the basic prefix tree, the frequency ratio is obtained by calculating the ratio of the frequency of the node's character to the sum of the frequencies of all nodes' characters. The information entropy is obtained by calculating the logarithm of the ratio of the total number of words to the number of words containing the character of that node. The node weight is obtained by multiplying the frequency ratio by the information entropy. Nodes with a weight lower than a preset weight threshold are merged and compressed. The parent-child relationships are recalculated and the node depth is updated for the merged and compressed nodes to obtain the character prefix tree.

[0057] Business attribute information is categorized according to attribute type. For enumeration type attribute values, a mapping dictionary is established to map enumeration values ​​to integer identifiers. For text type attribute values, their feature vectors are calculated and converted to numerical identifiers. For time type attribute values, they are converted to timestamps. A multi-level hash structure is constructed, and a corresponding weight coefficient is set for each level of the hash structure. The converted business attribute information is sequentially processed through each level of the hash structure to obtain multiple hash values. The multiple hash values ​​are multiplied by their corresponding weight coefficients and summed to obtain the hash position of the business attribute information. The business attribute information is then stored in an attribute association table based on the hash position.

[0058] In practical applications, one can first obtain the set of entity words to be processed and their corresponding business attribute information. For example, the set of product entity words for an e-commerce platform includes entity words such as "mobile phone", "watch", and "smartwatch", as well as their corresponding attributes such as category, price range, and sales status.

[0059] When building a character prefix tree in the vertical dimension, the character sequence is inserted sequentially into the empty prefix tree. Taking "phone", "watch", and "smartwatch" as examples, first, a root node is created. Then, for "phone", a "hand" node is created as a child node of the root node, and then a "machine" node is created as a child node of the "hand" node. Similarly, a path is created for "watch". Since the "hand" node already exists, only the "watch" node needs to be added as a child node of the "hand" node. For "smartwatch", a path is created for "smart", "power", "hand", and "watch". The "hand" and "watch" nodes already exist, so only the "smart" and "power" nodes need to be created and their corresponding parent-child relationships established. This yields the basic prefix tree.

[0060] Next, calculate the weight of each node. Take the "hand" node as an example. Suppose among all entity words, the word "hand" appears 3 times, and the total frequency of all node characters is 10. Then the frequency ratio of the "hand" node is 3 / 10 = 0.3. Suppose the total number of words is 3, and the number of words containing the word "hand" is 3. Then the information entropy is log(3 / 3) = 0. Multiply the frequency ratio by the information entropy to get the weight of the "hand" node as 0.3×0 = 0.

[0061] Suppose the preset weight threshold is 0.1. When it is found that the weights of some nodes are lower than the threshold, merge and compress them with their parent nodes. If the weight of the "able" node is 0.05, which is lower than the threshold 0.1, then merge the "able" node with its parent node "intelligent" to form a new node "intelligent". After the merge, recalculate the parent-child relationship and update the node depth. For example, the original depth of the "hand" node was 2, and it becomes 1 after the merge. In this way, a compressed character prefix tree is obtained, which not only retains the key character information but also reduces the scale of the tree.

[0062] When constructing the attribute association table in the horizontal dimension, first classify the business attribute information according to the attribute type. For the products on an e-commerce platform, they can be divided into enumerated types (such as product category, sales status), text types (such as product description), and time types (such as listing time), etc.

[0063] For the attribute values of the enumerated type, establish a mapping dictionary to convert the enumerated values into integer identifiers. For example, for the product category attribute, a mapping relationship can be established: {"electronic products": 1, "clothing": 2, "home furnishings": 3}. When processing the product category "electronic products" of "mobile phone", convert it to the integer 1.

[0064] For the attribute values of the text type, calculate their feature vectors and then convert them into numerical identifiers. For example, for the product description "high-performance smartphone", the feature vector [0.2, 0.5, 0.3] can be calculated through a text feature extraction algorithm, and then this vector is converted into a numerical identifier 4578 through a dimensionality reduction or hashing algorithm.

[0065] For the attribute values of the time type, directly convert them into timestamps. For example, "2023-05-01 12:00:00" can be converted into the timestamp 1682928000.

[0066] When constructing a multi-level hash structure, a three-level hash structure can be set, each corresponding to different attribute characteristics. A weight coefficient is assigned to each level of the hash structure; for example, the first level has a weight of 0.5, the second level 0.3, and the third level 0.2. Assuming we are processing the attribute information of "mobile phone," the hash values ​​1000, 2000, and 3000 are calculated using the three-level hash functions. Multiplying these values ​​by their corresponding weights and summing them: 1000 × 0.5 + 2000 × 0.3 + 3000 × 0.2 = 1700, yielding the hash position 1700.

[0067] Based on the calculated hash position 1700, the corresponding position is located in the attribute association table and the business attribute information for "mobile phone" is stored. If data already exists at that position, hash collisions are resolved using a linked list or open addressing. Ultimately, each position in the attribute association table stores the complete business attribute information for the entity word.

[0068] In practical applications, when querying information related to "smartwatch," the system first searches for a matching path in the character prefix tree to locate the corresponding leaf node. Then, based on the index associated with that node, it retrieves all business attribute information for "smartwatch" from the attribute association table, such as the category being "electronic products" (integer identifier 1), the price range being "mid-to-high-end" (integer identifier 4), and the listing date being "2023-06-15" (timestamp 1686801600). This combined vertical and horizontal data structure design efficiently stores character sequence information while facilitating rapid retrieval and updating of business attributes, making it particularly suitable for scenarios involving large-scale entity words and their complex attribute information.

[0069] In one optional implementation, a cross-reference pointer is established to associate the character prefix tree with the attribute association table. An attribute inheritance mechanism is set up to store the common business attributes of entity words with the same prefix in the parent node. When a dictionary query is executed, the cross-reference pointer is used for bidirectional retrieval. The bidirectional retrieval results are then processed through the attribute inheritance mechanism to obtain attribute-enhanced entity words, including:

[0070] The tree pointer pointing to the character prefix tree node and the table pointer pointing to the attribute association table entry are combined into a cross-reference pointer; the reference weight of the cross-reference pointer is obtained by weighting the tree node depth, the number of associated attributes, and the reference frequency.

[0071] The nodes in the character prefix tree are layered according to the reference weight. In each layer, parent and child node pairs are identified and feature vectors are constructed based on their attribute sets. The attribute similarity is obtained by calculating the cosine similarity between the feature vectors of the parent node and the feature vectors of the child node, and the character similarity is calculated based on the character edit distance between nodes. The weighted sum of the attribute similarity and the character similarity is used as the attribute matching degree. When the attribute matching degree is greater than a preset inheritance threshold, the matching attribute of the parent node and its corresponding reference weight are stored in the parent attribute of the child node, forming a multi-level attribute inheritance chain with weights.

[0072] For nodes obtained by bidirectional retrieval, their attribute enhancement weights are calculated. The attribute enhancement weights are inversely proportional to the inheritance depth of the node in the multi-level attribute inheritance chain and directly proportional to the usage frequency and inheritance confidence of the attribute in the multi-level attribute inheritance chain. Based on the attribute enhancement weights, the attributes on the multi-level attribute inheritance chain are prioritized and ranked. Attributes whose ranking results are higher than a preset priority threshold are merged into the retrieval nodes to generate attribute-enhanced entity words.

[0073] The character prefix tree stores entity words, with each node containing character information, an identifier indicating whether it is an endpoint, a set of child node pointers, and a node depth value. The attribute association table stores the business attributes of entity words, including entity word ID, attribute type, attribute value, and reference frequency fields. To link the two data structures, cross-reference pointers are constructed, with each pointer containing two parts: a tree pointer and a table pointer, pointing to a node in the character prefix tree and an entry in the attribute association table, respectively.

[0074] The construction process of cross-reference pointers is as follows: For a terminal node representing a complete entity word in the prefix tree, extract its corresponding entity word and retrieve all attribute records of that entity word in the attribute association table. Create a table pointer for each attribute record, and simultaneously create a tree pointer pointing to the terminal node. Combine the two to form a cross-reference pointer. For example, if the entity word "smartphone" corresponds to a terminal node in the prefix tree, and has two attribute records in the attribute association table: "Category: Electronic Products" and "Function: Communication", then create two cross-reference pointers, linking the node to these two attribute records respectively.

[0075] The reference weight for cross-reference pointers is calculated using a weighted combination method: Weight = 0.4 × tree node depth + 0.3 × number of associated attributes + 0.3 × reference frequency. Taking the entity word "smartphone" as an example, if its corresponding node depth is 5, the number of associated attributes is 8, and the reference frequency is 120, then its reference weight is calculated as 0.4 × 5 + 0.3 × 8 + 0.3 × 120 = 38.8.

[0076] Hierarchically classify the nodes of the character prefix tree based on citation weights. The weight range of 0 - 10 is the first layer, 10 - 30 is the second layer, 30 - 60 is the third layer, and above 60 is the fourth layer. In each layer, identify parent - child node pairs with an inclusion relationship. For example, in the third layer, "smartphone" and "smartwatch" correspond to different nodes respectively, and they share the prefix "smart", forming a parent - child relationship.

[0077] For the identified parent - child node pairs, construct a feature vector to represent their set of attributes. The dimension of the feature vector is equal to the number of all attribute types in the system, and each element in the vector represents the importance score of that attribute type. The calculation steps of the importance score are as follows: First, count the total frequency of occurrence F_total of this attribute in all entity words; then, calculate the frequency of occurrence F_current of this attribute in the entity word of the current node; next, calculate the frequency ratio R = F_current / F_total; finally, apply logarithmic normalization processing, score = log(1 + R) / log(2), and limit the result between 0 and 1. For example, if the "function" attribute appears 500 times in all entity words and 100 times in the entity words related to "smartphone", then its frequency ratio is 100 / 500 = 0.2, and the score after logarithmic normalization is approximately 0.26. For example, if there are 10 attribute types such as "category", "function", "price", "brand", etc., the feature vector is 10 - dimensional. The feature vector of the "smartphone" node is [0.8, 0.9, 0.5, 0.7, 0, 0, 0.3, 0, 0.4, 0], representing the importance scores of each attribute type.

[0078] Calculate the cosine similarity between the feature vectors of the parent node and the child node to obtain the attribute similarity. For the "smart" node and the "smartphone" node, assuming their feature vectors are [0.7, 0.8, 0.2, 0.5, 0, 0, 0.2, 0, 0.3,0] and [0.8, 0.9, 0.5, 0.7, 0, 0, 0.3, 0, 0.4, 0] respectively, the calculated attribute similarity is 0.96.

[0079] Calculate the character similarity based on the character edit distance between nodes. Character similarity = 1 - edit distance / longest string length. The edit distance between "smart" and "smartphone" is 2, and the longest string length is 4. Therefore, the character similarity = 1 - 2 / 4 = 0.5.

[0080] Weight - combine the attribute similarity and the character similarity to obtain the attribute matching degree: Attribute matching degree = 0.7×attribute similarity + 0.3×character similarity. For the above example, the attribute matching degree = 0.7×0.96 + 0.3×0.5 = 0.822.

[0081] An inheritance threshold of 0.75 is set. When the attribute matching degree is greater than this threshold, the matching attribute of the parent node and its corresponding reference weight are stored in the parent attribute of the child node. In the example, 0.822 > 0.75, so the attribute of the "Smart" node will be inherited by the "Smartphone" node. The inheritance process considers the matching degree of attributes. For example, the attributes "Category: Electronic Products" and "Function: Communication" have a high matching degree with the parent node's attributes and will be inherited and their reference weights recorded. This forms a multi-level attribute inheritance chain, with each node inheriting attributes from multiple parent nodes.

[0082] When performing a dictionary lookup, cross-reference pointers are used for bidirectional retrieval. Starting from an entity word, the corresponding node in the character prefix tree is found through the tree pointer, and then the relevant attribute in the attribute association table is found through the cross-reference pointer of that node; or starting from an attribute, a specific attribute item in the attribute association table is found through the table pointer, and then all entity words containing that attribute in the character prefix tree are found through the cross-reference pointer.

[0083] The attribute enhancement weight is calculated for nodes obtained from bidirectional retrieval: Attribute Enhancement Weight = Reference Frequency × (1 / Inheritance Depth) × Inheritance Confidence. Here, reference frequency represents the number of times the attribute is used, inheritance depth represents the number of levels the attribute is inherited from the original node, and inheritance confidence represents the reliability of attribute inheritance, determined by attribute matching degree. For example, the "Smartphone" node inherits the "Category: Electronic Products" attribute from the "Smart" node. If the reference frequency is 50, the inheritance depth is 1, and the inheritance confidence is 0.85, then the attribute enhancement weight = 50 × (1 / 1) × 0.85 = 42.5.

[0084] The attributes in the inheritance chain are prioritized based on attribute enhancement weights, with a priority threshold of 30. Attributes higher than the threshold are merged into the retrieval node. In the example above, the attribute weight "Category: Electronic Products" is 42.5, which is greater than 30. Therefore, it will be merged into the attribute set of the "Smartphone" node, forming the attribute-enhanced entity word.

[0085] Through the above method, this invention realizes entity word attribute enhancement based on cross-reference pointers and attribute inheritance mechanism, thereby improving the semantic richness and application value of entity words.

[0086] In one optional implementation, the attribute-enhanced entity words are divided into multiple sub-dictionary segments according to business domains, multiple independent element parsing channels are set, the semantic association strength between the sub-dictionary segments is calculated, and sub-dictionary segments with semantic association strength exceeding a preset association threshold are assigned to the same element parsing channel, including:

[0087] A multi-dimensional feature vector is established based on the characteristics of the business domain. The attribute-enhanced entity words are initially divided into multiple basic segments using the multi-dimensional feature vector as the feature benchmark. The segment density is calculated based on the ratio of the total usage frequency of the words in the basic segments to the number of words. The basic segments are then uniformly divided based on the segment density to obtain sub-dictionary segments.

[0088] Feature vectors from the sub-dictionary segments are extracted to construct a high-order tensor. This high-order tensor is then decomposed and dimensionality reduced to obtain a semantic feature matrix. The cosine distance of the semantic feature matrix is ​​calculated to obtain the domain relevance. Simultaneously, the sub-dictionary segments are constructed as a graph structure. Random walk sampling is performed on the graph structure to obtain a node sequence. The Skip-gram algorithm is used to map the node sequence to a low-dimensional vector space to obtain topological features. The structural relevance is calculated based on the Euclidean distance of the topological features. The domain relevance and the structural relevance are then dynamically fused nonlinearly to obtain the semantic relevance strength. Sub-dictionary segments with semantic relevance strength exceeding a preset relevance threshold are grouped into the same cluster.

[0089] Multiple feature parsing channels are established, and an independent parsing thread pool is allocated to each parsing channel. Each parsing thread pool contains multiple parallel processing threads. Sub-dictionary fragments in the same cluster are assigned to the corresponding feature parsing channels, and the parallel processing threads in the parsing thread pools are used to perform feature parsing operations on the sub-dictionary fragments.

[0090] A multi-dimensional feature vector is established as a classification benchmark based on the characteristics of the business domain. Taking the financial field as an example, a feature vector containing dimensions such as "transaction type", "risk level", "applicable scenario", and "regulatory requirements" can be constructed. For the attribute-enhanced entity term "high-risk cross-border payment", its multi-dimensional feature vector can be represented as [0.8, 0.9, 0.5, 0.7], representing the feature strength of the term in the four dimensions of transaction type, risk level, applicable scenario, and regulatory requirements.

[0091] Based on the aforementioned multidimensional feature vectors, attribute-enhanced entity words are initially segmented, resulting in multiple basic segments. For example, in the financial business domain, basic segments such as "payment," "credit," and "investment and wealth management" are formed. The "payment" basic segment includes entity words such as "online payment," "cross-border payment," and "quick payment." The segment density is calculated for each basic segment, using the ratio of the total frequency of use of terms within that segment to the number of terms. For example, the "payment" segment contains 100 terms with a total frequency of 5000, resulting in a segment density of 50. The "investment and wealth management" segment contains 80 terms with a total frequency of 2400, resulting in a segment density of 30.

[0092] The partitioning is performed uniformly based on the partition density to ensure a relatively balanced processing load across all sub-dictionary partitions. The specific partitioning steps are as follows: First, calculate the density values ​​and statistical distribution of all basic partitions to obtain the density mean μ and standard deviation σ. Then, set an upper limit μ+σ and a lower limit μ-σ for the density threshold. For basic partitions with a density exceeding the upper limit, a feature vector-based clustering algorithm splits them into multiple sub-parts, with the number of splits N calculated as ⌈current density / (μ+0.5σ)⌉. For example, if a partition has a density of 100 and μ+0.5σ=40, it is split into ⌈100 / 40⌉=3 sub-parts. For basic partitions with a density below the lower limit, calculate their feature similarity to other partitions, and merge the partitions with the highest similarity and a combined density not exceeding the upper limit. Iteratively perform the above splitting and merging operations until the density values ​​of all partitions fall within the [μ-σ, μ+σ] interval, or the number of iterations reaches the preset upper limit of 10. The final result is a set of sub-dictionary fragments with a relatively balanced density distribution.

[0093] In terms of semantic features, feature vectors from each sub-dictionary segment are extracted to construct a higher-order tensor. Taking the "cross-border payment" segment as an example, this segment contains terms such as "SWIFT remittance," "telegraphic transfer," and "international payment." Semantic features of each term are extracted, forming a third-order tensor with dimensions of [number of segments × number of terms × feature dimension]. For the constructed higher-order tensor, tensor decomposition is used to decompose it into a combination of core tensors and factor matrices, and then dimensionality reduction is performed to obtain the semantic feature matrix. For example, the original tensor dimension is [10 × 500 × 100], and after tensor decomposition and dimensionality reduction, a semantic feature matrix with dimension [10 × 50] is obtained, where 10 represents the number of segments and 50 represents the feature dimension after dimensionality reduction.

[0094] The domain relevance is obtained by calculating the cosine distance based on the semantic feature matrix. For example, the cosine distance between the semantic feature vectors of the "cross-border payment" segment and the "foreign exchange transaction" segment is 0.82, indicating a high degree of domain relevance between the two; while the cosine distance with the "consumer credit" segment is 0.31, indicating a low degree of relevance.

[0095] In terms of structural association, the sub-dictionary fragments are constructed as a graph structure, with each fragment as a node and the reference relationships between fragments as edges. For example, if entries in the "cross-border payment" fragment frequently co-occur with entries in the "foreign exchange transaction" fragment, an edge is established between the two nodes. A random walk sampling is performed on the constructed graph structure; for example, starting with the "cross-border payment" node, the next node to be visited is randomly selected according to the edge weights, generating a node sequence such as ["cross-border payment"-"foreign exchange transaction"-"international clearing"-"cross-border payment"].

[0096] The Skip-gram algorithm is used to process the above node sequence, mapping each node to a low-dimensional vector space to obtain the topological feature representation of each sub-dictionary segment. The specific steps of the Skip-gram algorithm are as follows: First, the node sequence generated by the random walk is used as the input corpus, and each node is considered as a "word". Then, the sliding window size is set to 5, and for each central node in the sequence, its neighboring nodes within the context window are extracted. Next, two sets of vectors are initialized for each node: an input vector and an output vector, with a dimension of 100. For each pair of central and context nodes, their vector dot product is calculated and converted into a probability distribution using the softmax function. The cross-entropy between this probability distribution and the actually observed context distribution is used as the loss function. The vector representation of all nodes is iteratively optimized using the stochastic gradient descent algorithm, with the initial learning rate set to 0.025 and gradually decreasing with each iteration. After 50 iterations, the input vector of each node is extracted as its final topological feature representation. For example, in the 100-dimensional vector obtained after training the "cross-border payment" node, the first 5 dimensions are [-0.25, 0.4, 0.1, -0.6, 0.3].

[0097] The final semantic association strength is obtained by combining domain relevance and structural relevance using a nonlinear dynamic fusion method. The specific description of the nonlinear dynamic fusion method is as follows: Let the domain relevance be S_sem, the structural relevance be S_str, and the fusion formula be S = α·S_sem p + (1-α)·S_str q Where α is the dynamic weighting coefficient, and p and q are nonlinear adjustment exponents. The dynamic weighting coefficient α is adaptively calculated based on the fragmentation characteristics: α = 0.5 + 0.3·tanh(F_sem - F_str), where F_sem and F_str represent the semantic feature strength and structural feature strength of the fragmentation, respectively. When the semantic features of the fragmentation are more significant, the value of α increases, assigning higher weight to the domain relevance; conversely, the value of α decreases. The nonlinear adjustment exponents p and q are used to adjust the sensitivity of relevance: when the relevance is generally high, p and q are greater than 1, amplifying the difference; when the relevance is generally low, p and q are less than 1, reducing the difference. The values ​​of p and q are dynamically set according to the statistical characteristics of the relevance distribution of the current batch of fragments, usually within the range of [0.8, 1.2]. For example, for the two shards "cross-border payments" and "foreign exchange transactions", the calculated values ​​are S_sem=0.82, S_str=0.85, F_sem=0.75, and F_str=0.70. Therefore, α=0.5+0.3·tanh(0.05)≈0.515, p=1.1, q=0.9, and the final fusion strength is S=0.515·0.82. 1.1 +(1-0.515)·0.85 0.9≈0.84. If the preset association threshold is 0.65, then these two fragments will be assigned to the same cluster.

[0098] Multiple feature parsing channels are established based on the clustering results. For example, three parsing channels are established to handle the three clusters of "Payment and Settlement", "Credit and Financing", and "Investment and Wealth Management" respectively. An independent parsing thread pool is allocated to each parsing channel, such as allocating a thread pool containing 8 parallel processing threads to the "Payment and Settlement" parsing channel.

[0099] Sub-dictionary fragments within the same cluster are assigned to corresponding feature parsing channels, and parallel processing threads in the parsing thread pool are used to perform feature parsing operations on the sub-dictionary fragments. For example, when the input text contains "customer applies for SWIFT international remittance service," it is identified that "SWIFT international remittance" belongs to the "cross-border payment" fragment, which belongs to the "payment clearing" cluster. Therefore, it is processed by the "payment clearing" parsing channel, which parses feature information such as "business type: international remittance" and "channel: SWIFT" from this entry. Through this divide-and-conquer parallel processing mechanism, the processing efficiency and accuracy of feature parsing are significantly improved.

[0100] In one optional implementation, each feature parsing channel performs attribute extraction and value range verification in parallel to obtain feature parsing results, and the integrity of the feature parsing results is guaranteed by a consistent hashing algorithm, including:

[0101] The dependency strength vector between sub-dictionary segments is calculated based on a preset context dependency relationship template. The co-occurrence probability vector between sub-dictionary segments is calculated based on word frequency statistics. The cosine similarity between the dependency strength vector and the co-occurrence probability vector is calculated to obtain the context relevance.

[0102] Using the sub-dictionary fragments as nodes, connections are established between nodes and the context relevance is used as the connection weight. A semantic network is obtained by iteratively establishing connections between all nodes. A depth-first search algorithm is used to extract association paths in the semantic network, and attribute features are extracted based on the association paths to obtain the attribute to be verified.

[0103] The attribute type consistency score is calculated as a static rule constraint value based on the preset attribute type constraint rules, the attribute dependency integrity score is calculated as a relation constraint value based on the attribute dependency rules, and the attribute temporal continuity score is calculated as a temporal constraint value based on the attribute temporal rules. The static rule constraint value, the relation constraint value, and the temporal constraint value are used to construct a constraint set, and the attribute to be verified that satisfies the constraint set is used as the element parsing result.

[0104] Multiple virtual nodes are evenly distributed on a hash ring. The element parsing result is mapped to the virtual node using a consistent hashing algorithm. The integrity of the element parsing result is verified by calculating and comparing the hash values ​​of the element parsing result on different virtual nodes.

[0105] The input data is preprocessed by segmenting the original text data into word sequences. These word sequences are then divided into multiple sub-dictionary segments based on word frequency and semantic relevance. For example, for a financial product introduction text, keywords such as "investment period," "minimum investment amount," and "expected rate of return" can be divided into different sub-dictionary segments.

[0106] For the predefined sub-dictionary segments, a dependency strength vector between them is calculated based on a predefined contextual dependency relation template. Specifically, a set of contextual dependency relation templates is predefined, such as "attribute-value" relation templates and "condition-result" relation templates. For any two sub-dictionary segments A and B, by traversing these templates, the frequency of words in A and B satisfying each template is counted, forming an N-dimensional dependency strength vector (N is the number of templates). For example, for the sub-dictionary segments "investment period" and "3 months", the dependency strength under the "attribute-value" template is 0.95, indicating that they highly conform to this dependency relation.

[0107] The co-occurrence probability vector between sub-dictionary segments is calculated based on word frequency statistics. For any two sub-dictionary segments, the frequency of their co-occurrence in different contexts (such as sentences, paragraphs, and documents) is counted, forming an M-dimensional co-occurrence probability vector (M being the number of context types considered). For example, the probability of "expected rate of return" and "4.5%" co-occurring in the same sentence is 0.88, and the probability of them co-occurring in the same paragraph is 0.92.

[0108] The cosine similarity of the dependency strength vector and co-occurrence probability vector obtained above is used to calculate the contextual relevance between sub-dictionary segments. Specifically, assuming the dependency strength vector of sub-dictionary segments A and B is VA and the co-occurrence probability vector is VB, their contextual relevance is calculated as the dot product of the two vectors divided by the product of their magnitudes. For example, the contextual relevance of "investment period" and "3 months" is calculated to be 0.91.

[0109] After obtaining the contextual relevance, sub-dictionary fragments are used as nodes. Connections are established between nodes, and the contextual relevance is used as the connection weight. By iteratively establishing all node connections, a semantic network is obtained. For example, in a financial product analysis scenario, a semantic network is constructed with "product name" → "investment period" → "expected rate of return" → "minimum investment amount" → "risk level". The weight of each connection reflects the contextual relevance between nodes.

[0110] Starting from each node, a depth-first search algorithm is used to record the sequence of nodes traversed during the search process as the associated path. For example, starting from the "product name" node, the associated path "product name → investment period → expected rate of return" can be extracted, indicating that there is a strong semantic relationship between these three elements.

[0111] Based on the extracted association paths, attribute features are further extracted to obtain the attributes to be verified. The specific steps are as follows: First, all node pairs are identified from the association paths, and each pair of adjacent nodes is considered a potential attribute-value relationship. Then, syntactic analysis is performed on each node pair to extract part-of-speech tags and syntactic dependency relations, identifying which node is the attribute and which is the value. Next, semantic pattern matching is applied to calculate the matching degree between the node pair and predefined attribute-value patterns (such as "noun + number", "attribute word + descriptive word", etc.). Subsequently, the attribute confidence score is calculated, which is based on the node's centrality in the semantic network, its similarity to known attributes, and the degree of contextual matching. Finally, attribute-value pairs with high confidence are selected according to a confidence threshold (e.g., 0.75) to form the set of attributes to be verified. For example, from the association path "investment period → 3 months", after the above steps, "investment period" is confirmed as the attribute name, "3 months" as the attribute value, and the confidence score is 0.92, therefore it is included in the set of attributes to be verified.

[0112] For the extracted attributes to be verified, multi-dimensional verification is required. The attribute type consistency score is calculated based on preset attribute type constraint rules and used as the static rule constraint value. For example, for the "expected rate of return" attribute, its value should be a percentage. If the extracted value is "4.5%", the type consistency score is 1.0; if the extracted value is "three months", the type consistency score is 0.

[0113] The dependency integrity score is calculated based on the dependency rules between attributes and used as the relation constraint value. For example, if the rule defines "risk level" as dependent on "expected rate of return", the dependency integrity score is 1.0 if both attributes are successfully extracted and their value ranges are reasonable; if one attribute is missing, the score is reduced.

[0114] Attribute-based time-series rules calculate the attribute time-series coherence score as the temporal constraint value. For example, if the rule defines that the "product release date" should be earlier than the "product expiration date", and the extracted two dates satisfy this time-series relationship, then the time-series coherence score is 1.0; otherwise, it is 0.

[0115] Static rule constraints, relational constraints, and temporal constraints are used to construct a constraint set, and a threshold (e.g., 0.8) is set. Attributes that satisfy the constraint set are used as the feature parsing results. For example, if all constraint values ​​for "Investment period: 3 months" exceed the threshold, it will be included in the final feature parsing results.

[0116] To ensure the integrity of the feature parsing results, multiple virtual nodes are evenly distributed on the hash ring (e.g., 256 virtual nodes). The specific steps for mapping the feature parsing results to the virtual nodes using a consistent hashing algorithm are as follows: First, each feature parsing result (attribute-value pair) is serialized into a string in the format "attribute name: attribute value". Then, a hash value is calculated for each serialized string using the MurmurHash3 algorithm to generate a 32-bit unsigned integer. Next, the hash value is modulo the total number of virtual nodes to obtain the virtual node index. An array of virtual nodes is maintained, with each node containing a node ID and a list of stored feature parsing results. Based on the calculated virtual node index, the feature parsing result is added to the result list of the corresponding virtual node. To improve system reliability, each feature parsing result is also copied to the next three virtual nodes in a clockwise direction (replication factor of 3). For example, the hash value of the parsed result "Product Name: Stable Financial Product" is 175634. Taking the modulo of 256 gives 122. Therefore, this result is stored in four virtual nodes with indices of 122, 123, 124 and 125.

[0117] The integrity of the feature parsing results is verified by calculating and comparing the hash values ​​of the parsing results on different virtual nodes. Multiple different hash functions (such as MD5, SHA-1, SHA-256, etc.) are selected to calculate the hash values ​​of the feature parsing results, and these hash values ​​are mapped to different virtual nodes. If the feature parsing results on these virtual nodes are consistent, the feature parsing results are considered complete; if inconsistencies exist, it indicates that information loss or errors occurred during the parsing process, and the parsing process needs to be re-executed.

[0118] In one optional implementation, the feature parsing results of multiple independent feature parsing channels are merged to generate a structured data feature set, and the structured data feature set is encapsulated into an independent functional component, including:

[0119] A bidirectional attention mechanism is used to extract features from the feature parsing results to obtain an attention weight matrix. The information entropy is then calculated by performing matrix multiplication on the attention weight matrix and the feature parsing results to obtain the confidence score of the feature parsing channel.

[0120] The confidence score of each element parsing channel is weighted and summed with the corresponding element parsing result to obtain a fusion result containing multiple elements and their associated information. Based on the semantic information of the elements in the fusion result, the elements are subjected to semantic dependency analysis according to a preset attribute template to construct an association network between elements. Based on the association network, the elements are organized according to a hierarchical structure of category-attribute-value to obtain a structured data element set.

[0121] The structured data element set is serialized and encoded to obtain a binary data stream. A component framework is constructed according to a preset component template. The binary data stream is encapsulated in the component framework to obtain a data layer. The binary data stream is parsed and a data operation interface and a function call interface are constructed to obtain an interface layer. The function call interface is encapsulated as a function execution module to obtain a function layer. The interface layer, the function layer and the data layer are packaged into an independent functional component.

[0122] like Figure 2 As shown, the method includes:

[0123] For the output of each feature parsing channel, a query vector, key vector, and value vector are constructed. The query vector represents the feature representation of the current feature, the key vector represents the feature representation of the context features, and the value vector contains the actual semantic information of the feature. By calculating the similarity between the query vector and the key vector, the attention level between each feature is obtained, forming an attention weight matrix. For example, for the two identified features "OLED screen" and "6.7 inches", the attention mechanism can calculate the association strength between them, determining that "6.7 inches" is an attribute value of "OLED screen".

[0124] The confidence score of the feature parsing channel is obtained by calculating the information entropy after performing matrix multiplication between the attention weight matrix and the feature parsing results. Specifically, the attention weight matrix is ​​multiplied with the feature parsing results of each channel to obtain the fused feature representation. The information entropy is calculated for these feature representations to evaluate the degree of uncertainty of the output results of each feature parsing channel. The lower the information entropy, the higher the certainty of the parsing result of that channel, and the higher the confidence. For example, if the information entropy of the named entity recognition channel for recognizing "smartphone" is 0.2, while the information entropy of the relation extraction channel for recognizing the same entity is 0.5, then the confidence of the named entity recognition channel is higher.

[0125] The confidence score of each element's parsing channel is weighted and summed with the corresponding element parsing result to obtain a fusion result containing multiple elements and their associated information. Specifically, for each identified element, a weighted average is calculated based on the confidence scores of each channel to obtain the final representation and associated information of that element. For example, if the confidence score of the named entity recognition channel for "OLED screen" is 0.8, and the confidence score of the attribute extraction channel for associating "OLED screen" with "6.7 inches" is 0.7, then in the fusion result, "OLED screen" is retained as an entity and associated with the attribute value "6.7 inches," with the association strength being the weighted result of the channel confidence scores.

[0126] Based on the semantic information of the elements in the fusion results, semantic dependency analysis is performed on the elements according to preset attribute templates to construct a network of relationships between them. Attribute templates define the relationships between different types of elements, such as a ternary relationship template of "device-configuration-parameter". Through semantic dependency analysis, the semantic dependencies between elements are determined, and a network of relationships is constructed. For example, "smartphone" as the core entity establishes a "configuration" relationship with "OLED screen", "Qualcomm Snapdragon 8 processor", and "5G network", while "6.7 inches" establishes a "parameter" relationship with "OLED screen".

[0127] Based on the relationship network, elements are organized according to a hierarchical structure of category-attribute-value to obtain a structured data element set. The constructed relationship network is transformed into a hierarchical structure to form a structured data element set. For example: {"Category":"Smartphone", "Attribute":[{"Name":"Screen", "Value":"OLED Screen", "Parameter":{"Size":"6.7 inches"}}, {"Name":"Processor", "Value":"Qualcomm Snapdragon 8 Processor"}, {"Name":"Network Support", "Value":"5G Network"}]}.

[0128] The structured data set is serialized and encoded into a binary data stream. Serialization tools such as JSON or Protocol Buffers are used to convert structured data into binary format for easier data transmission and storage. For example, the structured data above can be serialized into a binary stream "0x7B22E7B1BB...".

[0129] Build a component framework based on preset component templates. Component templates define the basic structure of functional components, including a data storage area, an interface definition area, and a function implementation area. Select a suitable component template based on the application scenario; for example, for data query scenarios, choose a component template with an efficient index structure.

[0130] The data layer is created by encapsulating binary data streams within the component framework. The serialized binary data stream is stored in the data area of ​​the component framework, forming the data layer. The data layer is responsible for managing and maintaining structured data, providing data persistence and consistency guarantees.

[0131] The interface layer is obtained by parsing the binary data stream and constructing data manipulation interfaces and function call interfaces. Based on the structure of the binary data, interfaces for adding, deleting, querying, and modifying data, as well as call interfaces for specific functions, are developed. For example, for smartphone data, interface functions such as getScreenSize() and getProcessorModel() can be provided.

[0132] The function call interface is encapsulated into a function execution module to obtain the function layer. Based on the function call interface defined in the interface layer, the corresponding functional logic is implemented to form the function execution module. For example, the comparePerformance() module can be implemented to compare the performance metrics of different processors.

[0133] Package the interface layer, functional layer, and data layer into independent functional components. Package the implementation code and data of these three layers to generate functional components that can be independently deployed and invoked. These components can be in the form of dynamic link libraries, JAR files, or microservice containers. The components possess data encapsulation and interface consistency, supporting reuse across different systems.

[0134] Through the above steps, the complete process of merging the results of multiple independent element parsing channels, generating a set of structured data elements, and encapsulating it into an independent functional component is completed.

[0135] This invention provides a dictionary-based system for parsing unstructured data elements in government and enterprise data. The system includes:

[0136] The first unit is used to perform word segmentation on unstructured data from government and enterprises, and to construct a feature matrix from the entity words to be matched in the word segmentation results.

[0137] The second unit is used to build a character prefix tree in the vertical dimension of the feature matrix to store the character sequence information of entity words, and to build an attribute association table in the horizontal dimension to record the business attribute information of entity words; to establish a cross-reference pointer to associate the character prefix tree with the attribute association table, to set an attribute inheritance mechanism, to store the common business attributes of entity words with the same prefix in the parent node, and to use the cross-reference pointer to perform bidirectional retrieval when performing dictionary query, and to obtain attribute-enhanced entity words through the attribute inheritance mechanism from the bidirectional retrieval results;

[0138] The third unit is used to divide the attribute-enhanced entity words into multiple sub-dictionary segments according to the business domain, set up multiple independent element parsing channels, calculate the semantic association strength between the sub-dictionary segments, and assign sub-dictionary segments with semantic association strength exceeding a preset association threshold to the same element parsing channel; each element parsing channel performs attribute extraction and value range verification in parallel to obtain element parsing results, and ensures the integrity of element parsing results through a consistent hashing algorithm;

[0139] The fourth unit is used to merge the feature parsing results of multiple independent feature parsing channels to generate a structured data feature set, and to encapsulate the structured data feature set into an independent functional component.

[0140] A third aspect of the present invention provides an electronic device, comprising:

[0141] processor;

[0142] Memory used to store processor-executable instructions;

[0143] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0144] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0145] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for implementing a dictionary-based government-enterprise unstructured data element parsing component, characterized in that, The method comprises the following steps: performing word segmentation on unstructured data of government and enterprise, and constructing entity words to be matched in a word segmentation result into a feature matrix; for the feature matrix, establishing a character prefix tree in a longitudinal dimension to store character sequence information of the entity words, and constructing an attribute association table in a transverse dimension to record business attribute information of the entity words; establishing a cross-reference pointer to associate the character prefix tree and the attribute association table, setting an attribute inheritance mechanism, storing common business attributes of entity words with the same prefix in a superior node, and when performing a dictionary query, using the cross-reference pointer to perform bidirectional retrieval, and obtaining attribute-enhanced entity words from the bidirectional retrieval result through the attribute inheritance mechanism; dividing the attribute-enhanced entity words into multiple sub-dictionary shards according to business fields, setting multiple independent element analysis channels, calculating semantic association strengths between the sub-dictionary shards, and assigning sub-dictionary shards with a semantic association strength exceeding a preset association threshold to the same element analysis channel; each element analysis channel performs attribute extraction and value range verification in parallel to obtain element analysis results, and a consistency hash algorithm is used to ensure the integrity of the element analysis results; merging element analysis results of multiple independent element analysis channels to generate a structured data element set, and encapsulating the structured data element set into an independent functional component.

2. The method of claim 1, wherein, The method comprises the following steps: inserting character sequences into an empty prefix tree in order to establish parent-child relationships between nodes and obtain a basic prefix tree; for each node in the basic prefix tree, calculating a frequency ratio of a node character frequency to a total sum of all node character frequencies to obtain a frequency ratio, calculating a logarithm of a ratio of a total word number to a number of words containing the node character to obtain an information entropy, and multiplying the frequency ratio and the information entropy to obtain a node weight; merging and compressing nodes with a node weight lower than a preset weight threshold, recalculating parent-child relationships of the merged and compressed nodes and updating node depths to obtain a character prefix tree; classifying business attribute information according to attribute types, establishing a mapping dictionary for attribute values of an enumeration type, mapping and converting enumeration values into integer identifiers; calculating a feature vector of an attribute value of a text type, converting the feature vector into a numerical identifier; converting an attribute value of a time type into a time stamp; constructing a multi-level hash structure, setting a corresponding weight coefficient for each level of the hash structure; sequentially passing the converted business attribute information through each level of the hash structure to calculate multiple hash values, multiplying the multiple hash values and the corresponding weight coefficients and summing them to obtain a hash position of the business attribute information, and storing the business attribute information in the attribute association table according to the hash position.

3. The method of claim 1, wherein, The method comprises the following steps: establishing a cross-reference pointer to associate the character prefix tree and the attribute association table, setting an attribute inheritance mechanism, storing common business attributes of entity words with the same prefix in a superior node, and when performing a dictionary query, using the cross-reference pointer to perform bidirectional retrieval, and obtaining attribute-enhanced entity words from the bidirectional retrieval result through the attribute inheritance mechanism. Combining a tree pointer pointing to a character prefix tree node and a table pointer pointing to an attribute association table item into a cross-reference pointer; combining a tree node depth, an association attribute quantity and a frequency of citation into a reference weight of the cross-reference pointer; According to the size of the reference weight, nodes in the character prefix tree are layered, in each layer, a parent-child node pair is identified and a feature vector is constructed based on the attribute set thereof, an attribute similarity is obtained by calculating a cosine similarity between a parent node feature vector and a child node feature vector, and a character similarity is calculated based on a character edit distance between nodes; a weighted sum of the attribute similarity and the character similarity is taken as an attribute matching degree; when the attribute matching degree is greater than a preset inheritance threshold, a matching attribute of the parent node and a corresponding reference weight thereof are stored into a superior attribute of the child node, forming a multi-level attribute inheritance chain with weights; For a node obtained through bidirectional retrieval, an attribute enhancement weight is calculated, the attribute enhancement weight is inversely proportional to an inheritance depth of the node in the multi-level attribute inheritance chain, and is proportional to a usage frequency and an inheritance confidence of the attribute in the multi-level attribute inheritance chain; based on the attribute enhancement weight, attributes on the multi-level attribute inheritance chain are prioritized, and attributes with a sorting result higher than a preset priority threshold are merged into the retrieval node, generating an attribute-enhanced entity word.

4. The method of claim 1, wherein, The attribute-enhanced entity word is divided into multiple sub-dictionary shards according to business domains, multiple independent element analysis channels are set, semantic association strengths between the sub-dictionary shards are calculated, and sub-dictionary shards with a semantic association strength exceeding a preset association threshold are allocated to the same element analysis channel. A multi-dimensional feature vector is established according to business domain characteristics, the attribute-enhanced entity word is preliminarily divided into multiple basic shards based on the multi-dimensional feature vector as a feature reference, a shard density is calculated according to a ratio of a total usage frequency of entries in the basic shard to a number of entries, the basic shard is uniformly divided based on the shard density, and a sub-dictionary shard is obtained; A feature vector of the sub-dictionary shard is extracted to construct a high-order tensor, the high-order tensor is decomposed by tensor decomposition and dimension reduction processing to obtain a semantic feature matrix, a domain association degree is calculated by calculating a cosine distance of the semantic feature matrix, the sub-dictionary shard is constructed into a graph structure, a node sequence is obtained by random walk sampling on the graph structure, a topology feature is obtained by mapping the node sequence to a low-dimensional vector space using a Skip-gram algorithm, a structure association degree is calculated based on an Euclidean distance of the topology feature; the domain association degree and the structure association degree are fused by a non-linear dynamic fusion to obtain a semantic association strength, and sub-dictionary shards with a semantic association strength exceeding a preset association threshold are divided into the same cluster; A plurality of element analysis channels are established, an independent analysis thread pool is allocated to each analysis channel, and each analysis thread pool includes a plurality of parallel processing threads; sub-dictionary shards in the same cluster are allocated to corresponding element analysis channels, and element analysis operations are performed on the sub-dictionary shards by using parallel processing threads in the analysis thread pool.

5. The method of claim 1, wherein, The element analysis channel performs attribute extraction and value range verification in parallel to obtain an element analysis result, and a consistent hashing algorithm is used to ensure the integrity of the element analysis result, including: Based on the preset context dependency relationship template, the dependency strength vector between the sub-dictionary shards is calculated, the co-occurrence probability vector between the sub-dictionary shards is calculated based on word frequency statistics, and the cosine similarity of the dependency strength vector and the co-occurrence probability vector is calculated to obtain the context correlation degree; The sub-dictionary shards are taken as nodes, a connection relationship is established between the nodes, and the context correlation degree is taken as a connection weight, and the connection relationship of all nodes is established through iteration to obtain a semantic network, an associated path is extracted in the semantic network using a depth-first search algorithm, and an attribute feature is extracted based on the associated path to obtain a to-be-verified attribute; Based on the preset attribute type constraint rule, an attribute type consistency score is calculated as a static rule constraint value, based on the dependency rule between attributes, an attribute dependency integrity score is calculated as a relationship constraint value, and based on the time sequence rule of the attribute, an attribute time sequence continuity score is calculated as a time constraint value, the static rule constraint value, the relationship constraint value and the time constraint value are constructed as a constraint set, and the to-be-verified attribute satisfying the constraint set is taken as an element analysis result; Uniformly distribute a plurality of virtual nodes on a hash ring, map the element analysis result to the virtual nodes through a consistent hashing algorithm, calculate the hash values of the element analysis result on different virtual nodes and compare them, and verify the integrity of the element analysis result.

6. The method of claim 1, wherein, Merge the element analysis results of a plurality of independent element analysis channels to generate a structured data element set, and encapsulate the structured data element set as an independent functional component, including: A bidirectional attention mechanism is used to extract features from the element analysis result to obtain an attention weight matrix, and the information entropy is calculated after matrix multiplication operation of the attention weight matrix and the element analysis result to obtain the confidence score of the element analysis channel; The confidence score of each element analysis channel and the corresponding element analysis result are weighted and added to obtain a fusion result containing a plurality of elements and their associated information; according to the semantic information of the elements in the fusion result, the elements are analyzed according to a preset attribute template to construct an associated relationship network between the elements, and the elements are organized according to a class-attribute-value hierarchical structure based on the associated relationship network to obtain a structured data element set; The structured data element set is serialized and encoded to obtain a binary data stream, a component framework is constructed according to a preset component template, the binary data stream is encapsulated in the component framework to obtain a data layer, the binary data stream is parsed and a data operation interface and a function calling interface are constructed to obtain an interface layer, the function calling interface is encapsulated as a function execution module to obtain a function layer, and the interface layer, the function layer and the data layer are packaged as an independent functional component.

7. A dictionary-based government-enterprise unstructured data element parsing component implementation system for implementing the method of any one of claims 1-6, characterized in that, including: A first unit is configured to perform word segmentation processing on the government-enterprise unstructured data, and construct a feature matrix from a to-be-matched entity word in a word segmentation result; The second unit is configured to, for the feature matrix, establish a character prefix tree in a longitudinal dimension, store character sequence information of entity words, construct an attribute association table in a transverse dimension, record business attribute information of the entity words, establish a cross-reference pointer, associate the character prefix tree with the attribute association table, set an attribute inheritance mechanism, store common business attributes of entity words with the same prefix in a superior node, use the cross-reference pointer for bidirectional retrieval when performing a dictionary query, and obtain attribute-enhanced entity words through the attribute inheritance mechanism from bidirectional retrieval results. The third unit is configured to divide the attribute-enhanced entity words into a plurality of sub-dictionary shards according to business fields, set a plurality of independent element analysis channels, calculate semantic correlation strengths between the sub-dictionary shards, and assign sub-dictionary shards with a semantic correlation strength exceeding a preset correlation threshold to the same element analysis channel. Each element analysis channel performs attribute extraction and value domain verification to obtain element analysis results, and a consistency hashing algorithm is used to ensure the integrity of the element analysis results. The fourth unit is configured to combine element analysis results of the plurality of independent element analysis channels to generate a structured data element set, and encapsulate the structured data element set as an independent functional component.

8. An electronic device, comprising: The computer program instructions are executed by the processor to implement the method of any one of claims 1-6. The computer program instructions are executed by the processor to implement the method of any one of claims 1-6. ​ ​ 9. A computer-readable storage medium having stored thereon computer program instructions, wherein, ​

Images