A material similarity analysis method based on a semantic library and a knowledge graph
By combining semantic databases and knowledge graphs in the material similarity analysis method, the problems of time-consuming, labor-intensive, and inaccurate material duplicate code identification have been solved, achieving efficient and accurate identification of material similarity.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CNNC NUCLEAR POWER OPERATION MANAGEMENT CO LTD
- Filing Date
- 2022-09-08
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies for identifying duplicate codes of materials are time-consuming, labor-intensive, and lack accuracy, especially when it comes to materials with small differences in coding or physical attributes but large differences in actual application.
A material similarity analysis method based on semantic database and knowledge graph is adopted. By judging the similarity of decisive attributes, calculating the similarity of non-deterministic attributes, and combining the weight evaluation of knowledge graph paths, the similarity of materials is finally calculated comprehensively.
It improves the accuracy and efficiency of identifying duplicate codes for materials, effectively distinguishing materials with small differences in codes or physical attributes but large differences in actual applications, thus enhancing the accuracy and reliability of similarity analysis.
Smart Images
Figure CN116304726B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of similarity algorithm analysis technology, and in particular to a material similarity analysis method based on semantic databases and knowledge graphs. Background Technology
[0002] Material master data, as one of the fundamental data in the supply chain field, is playing an increasingly important role. However, a low-quality data foundation is gradually becoming a constraint hindering the smooth progress of work. Solving the problem of duplicate codes mainly involves searching and comparing different material data, which is currently time-consuming and labor-intensive if done by power plant professionals or external vendors. Furthermore, human factors also pose a significant obstacle in the process of identifying duplicate codes.
[0003] Current similarity algorithms on the market include cosine similarity analysis. With the development of big data technology and the maturity of semantic databases and knowledge graphs, semantic databases are becoming increasingly important. Semantic databases are a crucial foundational language resource, providing rich corpus knowledge for natural language processing tasks. They are widely used in tasks such as word sense disambiguation, machine translation, information retrieval, and automatic question answering, and are an important component of intelligent knowledge management systems. Knowledge graphs are relational networks that connect all different types of information. They provide the ability to analyze problems from a relational perspective. Summary of the Invention
[0004] The purpose of this invention is to provide a material similarity analysis method based on semantic databases and knowledge graphs, which can solve the problem that although the differences in coding or physical attributes are small, the differences in actual applications are large.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A material similarity analysis method based on semantic database and knowledge graph includes the following steps:
[0007] S1: Determine whether the decisive attributes of two materials are similar. If either of the decisive attributes is not similar, then the two materials are non-duplicate materials; otherwise, proceed to S2.
[0008] S2: Calculate the similarity of field parts based on the weights of non-deterministic attributes and the distance of each field;
[0009] S3: Calculate the similarity of the graph parts by combining all knowledge graph paths and weight values;
[0010] S4: Add the similarity of the field part and the similarity of the graph part to obtain the final material similarity. Use the final material similarity to determine whether two materials are highly similar.
[0011] In S1, the decisive attributes are the hazard number attribute and whether it is a tool or appliance attribute.
[0012] In S2, non-deterministic attributes include material description attribute, material name attribute, basic material attribute, manufacturer part number attribute, and manufacturer name attribute.
[0013] Furthermore, the similarity of the material description attributes is calculated using a cosine similarity algorithm.
[0014] The steps of the cosine similarity algorithm calculation include:
[0015] S21: String preprocessing, converting uppercase and lowercase letters in the English and Chinese parts of the material name, and deleting special characters from the material description;
[0016] S22: Word segmentation processing, performing word segmentation on the preprocessed text content based on a dictionary.
[0017] S23: Synonym replacement, using a thesaurus to search for and replace words with synonyms;
[0018] S24: Cosine similarity calculation: The segmented words are deduplicated and sorted. The original word positions are replaced with the sorted indices. The number of indices is counted, and one-hot encoding is used to convert them into corresponding vectors. The final cosine similarity is obtained by dividing the inner product of the vectors by their respective modulo.
[0019] Furthermore, the similarity of the material name attribute, basic material attribute, manufacturer part number attribute, and manufacturer name attribute is calculated by the logical distance algorithm.
[0020] In S3, there are 5 knowledge graph paths. The importance of the 5 knowledge graph paths is initially assessed by using the AHP method, and the connection path weights of the two materials are assigned.
[0021] The calculation method for knowledge graph paths is as follows:
[0022] S31: Each path generates a final BOM node list through a knowledge graph, and each material has 0-5 BOM node lists;
[0023] S32: Count and record the number of paths connecting each BOM node in the list;
[0024] S33: Set a threshold value M for the number of paths in each BOM node. When the number of paths in a BOM node is greater than M, the BOM node is determined to be a valid node. When the number of paths is less than or equal to M, the BOM node is determined to be an invalid node.
[0025] S34: Perform cross-comparison calculations between the BOM node lists of two materials, take the intersection of the BOM nodes, and generate several sets of comparison results.
[0026] S35: When the BOM node lists obtained from two paths have an intersection of valid nodes, they are determined to be validly connected; if the intersection consists entirely of invalid nodes, they are determined to be invalidly connected; if there is no intersection of any nodes, they are determined to be disconnected; the three connectivity results for each path connectivity method correspond to three weights respectively.
[0027] S36: Calculate the similarity of the graph parts by combining the weight values of all knowledge graph paths.
[0028] In S31, the BOM node list includes the BOM number and the number of paths.
[0029] In S35, each path connectivity method has three connectivity results: valid connectivity, invalid connectivity, and no connectivity.
[0030] Compared with existing technologies, the material similarity analysis method based on semantic databases and knowledge graphs provided by this invention has the following advantages:
[0031] This invention, through algorithm optimization, breaks away from traditional similarity analysis algorithms and solves the problem of failing to identify small differences in physical attributes that lead to significant discrepancies in reality. It optimizes the accuracy of the similarity algorithm by leveraging a semantic database.
[0032] Furthermore, this invention enhances the accuracy and reliability of identification by leveraging the material usage history and location.
[0033] Furthermore, by incorporating historical data usage into similarity analysis, the reliability of similarity analysis based on the inherent attributes of materials is greatly improved due to the poor quality and low standardization of the initial coding data. Historical data enhances the persuasiveness of similarity results. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0035] Figure 1 This is a flowchart illustrating the calculation of total similarity provided in an embodiment of the present invention. Detailed Implementation
[0036] The following detailed description provides further details on specific implementation methods.
[0037] like Figure 1 As shown, this invention provides a material similarity analysis method based on semantic databases and knowledge graphs, including the following steps:
[0038] S1: Determine whether the decisive attributes of two materials are similar. If either of the decisive attributes is not similar, then the two materials are non-duplicate materials; otherwise, proceed to S2.
[0039] S2: Calculate the similarity of field parts based on the weights of non-deterministic attributes;
[0040] S3: Calculate the similarity of the graph parts by combining all knowledge graph paths and weight values;
[0041] S4: Add the similarity of the field part and the similarity of the graph part to obtain the final material similarity. Use the final material similarity to determine whether two materials are highly similar.
[0042] This invention can solve the problem of small differences in coding or physical attributes, but significant differences in actual applications. For example, consider a pressure transmitter with material code P1101 and a temperature transmitter with material code T1101. The editing distance between their codes is very small, but they represent completely different devices. The analysis results of this invention can effectively distinguish between the two.
[0043] Specifically, the calculation based on the material attribute semantic database and similarity algorithm is as follows:
[0044] The duplicate code analysis based on material attributes involves seven attributes, and different similarity comparison methods are used depending on the data type. Different weights are obtained for each attribute based on statistical analysis of business data. Finally, the similarity between two materials is the sum of the similarities of each attribute multiplied by its weight, as shown in the following formula:
[0045] Total attribute similarity = ∑ attribute i weight × attribute i similarity
[0046] Where i is the attribute number in Table 1.
[0047] The weights corresponding to the seven attributes obtained through calculation are shown in Table 1.
[0048] Table 1 Material Attributes
[0049] Serial Number Attribute Name Data types Weight 1 Material Description Chinese and English, special symbols 24％ 2 Material Name Chinese, English, and other languages 18％ 3 Basic materials Chinese and English, special symbols 16％ 4 Dangerous Goods Number English and numeric codes 13％ 5 Manufacturer Part Number English and numeric codes 13％ 6 Manufacturer Name Chinese and English 10％ 7 Tools and equipment English encoding 6％
[0050] "Hazardous Goods Number" and "Whether it is a Tool or Equipment" are the decisive attributes and prerequisites for determining whether two materials are duplicates. That is, if either of these two attributes is dissimilar, the two materials are considered non-duplicate. A full match method is used to determine whether two materials are the same.
[0051] Non-deterministic attributes include material description attributes, material name attributes, basic material attributes, manufacturer part number attributes, and manufacturer name attributes.
[0052] Attributes such as "Material Name," "Basic Material," "Manufacturer Part Number," and "Manufacturer Name" involve Chinese, English, numbers, or other characters. Similarity is calculated using the shortest edit distance algorithm. The shortest edit distance (EditDistance, also known as Levenshtein Distance) refers to the minimum number of editing operations required to transform one string into another. Permitted editing operations include replacing one character with another, inserting a character, and deleting a character. Generally, the smaller the edit distance, the greater the similarity between the two strings.
[0053] The "material description" attribute is composed of multiple attributes describing materials, involving diverse expressions and rich semantics. Therefore, this attribute is first processed through word segmentation and word vectorization, and then its similarity is calculated using the cosine similarity algorithm. The main processing steps are as follows:
[0054] S21: String Preprocessing
[0055] Convert the uppercase and lowercase letters in the English and Chinese parts of the material name, and remove special symbols from the material description.
[0056] S22: Word segmentation processing
[0057] Using the built-in dictionary in the Jieba word segmentation tool and a professional dictionary in the nuclear power field, the text content after string preprocessing is segmented.
[0058] S23: Synonym Replacement
[0059] By utilizing a compiled thesaurus of nuclear power related terms, synonym searches and replacements are performed on the words after Jieba word segmentation. This eliminates the impact of word differences on cosine similarity calculation when using bag-of-words deduplication.
[0060] S24: Cosine Similarity Calculation
[0061] After the material description is segmented by Jieba, the segmented words are deduplicated and sorted. The original word positions are replaced with the sorted indices. The number of indices is counted, and one-hot encoding is used to convert them into corresponding vectors. The final cosine similarity is obtained by dividing the inner product of the vectors by their respective modulo.
[0062] The following is a duplicate code analysis based on knowledge graphs:
[0063] This analysis, which compares and analyzes the usage scenarios of materials to determine if they are duplicate materials, supplements and supports the results of duplicate material attribute-based analysis. If two materials can be used on the same type of equipment in different or the same scenarios, they have a certain probability of being duplicated, especially when the two materials have high attribute similarity. This method utilizes path exploration and other methods in knowledge graphs to help materials search for paths that match their usage scenarios and complete the calculation of graph similarity. The similarity graph paths are shown in Table 2.
[0064] Table 2 Similarity Map Paths
[0065] Serial Number Graph path Path 1 Materials — QDR — Equipment — BOM Path 2 Materials – Spare parts consumption for work orders – Work order tasks – Equipment – Bill of Materials Path 3 Materials — MR — Work Orders — Equipment — BOM Path 4 Materials — MR — Work Order — Work Item — Equipment — BOM Path 5 Supplies - BOM
[0066] Among them, QDR stands for Quality Defect Report, MR stands for Material Request for Work Order, and BOM stands for Bill of Materials.
[0067] The core weighting approach is as follows: the weight of each path does not decrease linearly in the total weight. Usually, the top N paths are more important and the differences in importance are more obvious. Therefore, a hierarchical decision-making method is used to divide the target paths into layers, and the weights of different layers decrease geometrically.
[0068] Five pathways were involved. The importance of these five pathways was initially assessed using the Analytic Hierarchy Process (AHP), and the connecting pathways between two resources were assigned weights, as shown in Table 3. AHP is a multi-option or multi-objective decision-making method, combining qualitative and quantitative approaches. It is commonly used for complex, unstructured decision-making and authority assignment problems involving multiple objectives, criteria, factors, and levels. The decision objective, considered factors (decision criteria), and decision objects are hierarchically stratified according to their interrelationships, and the relative importance of all factors at a given level to the overall objective is determined by assigning weights.
[0069] Table 3 Weight Allocation of Material Connection Paths
[0070]
[0071]
[0072] The knowledge graph path calculation method is as follows (taking five paths pointing to the BOM as an example):
[0073] (1) Each path is used to obtain the final BOM node list (the list contains the BOM number and the number of paths). Each material has 0-5 BOM node lists (0 if there are no paths connected, and 5 if all paths are connected).
[0074] (2) Count and record the number of paths connecting each BOM node in the list (for example, if the BOM node list result obtained by material A through path one is BOM A, and the number of paths is 20, then it means that there are 20 paths connecting material A and BOM A through path one).
[0075] (3) Set a threshold value M for the number of BOM nodes for each path (the threshold M is adjusted by sampling the data and using statistical methods; M can be 0). When the number of paths of a BOM node is greater than M, the BOM node is determined to be a valid node. When the number of paths is less than or equal to M, the BOM node is determined to be an invalid node.
[0076] (4) Cross-compare the BOM node lists of the two materials, take the intersection of the BOM nodes, and generate several sets of comparison results.
[0077] (5) Each path connectivity method has three connectivity results: valid connectivity, invalid connectivity, and no connectivity. When the BOM node lists obtained from two paths have an intersection of valid nodes, they are determined to be validly connected; if the intersection consists entirely of invalid nodes, they are determined to be invalidly connected; if there is no intersection of nodes, they are determined to be no connectivity. The three connectivity results of each path connectivity method correspond to three weights. In the two material calculation and analysis processes, each path connectivity method will only generate one weight.
[0078] (6) Combine the weights of all paths to derive the similarity of the graph parts.
[0079] Therefore, the similarity of the field part and the similarity of the graph part are added together to obtain the final material similarity. The final material similarity is used to determine whether two materials are highly similar.
[0080] We collected 1000 known items containing both different and similar items, calculated their similarity scores, and sorted them according to the similarity results. Based on the actual similarity of the items and the calculated similarity scores, we determined that data with a similarity greater than 0.6 is mostly a list of suspected duplicate items, while data with a similarity greater than 0.8 is considered to have high similarity. Therefore, items with a similarity greater than 0.6 are included in the list of suspected duplicate items, and items with a similarity greater than 0.8 are considered to have high similarity.
[0081] This invention combines the physical attributes of materials with their usage scenarios to conduct a multi-dimensional analysis of duplicate codes. Specifically, the analysis based on physical attributes compares various fields describing the material to determine if there are duplicate codes. The analysis based on usage scenarios traces and analyzes the material's historical usage, i.e., whether the material was used on the same equipment, to determine if there are duplicate codes. Finally, the results of both analyses are considered to determine if a material has a duplicate code. The final result is a weighted sum of the two analyses, with weights obtained, for example, from known samples and calculation results.
[0082] In other words, duplicate code analysis of two materials requires combining similarity calculation of duplicate code identification fields with knowledge graph path calculation. The entities involved in material duplicate code analysis include materials, personnel, equipment, manufacturers, work orders / tasks, MR, BOM, QDR, work items, and spare parts consumption of work orders.
[0083] This method, in analyzing the similarity of material attributes, leverages semantic databases to normalize terms with the same pronouns and employs cosine similarity algorithm for similarity analysis. It then uses the KL divergence algorithm to compare changes in KL divergence under different weight proportions, thereby obtaining a relatively optimal initial weight proportion. Further adjustments to the weight proportions are made based on expert advice, resulting in the final weight proportions for each attribute. In knowledge graph path similarity analysis, the Analytic Hierarchy Process (AHP) is used to assign weights to paths, and this is then combined with practical business considerations to arrive at the final weight proportions for each path.
[0084] The duplicate code analysis based on knowledge graph technology in this invention is not only an analysis of the material master data itself, but also introduces related business data and other master data as the data foundation for data analysis, associates various data objects of business data and master data, and assigns duplicate code analysis weights.
[0085] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for material similarity analysis based on semantic databases and knowledge graphs, characterized in that, Includes the following steps: S1: Determine whether the decisive attributes of two materials are similar. If either of the decisive attributes is not similar, then the two materials are non-duplicate materials; otherwise, proceed to S2. S2: Calculate the similarity of field parts based on the weights of non-deterministic attributes and the distance of each field; S3: Calculate the similarity of the knowledge graph parts by combining all knowledge graph paths and weight values; the calculation method for knowledge graph paths is as follows: S31: Each path generates a final BOM node list through a knowledge graph, and each material has 0-5 BOM node lists; S32: Count and record the number of paths connecting each BOM node in the list; S33: Set a threshold value M for the number of paths in each BOM node. When the number of paths in a BOM node is greater than M, the BOM node is determined to be a valid node. When the number of paths is less than or equal to M, the BOM node is determined to be an invalid node. S34: Perform cross-comparison calculations between the BOM node lists of two materials, take the intersection of the BOM nodes, and generate several sets of comparison results. S35: When the BOM node lists obtained from two paths have an intersection of valid nodes, they are determined to be validly connected; if the intersection consists entirely of invalid nodes, they are determined to be invalidly connected; if there is no intersection of any nodes, they are determined to be disconnected; the three connectivity results for each path connectivity method correspond to three weights respectively. S36: Calculate the similarity of the graph parts by combining the weight values of all knowledge graph paths; S4: Add the similarity of the field part and the similarity of the graph part to obtain the final material similarity. Use the final material similarity to determine whether two materials are highly similar.
2. The material similarity analysis method based on semantic database and knowledge graph according to claim 1, characterized in that, In S1, the decisive attributes are the hazard number attribute and whether it is a tool or appliance attribute.
3. The material similarity analysis method based on semantic database and knowledge graph according to claim 1, characterized in that, In S2, non-deterministic attributes include material description attribute, material name attribute, basic material attribute, manufacturer part number attribute, and manufacturer name attribute.
4. The material similarity analysis method based on semantic database and knowledge graph according to claim 3, characterized in that, The similarity of the material description attributes is calculated using the cosine similarity algorithm.
5. The material similarity analysis method based on semantic database and knowledge graph according to claim 4, characterized in that, The steps of the cosine similarity algorithm calculation include: S21: String preprocessing, converting uppercase and lowercase letters in the English and Chinese parts of the material name, and deleting special characters from the material description; S22: Word segmentation processing, performing word segmentation on the preprocessed text content based on a dictionary. S23: Synonym replacement, using a thesaurus to search for and replace words with synonyms; S24: Cosine similarity calculation: The segmented words are deduplicated and sorted. The original word positions are replaced with the sorted indices. The number of indices is counted, and one-hot encoding is used to convert them into corresponding vectors. The final cosine similarity is obtained by dividing the inner product of the vectors by their respective modulo.
6. The material similarity analysis method based on semantic database and knowledge graph according to claim 3, characterized in that, The similarity of the material name attribute, basic material attribute, manufacturer part number attribute, and manufacturer name attribute was calculated using the shortest edit distance algorithm.
7. The material similarity analysis method based on semantic database and knowledge graph according to claim 1, characterized in that, In S3, there are 5 knowledge graph paths. The importance of the 5 knowledge graph paths is initially assessed by using the AHP method, and the connection path weights of the two materials are assigned.
8. The material similarity analysis method based on semantic database and knowledge graph according to claim 1, characterized in that, In S31, the BOM node list includes the BOM number and the number of paths.
9. The material similarity analysis method based on semantic database and knowledge graph according to claim 1, characterized in that, In S35, each path connectivity method has three connectivity results: valid connectivity, invalid connectivity, and no connectivity.