Industrial knowledge extraction and editing system and method based on man-machine cooperation

Abstract

The invention discloses an industrial knowledge extraction and editing system and method based on man-machine collaboration, and the method comprises the steps: preprocessing an original industrial text, inputting the preprocessed original industrial text into a knowledge extraction model, obtaining model output, and obtaining a plurality of pieces of structured knowledge and initial credibility based on the model output; collecting a plurality of historical modification records of the structured knowledge, and obtaining change features according to the historical modification records; calculating expert credibility according to historical auditing data of the plurality of experts, and obtaining auditing priorities according to the expert credibility and the change indexes; and displaying and modifying the structured knowledge according to the auditing priority, and updating the credibility of the structured knowledge and the credibility of the expert according to the expert modification record. The invention relates to the technical field of industrial knowledge management, and solves the technical problems that an existing industrial support extraction and editing system is insufficient in semantic comprehension ability, difficult to consider extraction efficiency and accuracy and single in man-machine cooperation function.

Description

Technical Field

[0001] The present invention belongs to the field of industrial knowledge management and relates to deep learning technology, specifically an industrial knowledge extraction and editing system and method based on human-machine collaboration. Background Art

[0002] As the digital transformation of industry deepens, the industrial knowledge management system, as the core infrastructure of intelligent manufacturing, undertakes the important function of structured storage of equipment parameters, process standards, fault cases and other knowledge, providing key decision support for production optimization, equipment maintenance and supply chain management. However, the multi-source heterogeneous characteristics of industrial knowledge (covering ERP master data, technical documents, maintenance logs, etc.), the density of professional terms and the frequent dynamic updates make traditional technical solutions face significant challenges in the knowledge extraction and maintenance links.

[0003] The current mainstream technology mainly relies on two methods: rule-based text mining and expert manual maintenance. In rule-based text mining technology, although the natural language processing model can be used to extract entities and relationships from unstructured text, its semantic understanding ability has obvious limitations. The general pre-training model does not have sufficient contextual representation of professional terms in the industrial field, which can easily lead to entity recognition bias, such as confusing equipment models with irrelevant codes. In addition, the extraction results often cause knowledge fragmentation problems due to the lack of cross-document association verification, and automatic resolution cannot be achieved when parameter conflicts occur in different data sources. Although the expert manual annotation and maintenance mode can ensure the accuracy of knowledge, it faces efficiency bottlenecks and collaboration cost problems. The manual processing speed is difficult to match the real-time update scale of industrial data, and cross-departmental knowledge collaboration requires a complex approval process, resulting in delayed updates of key information.

[0004] The above defects together lead to the "quality-efficiency" paradox of industrial knowledge management systems: if the erroneous parameters extracted by automation are not corrected in time, the credibility of the knowledge base will continue to decline; and high-precision manual maintenance is accompanied by a surge in resource costs. In addition, low-quality knowledge may lead to intelligent decision-making errors, resulting in major risks such as production interruptions. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an industrial knowledge extraction and editing system and method based on human-computer collaboration, which is used to solve the technical problems of insufficient semantic understanding ability, difficulty in balancing extraction efficiency and accuracy, and single human-computer collaboration function in the existing industrial knowledge management system.

[0006] To achieve the above objectives, the first aspect of the present invention provides an industrial knowledge extraction and editing system based on human-machine collaboration, comprising: Knowledge extraction module: used to pre-process the original industrial text and input it into the knowledge extraction model to obtain the model output, and obtain some structured knowledge and initial credibility based on the model output; the knowledge extraction model is built based on the deep learning algorithm; Change analysis module: used to collect historical modification records of some structured knowledge and obtain change features based on the historical modification records; the change features include knowledge change indicators and knowledge vulnerability scores; Audit decision module: used to calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; User interaction module: used to provide an interactive page for expert review, display and modify structured knowledge according to the review priority, and update the credibility of structured knowledge and expert credibility according to the expert modification records.

[0007] It should be noted that the knowledge extraction module, change analysis module, review decision module and user interaction module of the present invention are in communication connection.

[0008] Furthermore, the knowledge extraction model is constructed based on a deep learning algorithm, including: A1-1, add Bi-LSTM layer and CRF layer to the BERT output layer in the BERT-Base-Chinese model to build the initial model; A1-2, collect a number of industrial texts, preprocess and entity annotate the industrial texts, and obtain an unannotated text set and an annotated data set; wherein the annotated data set includes a text set and a corresponding annotation set; A1-3, pre-train the initial model using the unlabeled text set to obtain a pre-trained model; A1-4, input the labeled data set into the pre-training model for supervised training to obtain a knowledge extraction model; wherein the input of the knowledge extraction model is industrial text, and the output is industrial labels and label prediction probabilities, and the industrial labels include material name, brand, model, specification parameters and others.

[0009] In the construction of the knowledge extraction model, the contextual understanding ability of BERT is combined with the sequence modeling ability of Bi-LSTM, which can effectively handle long-distance dependencies in complex industrial texts and improve the accuracy of feature extraction. At the same time, a model training strategy that combines pre-training and supervised training is adopted to make full use of the domain-general features of unlabeled data, reduce dependence on labeled data, and significantly improve the generalization ability of the model in industrial scenarios, ensuring that it can adapt to diverse industrial text inputs.

[0010] Furthermore, the model output is based on obtaining a number of structured knowledge and initial credibility, including: A2-1, define the necessary attributes, specification attributes and characteristic attributes of the standard knowledge template to obtain the standard knowledge template; A2-2, unify the format of industrial labels in the model output according to the standard knowledge template to obtain some structured knowledge; A2-3, calculate the text features of structured knowledge based on the standard knowledge template; the text features include completeness C, standardization S, consistency K, source level L and timeliness score F; A2-4, calculate the text quality score TQS of the original industrial text according to the text quality calculation formula: TQS=α1×C+α2×S+α3×K; A2-5, calculate the data source score SCS of the original industrial text according to the data source score calculation formula: SCS = β1 × L + β2 × F; A2-6, according to the formula MCS = 1 / n∑P i The model confidence MCS is calculated; where n represents the number of industrial tags, P i represents the predicted probability of the i-th industrial label; A2-7, using the weighted summation method, the initial credibility IC of some structured knowledge is calculated according to the text quality score TQS, the data source score and the model confidence MCS; among them, α1, α2, α3 and β1, β2 represent the weight coefficients corresponding to each formula respectively, and the sum of the corresponding weight coefficients is 1.

[0011] In order to solve the problems of chaotic industrial text formats and fragmented knowledge, the standard knowledge template is defined to unify the format, and the text quality is quantified from the dimensions of completeness, standardization, consistency, etc. At the same time, the reliability of the data is evaluated by combining the data source level and timeliness, and the model confidence is used to reflect the stability of the prediction results. Finally, the initial credibility is calculated by combining multiple factors through the weighted sum method to fully reflect the reliability of knowledge, providing clear quantitative evaluation indicators for subsequent review and application.

[0012] Furthermore, the text features of structured knowledge calculated according to the standard knowledge template include: The total number of attributes actually filled in the structured knowledge is N total The ratio of the number of necessary attributes to the standard knowledge template is marked as completeness C; The number of attributes in structured knowledge that conform to the standard knowledge template is divided by the total number of attributes N total The ratio of is marked as the standard degree S; The number of attributes in structured knowledge that do not conform to the standard knowledge template is marked as the number of contradictory attributes N. C , and according to the formula K=1-N C / N total Calculate the consistency K; Divide structured knowledge according to the source of industrial text data and set the source level to obtain the source level L; And according to the time interval between the generation time of industrial text data and the current time Calculate the timeliness score F, the formula is: .

[0013] Furthermore, obtaining the change feature according to the historical modification record includes: B1-1, the modification frequency score FS is calculated based on the number of modifications of structured knowledge in the historical modification records. The formula is: ;in, Indicates the modification frequency within a preset short period of time. Indicates the frequency of modification within the preset medium term. Indicates the modification frequency within a preset long-term period. , , represents the weight coefficient; B1-2, divide the structured knowledge into numerical knowledge and categorical knowledge according to data type, and collect the structured knowledge V before modification in the historical modification records old and the modified structured knowledge V new ; B1-3, for numerical knowledge, according to the formula Calculate the modification score AS; for categorical knowledge, according to the formula The modification score AS is calculated; where: Indicates the preset reference value, represents the structured knowledge vector before modification, represents the modified structured knowledge vector; B1-4, establishing a knowledge graph association relationship based on a number of structured knowledge, and determining the direct associated node ND and the indirect associated node NI of each node; wherein, in the knowledge graph association relationship, the structured knowledge items are nodes; B1-5, according to the formula The influence range score IS is calculated; where: Indicates the node importance of the directly associated node, Indicates the node importance of indirectly related nodes, represents the level of indirectly related nodes, i represents the structured knowledge index, I max Indicates the maximum value of node importance among several associated nodes; B1-6, perform weighted summation on the knowledge change indicators to obtain the knowledge vulnerability score VS; wherein the knowledge change indicators include the modification frequency score FS, the modification amplitude score AS and the impact range score IS.

[0014] The change characteristics evaluate the stability of knowledge from three dimensions: modification frequency, magnitude, and impact range by analyzing historical modification records. Among them, the knowledge graph association calculates the impact range through the node importance, quantifies the chain effect of knowledge changes; the differentiated processing of numerical and categorical knowledge ensures the accurate calculation of the modification range; the final knowledge vulnerability score provides a basis for the audit priority, helping the system to dynamically track the risk of knowledge changes, identify frequently modified or high-impact knowledge, and prioritize potential problems.

[0015] Furthermore, the calculation of the expert credibility based on the historical review data of several experts includes: C1-1, collect historical audit data of several experts, including the number of tasks assigned N A 、Number of tasks accepted R , audit content, audit results, audit records, audit quantity N T ; Among them, historical audit data is stored in the audit database; C1-2, based on the audit results, audit quantity, and the time difference between the audit time in the audit record and the current time , calculate the accuracy score EA of the current expert, the formula is: ;in, Indicates the audit results, and Represents structured knowledge of correct review, Represents structured knowledge of error review, represents the total number of reviews, i represents the structured knowledge index; C1-3, using the cosine similarity calculation formula, count the similar audit records whose similarity with the current expert's audit content is greater than the preset similarity threshold, and get the total number of similar cases , Number of cases with consistent audit results and the number of cases that agree with the high confidence expert review results ; Wherein, the highly credible experts include system-defined experts and experts whose expert credibility obtained by updating the expert credibility is greater than a preset credibility threshold; C1-4, according to the formula Calculate the consistency score EC of the current expert; C1-5, according to the formula Calculate the current expert's activity score EAct; where, Indicates the number of tasks with detailed descriptions included in the audit results; C1-6, using the weighted summation method, according to the accuracy score EA, consistency score EC and activity score EAct of the current expert, the expert credibility E is calculated; where, , and , , They represent the weight coefficients of the corresponding formulas respectively.

[0016] In the calculation formula of expert credibility, the accuracy score reflects the correctness of the expert's review results; the consistency score considers the consistency of the expert's review results with other experts (especially high-credibility experts); and the activity score combines the timely response rate. , task completion rate and detailed description rate , which reflects the enthusiasm and input of experts in participating in the audit work, and can more comprehensively and accurately reflect the experts' actual capabilities and work performance.

[0017] Furthermore, the review priority is obtained according to the expert credibility and the change index, including: C2-1, weighted sum of the modification frequency score FS, impact scope score IS and knowledge vulnerability score VS in the change index to obtain the change risk CR; C2-2, according to the formula The audit priority Pr of structured knowledge is calculated; among them, , Represents the weight coefficient, and the sum is 1.

[0018] In the process of determining the audit priority, we ensure that high-risk knowledge is given priority by changing indicators such as the vulnerability of comprehensive risk knowledge and the frequency of modification. We also ensure that highly capable experts handle key tasks in combination with the credibility of experts, balance the efficiency and accuracy of the audit, and achieve efficient allocation of audit resources to improve the reliability of the audit results.

[0019] Furthermore, the display and modification of structured knowledge according to the audit priority includes: Sort and display structured knowledge according to the review priority, and obtain a number of assigned experts for structured knowledge according to the preset expert assignment rules; Multiply the assigned expert's expert credibility E by the domain weight to get the expert weight W j ; where j represents the expert index; For numerical knowledge, collect the proposed modification values ​​Vp of several assigned experts and the expert approval values ​​Ve of highly credible experts, and calculate them according to the formula Eo j =1- Vp-Ve / Vp calculates the expert opinion value Eo of the jth expert j ; For categorical knowledge, when the assigned experts agree to modify, the expert opinion value Eo j =1; when the assigned expert disagrees with the modification, the expert opinion value Eo j =0; According to the formula Calculate and obtain the fusion result value FinalScore; According to the formula The degree of disagreement DS is calculated; Determine whether the disagreement degree DS is greater than the preset disagreement threshold; if yes, use the expert opinion value with the highest expert weight to modify the structured knowledge; if no, use the fusion result value to modify the structured knowledge.

[0020] Further, updating the structured knowledge credibility and the expert credibility according to the expert modification record includes: According to the formula IC new_i =IC i ×(1+0.1×(1-DS)) to calculate the updated structured knowledge credibility IC new Among them, IC i represents the initial credibility of the i-th structured knowledge; According to formula E new_j =E j ×(1+0.05×(1- Eo j -FinalScore )) Calculate the updated expert credibility E of the jth expert new_j ; where E j represents the expert credibility of the jth expert.

[0021] The smaller the difference in opinion, the greater the credibility of structured knowledge, which incentivizes experts to reach a consensus; the smaller the opinion deviation value, the greater the credibility of experts, which encourages experts to provide accurate opinions; through this dynamic update mechanism, the quality of knowledge and expert capabilities are continuously optimized, forming a closed-loop feedback system to reduce the need for repeated audits, further improve the quality of audits, and promote continuous improvement of the overall performance of the system.

[0022] A second aspect of the present invention provides an industrial knowledge extraction and editing method based on human-machine collaboration, comprising: The original industrial text is preprocessed and input into the knowledge extraction model to obtain the model output, and some structured knowledge and initial credibility are obtained based on the model output; wherein the knowledge extraction model is constructed based on the deep learning algorithm; Collect historical modification records of some structured knowledge, and obtain change features based on the historical modification records; wherein the change features include knowledge change indicators and knowledge vulnerability scores; Calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; The structured knowledge is displayed and modified according to the review priority, and the structured knowledge credibility and expert credibility are updated according to the expert modification records.

[0023] Compared with the prior art, the present invention has the following beneficial effects: The present invention builds a knowledge extraction model based on deep learning algorithms (Bi-LSTM layer and CRF layer are added to the BERT output layer in the BERT-Base-Chinese model in sequence), giving full play to the powerful context understanding ability of BERT and the advantage of Bi-LSTM in capturing long-distance dependencies, and can deeply understand the professional terms and complex concepts in industrial texts. After pre-training and supervised training, the model can more accurately identify entities and relationships in industrial texts, significantly improve the accuracy of semantic understanding, and provide a reliable foundation for subsequent knowledge extraction and application; The present invention adopts a human-computer collaborative approach. The knowledge extraction module first uses the knowledge extraction model to automatically preprocess and extract knowledge from the original industrial text to obtain structured knowledge and initial credibility, thereby achieving efficient extraction of most knowledge. At the same time, the change analysis module analyzes the change characteristics of knowledge, the review decision module determines the review priority based on expert credibility and change indicators, and the user interaction module allows experts to review and modify structured knowledge based on the review priority. This approach combines the high efficiency of automatic extraction with the accuracy of expert review, which not only improves the extraction efficiency but also ensures the extraction quality, effectively solving the problem of the difficulty in balancing efficiency and accuracy in the prior art. The user interaction module of the present invention provides an interactive page for expert review, and experts can display and modify structured knowledge according to the review priority. During the modification process, there are different processing methods for numerical knowledge and categorical knowledge, and by calculating indicators such as expert opinion value, fusion result value and disagreement degree, the opinions of experts are comprehensively considered to determine the final modification plan. This interactive knowledge editing method provides experts with powerful editing functions, allowing experts to easily make fine corrections and supplements to the extracted knowledge, better meeting the experts' needs for knowledge optimization, thereby making the knowledge system more complete and accurate, and improving the application value of knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A framework diagram of the industrial knowledge extraction and editing system based on human-machine collaboration provided by the present invention; Figure 2 A schematic diagram of the process flow of the industrial knowledge extraction and editing method based on human-machine collaboration provided by the present invention; Figure 3 A schematic diagram of the construction process of the knowledge extraction model provided by the present invention; Figure 4 A schematic diagram of the workflow of the change analysis module provided by the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figure 1-Figure 4 The first embodiment of the present invention provides an industrial knowledge extraction and editing system based on human-machine collaboration, including: Knowledge extraction module: used to pre-process the original industrial text and input it into the knowledge extraction model to obtain the model output, and obtain some structured knowledge and initial credibility based on the model output; the knowledge extraction model is built based on the deep learning algorithm; Change analysis module: used to collect historical modification records of some structured knowledge and obtain change features based on the historical modification records; the change features include knowledge change indicators and knowledge vulnerability scores; Audit decision module: used to calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; User interaction module: used to provide an interactive page for expert review, display and modify structured knowledge according to the review priority, and update the credibility of structured knowledge and expert credibility according to the expert modification records.

[0028] Based on the above technical modules, the present invention realizes efficient automatic extraction and dynamic optimization management of industrial knowledge. Specifically, the present invention significantly improves the accuracy and domain adaptability of knowledge extraction through the closed-loop collaboration mechanism of deep learning model and expert review. The system innovatively constructs a multi-dimensional credibility evaluation system, dynamically associates knowledge vulnerability scores with expert credibility, and realizes intelligent scheduling and priority decision-making of audit resources. Through the deep mining of knowledge evolution laws by the change feature analysis module, the weak links in the knowledge system are effectively identified, providing data support for the continuous optimization of the industrial knowledge base. Its human-machine collaborative mechanism not only reduces the cost of manual review, but also corrects the knowledge credibility weight in real time through expert feedback, forming a positive cycle of iterative improvement of knowledge quality, and finally constructs an industrial knowledge extraction and editing system with self-evolution capability: Knowledge extraction module: used to pre-process the original industrial text and input it into the knowledge extraction model to obtain the model output, and obtain some structured knowledge and initial credibility based on the model output; In order to achieve the accurate transformation of industrial knowledge from unstructured text to standardized semantic representation, in the knowledge extraction module, a knowledge extraction model based on deep learning algorithm is used to intelligently parse multi-source industrial texts. Then, based on the standard knowledge template, the model output is transformed into structured knowledge of industrial texts, and the initial credibility of structured knowledge is obtained according to the text features of structured knowledge. In one implementation, the process of constructing the knowledge extraction model in the knowledge extraction module may include the following steps: First, prepare and preprocess the data: In order to achieve comprehensive coverage of industrial knowledge, the system has built a multi-source heterogeneous data fusion mechanism, integrating the following core data sources: Large-scale industrial material text corpus (covering machinery, electrical, automation and other fields); ERP system material master data (including 150,000+ material codes and their classification system); Purchase order structured data (parse supplier delivery parameters in XML / JSON format); Material technical specifications (including PDF / scanned copies processed by OCR); Equipment maintenance manual (including fault code mapping table, maintenance record time series data); The following preprocessing process is performed on multi-source industrial text data: ① Text cleaning: Use regular expressions to remove special symbols (such as "■", "※"), develop standardized converters for measurement units (such as unifying "380 volts → 380V", "50 hertz → 50Hz", "3.7 kilowatts → 3.7kW"), and build a standardized industrial text library; ② Field segmentation: Improve Jieba segmenter based on industrial dictionary and establish a protection list of professional terms (such as overall recognition of "harmonic reducer" to avoid incorrect segmentation into "harmonic / reducer"); ③ Annotation data construction: Use the BIO annotation scheme to annotate entity boundaries and types, and obtain the following annotation set example: B-NAME, I-NAME, / / Material name B-BRAND, I-BRAND, / / Brand B-MODEL, I-MODEL, / / Model B-SPEC, I-SPEC, / / Specifications O / / Other } This preprocessing process forms a closed loop with the subsequent model architecture - the cleaned data is directly input into the domain pre-training module, and the labeled data is used as a supervisory signal to guide the fine-tuning stage.

[0029] Next, design the model architecture: Based on the particularity of industrial scenarios, a progressive domain adaptation architecture is adopted, and its technical evolution path is as follows: A[General language understanding]-->B[Industrial semantic enhancement]-->C[Sequence labeling optimization]; Based on the above technical evolution path, in this embodiment, BERT-Base-Chinese is used as the starting point to perform domain enhancement improvements, mainly including: ① Vocabulary expansion: The top 5% of high-frequency professional terms are extracted from several manuals through the TF-IDF algorithm, and several new professional terms are added (such as "Modbus-TCP protocol", "IP protection level", etc.); ②Domain pre-training: Continuous pre-training is performed on several industrial texts, and parameter contrast loss is introduced: ;in, represents input text sample pairs, which are different description fragments of the same device, such as the "rated voltage" field in the technical manual and the "input voltage" field in the purchase order); P is a set of positive sample pairs, which represents text pairs from the same device model and the same parameter category, such as "380V" and "380 volts" both describe voltage values; N is a set of negative sample pairs, which represents texts from different devices or different parameter categories, such as taking the voltage parameter "380V" and the power parameter "3.7KW" as negative pairs; Represents the exponential similarity value, usually using cosine similarity to measure the semantic similarity between two text descriptions. Represents the model output; the positive sample pair The negative samples N are randomly sampled from the text descriptions of the same device model, forcing the model to learn the deep semantic associations of device parameters; Next, we designed a hierarchical decoding architecture based on the common multi-valued parallel and cross-sentence association characteristics in industrial texts: Bi-LSTM layer (hidden_size=256, layers=2): captures long-distance dependencies, for example, establishing an association between temperature values ​​and condition descriptions in "Operating temperature range: -20℃~+60℃ (under normal pressure conditions)"; CRF layer: Constrain the label sequence through the transfer matrix to prohibit illegal combinations such as "I-BRAND" appearing after the "O" label; Then, a phased progressive training framework is adopted to address the dual challenges of scarcity of annotated data and semantic complexity in the industrial field. It mainly includes two stages: domain adaptation pre-training and entity recognition fine-tuning: Phase 1: Use unlabeled data for domain adaptation pre-training. In addition to the standard MLM task, a new entity boundary prediction task is added. By randomly masking 15% of the entities in the unlabeled industrial text (such as "[MASK] inverter"), the model is required to predict the type of the masked entity, allowing the model to learn the common features of the industrial field and the relevant characteristics of the entities in a large amount of unlabeled industrial text. Stage 2: Fine-tune entity recognition using labeled data and use dynamic loss function Perform optimization training of the model, where The weight coefficient of the loss function can be gradually decayed from 0.8 to 0.3 with the training rounds, gradually strengthening the sequence constraint. represents the cross entropy loss function, represents the conditional random field loss function; finally, after two-stage model training, the knowledge extraction model of this embodiment is obtained.

[0030] In one implementation, obtaining a number of structured knowledge and initial credibility based on the model output includes the following steps: A2-1, define the necessary attributes, specification attributes and characteristic attributes of the standard knowledge template to obtain the standard knowledge template; A2-2, unify the format of industrial labels in the model output according to the standard knowledge template to obtain some structured knowledge; A2-3, calculate the text features of structured knowledge based on the standard knowledge template; the text features include completeness C, standardization S, consistency K, source level L and timeliness score F; A2-4, the text quality score TQS of the original industrial text is calculated according to the text quality calculation formula: TQS=α1×C+α2×S+α3×K; where α1, α2, and α3 are weight coefficients, and the default values ​​are: α1=0.4, α2=0.3, and α3=0.3; A2-5, the data source score SCS of the original industrial text is calculated according to the data source score calculation formula: SCS=β1×L+β2×F; where β1 and β2 are weight coefficients, and the default values ​​are: β1=0.6, β2=0.4; A2-6, according to the formula MCS = 1 / n∑P i The model confidence MCS is calculated; where n represents the number of industrial tags, P i represents the predicted probability of the i-th industrial entity label; A2-7, using the weighted summation method, the initial credibility IC of some structured knowledge is calculated according to the text quality score TQS, the data source score SCS and the model confidence MCS, and the default values ​​of the weight coefficients are: W TQS =0.3, W SCS =0.4, W MCS =0.3; Among them, the text features of structured knowledge include: The total number of attributes actually filled in the structured knowledge is N total The ratio of the number of necessary attributes to the standard knowledge template is marked as completeness C; The ratio of the number of attributes that conform to the standard knowledge template in structured knowledge to the total number of attributes Ntotal is marked as the standardization degree S; The number of attributes in structured knowledge that do not conform to the standard knowledge template is marked as the number of contradictory attributes N. C , and according to the formula K=1-N C / N total Calculate the consistency K; Divide structured knowledge according to the source of industrial text data and set the source level, including: Level 1 (L=1.0): original technical documentation; Level 2 (L=0.8): ERP master data; Level 3 (L=0.6): purchase order data; Level 4 (L=0.4): manual data entry; Get source level L; And according to the time interval between the generation time of industrial text data and the current time Calculate the timeliness score F, the formula is: .

[0031] In one implementation, the definition of the standard knowledge template in step A2-1 can be expressed as follows: { "template_version": "1.0", "material_type": { # Material type "electrical_equipment": { # Electrical equipment "required_fields": ["name", "brand", "model"], # Required attributes "spec_fields": { # Specification attributes "electrical": ["voltage", "current", "power"], "physical": ["length", "width", "height", "weight"], "environmental": ["temperature", "humidity"] }, "feature_fields": ["functions", "certificates", "accessories"] # Feature attributes }, "mechanical_parts": { # Mechanical parts "required_fields": ["name", "brand", "model"], # Required attributes "spec_fields": { # Specification attributes "dimensional": ["size", "thickness", "diameter"], "material": ["material_type", "hardness", "finish"] }, "feature_fields": ["usage", "compatibility"] # Feature attributes } / / Other material type definitions } } In one implementation, the structured knowledge after formatting is unified by the standard knowledge template in step A2-2 may be as follows: { # Basic information "material_id": "EE20231201001", "material_type": "electrical_equipment", "base_info": { # Required attributes "name": "Inverter", "brand": "Yaskawa", # "Yaskawa" means the brand name "model": "CIMR-HB4A0009FBA", "category_code": "EE-VFD-001" }, "specifications": { # Specification attributes "electrical": { "voltage": { "value": 380, "unit": "V", "standard_unit": "V", "confidence": 0.98 }, "power": { "value": 3.7, "unit": "KW", "standard_unit": "KW", "confidence": 0.95 } } }, "features": { # Feature attributes "functions": ["Braking unit"], "certificates": [], "accessories": [] }, "quality_metrics": { # Quality indicators "completeness": 0.92, "consistency": 0.95, "standardization": 0.90 }, "meta": { # Metadata information "process_time": "2023-12-01 10:30:00", "template_version": "1.0", "source": "text_extraction" } } In this embodiment, the knowledge extraction module builds a multi-source heterogeneous data fusion mechanism, integrates multiple types of data sources such as ERP master data and technical specifications, and converts the original text into high-quality input through pre-processing such as text cleaning, domain segmentation, and BIO annotation; based on the progressive domain adaptation architecture, a high-precision knowledge extraction model is built through vocabulary expansion, domain pre-training, and hierarchical decoding (Bi-LSTM+CRF captures long-distance dependencies); then the industrial labels output by the model are mapped to structured data through standard knowledge templates, and the initial credibility is calculated by weighted calculation of the three dimensions of text features, data sources, and model confidence, providing a basis for subsequent review priority sorting.

[0032] Change analysis module: used to collect historical modification records of some structured knowledge and obtain change features based on the historical modification records; the change features include knowledge change indicators and knowledge vulnerability scores; In order to achieve accurate identification and risk assessment of industrial knowledge changes, in the change analysis module, the system first collects the historical modification records of the extracted structured knowledge (including modification time, modification content, modifier, etc.), and then quantitatively analyzes its change characteristics through multiple dimensions, thereby locating high-risk knowledge items and generating vulnerability scores, providing a basis for audit priority sorting.

[0033] Specifically, the workflow of the change analysis module includes: 1. Calculate the modification frequency score: Extract the modification times of each structured knowledge from the historical modification records and divide them into the following types according to the time window: N modification times in the short term (within 7 days) short ;N number of mid-term (within 30 days) modifications mid and the number of long-term (180 days) modifications N long ; Then the ratio calculation is performed with each time window to obtain the modification frequency within the preset short period of time , preset modification frequency in the medium term , preset the modification frequency in the long term ; According to the formula The modification frequency score FS is calculated; where: , , Represents the weight coefficient, the default value is: =0.5, =0.3, =0.3; 2. Calculate the modification score: The structured knowledge is divided into numerical knowledge and categorical knowledge according to data type, and the structured knowledge V before modification in the historical modification records is collected. old and the modified structured knowledge V new ; For numerical knowledge, according to the formula The modification amplitude score AS is calculated; For categorical knowledge, according to the formula The modification score AS is calculated; where: Represents a preset benchmark value, based on attribute ratings, industry standards, or historical statistics. represents the structured knowledge vector before modification, represents the modified structured knowledge vector; 3. Calculate the impact range score: With structured knowledge items as nodes, a knowledge graph is established through attribute association, and the direct related nodes ND and indirect related nodes NI of each node are determined; the nodes are mainly divided into three levels, and the corresponding importance is as follows: Core nodes (such as device models): 3; important nodes (such as key parameters): 2; ordinary nodes (such as attachments): 1; Then according to the formula The influence range score IS is calculated; where: Indicates the node importance of the directly associated node, Indicates the node importance of indirectly related nodes, represents the level of indirectly related nodes, i represents the structured knowledge index, I max represents the maximum value of the node importance among several associated nodes, that is, in this embodiment, I max =3 (core node); 4. Calculate the knowledge vulnerability score According to the weighted summation formula VS=W FS ×FS+W AS ×AS+W IS×IS calculates the knowledge vulnerability score VS; where the default values ​​of the weight coefficients are: W FS =0.4, W AS =0.3, W IS =0.3; And they are divided into levels according to the knowledge vulnerability score: High vulnerability: VS>=0.7; Medium vulnerability: 0.4<=VS<0.7; Low vulnerability: VS<0.4; Based on the above workflow, in the change analysis module, the system achieves accurate identification and hierarchical management of industrial knowledge change risks by quantitatively analyzing historical modification records. Specifically, the change analysis module identifies the change frequency of knowledge items in the short, medium, and long term by calculating the modification frequency score; quantifies the severity of numerical or categorical changes by the modification amplitude score; evaluates the chain effect of knowledge changes in the knowledge graph by the impact range score; and finally obtains the knowledge vulnerability score by the weighted summation method; it not only achieves the traceability of knowledge changes, but also provides a scientific basis for audit decisions through quantitative indicators, ensuring that high-risk knowledge (such as key parameters with high frequency modifications) is reviewed by experts first, thereby improving the stability and reliability of the industrial knowledge base.

[0034] Audit decision module: used to calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; In the audit decision module, in order to achieve intelligent matching between expert capabilities and knowledge risks, the system quantifies expert credibility in multiple dimensions and combines knowledge change indicators to dynamically generate audit priorities, ensuring that high-risk knowledge is handled first by high-credibility experts; In one implementation, the calculation process of the expert credibility in the audit decision module may include the following steps: C1-1, collect historical audit data of several experts, including the number of tasks assigned N A 、Number of tasks accepted R , audit content, audit results, audit records, audit quantity N T ; Among them, historical audit data is stored in the audit database; C1-2, based on the audit results, audit quantity, and the time difference between the audit time in the audit record and the current time , calculate the accuracy score EA of the current expert, the formula is: ;in, Indicates the audit results, and Represents structured knowledge of correct review, Represents structured knowledge of error review, represents the total number of reviews, i represents the structured knowledge index; C1-3, using the cosine similarity calculation formula, count the similar audit records whose similarity with the current expert's audit content is greater than the preset similarity threshold, and get the total number of similar cases , Number of cases with consistent audit results and the number of cases that agree with the high confidence expert review results ; Wherein, the highly credible expert refers to an expert whose credibility is greater than a preset credibility threshold; C1-4, according to the formula Calculate the consistency score EC of the current expert; the default values ​​of the weight coefficients are: =0.6, =0.4; C1-5, according to the formula Calculate the current expert's activity score EAct; where, Indicates the number of tasks with detailed descriptions in the review results. The default values ​​of the weight coefficients are: =0.4, =0.3, =0.3; C1-6, using the weighted summation method, according to the accuracy score EA, consistency score EC and activity score EAct of the current expert, the expert credibility E is calculated; and the default values ​​of the weight coefficients of each indicator are: W EA =0.8,W EC =0.15, W EAct =0.05.

[0035] In the expert credibility calculation formula, the accuracy score is combined with the time decay factor to give priority to recent audit performance, avoid the impact of outdated historical data on the assessment, and truly reflect current capabilities; the consistency score matches similar cases through cosine similarity and compares the consistency with highly credible experts to ensure the uniformity of audit standards and enhance the authority of audit results; in addition, the activity score comprehensively considers response speed, task completion rate, and detailed description rate to encourage experts to actively participate in audits and promote the timely completion of high-quality audits.

[0036] In one implementation, the audit priority in the audit decision module can be obtained by the following steps: C2-1, according to the formula CR=W FS ×FS+W IS ×IS+W VS ×VS is used to calculate the change risk CR; the default values ​​of the weight coefficients are: W FS =0.4, W IS =0.3, W VS =0.3 C2-2, according to the formula The audit priority Pr of structured knowledge is calculated; among them, , Represents the weight coefficient, and the default values ​​are: =0.6, =0.4.

[0037] The audit decision module realizes intelligent scheduling of audit tasks by quantifying expert capabilities and knowledge risks. For example, high-priority tasks are automatically matched with high-credibility experts (E≥0.8) to ensure the audit quality of key knowledge; while low-priority tasks can be assigned to novice experts for capacity building.

[0038] User interaction module: used to provide an interactive page for expert review, display and modify structured knowledge according to the review priority, and update the credibility of structured knowledge and expert credibility according to the expert modification records.

[0039] Finally, in the user interaction module, in order to ensure the accuracy and efficiency of industrial knowledge review and the dynamic update of knowledge and expert credibility, an interactive page for expert review is provided, structured knowledge is displayed and modified according to the review priority, and relevant credibility is updated according to the expert modification records, thus realizing the intelligent and scientific knowledge review process.

[0040] In one implementation, displaying and modifying structured knowledge according to the review priority may include the following steps: First, the structured knowledge is sorted and displayed according to the review priority, so that experts can clearly see the knowledge items of different priorities. Then, according to the preset expert allocation rules, a number of assigned experts are determined for each structured knowledge, for example: High risk (Pr>=0.8): 3 experts are needed; Medium risk (0.5<=Pr<0.8): 2 experts are needed; Low risk (Pr<0.5): 1 expert is needed; For each assigned expert, multiply their expert credibility E by the domain weight to obtain the expert weight W j (j represents the expert index). The field weight reflects the expert's professionalism in a specific field, and can comprehensively consider the expert's overall credibility and authority in the field. It is mainly assigned based on the expert's education and work background (including education background, work experience, professional qualifications, etc.); For numerical knowledge, collect the proposed modification values ​​Vp of several assigned experts and the expert approval values ​​Ve of highly credible experts, and calculate them according to the formula Eo j =1- Vp-Ve / Vp calculates the expert opinion value Eo of the jth expert j, to measure the degree of difference between the proposed modification value of the current expert and the value recognized by the highly credible expert. The smaller the difference, the higher the opinion value of the current expert; For categorical knowledge, when the assigned experts agree to modify, the expert opinion value Eo j =1; when the assigned expert disagrees with the modification, the expert opinion value Eo j =0; Then calculate the fusion result value and the disagreement degree: According to the formula and formula The fusion result value FinalScore and the degree of disagreement DS are calculated respectively; Finally, determine whether the disagreement degree DS is greater than the preset disagreement threshold; if yes, use the expert opinion value with the highest expert weight to modify the structured knowledge to ensure that when there is a large disagreement, the most authoritative expert will lead the modification; if no, use the fusion result value to modify the structured knowledge to reflect the comprehensive opinions of the majority of experts.

[0041] In one implementation, updating the structured knowledge credibility and the expert credibility according to the expert modification record may include the following steps: According to the formula IC new_i =IC i ×(1+0.1×(1-DS)) to calculate the updated structured knowledge credibility IC new Among them, IC i represents the initial credibility of the i-th structured knowledge; According to formula E new_j =E j ×(1+0.05×(1- Eo j -FinalScore )) Calculate the updated expert credibility E of the jth expert new_j ; where E j represents the expert credibility of the jth expert.

[0042] In the user interaction module, by displaying structured knowledge in order of audit priority, experts can quickly focus on high-risk knowledge and improve audit efficiency; by calculating expert weights, expert opinion values, fusion result values ​​and disagreement degrees, it is possible to comprehensively consider the professional capabilities and opinions of different experts and make more reasonable modification decisions; and by updating the credibility of structured knowledge and expert credibility, the evaluation of knowledge and experts can be continuously adjusted and optimized during the audit process, thus building a continuously updated audit system and improving the quality and level of industrial knowledge management.

[0043] The second embodiment of the present invention provides an industrial knowledge extraction and editing method based on human-machine collaboration, including: The original industrial text is preprocessed and input into the knowledge extraction model to obtain the model output, and some structured knowledge and initial credibility are obtained based on the model output; wherein the knowledge extraction model is constructed based on the deep learning algorithm; Collect historical modification records of some structured knowledge, and obtain change features based on the historical modification records; wherein the change features include knowledge change indicators and knowledge vulnerability scores; Calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; The structured knowledge is displayed and modified according to the review priority, and the structured knowledge credibility and expert credibility are updated according to the expert modification records.

[0044] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0045] Working principle of the present invention: The knowledge extraction module uses deep learning models to preprocess industrial texts, perform entity recognition and structural transformation, and generate structured knowledge with initial credibility; The change analysis module calculates the modification frequency, magnitude, and impact range of knowledge by analyzing historical modification records, generates knowledge vulnerability scores, and evaluates knowledge stability; The audit decision module dynamically calculates the expert credibility based on the expert's historical audit data (accuracy, consistency, activity), and determines the audit priority in combination with the knowledge vulnerability score; Finally, the user interaction module displays knowledge items according to priority, and experts review and modify them through an interactive interface. Then, based on expert opinions, the knowledge credibility and expert credibility are dynamically updated, forming a cycle of update and optimization.

[0046] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. Industrial knowledge extraction and editing system based on human-machine collaboration, characterized by: include: Knowledge extraction module: used to pre-process the original industrial text and input it into the knowledge extraction model to obtain the model output, and obtain some structured knowledge and initial credibility based on the model output; the knowledge extraction model is built based on the deep learning algorithm; Change analysis module: used to collect historical modification records of some structured knowledge and obtain change features based on the historical modification records; the change features include knowledge change indicators and knowledge vulnerability scores; Audit decision module: used to calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; User interaction module: used to provide an interactive page for expert review, display and modify structured knowledge according to the review priority, and update the credibility of structured knowledge and expert credibility according to the expert modification records.

2. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 1 is characterized in that: The knowledge extraction model is built based on a deep learning algorithm, including: A1-1, add Bi-LSTM layer and CRF layer to the BERT output layer in the BERT-Base-Chinese model to build the initial model; A1-2, collect a number of industrial texts, preprocess and entity annotate the industrial texts, and obtain an unannotated text set and an annotated data set; wherein the annotated data set includes a text set and a corresponding annotation set; A1-3, pre-train the initial model using the unlabeled text set to obtain a pre-trained model; A1-4, input the labeled data set into the pre-training model for supervised training to obtain a knowledge extraction model; wherein the input of the knowledge extraction model is industrial text, and the output is industrial labels and label prediction probabilities, and the industrial labels include material name, brand, model, specification parameters and others.

3. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 1 is characterized in that: The model-based output obtains several structured knowledge and initial credibility, including: A2-1, define the necessary attributes, specification attributes and characteristic attributes of the standard knowledge template to obtain the standard knowledge template; A2-2, unify the format of industrial labels in the model output according to the standard knowledge template to obtain some structured knowledge; A2-3, calculate the text features of structured knowledge based on the standard knowledge template; the text features include completeness C, standardization S, consistency K, source level L and timeliness score F; A2-4, calculate the text quality score TQS of the original industrial text according to the text quality calculation formula: TQS=α1×C+α2×S+α3×K; A2-5, calculate the data source score SCS of the original industrial text according to the data source score calculation formula: SCS = β1 × L + β2 × F; A2-6, according to the formula MCS = 1 / n∑P i The model confidence MCS is calculated; where n represents the number of industrial tags, P i represents the predicted probability of the i-th industrial label; A2-7, using the weighted summation method, the initial credibility IC of some structured knowledge is calculated according to the text quality score TQS, the data source score and the model confidence MCS; among them, α1, α2, α3 and β1, β2 represent the weight coefficients corresponding to each formula respectively, and the sum of the corresponding weight coefficients is 1.

4. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 3 is characterized in that: The text features of structured knowledge calculated according to the standard knowledge template include: The total number of attributes actually filled in the structured knowledge is N total The ratio of the number of necessary attributes to the standard knowledge template is marked as completeness C; The number of attributes in structured knowledge that conform to the standard knowledge template is divided by the total number of attributes N total The ratio of is marked as the standard degree S; The number of attributes in structured knowledge that do not conform to the standard knowledge template is marked as the number of contradictory attributes N. C , and according to the formula K=1-N C / N total Calculate the consistency K; Divide structured knowledge according to the source of industrial text data and set the source level to obtain the source level L; And according to the time interval between the generation time of industrial text data and the current time Calculate the timeliness score F, the formula is: .

5. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 1 is characterized in that: The change characteristics obtained according to the historical modification records include: B1-1, the modification frequency score FS is calculated based on the number of modifications of structured knowledge in the historical modification records. The formula is: ;in, Indicates the modification frequency within a preset short period of time. Indicates the frequency of modification within the preset medium term. Indicates the modification frequency within a preset long-term period. , , represents the weight coefficient; B1-2, divide the structured knowledge into numerical knowledge and categorical knowledge according to data type, and collect the structured knowledge V before modification in the historical modification records old and the modified structured knowledge V new ; B1-3, for numerical knowledge, according to the formula Calculate the modification score AS; for categorical knowledge, according to the formula The modification score AS is calculated; where: Indicates the preset reference value, represents the structured knowledge vector before modification, represents the modified structured knowledge vector; B1-4, establishing a knowledge graph association relationship based on a number of structured knowledge, and determining the direct associated node ND and the indirect associated node NI of each node; wherein, in the knowledge graph association relationship, the structured knowledge items are nodes; B1-5, according to the formula The influence range score IS is calculated; where: Indicates the node importance of the directly associated node, Indicates the node importance of indirectly related nodes, represents the level of indirectly related nodes, i represents the structured knowledge index, I max Indicates the maximum value of node importance among several associated nodes; B1-6, perform weighted summation on the knowledge change indicators to obtain the knowledge vulnerability score VS; wherein the knowledge change indicators include the modification frequency score FS, the modification amplitude score AS and the impact range score IS.

6. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 5 is characterized in that: The expert credibility is calculated based on the historical review data of several experts, including: C1-1, collect historical audit data of several experts, including the number of tasks assigned N A 、Number of tasks accepted R , audit content, audit results, audit records, audit quantity N T ; Among them, historical audit data is stored in the audit database; C1-2, based on the audit results, audit quantity, and the time difference between the audit time in the audit record and the current time , calculate the accuracy score EA of the current expert, the formula is: ;in, Indicates the audit results, and Represents structured knowledge of correct review, Represents structured knowledge of error review, represents the total number of reviews, i represents the structured knowledge index; C1-3, using the cosine similarity calculation formula, count the similar audit records whose similarity with the current expert's audit content is greater than the preset similarity threshold, and get the total number of similar cases , Number of cases with consistent audit results and the number of cases that agree with the high confidence expert review results ; Wherein, the highly credible experts include system-defined experts and experts whose expert credibility obtained by updating the expert credibility is greater than a preset credibility threshold; C1-4, according to the formula Calculate the consistency score EC of the current expert; C1-5, according to the formula Calculate the current expert's activity score EAct; where, Indicates the number of tasks with detailed descriptions included in the audit results; C1-6, using the weighted summation method, according to the accuracy score EA, consistency score EC and activity score EAct of the current expert, the expert credibility E is calculated; where, , and , , They represent the weight coefficients of the corresponding formulas respectively.

7. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 6 is characterized in that: The review priorities are determined based on expert credibility and change indicators, including: C2-1, weighted sum of the modification frequency score FS, impact scope score IS and knowledge vulnerability score VS in the change index to obtain the change risk CR; C2-2, according to the formula The audit priority Pr of structured knowledge is calculated; among them, , Represents the weight coefficient, and the sum is 1.

8. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 6 is characterized in that: The display and modification of structured knowledge according to the audit priority includes: Sort and display structured knowledge according to the review priority, and obtain a number of assigned experts for structured knowledge according to the preset expert assignment rules; Multiply the assigned expert's expert credibility E by the domain weight to get the expert weight W j ; Wherein, j represents the expert index; Wherein, the field weight is the weight value preset by the system, which is used to evaluate the professional level of experts in several fields; For numerical knowledge, collect the proposed modification values ​​Vp of several assigned experts and the expert approval values ​​Ve of highly credible experts, and calculate them according to the formula Eo j =1- Vp-Ve / Vp calculates the expert opinion value Eo of the jth expert j ; For categorical knowledge, when the assigned expert agrees to modify, the expert opinion value Eo j =1; when the assigned expert disagrees with the modification, the expert opinion value Eo j =0; According to the formula Calculate and obtain the fusion result value FinalScore; According to the formula The degree of disagreement DS is calculated; Determine whether the disagreement degree DS is greater than the preset disagreement threshold; if yes, use the expert opinion value with the highest expert weight to modify the structured knowledge; if no, use the fusion result value to modify the structured knowledge.

9. The industrial knowledge extraction and editing system based on human-machine collaboration according to claim 8 is characterized in that: The updating of structured knowledge credibility and expert credibility according to the expert modification record includes: According to the formula IC new_i =IC i ×(1+0.1×(1-DS)) to calculate the updated structured knowledge credibility IC new Among them, IC i represents the initial credibility of the i-th structured knowledge; According to formula E new_j =E j ×(1+0.05×(1- Eo j -FinalScore )) Calculate the updated expert credibility E of the jth expert new_j ; where E j represents the expert credibility of the jth expert.

10. The method for extracting and editing industrial knowledge based on human-machine collaboration is applied to the system for extracting and editing industrial knowledge based on human-machine collaboration as claimed in any one of claims 1 to 9, characterized in that: include: The original industrial text is preprocessed and input into the knowledge extraction model to obtain the model output, and some structured knowledge and initial credibility are obtained based on the model output; wherein the knowledge extraction model is constructed based on the deep learning algorithm; Collect historical modification records of some structured knowledge, and obtain change features based on the historical modification records; wherein the change features include knowledge change indicators and knowledge vulnerability scores; Calculate the expert credibility based on the historical audit data of several experts, and obtain the audit priority based on the expert credibility and change indicators; The structured knowledge is displayed and modified according to the review priority, and the structured knowledge credibility and expert credibility are updated according to the expert modification records.

Images