A large-model-assisted self-improving multimodal industrial equipment knowledge graph construction method
The method of constructing a self-improving multimodal industrial equipment knowledge graph with the assistance of a large model utilizes multimodal data and equipment topology information to build an initial graph and perform self-improving updates. This solves the problem of misassociations after major overhaul of chemical compressor units, achieves high-precision anomaly identification and graph optimization, and ensures the stability and accuracy of the knowledge graph in complex scenarios.
Patent Information
- Application Number
- CN202610629472.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-09
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-05-09
AI Technical Summary
In short-cycle resumption scenarios after major overhauls of chemical compressor units, existing knowledge graph construction methods are prone to forming stable false associations, making it difficult to identify rare and harmful early abnormal knowledge such as minor sealing failures. Furthermore, the high rate of false associations leads to instability of knowledge graphs in complex industrial scenarios.
A self-boosting multimodal industrial equipment knowledge graph construction method with large model assistance is proposed. By acquiring data such as voice, image, vibration, alarm and work order of the equipment, and combining the equipment topology snapshot and tag number evolution information, a candidate set of identical objects and a set of constraints that prohibit merging are generated. The large model is used to extract evidence anchor points and timestamps to construct an initial graph. By comparing the local counterfactual fragment set with the original fragment, the difference in relationship establishment and constraint violation value are calculated. Credible subgraphs are selected for self-boosting update to optimize the graph.
Significantly reduces the false association rate, improves the structural consistency and physical interpretability of the knowledge graph, enables accurate identification and high-precision relationship extraction in complex industrial scenarios, and ensures stable convergence of the graph under version evolution and topological perturbation.
Smart Images

Figure CN122154887B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial equipment knowledge graph construction, specifically to a method for constructing a self-improving multimodal industrial equipment knowledge graph assisted by a large model. Background Technology
[0002] In short-cycle restart scenarios after major overhauls of chemical compressor units, it is common to see situations where the same measuring point uses the old tag number, spare parts are replaced and the old component abbreviation is used, and inspection voice recordings are made with habitual terms while the DCS alarms with new codes. This causes infrared hotspots, vibration side frequencies and work order descriptions to appear simultaneously in time but be incorrectly assigned to adjacent components. For minor sealing failure symptoms that only occur within a few minutes of start-stop switching, existing graph construction methods usually build edges directly based on co-occurrence or semantic similarity, which easily forms stable erroneous associations. These associations become more and more solidified with subsequent iterations, making it difficult to discover such rare and harmful early anomalies. Summary of the Invention
[0003] The purpose of this invention is to provide a large-model-assisted self-improving multimodal industrial equipment knowledge graph construction method to address the shortcomings in the prior art.
[0004] To achieve the above objectives, this invention provides the following technical solution: a large-model-assisted self-improving multimodal industrial equipment knowledge graph construction method, comprising:
[0005] S100: Acquire the voice, image, vibration, alarm, work order and maintenance records of the equipment during the target time period, and extract the equipment topology snapshot and tag number evolution information corresponding to the target time period to obtain the multimodal event fragment set M; S200, based on the device topology snapshot and tag number evolution information, generate a candidate set H of identical objects and a set of prohibited merging constraints J for M; S300: Input M, H and J into the large model and extract the entity relationship candidate set C with modal evidence anchors, timestamps and version identifiers; S400, construct an initial graph G based on C, and replace the version identifier, adjacent topological nodes or the order of events around the evidence anchor point to generate a local counterfactual fragment set F; S500, calculate the difference between the establishment of the relationship between G and F in the original fragment and the constraint violation value respectively, obtain the anti-confusion score Z of each candidate relationship, and retain the stable relationship that satisfies the threshold to form a reliable subgraph K; S600 uses K as positive samples and the negated relations in F as difficult negative samples to perform self-upgrading updates on the large model. It then uses the updated large model to re-extract and complete the graph of unconfirmed fragments, and outputs the target industrial equipment knowledge graph.
[0006] Preferably, the step of generating a candidate set H of identical objects and a set of constraints J to prohibit merging for M includes: S210, extracting the installation location, upstream connection node, downstream connection node and measurement object for each event segment based on the device topology snapshot, and constructing a segment topology fingerprint; S220, mapping each event segment to a corresponding historical tag chain and replacement tag chain based on the tag evolution information; S230, pairing event segments with consistent segment topology fingerprints and whose historical tag chains and replacement tag chains meet the time connection condition, and generating a candidate set H of identical objects; S240, applying a prohibition-to-merge mark to event segments with conflicting installation locations, conflicting measurement objects, or overlapping times and broken tag chains, and generating a set of constraints J to prohibit merging.
[0007] Preferably, the extraction of the entity relation candidate set C with modal evidence anchors, timestamps, and version identifiers includes: S310, binding each event fragment in the multimodal event fragment set M with the same object candidate set H and the prohibited merging constraint set J in chronological order, and constructing a constraint input sequence containing the original content index, fragment topological fingerprint, and position of the tag chain; S320, inputting the constraint input sequence into the large model, and inserting modal boundary markers for voice, image, vibration, alarm, and text respectively, generating modal evidence anchors corresponding to the original content index; S330, extracting entity relation triples and corresponding timestamps based on the modal evidence anchors and event fragment time intervals, and assigning version identifiers based on the historical tag chain and the replacement tag chain; S340, removing entity relation triples that violate the prohibited merging constraint set J, and writing the remaining results into the entity relation candidate set C.
[0008] Preferably, the construction of the initial graph G based on C includes: S410, constructing directed edges with timestamps and version identifiers according to the head entity, relation and tail entity in the entity relation candidate set C, and using modal evidence anchor points as edge attributes to form the initial graph G.
[0009] Preferably, generating a local counterfactual fragment set F includes: S420, locating the corresponding event fragment using the modal evidence anchor point, replacing its version identifier to generate a version perturbation fragment, and keeping the other fields unchanged; S430, based on the device topology snapshot, selecting the upstream or downstream connection node of the node corresponding to the modal evidence anchor point in the initial map G, replacing the head entity or tail entity, and generating a topology perturbation fragment; S440, rearranging and combining the version perturbation fragment and the topology perturbation fragment according to the chronological order of events to form a local counterfactual fragment set F.
[0010] Preferably, obtaining the anti-confusion score Z and the credible subgraph K includes: S510, based on modal evidence anchor points, extracting the probability of each candidate relationship being established in the original fragment and the probability of the relationship being established in the local counterfactual fragment set F from the initial graph G, and calculating the difference between the two as the relationship establishment difference; S520, based on the prohibited merging constraint set J and the device topology snapshot, statistically analyzing the installation location conflict, measurement object conflict, and tag chain breakage triggered by the candidate relationship in the local counterfactual fragment set F to obtain the constraint violation value; S530, combining the relationship establishment difference and the constraint violation value according to a preset weight coefficient to obtain the anti-confusion score Z; S540, screening candidate relationships with an anti-confusion score Z greater than a preset threshold and constructing the credible subgraph K.
[0011] Preferably, step S520 further includes: S521, for each candidate relationship, locating all corresponding segments in the local counterfactual segment set F based on modal evidence anchor points, and extracting the installation location, measurement object, and tag chain information of the corresponding event segment; S522, comparing the installation location with the node connection relationship in the device topology snapshot, marking it as an installation location conflict when there is no direct connection or hierarchical relationship, and matching the measurement object with a predefined set of physical quantity categories, marking it as a measurement object conflict when they are inconsistent; S523, according to the tag chain continuity determination rule, marking segments with overlapping time and no sequential replacement relationship of tag chains as tag chain breaks, and counting the number of occurrences of each type of conflict; S524, assigning different weight coefficients to installation location conflicts, measurement object conflicts, and tag chain breaks respectively, and calculating the constraint violation value by summing the product of the number of conflicts and the corresponding weight coefficient.
[0012] Preferably, step S540 further includes: S541, normalizing the anti-obfuscation score Z of each candidate relationship; S542, constructing a joint screening index based on the normalized anti-obfuscation score Z, combined with the timestamp density and modal evidence anchor point number corresponding to the candidate relationship; S543, comparing the joint screening index with a preset threshold to screen out candidate relationships that meet the conditions; S544, aggregating the screened candidate relationships according to the timestamp order and device topology connection relationship to construct a reliable subgraph K.
[0013] Preferably, the self-improvement update of the large model using K as positive samples and the negated relations in F as hard-to-negative samples includes: S610, extracting relation edges and their modal evidence anchors, timestamps, and version identifiers from the credible subgraph K to construct a positive sample set; S620, selecting fragments from the local counterfactual fragment set F that are consistent with the head or tail entities of the positive sample set and whose relations are negated to construct a hard-to-negative sample set; S630, inputting the positive sample set and the hard-to-negative sample set into the large model according to the evidence anchor alignment method, so that the large model learns to distinguish between real relations and perturbation relations, and completes the self-improvement update; S640, using the updated large model to re-extract entity relations from unconfirmed fragments and merge them with the credible subgraph K to output the target industrial equipment knowledge graph.
[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention introduces a comparison mechanism between a local counterfactual fragment set F and the original fragment, constructing a joint evaluation system of relationship establishment difference and constraint violation value. This effectively distinguishes between "true stable relationships" and "pseudo-relationships caused by temporal co-occurrence, topological proximity, or tag replacement." Compared to traditional graph construction methods that rely solely on semantic similarity or co-occurrence frequency, this invention explicitly introduces equipment topology snapshots, tag evolution information, and a prohibited merging constraint set J during the relationship determination process. This allows for accurate identification of the same physical object and its true relationship even in complex industrial scenarios such as short-cycle resumption of production after maintenance, frequent measurement point replacement, and mixed use of colloquial aliases. Consequently, it significantly reduces the false association rate and improves the structural consistency and physical interpretability of the knowledge graph.
[0015] 2. This invention employs a self-boosting update mechanism that uses a trustworthy subgraph K as positive samples and denied relations in a local counterfactual fragment set as difficult negative samples. This enables the large model to continuously learn "easily confused relation boundaries" and achieve higher-precision relation extraction and graph completion in subsequent unconfirmed fragments. This approach overcomes the limitations of existing methods that rely on static labeled data, achieving closed-loop iterative optimization for multimodal data of industrial equipment. This ensures the knowledge graph maintains stable convergence even under version evolution, topological perturbations, and short-term anomalies. The resulting target industrial equipment knowledge graph possesses not only higher accuracy but also greater robustness. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] For examples, please refer to Figure 1 As shown in this embodiment, the large model-assisted self-improving multimodal industrial equipment knowledge graph construction method includes: Taking the anomaly identification of a pyrolysis gas centrifugal compressor unit within 72 hours of restarting production after a major overhaul of an ethylene plant as an example, the equipment includes components such as a main motor, speed increaser, compressor body, primary seal, secondary seal, bearing housing, inlet pipe section, and outlet pipe section. The target time period can be preset according to application needs; in this embodiment, it is set from 30 minutes before the equipment starts up to 6 hours after startup.
[0020] In step S100, the equipment first acquires multi-source raw data during the target time period. Voice data comes from inspection terminals and intercom systems, including inspection reports, shift handover records, and defect repetitions; image data includes visible light images and infrared thermal images, sourced from fixed cameras, handheld inspection devices, or robots; vibration data comes from acceleration sensors installed on bearing housings, sealing ends, and the casing; alarm data comes from DCS, PLC, or SIS systems, including at least alarm tag number, alarm type, trigger time, recovery time, and alarm value; work order data comes from the equipment management system, including at least work order number, equipment code, processing content, handler, and processing time; maintenance records come from the maintenance system, including at least replaced parts, old tag number, new tag number, disassembly / reassembly time, calibration results, and trial run conclusions.
[0021] The data from all the aforementioned sources are processed using a unified time reference. Preferably, the DCS server time is used as the primary clock for clock correction of the voice terminals, camera equipment, vibration data acquisition devices, and work order system logs. For data acquisition devices without a unified clock, compensation can be made based on upload time, log write time, or trigger event time. All data is uniformly mapped to millisecond-level timestamps.
[0022] Subsequently, the data for each modality were segmented into multimodal event segments. Speech data underwent speech activity detection and automatic speech recognition processing to obtain text transcription results, which were then segmented at the sentence level or by fixed-duration windows. Image data was segmented by shooting time, shooting location, and target device area, and the image file index, shooting pose, and detected component bounding boxes were recorded. Vibration data was segmented by sliding time windows, and features such as root mean square, kurtosis, envelope spectrum dominant frequency, and sideband energy ratio were extracted for each window. Alarm data was segmented by single alarm event. Work orders and maintenance records were segmented by record entries. Each event segment included at least: segment number, modality type, original data index, start and end times, involved equipment code, involved tag number or component name, and quality marker.
[0023] Further extract the equipment topology snapshot and tag number evolution information corresponding to the target time period. The equipment topology snapshot is generated by combining the valid P&ID diagram, equipment ledger, sensor placement table, and control point table within the target time period, forming a node-connection relationship structure. Nodes include at least equipment, components, measuring points, valves, pipe sections, and alarm objects, and connection relationships include at least installation, connection, measurement, dependency, and control relationships. Tag number evolution information is extracted based on maintenance records, work order closure records, and ledger change records, and includes at least the old tag number, new tag number, effective time, failure time, reason for change, and corresponding component. For example, if after maintenance "PT-102A" is changed to "PT-102A-R1", and the primary sealing vibration measuring point is moved from "SV-1" to "SV-1B", then this change is written into the tag number evolution information.
[0024] Finally, the modal segments after time unification and event segmentation are associated with the device topology snapshot and tag evolution information to obtain a multimodal event segment set M. Any event segment in M can be represented as: m_i =<id_i,type_i,ts_i,te_i,dev_i,tag_i,part_i,content_i,feature_i,q_i> Where id_i is the fragment number, type_i is the modality type, ts_i and te_i are the start and end times respectively, dev_i is the device code, tag_i is the tag number, part_i is the component identifier, content_i is the original content index or transcribed text, feature_i is the corresponding feature, and q_i is the data quality marker. Through this method, a multimodal event fragment set M can be obtained that can be directly used for subsequent same-object identification, relationship extraction, and conflict verification.
[0025] S200, based on the device topology snapshot and tag evolution information, generate a candidate set H for identical objects and a set of constraints J for prohibiting merging for M. In this embodiment, steps S210 to S240 are used to identify pairs of event fragments pointing to the same physical object in the multimodal event fragment set M, and simultaneously exclude event fragment pairs that should be prohibited from merging. The specific implementation is as follows.
[0026] In step S210, firstly, an index table of device nodes, a index table of connection relationships, an index table of measurement objects, and an index table of installation locations are established based on the device topology snapshot. Then, for each event segment in the multimodal event segment set M, the installation location, upstream connection node, downstream connection node, and measurement object are extracted sequentially. For structured event segments, the tag number field, component field, and alarm field inherent to the event segment are read first. For unstructured event segments corresponding to voice, image description text, and work order text, text standardization is performed first, followed by word-by-word matching using a device name dictionary, tag number dictionary, and component synonym dictionary. When multiple candidate matching results exist, the candidate with the same device code within the target time period and the shortest connection path in the device topology snapshot is selected as the final result.
[0027] In this context, the installation location is defined as the attachment point of the sensor, component, or maintenance position corresponding to the event segment in the equipment topology snapshot; the upstream and downstream connection nodes are defined as the nearest preceding and following nodes along the medium flow direction of this attachment point; and the measurement object is defined as one of the pressure, temperature, vibration, flow rate, displacement, or leakage states reflected by the event segment. After extraction, the installation location, upstream and downstream connection nodes, measurement object, and their hierarchical paths in the equipment topology snapshot are combined to construct the segment topology fingerprint. Segment topology fingerprint consistency means that the above items corresponding to two event segments are completely identical after standardization; if only one item is missing, it is allowed to be completed based on the uniquely reachable node in the equipment topology snapshot; if there are more than one reachable node, the event segment will not participate in the generation of the same object candidate set H.
[0028] In step S220, a historical tag number chain and a replacement tag number chain are established for each event segment based on the tag number evolution information. Specifically, the tag number evolution information is organized into directed association records sorted by effective time. Each association record includes at least the old tag number, new tag number, effective start time, effective end time, corresponding installation location, and reason for change.
[0029] For each event segment, if it contains a tag number, a historical tag number chain is obtained by tracing backwards in time from that tag number, and a replacement tag number chain is obtained by tracing forwards in time. If it does not contain a tag number, the installation location and measurement object obtained in step S210 are used as search conditions, and a tag number that matches both the installation location and measurement object is searched in the tag number evolution information as the starting point. Adjacent tag numbers in the historical tag number chain must meet a time connection condition, which is defined as the time interval between the effective end time of the previous tag number and the effective start time of the next tag number not exceeding 24 hours. If the occurrence time of an event segment falls within the effective interval of a certain tag number, or if the overlap between the event segment's time interval and the effective interval is not less than 0.5 times the shorter time interval, then the event segment is determined to be mapped to that tag number. The above-mentioned ratio threshold is determined based on the playback of historical maintenance data, preferably 0.5, and can be selected within the range of 0.3 to 0.8.
[0030] In step S230, event fragments in the multimodal event fragment set M are paired up. When the fragment topological fingerprints of two event fragments are consistent, and one event fragment maps to an adjacent bit on the historical bit chain or replacement bit chain of the other event fragment, or both are mapped to two bits on the same historical bit chain and replacement bit chain that satisfy the time connection condition, it is determined that the two event fragments point to the same physical object, and the event fragment pair is written into the same object candidate set H.
[0031] For example, the tag number of a compressor's primary sealing pressure measuring point was "PDT-311" before maintenance, and was replaced with "PDT-311A" after maintenance. The old tag number's effective end time was 10:00 AM on March 18, 20xx, and the new tag number's effective start time was 10:20 AM on March 18, 20xx. If the voice event segment at 10:25 AM is described as "primary sealing pressure differential fluctuation", and the alarm event segment at 10:28 AM corresponds to the tag number "PDT-311A", and both have the same installation location, upstream connection node, downstream connection node, and measurement object, then because they satisfy the conditions of consistent segment topological fingerprint and time connection, a pair of event segments should be generated in the candidate set H of the same object.
[0032] In step S240, a set of constraints J prohibiting merging is generated. If two event segments have any of the following conditions, a prohibition-merging flag is applied to the event segments: First, installation location conflict, that is, the installation locations corresponding to the two event segments are different, and the two do not constitute a direct superior-subordinate node relationship in the device topology snapshot; Second, measurement object conflict, that is, the measurement objects of the two event segments belong to different physical quantity categories; Third, time overlap and tag chain break, that is, the time intervals of the two event segments overlap, and the proportion of the overlap duration to the shorter time interval is not less than 0.3, but there is no replacement relationship between the historical tag chain and the replacement tag chain corresponding to the two, and there are no consecutive tag numbers that meet the time connection conditions.
[0033] Specifically, the term "broken position chain" refers to two position chains that have neither a common position number nor a continuous evolution record from the old position number to the new position number, or where there is an evolution record but the time interval between adjacent position numbers exceeds 24 hours. In addition to storing event fragments, the prohibited merging constraint set J also stores the type and source of evidence that triggers the prohibited merging, for direct retrieval during subsequent relationship verification.
[0034] For example, one event segment is described as a first-level sealing pressure difference abnormality, and another event segment corresponds to a high inlet temperature. Although the two occur within the same time window, they are different in installation location, measurement object, and there is no continuous replacement relationship in the tag number evolution information. Therefore, they should be written into the prohibited merging constraint set J, and are not allowed to enter the same object candidate set H.
[0035] In this embodiment, step S300 is used to extract a candidate set C of entity relationships with modal evidence anchors, timestamps, and version identifiers under the joint constraints of the multimodal event fragment set M, the candidate set of identical entities H, and the prohibited merging constraint set J. The specific implementation is as follows.
[0036] In step S310, the event segments in the multimodal event segment set M are first sorted from earliest to latest according to their start time. When two event segments have the same start time, they are arranged according to a fixed order of modal types, namely alarm, vibration, voice, image, and text. Then, the sorted event segments are bound one by one to the same-item candidate set H and the prohibited merging constraint set J to construct a constraint input sequence. Each record in the constraint input sequence includes at least: event segment number, original content index, modal type, segment topological fingerprint, start time, end time, historical position in the position chain, replacement position in the position chain, same-item candidate flag, and prohibited merging flag. The historical position in the position chain is defined as the sequence number of the corresponding position number of the event segment in the historical position chain, and the replacement position in the position chain is defined as the sequence number of the corresponding position number of the event segment in the replacement position chain. If an event segment is not mapped to a corresponding position chain, the corresponding position is recorded as 0.
[0037] In this embodiment, to avoid excessively long context affecting the extraction stability of the large model, the constraint input sequence is divided into sliding time windows. The preferred sliding time window length is 300 seconds, and the preferred step size is 60 seconds. This value is jointly calibrated based on the alarm duration of the refining unit, the average duration of the inspection voice, and the propagation duration of abnormal vibration, and can be adjusted within the range of 120 to 900 seconds. The constraint input sequence within each sliding time window retains the original chronological order of the event segments, thereby enabling the subsequent large model to use temporal continuity to determine entity relationships.
[0038] In step S320, the constraint input sequence is input into the large model, and modal boundary markers are inserted for speech, image, vibration, alarm, and text respectively, generating modal evidence anchor points corresponding to the original content index. The large model adopts an autoregressive sequence generation structure, with a parameter scale preferably between 7 billion and 34 billion. The input is organized using an instruction template, which at least includes a task description, terminology definition, output format, rules prohibiting merging, and version identification rules. Specifically, for speech event segments, modal boundary markers of "speech start + event segment number" and "speech end + event segment number" are inserted before and after the transcribed text; for image event segments, modal boundary markers of "image start + event segment number" and "image end + event segment number" are inserted before and after the image description text, and the target region coordinates, target component name, and visible anomaly labels are retained in the description text; for vibration event segments, the root mean square, kurtosis, envelope spectrum dominant frequency, and sideband energy ratio are written as feature text in a fixed field order, and modal boundary markers of "vibration start + event segment number" and "vibration end + event segment number" are inserted before and after them; for alarm event segments, the alarm tag number, alarm type, trigger value, trigger time, and recovery time are retained, and corresponding modal boundary markers are inserted; for text event segments, the work order text or maintenance record text is retained, and corresponding modal boundary markers are inserted. Modal evidence anchor points are defined as the traceable positions of entities or relations in the constrained input sequence, specifically represented by the event segment number, original content index, start character position, and end character position.
[0039] In this embodiment, for image event segments, the modal evidence anchor points also record the coordinates of the target region; for vibration event segments, the modal evidence anchor points also record the time window to which the feature belongs. To ensure consistency in input scale between different modalities, a minimum-to-maximum normalization method is used for numerical features, with the normalization interval fixed at 0 to 1; the minimum and maximum values are obtained from the statistical analysis of the target device's stable operation data over the past 30 days, and if the sample is insufficient, it is supplemented by historical data from the same model of device.
[0040] In step S330, based on the modal evidence anchor point and the event segment time interval, entity relation triples and corresponding timestamps are extracted, and version identifiers are assigned according to the historical tag chain and replacement tag chain. Specifically, the output format of the large model is fixed as structured text, and each extraction result includes a head entity, relation, tail entity, head entity modal evidence anchor point, tail entity modal evidence anchor point, relation trigger word position, timestamp, version identifier, and extraction confidence value. The head entity and tail entity correspond to equipment, component, measurement point, abnormal phenomenon, operating condition, or maintenance action, respectively; relations include at least belonging to, installed at, measured, indicated, caused, replaced, associated with, and occurred at. The rules for determining the timestamp are as follows: when the head entity and tail entity come from the same event segment, the midpoint between the start time and end time of the event segment is taken as the timestamp; when the head entity and tail entity come from different event segments, the midpoint between the overlapping time intervals of the two event segments is taken as the timestamp; if the two event segments do not have overlapping time intervals, the midpoint between the event segments corresponding to the shorter time interval is taken as the timestamp.
[0041] For example, the voice event segment at 10:25 describes the pressure differential fluctuation of the first-level seal, and the alarm event segment at 10:28 corresponds to the tag number "PDT-311A". If the large model extracts the entity relationship triplet of "first-level seal, indication, pressure differential fluctuation", then the timestamp is preferably the overlapping or nearest time between 10:26:30 and 10:28. The version identifier is defined as a combination string of "tag number identifier + tag number chain type + tag number chain position + effective time interval"; where the tag number chain type can only be the historical tag number chain or the replacement tag number chain. If the head entity and the tail entity are mapped to the same historical tag number chain, then the version identifier corresponding to the historical tag number chain is used; if they are mapped to the same replacement tag number chain, then the version identifier corresponding to the replacement tag number chain is used; if the head entity and the tail entity are located on the historical tag number chain and the replacement tag number chain respectively, and they meet the time connection condition in step S230, then the replacement occurrence time is used as the version switching point, and the version identifier simultaneously records the version before the switch and the version after the switch.
[0042] In this embodiment, to improve the protectability and verifiability of the entity relation candidate set C, an extraction confidence value is calculated for each entity relation triple. The extraction confidence value consists of two parts: the first part is the conditional generation probability when the large model outputs the entity relation triple, which is the geometric mean of the probabilities of the corresponding terms in each output field; the second part is the evidence consistency score. The evidence consistency score is defined as a combined score of the temporal proximity of the head entity modal evidence anchor and the tail entity modal evidence anchor within the same sliding time window, the consistency of the fragment topological fingerprint, and the consistency of the same-item candidate marker. The temporal proximity is calculated by subtracting the time difference between the two event fragments from 1 and dividing by the sliding time window length, and is limited to between 0 and 1; the fragment topological fingerprint consistency is 1 when completely consistent, 0.8 when only one item is missing and uniquely completed by the device topology snapshot, and 0 otherwise; the same-item candidate marker consistency is 1 when the event fragments containing the head entity and tail entity belong to the same-item candidate set H, and 0.5 otherwise. The final extraction confidence value is calculated as "0.7 multiplied by the conditional generation probability plus 0.3 multiplied by the evidence consistency score," with the weighting coefficients 0.7 and 0.3 determined by grid search calibration on the labeled sample set. The weight of the conditional generation probability can be adjusted from 0.5 to 0.8, and the weight of the evidence consistency score can be adjusted from 0.2 to 0.5. Entity relation triples with an extraction confidence value of not less than 0.65 are retained; this threshold is determined based on the balance point of precision and recall on historical labeled data and can be adjusted from 0.55 to 0.8.
[0043] In step S340, entity relation triples that violate the prohibition on merging constraint set J are removed, and the remaining results are written into the entity relation candidate set C. Specifically, for each entity relation triple, the event fragment pair corresponding to its head entity modal evidence anchor point and tail entity modal evidence anchor point is read, and the existence of the event fragment pair in the prohibition on merging constraint set J is checked; if it exists, the entity relation triple is directly removed. For cases where the head entity and tail entity correspond to multiple modal evidence anchor points, the set of modal evidence anchor points with the highest confidence value is selected for retrieval; if the highest set of modal evidence anchor points triggers the prohibition on merging flag, the entity relation triple is removed as a whole, and no other modal evidence anchor points are used to replace it. To prevent false connections caused by temporal proximity, a judgment rule is added: when the installation location of the event fragments corresponding to the head entity and tail entity conflict, the measurement object conflicts, or the time overlaps and the tag chain is broken, even if the large model outputs an "association" or "cause" relationship, it must be removed.
[0044] For example, if a voice event segment describes "abnormal level 1 sealing pressure difference" and another alarm event segment describes "high inlet temperature," and both occur within the same sliding time window but at different installation locations and measuring objects, then the entity relation triplet "abnormal level 1 sealing pressure difference, resulting in, high inlet temperature" should be removed because it violates the prohibition on merging constraint set J. After removal, the remaining entity relation triplets are written into the entity relation candidate set C according to their timestamps from earliest to latest; if the timestamps are the same, they are sorted according to the extraction confidence value from highest to lowest.
[0045] In this embodiment, step S400 is used to construct an initial graph G based on the entity relationship candidate set C, and generate a local counterfactual fragment set F around the modal evidence anchor point, so as to subsequently identify unstable relationships in the entity relationship candidate set C that rely on accidental co-occurrence, incorrect version mapping, or adjacent missubstitution. Step S400 specifically includes steps S410 to S440.
[0046] In step S410, directed edges with timestamps and version identifiers are constructed according to the head entity, relation and tail entity in the entity relation candidate set C, and the modal evidence anchor point is used as the edge attribute to form the initial graph G.
[0047] Specifically, the head and tail entities in the candidate set C of entity relationships are first standardized and mapped. The key values used for standardization mapping include equipment code, installation location, measurement object, tag number, and version identifier retained in step S300. When two entities are identical in equipment code, installation location, measurement object, and version identifier, they are determined to be the same graph node. Relationships are represented by directed edges, and the direction of the edges is determined according to the direction from the head entity to the tail entity in the candidate set C of entity relationships. For example, "measurement point, indication, abnormal phenomenon" and "component, installed at, equipment" retain their original directions and are not rewritten in reverse. Each directed edge contains at least the following attributes: relationship type, timestamp, version identifier, modal evidence anchor point, extraction confidence value, and source event fragment number.
[0048] In this context, modal evidence anchor points are stored as edge attributes, meaning that the directed edge must be able to trace back to the original content index and location that triggered the relationship. For speech and text, this corresponds to the start and end character positions; for images, it corresponds to the target area coordinates; for vibration, it corresponds to the time window to which the feature belongs; and for alarms, it corresponds to the alarm record number and trigger time. To avoid the same fact being repeatedly written into the initial graph G, multiple directed edges with the same head entity, the same relationship, the same tail entity, and a timestamp interval of no more than 300 seconds are merged. The merging rule is to retain the directed edge with the highest extracted confidence value and incorporate the modal evidence anchor points of the remaining directed edges into the evidence list of that directed edge. The 300 seconds is determined statistically based on the duration of the compressor unit alarm, the lag time of the inspection verbal statement, and the work order supplementary recording time, and can be adjusted within the range of 60 seconds to 900 seconds.
[0049] In step S420, the corresponding event fragment is located using the modal evidence anchor point, and its version identifier is replaced to generate a version perturbation fragment, while keeping the other fields unchanged. Specifically, based on the source event fragment number and original content index in the modal evidence anchor point, the corresponding event fragment in the multimodal event fragment set M is traced back, and then the position of the event fragment in the historical position chain and the replacement position chain is read. The version identifier replacement is not arbitrary, but is only allowed to be performed at a position adjacent to the current version identifier. The adjacent position is defined as: the previous and next positions in the same historical position chain, or the previous and next positions in the same replacement position chain; if the current version identifier is at the beginning or end of the position chain, replacement is only allowed to be performed on the existing side. During replacement, the head entity name, relationship, tail entity name, timestamp, and modal evidence anchor point remain unchanged, and only the version identifier is rewritten to the adjacent version identifier, thereby generating a version perturbation fragment.
[0050] For example, if the version identifier of a directed edge in a candidate set C of entity relations corresponds to "PDT-311A, replacement bit chain, 2nd bit, from 10:20 AM on March 18, 20xx to 8:00 AM on March 21, 20xx", and its preceding bit in the replacement bit chain is "PDT-311, replacement bit chain, 1st bit, from 8:00 AM on March 10, 20xx to 10:20 AM on March 18, 20xx", then the version identifier in the original directed edge can be replaced with the preceding version identifier, forming a version perturbation fragment. To ensure the comparability of the version perturbation fragment, the time deviation between the replaced version identifier and the original timestamp must not exceed 24 hours; this 24-hour period must be consistent with the time connection condition in step S220, and can be calibrated within a range of 1 hour to 72 hours based on maintenance switch records. If it exceeds this range, no corresponding version perturbation fragment will be generated.
[0051] In step S430, based on the equipment topology snapshot, upstream or downstream connecting nodes of the nodes corresponding to the modal evidence anchor points are selected in the initial map G to replace the head or tail entities, generating a topology disturbance fragment. Specifically, the source event fragment that triggered the directed edge is first traced back to the modal evidence anchor point, and then the corresponding node is located in the equipment topology snapshot according to the installation location in the source event fragment. Adjacent topology nodes are limited to upstream or downstream connecting nodes that have a direct connection relationship with the corresponding node and a connection step size of 1. Replacement is not allowed to span more than 2 connection steps to ensure that the disturbance remains within a local range. The replacement of head or tail entities follows the principle of consistency of measurement object and the principle of consistency of entity type. The principle of consistency of measurement object means that if the original tail entity represents a pressure anomaly, the candidate node after replacement must also belong to the node that can bear the pressure measurement object; if the original head entity represents a first-level sealing pressure measuring point, the candidate node after replacement must also be a measuring point node, and cannot be replaced by an equipment node or maintenance action node.
[0052] The principle of entity type consistency means that equipment is replaced only with equipment, components are replaced only with components, measuring points are replaced only with measuring points, and abnormal phenomena are replaced only with abnormal phenomena. To prevent the generation of obviously unreasonable topology disturbance segments, a topology replacement constraint is also set: if the adjacent topology node differs from the original node by more than one level in terms of installation location hierarchy, or if the measurement object of the adjacent topology node is inconsistent with that of the original node, then the replacement is discarded.
[0053] For example, if the original directed edge is "Level 1 sealing pressure measurement point, indication, differential pressure fluctuation", and the upstream connected node of the modal evidence anchor point in the equipment topology snapshot is "Level 1 sealing inlet pipe section measurement point", and the downstream connected node is "Level 1 sealing outlet pipe section measurement point", and both are pressure measurement objects, then the original head entity can be replaced to generate two topology disturbance fragments respectively; if the adjacent node is "inlet temperature measurement point", since its measurement object is temperature, it does not meet the principle of measurement object consistency, so no corresponding topology disturbance fragment is generated.
[0054] In step S440, the version perturbation fragment and the topology perturbation fragment are rearranged and combined according to the chronological order of events to form a local counterfactual fragment set F. The chronological order of events refers to the temporal relationship determined by the start and end times of the source event fragments corresponding to each fragment. In specific implementation, firstly, taking the timestamp corresponding to the original directed edge as the center, other directed edges with direct temporal correlation within a 300-second range before and after it are selected to form a local temporal fragment set; the 300 seconds can be calibrated according to the abnormal propagation time of the device and adjusted within the range of 120 seconds to 900 seconds. Then, the version perturbation fragment generated in step S420 and the topology perturbation fragment generated in step S430 are respectively embedded into this local temporal fragment set, while keeping the temporal order of other directed edges that have not been replaced unchanged. If the original local time-series segment set contains "10:25 AM, Level 1 sealing pressure measurement point, indication, differential pressure fluctuation" and "10:28 AM, differential pressure fluctuation, resulting in Level 1 sealing alarm," then after the version disturbance segment is replaced, the former will still precede the latter in time; the same priority relationship will be maintained after the topology disturbance segment is replaced. During rearrangement and combination, only three types of cases are allowed: replacement of a single version identifier, replacement of a single adjacent topology node, and sequential combination of both. Replacing more than two adjacent topology nodes simultaneously within the same local time-series segment set is not allowed, nor is splicing across different time windows permitted, to avoid uncontrolled growth of the local counterfactual segment set F.
[0055] To control the number of local counterfactual fragments F, a maximum of 6 local counterfactual fragments are retained for each original directed edge. Priority is given to fragments with the smallest timestamp difference from the original directed edge, the most direct topological connection, and the shortest version replacement distance. Here, the shortest version replacement distance means a position difference of 1 in the tag chain, and the most direct topological connection means a connection step size of 1 and the smallest difference in installation location hierarchy. The upper limit of 6 fragments is determined by the differentiation effect between stable and erroneous relationships in historical samples and can be adjusted within the range of 3 to 10 fragments.
[0056] In this embodiment, each fragment in the local counterfactual fragment set F maintains the same data structure as the original directed edges, including at least a head entity, relation, tail entity, timestamp, version identifier, modal evidence anchor point, source event fragment number, and perturbation type. The perturbation type is explicitly marked as a version perturbation fragment, a topology perturbation fragment, or a sequential combination perturbation fragment.
[0057] In this embodiment, obtaining the anti-obfuscation score Z and the credible subgraph K includes steps S510 to S540, wherein step S520 further includes steps S521 to S524, and step S540 further includes steps S541 to S544. The purpose of this part is to utilize the difference between the original evidence and the local counterfactual evidence to identify candidate relationships that are stable only in the real context but no longer stable after version replacement, replacement of adjacent topological nodes, or perturbation of the order of events, thereby forming the credible subgraph K.
[0058] In step S510, based on the modal evidence anchor point, the probability of each candidate relation being established in the original fragment and the probability of being established in the local counterfactual fragment set F are extracted from the initial graph G, and the difference between the two is calculated as the relation establishment difference. Specifically, for each candidate relation in the initial graph G, the corresponding original event fragment is first traced back based on its modal evidence anchor point, and then all local counterfactual fragments that are from the same source as the candidate relation are retrieved from the local counterfactual fragment set F. "From the same source" means that the local counterfactual fragment has the same source event fragment number and the same modal evidence anchor point position as the original candidate relation; only version identifiers, adjacent topological nodes, or event order are allowed to be replaced.
[0059] The probability of a relationship being established is calculated using a unified scoring rule. The scoring input includes four items: semantic support, temporal consistency, topological consistency, and version consistency. Semantic support is defined as the degree to which the content pointed to by the modal evidence anchor directly supports the head entity, relation, and tail entity. When all three entities can be clearly located in the original content covered by the modal evidence anchor, the semantic support is recorded as 1; when only two of them can be located, it is recorded as 0.6; when only one of them can be located, it is recorded as 0.3; and when none of them can be located, it is recorded as 0. Time consistency is calculated based on the relationship type. For "cause" relationships, the timestamp of the cause event must be earlier than the timestamp of the result event, and the interval between them must not exceed 600 seconds, which is recorded as 1. If the interval is between 600 and 1800 seconds, it decreases linearly from 1 to 0. If the timestamps are in reverse order, it is recorded as 0. For "indicate" and "associate" relationships, the timestamps of the head entity and the tail entity must be within the same 300-second sliding time window, which is recorded as 1. If the interval exceeds 300 seconds but does not exceed 900 seconds, it decreases linearly, and if it exceeds 900 seconds, it is recorded as 0. For "installed at" and "belongs to" relationships, it is recorded as 1 as long as the timestamp falls within the effective time interval of the corresponding version identifier, otherwise it is recorded as 0. Topology consistency is calculated based on the device topology snapshot. When the head entity node and the tail entity node meet the allowed connection mode of the relationship type in the device topology snapshot, it is recorded as 1, otherwise it is recorded as 0.
[0060] The allowed connection modes are predefined by the relation type constraint table. For example, "Installed at" requires the head entity to be a component or measuring point and the tail entity to be a device or component, with a direct connection or a level 1 hierarchical relationship between the two. "Indication" requires the head entity to be a measuring point or alarm object and the tail entity to be a physical quantity status or abnormal phenomenon. Version consistency is calculated based on the matching of the version identifier and the timestamp. A timestamp within the effective time interval corresponding to the version identifier is recorded as 1, otherwise as 0. The above four items are multiplied by 0.4, 0.25, 0.2, and 0.15 respectively and then summed to obtain the probability of the relationship being established. The weight coefficients are determined by calibration using a manually labeled relation sample set. Preferably, no less than 2000 candidate relations are used as training calibration samples, and the combination that maximizes the accuracy of relation determination is selected through a grid search method. The semantic support weight can be in the range of 0.3 to 0.5, the time consistency weight can be in the range of 0.15 to 0.3, the topology consistency weight can be in the range of 0.1 to 0.3, and the version consistency weight can be in the range of 0.1 to 0.2.
[0061] For the probability of a relationship being valid in the local counterfactual fragment set F, the same scoring rule as for the original fragments is used. The probability of a relationship being valid is calculated for each of the local counterfactual fragments corresponding to the candidate relationship, and then the arithmetic mean of all the probability of a relationship being valid is taken to obtain the probability of a candidate relationship being valid in the local counterfactual fragment set F. If no corresponding local counterfactual fragment is found, the probability of a candidate relationship being valid in the local counterfactual fragment set F is recorded as 0.5 to maintain the neutrality of subsequent judgments. The relationship validity difference is defined as the probability of a relationship being valid in the original fragment minus the probability of a relationship being valid in the local counterfactual fragment set F; when this difference is less than 0, it is recorded as 0. The reason for using this rule is that a truly stable candidate relationship should have a high probability of being valid in the original fragment, but significantly decrease in the local counterfactual fragment set F due to version replacement, topological perturbation, or temporal perturbation. Taking "primary sealing pressure measuring point, indication, and differential pressure fluctuation" as an example, if the probability of its relationship being true in the original fragment is 0.91 and the average probability of its relationship being true in the local counterfactual fragment set F is 0.28, then the difference in the relationship being true is 0.63.
[0062] In step S520, based on the prohibited merging constraint set J and the device topology snapshot, the installation location conflict, measurement object conflict and tag chain breakage triggered by the candidate relationship in the local counterfactual fragment set F are statistically analyzed to obtain the constraint violation value.
[0063] In step S521, for each candidate relationship, all corresponding segments in the local counterfactual segment set F are located based on the modal evidence anchor point, and the installation location, measurement object, and tag chain information of the corresponding event segments are extracted. Here, all corresponding segments refer to all local counterfactual segments generated from the same original candidate relationship through steps S420, S430, and S440. For each local counterfactual segment, the installation location field, measurement object field, historical tag chain position, replacement tag chain position, and effective time interval corresponding to the version identifier of the event segment corresponding to its head and tail entities are read, and a segment-by-segment record table is formed.
[0064] In step S522, the installation location is compared with the node connection relationship in the equipment topology snapshot. If no direct connection or hierarchical relationship exists, it is marked as an installation location conflict. Simultaneously, the measurement object is matched with a predefined set of physical quantity categories; if they are inconsistent, it is marked as a measurement object conflict. Specifically, a direct connection in the equipment topology snapshot refers to a connection edge between two nodes; a hierarchical relationship refers to a parent-child containment relationship between two nodes, with a hierarchy depth difference not exceeding 1. If the installation location corresponding to the head entity and the installation location corresponding to the tail entity in a local counterfactual fragment have neither a direct connection nor a hierarchical relationship, it is marked as an installation location conflict. The predefined set of physical quantity categories includes eight categories: pressure, temperature, vibration, flow rate, liquid level, displacement, leakage status, and electrical status. These categories are established based on the equipment measurement point log, alarm type table, and process variable table. If the measurement objects in a local counterfactual fragment do not belong to the same physical quantity category, or do not meet the physical quantity requirements of the relationship type constraint table for that relationship, it is marked as a measurement object conflict.
[0065] In step S523, according to the continuity determination rules of the tag number chain, segments with overlapping times and no successive replacement relationship in the tag number chain are marked as tag number chain breaks, and the number of occurrences of various conflicts is counted. The tag number chain continuity determination rules are defined as follows: the tag numbers corresponding to two event segments are in the same historical tag number chain or the same replacement tag number chain, and the position difference of the tag number chain is not greater than 1, or there is a direct replacement record from the old tag number to the new tag number between the two, and the interval between adjacent effective time intervals is not greater than 24 hours. The 24 hours are determined based on the maintenance records and instrument switching records and can be adjusted within the range of 1 hour to 72 hours. Time overlap is determined by the overlap ratio, specifically the intersection duration of the time intervals of the two event segments divided by the duration of the shorter event segment; when this ratio is not less than 0.3, it is considered as time overlap. If two time-overlapping event segments are not continuous on the tag number chain and there is no successive replacement relationship, they are marked as tag number chain breaks.
[0066] In step S524, different weight coefficients are assigned to installation location conflicts, measurement object conflicts, and tag number chain breaks, respectively. The constraint violation value is calculated by summing the products of the number of conflicts and the corresponding weight coefficients. Preferably, the weight coefficients for installation location conflicts, measurement object conflicts, and tag number chain breaks are set to 0.3, 0.25, and 0.45, respectively. The above weight coefficients are set based on the degree of damage to the mis-extraction relationship caused by various errors in manually labeled error samples. Among them, tag number chain breaks usually indicate that the replacement relationship is not valid at all and have the greatest impact on the result, so they are given the highest weight. In specific calculation, the occurrence frequency of installation location conflicts, measurement object conflicts, and tag number chain breaks in all corresponding segments is counted first, and then multiplied by the corresponding weight coefficients. The three results are then summed and divided by the total number of all corresponding segments to obtain the constraint violation value in the range of 0 to 1; when the calculated result is greater than 1, it is recorded as 1. For example, if a candidate relation has four local counterfactual segments, including one conflict in the installation location, one conflict in the measurement object, and two breaks in the tag chain, the constraint violation value is calculated using the method of "1 multiplied by 0.3, 1 multiplied by 0.25, 2 multiplied by 0.45, added together, and then divided by 4", resulting in 0.3625. The weighting coefficient for the installation location conflict can be adjusted from 0.2 to 0.4, the weighting coefficient for the measurement object conflict can be adjusted from 0.15 to 0.35, and the weighting coefficient for the tag chain break can be adjusted from 0.35 to 0.55. The specific values are obtained through calibration of the discriminative ability on historical mis-sampled data.
[0067] In step S530, the relationship establishment difference and constraint violation value are combined and calculated according to preset weighting coefficients to obtain the anti-misconception score Z. To ensure that the anti-misconception score Z exhibits a monotonic relationship of "the larger the value, the more stable it is," a reverse stabilizing factor is first taken for the constraint violation value, i.e., 1 is subtracted from the constraint violation value to obtain the constraint satisfaction level. Then, the relationship establishment difference and constraint satisfaction level are combined according to weighting coefficients. Preferably, the weighting coefficient for the relationship establishment difference is set to 0.65, and the weighting coefficient for the constraint satisfaction level is set to 0.35. The anti-misconception score Z is calculated using the method of "0.65 multiplied by the relationship establishment difference, plus 0.35 multiplied by the constraint satisfaction level." These two weighting coefficients are calibrated and determined using the precision priority principle from historical labeled samples, where the weighting coefficient for the relationship establishment difference can be adjusted within the range of 0.5 to 0.8, and the weighting coefficient for the constraint satisfaction level can be adjusted within the range of 0.2 to 0.5. Taking the aforementioned "primary sealing pressure measurement point, indication, and differential pressure fluctuation" as an example, if the relationship is valid with a difference of 0.63 and the constraint violation value is 0.10, then the constraint satisfaction level is 0.90. The anti-confusion score Z is calculated using the method of "0.65 multiplied by 0.63, plus 0.35 multiplied by 0.90", resulting in 0.7245. This candidate relationship indicates that it is stably valid in real evidence and is not easily confused in the local counterfactual fragment set F.
[0068] In step S540, candidate relations with anti-aliasing scores Z greater than a preset threshold are selected, and a reliable subgraph K is constructed.
[0069] In step S541, the anti-confusion score Z of each candidate relation is normalized. The normalization process uses a linear mapping from minimum to maximum value. Within the current batch of candidate relations, the minimum and maximum values of all anti-confusion scores Z are calculated. Then, the minimum value is subtracted from the anti-confusion score Z of any candidate relation, and the result is divided by the difference between the maximum and minimum values, thus mapping to the interval 0 to 1. If the maximum value in the current batch equals the minimum value, it indicates that the anti-confusion score Z of each candidate relation has no discriminative power. In this case, the normalized anti-confusion score Z is uniformly recorded as 0.5 to avoid generating artificially high or low results.
[0070] In step S542, a joint screening index is constructed based on the normalized anti-obfuscation score Z, combined with the timestamp density and the number of modal evidence anchors corresponding to the candidate relationships. Timestamp density is defined as the number of times a candidate relationship with the same head entity, relationship, tail entity, and version identifier appears within a sliding time window of 1800 seconds, centered on the timestamp of the candidate relationship. After calculating the timestamp density of all candidate relationships in the current batch, a normalized timestamp density is obtained using the same minimum-to-maximum linear mapping method as in step S541. The number of modal evidence anchors is defined as the total number of different modal evidence anchors supporting the candidate relationship. The deduplication rule is that only one instance is counted if the modal type + original content index is the same. For example, multiple character intervals in the same speech text may form multiple modal evidence anchors, but they are only counted as one speech modal evidence anchor. The number of modal evidence anchors is also normalized using a minimum-to-maximum linear mapping method. The joint screening index is calculated using the following method: 0.75 multiplied by the normalized anti-confusion score Z, plus 0.15 multiplied by the normalized timestamp density, plus 0.10 multiplied by the normalized number of modal evidence anchors. The weighting coefficients for the normalized anti-confusion score Z, Z, Z, and Z can be adjusted from 0.6 to 0.85; the normalized timestamp density weighting coefficients can be adjusted from 0.05 to 0.2; and the normalized number of modal evidence anchors weighting coefficients can be adjusted from 0.05 to 0.15.
[0071] In step S543, the joint screening index is compared with a preset threshold to filter out candidate relations that meet the conditions. The preset threshold is determined based on the historical labeled dataset according to the principle of balancing precision and recall, with a preferred value of 0.72, which can be adjusted within the range of 0.6 to 0.9. The calibration method is as follows: in the historical labeled dataset, the precision, recall, and harmonic index of the stable relation identification results are calculated for different preset thresholds, and the threshold that maximizes the harmonic index and makes the precision not less than 0.8 is taken as the preset threshold. When the joint screening index of a candidate relation is greater than or equal to the preset threshold, it is determined to be a stable candidate relation; when it is less than the preset threshold, it is determined to be an unstable candidate relation and is removed.
[0072] In step S544, the filtered candidate relationships are aggregated according to timestamp order and device topology connection relationships to construct a reliable subgraph K. Specifically, all stable candidate relationships are first sorted from earliest to latest timestamp; when timestamps are the same, they are sorted from highest to lowest according to the normalized anti-aliasing score Z. Then, the head entity and tail entity are uniformly mapped according to device code, installation location, measurement object, and version identifier, and entity nodes with the same mapping result are merged into the same node. For multiple stable candidate relationships between the same pair of entity nodes, if the relationship type is the same and the timestamp difference is no more than 1800 seconds, the one with the highest joint screening index is retained as the main relationship edge, and the modal evidence anchor points of the remaining relationship edges are added to the evidence list of the main relationship edge; if the relationship types are different, they are retained separately to form multiple relationship edges. Finally, only the set of nodes and their relationship edges that contain at least one stable relationship edge and satisfy the direct connection or hierarchical reachability conditions in the device topology snapshot are retained to form the reliable subgraph K. Each relation edge in the trustworthy subgraph K retains a timestamp, version identifier, modal evidence anchor point, relation establishment difference, constraint violation value, anti-confusion score Z, and joint screening index for subsequent training updates and result traceability.
[0073] In this embodiment, step S600 is used to perform a self-boosting update on the large model using the credible subgraph K as positive samples and the denied relations in the local counterfactual fragment set F as difficult negative samples. The updated large model is then used to re-extract and complete the graph of unconfirmed fragments, outputting the target industrial equipment knowledge graph. Specifically, this includes steps S610 to S640.
[0074] In step S610, the relation edges in the trusted subgraph K, along with their modal evidence anchors, timestamps, and version identifiers, are extracted to construct a positive sample set. Specifically, all relation edges in the trusted subgraph K are traversed, and the head entity, relation, tail entity, modal evidence anchor, timestamp, version identifier, anti-confusion score Z, and joint screening index are read. Only relation edges with a joint screening index not less than a preset threshold and complete modal evidence anchors are retained.
[0075] Complete modal evidence anchor points mean that both the head entity and the tail entity can be traced back to the original content index, and have at least one of the following positioning forms: character range, target area coordinates, time window position, or alarm record number. For each retained relation edge, a supervisory sample record is constructed. The supervisory sample record includes at least the sample number, input context, target output, head entity position, tail entity position, relation trigger position, timestamp, version identifier, and sample weight. The input context consists of the original content corresponding to the modal evidence anchor point and its adjacent content. Preferably, event segments within 150 seconds are extracted on both sides of the time axis centered on the modal evidence anchor point and spliced in chronological order. For voice and text, an additional 128 characters are retained before and after the audio; for images, the target area coordinates and area description text are retained; for vibration, one additional time window feature is retained before and after the vibration; and for alarms, one additional alarm record before and after the alarm under the same device code is retained. The target output is fixed as "head entity, relation, tail entity, timestamp, version identifier, relation validity field," where the relation validity field of the positive sample record is marked as 1. The sample weights are calculated by multiplying the anti-confusion score Z by 0.6 and adding 0.4 by the joint screening index. The values of 0.6 and 0.4 are determined by calibration of historical labeled samples and can be adjusted within the ranges of 0.5 to 0.7 and 0.3 to 0.5, respectively.
[0076] In step S620, fragments that are identical to the head or tail entity of the positive sample set and whose relationships are negated are selected from the local counterfactual fragment set F to construct a hard-to-bear sample set. A relationship being negated means that the corresponding relationship does not enter the trustworthy subgraph K and satisfies at least one of the following: the probability of the relationship being true is less than 0.4, or the constraint violation value is greater than 0.5, or the anti-confusion score Z is less than a preset threshold minus 0.1. To ensure the deceptiveness of hard-to-bear samples, they must satisfy at least two of the following conditions with a positive sample record: identical head entity, identical tail entity, identical relationship type, timestamp interval not greater than 1800 seconds, and version identifier in an adjacent position in the same historical position chain or the same replacement position chain. For each positive sample record, a maximum of three hard-to-bear sample records are selected. The structure of the hard-to-bear sample records is identical to that of the positive sample records, but the relationship validity field is set to 0. The sample weight is calculated by multiplying the proximity normalization value by 0.5 and then multiplying it by the constraint violation value by 0.5. The proximity normalization value is obtained by accumulating the scores of the five matching results: head entity consistency, tail entity consistency, relation type consistency, timestamp interval not exceeding 300 seconds, and adjacent version identifiers, and then dividing by 5.
[0077] In step S630, the positive sample set and the hard negative sample set are input into the large model according to the evidence anchor point alignment method, enabling the large model to learn to distinguish between real relationships and perturbation relationships, and complete the self-boosting update. The evidence anchor point alignment method refers to explicitly inserting markers such as "head entity start", "head entity end", "tail entity start", "tail entity end", "relationship trigger start", and "relationship trigger end" into the input context, so that the large model can simultaneously learn the relationship results and the positions of the modal evidence anchor points that they must depend on.
[0078] The large model's self-boosting update employs a joint training approach of supervised fine-tuning and preference constraints. Supervised fine-tuning learns the structured output, while preference constraints make the large model more inclined to output positive sample records rather than hard-negative sample records under similar inputs. During training, the positive sample set and the hard-negative sample set are mixed in a 1:2 ratio, the preferred batch size is 16 to 64, the preferred learning rate is 0.000005 to 0.00005, and the preferred number of training epochs is 2 to 5. The supervised fine-tuning loss consists of the structured field prediction loss and the relation validity field determination loss, combined in the manner of "0.7 multiplied by the structured field prediction loss, plus 0.3 multiplied by the relation validity field determination loss". The preference constraints require that the average generation probability of the target output for a positive sample record is at least 0.1 higher than that of the corresponding hard-negative sample record; if this is not met, the model continues to participate in subsequent training.
[0079] To prevent the original extraction capabilities from being forgotten, at least 20% additional basic extracted samples are added to each training batch. After training, if the updated large model's relation recognition accuracy on the validation set is no less than 1.05 times that of the original large model, and the erroneous relation retention rate is no more than 0.85 times that of the original large model, then the update is considered valid.
[0080] In step S640, the updated large model is used to re-extract entity relations from the unconfirmed fragments and merge them with the trusted subgraph K to output the target industrial equipment knowledge graph. Unconfirmed fragments refer to event fragments that were not written into the trusted subgraph K during the aforementioned extraction process. Preferably, unconfirmed fragments satisfying at least one of the following conditions are selected: they have the same device code as entity nodes in the trusted subgraph K, are adjacent nodes in the same device topology snapshot, have a timestamp interval of no more than 3600 seconds with the relation edge in the trusted subgraph K, or are in the same historical index chain or the same replacement index chain as the version identifier in the trusted subgraph K. A constraint input sequence is constructed for the unconfirmed fragments in the manner described in step S310, and the updated large model is input to re-extract entity relations. Steps S510 to S540 are repeated on the re-extraction results to recalculate the relation establishment difference, constraint violation value, anti-confusion score Z, and joint screening index. Only results with a joint screening index not less than a preset threshold and that do not violate the prohibited merging constraint set J are included in the fusion scope. If the re-extraction result has the same head entity, relation, tail entity, and version identifier as the existing relation edge in the credible subgraph K, then the one with the higher joint screening index is retained as the main relation edge, and the modal evidence anchor point of the other one is merged into the main relation edge; if the version identifiers are different but are located in consecutive positions in the same historical position chain and replacement position chain, then they are retained as relation edges under different versions respectively.
[0081] Finally, the newly added stable relationship edges are fused with the trusted subgraph K according to the timestamp order and the device topology connection relationship to output the target industrial equipment knowledge graph. Each relationship edge in this target industrial equipment knowledge graph retains a timestamp, version identifier, modal evidence anchor point, anti-obfuscation score Z, and joint screening index, thereby achieving traceability of results and continuous self-improvement.
[0082] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A method for constructing a self-enhancing multimodal industrial equipment knowledge graph assisted by a large model, characterized by: include: S100: Acquire the voice, image, vibration, alarm, work order and maintenance records of the equipment during the target time period, and extract the equipment topology snapshot and tag number evolution information corresponding to the target time period to obtain the multimodal event fragment set M; S200, based on the equipment topology snapshot and tag evolution information, generate a candidate set H for identical objects and a set of constraints J for prohibiting merging for M, including: S210, based on the equipment topology snapshot, extract the installation location, upstream connection node, downstream connection node, and measurement object for each event segment, and construct a segment topology fingerprint; S220, based on the tag evolution information, map each event segment to a corresponding historical tag chain and a replacement tag chain; S230, pair event segments with consistent topology fingerprints and whose historical tag chains and replacement tag chains meet the time connection condition, and generate a candidate set H for identical objects; S240, apply a prohibition-merging mark to event segments with conflicting installation locations, conflicting measurement objects, or overlapping times and broken tag chains, and generate a set of constraints J for prohibiting merging. S300: Input M, H and J into the large model and extract the entity relationship candidate set C with modal evidence anchors, timestamps and version identifiers; S400, construct an initial graph G based on C, and replace the version identifier, adjacent topology nodes, or event sequence around the evidence anchor point to generate a local counterfactual fragment set F, including: S420, locate the corresponding event fragment with the modal evidence anchor point, replace its version identifier to generate a version perturbation fragment, and keep the other fields unchanged; S430, based on the device topology snapshot, select the upstream or downstream connection node of the node corresponding to the modal evidence anchor point in the initial graph G, replace the head entity or tail entity, and generate a topology perturbation fragment; S440, rearrange and combine the version perturbation fragment and the topology perturbation fragment according to the event sequence to form a local counterfactual fragment set F; S500: Calculate the relationship establishment difference and constraint violation value of G on the original fragment and F respectively to obtain the anti-confusion score Z of each candidate relationship, and retain stable relationships whose scores meet the threshold to form a credible subgraph K. This includes: S510: Based on modal evidence anchor points, extract the relationship establishment probability of each candidate relationship in the original fragment and the relationship establishment probability in the local counterfactual fragment set F from the initial graph G, and calculate the difference between the two as the relationship establishment difference; S520: Based on the prohibited merging constraint set J and the equipment topology snapshot, statistically analyze the installation part conflict, measurement object conflict and tag chain breakage triggered by the candidate relationship in the local counterfactual fragment set F to obtain the constraint violation value; S530: Combine the relationship establishment difference and constraint violation value according to the preset weight coefficient to obtain the anti-confusion score Z; S540: Filter candidate relationships with anti-confusion scores Z greater than the preset threshold to construct the credible subgraph K. S600: Using K as the positive sample, constructing a difficult-to-bear sample based on the segments in F whose relations are negated, performing self-boosting updates on the large model, and using the updated large model to re-extract and complete the graph of unconfirmed segments, outputting the target industrial equipment knowledge graph, including: S610: extracting relation edges and their modal evidence anchors, timestamps, and version identifiers from the credible subgraph K to construct a positive sample set; S620: selecting segments from the local counterfactual segment set F that are consistent with the head or tail entities of the positive sample set and whose relations are negated, constructing a difficult-to-bear sample set; S630: inputting the positive sample set and the difficult-to-bear sample set into the large model according to the evidence anchor alignment method, enabling the large model to learn to distinguish between real relations and perturbation relations, completing the self-boosting update; S640: using the updated large model to re-extract entity relations from unconfirmed segments and merge them with the credible subgraph K to output the target industrial equipment knowledge graph.
2. The method for constructing a large-model-assisted self-lifting multimodal industrial equipment knowledge graph according to claim 1, characterized in that: The extraction of the entity relation candidate set C with modal evidence anchors, timestamps, and version identifiers includes: S310, binding each event fragment in the multimodal event fragment set M with the same object candidate set H and the prohibited merging constraint set J in chronological order, and constructing a constraint input sequence containing the original content index, fragment topological fingerprint, and position of the tag chain; S320, inputting the constraint input sequence into the large model, and inserting modal boundary markers for voice, image, vibration, alarm, and text respectively, generating modal evidence anchors corresponding to the original content index; S330, extracting entity relation triples and corresponding timestamps based on the modal evidence anchors and event fragment time intervals, and assigning version identifiers based on the historical tag chain and the replacement tag chain; S340, removing entity relation triples that violate the prohibited merging constraint set J, and writing the remaining results into the entity relation candidate set C.
3. The method for constructing a self-lifting multimodal industrial equipment knowledge graph with large model assistance according to claim 1, characterized in that: The initial graph G is constructed based on C, including: S410, constructing directed edges with timestamps and version identifiers according to the head entity, relation and tail entity in the entity relation candidate set C, and using modal evidence anchor points as edge attributes to form the initial graph G.
4. The method for constructing a large-model-assisted self-lifting multimodal industrial equipment knowledge graph according to claim 1, characterized in that: Step S520 further includes: S521, for each candidate relationship, locating all corresponding segments in the local counterfactual segment set F based on modal evidence anchor points, and extracting the installation location, measurement object, and tag chain information of the corresponding event segment; S522, comparing the installation location with the node connection relationship in the device topology snapshot, marking it as an installation location conflict when there is no direct connection or hierarchical relationship, and matching the measurement object with the predefined physical quantity category set, marking it as a measurement object conflict when they are inconsistent; S523, according to the tag chain continuity determination rule, marking segments with overlapping time and no sequential replacement relationship of tag chains as tag chain breaks, and counting the number of occurrences of each type of conflict; S524, assigning different weight coefficients to installation location conflicts, measurement object conflicts, and tag chain breaks respectively, and calculating the constraint violation value by summing the product of the number of conflicts and the corresponding weight coefficient.
5. The method for constructing a large-model-assisted self-lifting multimodal industrial equipment knowledge graph according to claim 1, characterized in that: Step S540 further includes: S541, normalizing the anti-obfuscation score Z of each candidate relationship; S542, constructing a joint screening index based on the normalized anti-obfuscation score Z, combined with the timestamp density and modal evidence anchor point number corresponding to the candidate relationship; S543, comparing the joint screening index with a preset threshold to screen out candidate relationships that meet the conditions; S544, aggregating the screened candidate relationships according to the timestamp order and device topology connection relationship to construct a reliable subgraph K.
Citation Information
Patent Citations
Intelligent media asset multi-modal knowledge graph management method based on large model
CN119294506A
False information multi-source association reasoning system based on knowledge graph
CN121352022A
