Alumina process event map construction method based on multi-modal semantic parsing

Abstract

The present application relates to a kind of alumina process event atlas construction method based on multimodal semantic parsing, belong to knowledge atlas and industrial process information processing technical field.The method includes: obtaining at least two modal alumina process data and preprocessing;Each modal data semantic parsing generates candidate process event;Cross-modal candidate event matching score is calculated, by text semantic similarity, time interval overlap, topological position similarity and process parameter similarity are weighted and synthesized, the candidate event clustering is generated process event node to the matching score not less than threshold value;Under time constraint and adjacency constraint, generate candidate timing edge, according to evidence score, select by directed ring detection to constitute directed acyclic graph;Conflict node is created to multi-source conflict data, is clustered into evidence cluster according to evidence root identification, and same cluster multiple evidences are counted according to single piece;Along the confidence propagation of timing directed edge, the product of the confidence probability of each path is supplemented to the convergence of multiple evidence clusters.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph and industrial process information processing technology, and relates to a method for constructing an alumina process event graph based on multimodal semantic parsing. Background Technology

[0002] Alumina is the main raw material for electrolytic aluminum production, and its production process involves complex physicochemical reactions and multiple continuous operations. Taking the Bayer process for alumina production as an example, its typical flow includes bauxite crushing, wet milling and pulping, high-temperature and high-pressure leaching, red mud sedimentation and separation, seed crystal decomposition, aluminum hydroxide washing, and high-temperature calcination, ultimately obtaining the finished alumina. Among these, the high-temperature and high-pressure leaching process involves several key process parameters such as temperature, pressure, and alkali concentration. In the calcination process, aluminum hydroxide undergoes a phase transition to alumina, generating the transition phase γ- at different temperature ranges. and stable phase α- The phase type directly determines product performance. The complete thermal history of heating, holding, and cooling during the calcination process has a decisive impact on the final phase composition and product performance. With the development of industrial information technology, the use of knowledge graph technology for the structured organization, relational querying, and inference application of alumina process knowledge has become an important technical direction for improving process management and product quality control.

[0003] Currently, mainstream materials databases both domestically and internationally primarily employ a binary recording model of "material-property" or "material-phase stability." Taking alumina as an example, the same alumina product, calcined at 800℃ and calcined at 1100℃, yields γ-phase with a specific surface area not exceeding 200 m² / g, respectively. and α- with a specific surface area of ​​approximately 10 m² / g Significant differences in performance exist. However, existing databases typically only record the specific surface area of ​​alumina in general terms, without relating it to specific process conditions such as calcination temperature, holding time, and cooling rate, nor distinguishing the corresponding phase types. This results in the loss of information regarding the relationship between process conditions and product performance during the knowledge representation stage. While existing process knowledge graphs have incorporated entities such as processes, parameters, and equipment into their nodes, their relationships are mostly limited to the static hierarchy of processes and parameters. They have not yet established a complete path of association from raw materials through the process to the phase and finally to the final performance. The relationship between process conditions and product performance cannot be structurally expressed or computationally traceable.

[0004] Alumina production is a multi-stage, continuous operation with strict sequential and interdependent relationships between the stages. The temperature and pressure conditions in the leaching stage directly affect the red mud settling and separation efficiency, thus influencing seed crystal decomposition and the phase composition and properties of the final calcined product. In the calcination stage, the heating rate, holding temperature and time, and cooling rate constitute a complete thermal history. Even at the same maximum temperature, different heating and cooling paths result in significantly different phases and properties. Existing knowledge graphs typically express inter-stage relationships using only single sequence numbers or causal networks, failing to represent the temporal dependencies where one stage must precede another, or the cross-stage relationships where parameter values ​​in one stage affect the phase products of subsequent stages. Furthermore, simply arranging stages based on chronological order generates numerous meaningless associations detached from actual process conditions, and the temporal dependencies between stages lack a computationally verifiable, structured representation mechanism.

[0005] Data for alumina processes comes from diverse sources, including high-frequency continuous time-series data collected by distributed control systems (DCS), high-precision but sparse laboratory test data, process manuals and academic literature primarily providing qualitative descriptions and recommended values, and equipment operation logs. Data from different sources generally differ in accuracy, sampling frequency, and numerical values. For example, regarding leaching temperature, DCS time-series data shows an average of 250℃, the process manual specifies 250℃, and a certain optimization experiment reported 260℃. Traditional data processing methods, such as overwriting older data with the latest data, arithmetic averaging, or deleting so-called outliers, destroy the distribution characteristics, measurement conditions, and uncertainty information of the original data. Users cannot trace the data source or determine the confidence level of data from different sources. Furthermore, when data from the same source repeatedly enters reliability assessments through multiple presentation paths, such as original records and secondary analysis reports, duplicate counting can lead to an overestimation of the assessment results. Similarly, when different sources have citation or data reuse relationships, treating them as independent sources without distinction will also distort the reliability assessment.

[0006] In summary, existing technologies lack effective mechanisms for retaining conflicting data and assessing confidence levels from multiple sources, making it difficult to meet the reliability management requirements of multi-source process data. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method for constructing an alumina process event graph based on multimodal semantic parsing, which aims to solve the problems of: the inability to structurally trace the relationship between process conditions and product performance in existing alumina process knowledge representations, the inability to computably verify the temporal dependencies of multiple processes, and the lack of objective confidence assessment for multi-source conflicting data.

[0008] The method includes the following steps: Step 1: Acquire alumina process data in at least two different modes and preprocess them. The modes include text data, image data, time-series data, and structured data. Step 2: Perform semantic parsing on each modal data after preprocessing, corresponding to its modal type, to generate a set of candidate process events for each modality; When text data is included, alumina process entities and sequential relationships are identified from the text; when image data is included, process nodes and connection topology are identified from the image; when time-series data is included, candidate process events, their start time, end time, and process parameters are extracted through change point detection. Step 3: Calculate the matching score between cross-modal candidate events. The matching score is a weighted synthesis of the computable similarity items among text semantic similarity, time interval overlap, topological location similarity, and process parameter similarity. When a certain similarity item is not computable, that similarity item is not included in the calculation and its weight is normalized among the remaining computable similarity items. Construct a matching graph with candidate events as vertices and matching scores not lower than a preset threshold as edges. Perform consistency clustering on the matching graph such that the matching scores of any two candidate events within a cluster are not lower than the preset threshold. Each cluster generates a process event node. The process event node includes process type, start time, end time, process parameter set, multimodal evidence reference list, and attribute-level confidence. Step 4: Determine the time interval relationship between nodes based on the start and end times of each process event node; the generation of candidate time sequence edges simultaneously satisfies time constraints and adjacency constraints. The time constraint is that the end time of the previous event is earlier than the start time of the next event. The adjacency constraint is that the two events satisfy at least one of the following: direct connection in the flowchart, direct sequential indication in the text, direct material transfer relationship, equipment connection relationship, or adjacent process relationship in the alumina field. For each candidate temporal edge that meets the conditions, a multimodal evidence score is calculated. Candidate temporal edges are selected one by one according to the multimodal evidence score. After each selection, directed cycle detection is performed to obtain a set of target temporal directed edges that satisfy the acyclic constraint. Together with the process event nodes, they form a directed acyclic graph. The acyclic constraint only applies to the main process event temporal edges within the same production batch or the same material instance. Circular logistics relationships are stored separately as circular logistics edges. Step 5: Map the phase entities corresponding to the alumina material after processing through the time-series process path shown in the directed acyclic graph to the process event nodes, and map the phase entities to the corresponding performance entities to form the "process event path-phase-performance" association link. Step Six: When there are multiple conflicting data from different sources for the same process, the same process parameter, or the same phase, create a conflict node. Each conflicting data item and its corresponding evidence root identifier, source modality, and initial confidence level are stored in parallel under the conflict node. The conflicting items are clustered into evidence clusters based on the evidence root identifier. Multiple pieces of evidence within the same evidence cluster are counted as a single piece of evidence during confidence propagation. The confidence level is propagated along the directed edges of the time sequence; the overall confidence level of a single path is the minimum value of the node attribute confidence level and the edge confidence level participating in the current propagation on that path; when there are multiple independent paths of different evidence clusters pointing to the same node, the final overall confidence level is 1 minus the product of the confidence levels of each path.

[0009] Furthermore, the matching score S in step three is calculated using the following formula:

[0010] in, For text semantic similarity, The degree of overlap between time intervals. For topological location similarity, Similarity of process parameters; , , , These are indicators of whether each similarity item can be calculated; a value of 1 indicates that it can be calculated, and a value of 0 indicates that the similarity item cannot be calculated. , , , Preset weighting coefficients and satisfying When the sum of the weights of all similarity items is zero, the matching score of the candidate event pair is not calculated and the candidate event pair is determined to be unmergeable.

[0011] Furthermore, the consistent clustering in step three adopts a fully linked agglomerative clustering strategy: initially, each candidate event is treated as a separate cluster, and two clusters are merged successively. Merging is only performed when the matching scores of any two candidate events in the merged new cluster are not lower than the preset threshold; clustering is terminated when there are no mergeable cluster pairs that meet the above conditions.

[0012] Furthermore, the multimodal evidence score E of the candidate temporal edges in step four is calculated using the following formula:

[0013] in, This is the time interval constraint indicator value. This refers to the topological constraint indicator value of the flowchart. This is a text order constraint indicator value. This is the domain knowledge constraint indicator value. Each indicator function takes a value of 1 when it satisfies the corresponding constraint, and a value of 0 otherwise. , , , For the corresponding weight coefficients and satisfying .

[0014] Furthermore, in step six, the evidence root identifier is a unique code of the original data source, including at least one of the following: document digital object identifier, distributed control system channel identifier, and standard manual number; no weighted average or selective coverage is performed between different conflict items.

[0015] Furthermore, in step six, for evidence clusters with different root identifiers but with citation relationships or data reuse relationships, the correlation coefficient between evidence clusters is calculated. Only when the correlation coefficient is less than a preset correlation threshold, the two evidence clusters participate as independent clusters in the convergence and propagation in step six.

[0016] Furthermore, in step five, the mapping between the physical phase and the process event node is based on the following rule: when the calcination temperature is within the first temperature range, a point pointing to γ- is generated. Candidate phase assertion; when the calcination temperature is in the second temperature range, α-- is generated. Candidate phase assertion: When the calcination temperature is in the third temperature range and the cooling rate is significantly higher than the conventional cooling rate, α--- is generated. With γ- Candidate phase assertions for mixed phases; wherein the first temperature range is lower than the second temperature range, and the rules simultaneously record their applicability, source literature, and confidence level; the phase entities include α- γ- δ- ,θ- At least one of gibbsite and sodium aluminate; the properties include at least one of purity, true density, specific surface area, pore volume, pore size distribution, thermal stability and thermal shock resistance.

[0017] Furthermore, it also includes a dynamic incremental update step: Steps two through six are executed on the newly added data to generate new candidate process events, candidate edges, and candidate knowledge assertions; if the newly added process event node belongs to an existing process chain, the corresponding temporal position is inserted according to its time interval, and the edge selection in step four is re-executed only for the newly added node and its neighboring candidate edges; if the newly added data and the existing data in the conflicting node belong to the same conflict group, it is added as a new conflict item, and step six is ​​executed on the evidence clusters and propagation paths affected by the newly added data; the entire graph does not need to be reconstructed as a whole.

[0018] Furthermore, the attribute-level confidence in step three includes event existence confidence, type confidence, time interval confidence, and process parameter confidence, which are calculated independently based on the evidence sources supporting each attribute.

[0019] This invention also provides a system for constructing an alumina process event map based on multimodal semantic parsing, comprising: The data acquisition module is used to acquire and preprocess alumina process data in at least two different modes, including text data, image data, time-series data, and structured data. The semantic parsing module is used to perform semantic parsing on each modal data after preprocessing, corresponding to its modal type, and generate a set of candidate process events for each modality. The event fusion module is used to calculate the matching score between cross-modal candidate events. The matching score is a weighted synthesis of computable similarity items among text semantic similarity, time interval overlap, topological position similarity and process parameter similarity. A matching graph is constructed with candidate events as vertices and matching scores not lower than a preset threshold as edges. Consistent clustering is performed on the matching graph, and each cluster generates a process event node. The graph construction module is used to determine time constraints based on the start and end times of each process event node and generate candidate time-series edges by combining them with adjacency constraints. It calculates the multimodal evidence scores of the candidate time-series edges and selects them one by one. After directed cycle detection, it obtains a set of target time-series directed edges that satisfy the acyclic constraints, which together with the process event nodes constitute a directed acyclic graph. The knowledge chain generation module is used to map the phase entities corresponding to the alumina material after processing through the time-series process path shown in the directed acyclic graph to the process event nodes, and to map the phase entities to the corresponding performance entities, forming an association link of "process event path-phase-performance". The conflict resolution module is used to create conflict nodes when there are multiple conflicting data from different sources for the same process, the same process parameter, or the same phase, and to cluster conflicting items into evidence clusters based on evidence root identifiers. The confidence propagation module is used to propagate confidence along the time-series directed edges and converge them when there are multiple independent paths of different evidence clusters pointing to the same node.

[0020] Compared with the prior art, the present invention has the following advantages: This invention constructs a four-layer knowledge association structure of "material-process event sequence-phase-performance" by hierarchically organizing process event nodes, phase nodes, and performance nodes. This solves the problem of missing information on process conditions and phase evolution in the traditional binary model of "material-performance", and enables the structured expression and computable traceability of the association path between process conditions and product performance.

[0021] This invention overcomes the deficiency of incomplete information in single-modal data by establishing a unified cross-modal matching scoring formula for text, images, time series and structured data and using effective modal normalization processing to generate process event nodes by weighted fusion of evidence from each modality. This improves the recall and accuracy of process event identification.

[0022] This invention generates candidate temporal edges through the combined effect of time interval constraints and adjacency constraints, and selects the set of temporal edges using a directed acyclic graph as a global constraint. This enables the temporal dependencies between alumina processes to be expressed in a calculable and verifiable structure, while avoiding meaningless associations that are detached from the actual process based solely on the order of time.

[0023] This invention enables conflicting data from different sources to coexist independently through a conflict node mechanism, ending the traditional information loss model of overwriting, averaging, and deletion. By identifying evidence roots and merging evidence clusters, it eliminates the problem of overestimation caused by repeated counting of the same or highly related sources during confidence propagation, thus achieving a more objective quantitative assessment of knowledge reliability.

[0024] This invention reduces the long-term maintenance cost of knowledge graphs by resolving the local directed acyclic graph and retransmitting the local confidence scores, eliminating the need for overall graph reconstruction when new data is added.

[0025] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the overall method of the present invention; Figure 2 This is a schematic diagram illustrating candidate event matching and cluster fusion. Figure 3 This is a schematic diagram of the directed acyclic graph of the Bayer alumina process timing in Example 2; Figure 4 This is a schematic diagram of evidence cluster merging and confidence propagation in Example 3. Detailed Implementation

[0027] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] Example 1 The following is combined Figures 1 to 4 Taking the Bayer process for alumina production as an example, the specific embodiments of the present invention will be described in detail. Figure 1 As shown, this invention provides a method for constructing an alumina process event graph based on multimodal semantic parsing, including steps one to seven, and finally outputting an enhanced knowledge graph.

[0029] Step 1: Multimodal data acquisition and preprocessing.

[0030] Acquire alumina process data in at least two different modalities, including text data, image data, time-series data, and structured data.

[0031] As one implementation method, the acquired data includes at least one of time-series data and text data or image data to ensure that subsequent cross-modal fusion has a temporal and semantic reference.

[0032] For text data, OCR text recognition is performed on scanned documents, followed by text cleaning, paragraph segmentation, and entity boundary detection. For image data, image format and resolution are standardized, and image enhancement and contrast adjustment are performed. For time-series data, statistical methods are used to detect and correct outliers, missing values ​​are linearly imputed, and data from different sampling frequencies are resampled to the same time base. For structured data, field formats from various sources are standardized, data cleaning and field mapping are performed, and missing values ​​are handled.

[0033] Step 2: Multimodal semantic parsing and candidate event generation.

[0034] Semantic parsing corresponding to the modality type is performed on each of the preprocessed modal data.

[0035] Text semantic parsing: When the input data contains text, named entity recognition technology is used to identify alumina process-related entities from the text, including material entities such as bauxite, aluminum hydroxide, and alumina, as well as process entities such as leaching, sedimentation, decomposition, and calcination, equipment entities such as rotary kilns, ball mills, and autoclaves, parameter entities such as temperature, pressure, concentration, and time, and phase entities such as α- and γ-phase entities. Simultaneously, semantic relationships between entities are identified, including upstream / downstream relationships, parameter membership relationships, and sequence indication relationships.

[0036] Image semantic parsing: When the input data includes images, computer vision techniques are used to extract structured information from alumina process images. For flowcharts, object detection is used to identify process nodes and material flow arrows, extracting the spatial location, type labels, and directed connection topology between process nodes. For equipment images, image classification and object detection are used to identify equipment type and appearance features. For microscopic images, image segmentation is used to identify phase morphology and phase distribution characteristics.

[0037] Semantic parsing of time-series data: When the input data contains time-series data, candidate process events are extracted from the time-series data of sensors in the distributed control system. A change point detection algorithm is used to identify the moments of change in process states, including the start and end times of heating, cooling, isothermal control, pressurization and depressurization, and feeding and discharging operations. For each detected event interval, the start time, end time, event type, and statistical characteristics within the interval are extracted, including extreme temperatures, extreme pressures, rate of change, and steady-state duration. After this processing, each operation interval detected from the time-series data constitutes a candidate process event.

[0038] Structured data parsing: When the input data contains structured data, process identifiers, timestamps, process parameter records and performance indicators are extracted from production reports, quality inspection reports and equipment operation logs to form structured candidate events.

[0039] Preliminary aggregation of multimodal candidate events: The process entities extracted from the text, the process nodes detected in the flowchart, the operation intervals detected in the time series data, and the records extracted from the structured data are used as the sources of various candidate process events, each carrying its modal label and original evidence reference, and enter the cross-modal fusion in step three.

[0040] Step 3: Cross-modal evidence fusion and process event node generation.

[0041] For any two candidate events from different modalities, calculate their matching score S, indicating they belong to the same objective process event. This score is determined by text semantic similarity. Time interval overlap Topological location similarity Similarity with process parameters The similarity is weighted and synthesized across four dimensions, and calculated using effective modality normalization:

[0042] in, For text semantic similarity, cosine similarity is calculated based on event name, process type label, and context field vector; The overlap of time intervals is the ratio of the length of the intersection of the intervals to the length of the union of the intervals when both events have timestamps. Topological position similarity is calculated based on upstream and downstream adjacency relationships in the flowchart; The similarity of process parameters is calculated based on the relative deviation of the values ​​of key parameters such as temperature and pressure. , , , These are indicators of whether each similarity item can be calculated; a value of 1 indicates that it can be calculated, and a value of 0 indicates that the similarity item cannot be calculated. , , , For the preset weighting coefficients, satisfy .

[0043] When a certain similarity term cannot be calculated, the corresponding numerator and denominator terms are not included in the calculation, and the weights of the remaining valid modes are automatically normalized in the denominator. When the sum of the weights of all similarity terms is zero, the matching score of the candidate event pair is not calculated, and the candidate event pair is determined to be unmergeable.

[0044] like Figure 2 As shown, a candidate event matching graph is constructed, with each candidate event as a vertex and edges connecting fusionable candidate event pairs with a matching score S not lower than a preset threshold.

[0045] Consistent clustering is performed on the matching graph. As one implementation method, a fully linked agglomerative clustering strategy is adopted: initially, each candidate event is treated as a separate cluster. Two clusters are merged successively, and merging is only performed when the matching score S of any two candidate events in the merged new cluster is not lower than a preset threshold. Clustering is terminated when there are no mergeable cluster pairs that meet the above conditions. Each final cluster generates a process event node.

[0046] The merged node inherits information from all sources: the process type is the type label with the highest confidence among the sources; the time interval is calculated by weighted averaging to ensure that the weighted average time value falls at the weighted center of the time values ​​of each source, while retaining the original time value and interval uncertainty of each source.

[0047] Each process event node includes at least a unique identifier, process type, start and end times, a set of process parameters including temperature, pressure, concentration, and duration, a multimodal evidence citation list including literature citations, image file identifiers, distributed control system channel numbers and time periods, structured record numbers, source modality tags, and attribute-level confidence scores. The attribute-level confidence scores include event existence confidence, type confidence, time interval confidence, and process parameter confidence, calculated independently based on the evidence sources supporting each attribute. The initial value of each attribute's confidence score is determined according to the authority level of its evidence source.

[0048] Step 4: Generation of temporal directed edges and construction of directed acyclic graph.

[0049] Let the set of all process event nodes generated in step three be the vertex set. For any two process event nodes in the vertex set, the generation of candidate temporal edges must simultaneously satisfy both time constraints and adjacency constraints.

[0050] The time constraint is: the end time of the previous node is earlier than the start time of the next node.

[0051] The adjacency constraint is: two nodes must satisfy at least one of the following conditions: First, there is a directed arrow connection from the previous node to the next node in the flowchart; Second, there is a sequence indicator in the process description text; Third, there is a direct material transfer relationship or equipment connection relationship between the two nodes; Fourth, according to the inherent process sequence of the Bayer process, such as crushing, grinding, leaching, sedimentation, decomposition, washing, and calcination, the two nodes conform to the adjacent process relationship.

[0052] A candidate temporal edge is generated between two nodes and added to the edge set only if both nodes satisfy at least one of the above time constraints and adjacency constraints.

[0053] For each candidate edge, calculate its multimodal evidence score E:

[0054] in, This is the time interval constraint indicator value. This refers to the topology constraint indicator value of the flowchart. This is a text order constraint indicator value. This is the domain knowledge constraint indicator value. Each indicator function takes a value of 1 when it satisfies the corresponding constraint, and a value of 0 otherwise. , , , The corresponding weight coefficients and satisfying .

[0055] Based on directed graphs, this invention selects candidate edges according to the multimodal evidence score E of each candidate temporal edge under the global constraint of irreversible main process sequence within the same production batch of alumina, thereby obtaining a set of target temporal directed edges that satisfy acyclic constraints. Specifically, the selection method is as follows: candidate edges are processed sequentially in descending order of evidence score E. If a candidate edge does not generate a directed cycle after being added, it is retained; if a directed cycle is generated, the edge is discarded and marked as a temporal conflict. The acyclic constraint only applies to the temporal edges of main process events within the same production batch or the same material instance; cyclic logistics relationships are stored separately as cyclic logistics edges.

[0056] like Figure 3 As shown, after the above processing, the final output diagram is a directed acyclic graph, which fully represents the time sequence process path in the main alumina production process.

[0057] Step 5: Generation of a four-layer knowledge chain: "materials-process-phase-performance".

[0058] In the directed acyclic graph output in step four, the process event nodes are arranged in the order determined by the temporal directed edges, fully representing the process path experienced by the alumina material and the process parameters carried by each event node. Parameter information of key event nodes is extracted along the temporal process path, establishing a mapping between process events and phases, and between phases and properties. This expands the directed acyclic graph from a pure temporal process path into a complete knowledge graph encompassing phase evolution and performance indicators.

[0059] Alumina Process - Phase Mapping Rules: Traverse the process event nodes in the directed acyclic graph. When a node of type calcination is detected, extract the calcination temperature, duration, and cooling rate from the process parameter set of that node to trigger the phase mapping rule. Each rule has the following form: If the calcination temperature is within a specified range, the duration meets a given condition, and the cooling rate meets a given condition, then generate a candidate phase assertion pointing to the corresponding phase, and establish a directed edge from the calcination event node to the phase node in the graph.

[0060] As one implementation method, an example rule includes: if the calcination temperature is within a first temperature range, then generating a γ-... Candidate assertions; if the calcination temperature is within the second temperature range and the duration meets the condition, then a statement pointing to α- is generated. Candidate assertions; if the calcination temperature is in the third temperature range and the cooling rate is significantly higher than the conventional cooling rate, then α-pointing to the formation of the product is generated. With γ- Candidate assertions for the mixed phase. The first temperature range is lower than the second temperature range. Each rule also records its applicable conditions, source literature, and confidence level. The specific thresholds for the above temperature ranges and cooling rates can be adjusted according to actual operating conditions; this invention is not limited to the specific values ​​listed.

[0061] Phase-performance mapping rule: For generated phase nodes, the built-in phase-performance association rule is used to map the phase nodes to the corresponding performance nodes, and directed edges from phase nodes to performance nodes are established in the graph. As one implementation method, α- Related to purity and true density, γ- This is related to specific surface area, pore volume, and pore size distribution. The above performance thresholds are reference ranges based on typical alumina processes; specific thresholds may be adjusted according to the intended use of the product.

[0062] Through the above layer-by-layer mapping, a complete knowledge association chain from materials to final performance is finally formed on the basis of the directed acyclic graph: material nodes such as bauxite or aluminum hydroxide, and process event chains are chains formed by connecting multiple process event nodes through temporally directed edges. Phase nodes such as α- or γ-, and performance nodes such as purity and specific surface area.

[0063] Step Six: Conflict Knowledge Processing and Confidence Propagation.

[0064] This invention defines four types of conflicts: numerical conflict, which refers to different values ​​given by different sources for the same process parameter in the same process; phase attribution conflict, which refers to calcined products that meet the same process conditions being mapped to different phases by different sources; temporal conflict, which means that the process sequence described by different data sources is contradictory; and modal conflict, which means that different modal data have inconsistent parameter descriptions of the same event.

[0065] When a conflict is detected, the system creates a conflict node. Each conflict node contains a unique conflict identifier, a conflict subject, a conflict type, and a list of conflict items. Each item in the conflict item list independently records a data value from a source, along with the source information, initial confidence level, timestamp, and contextual description. Conflict items are stored independently, without weighted averaging, filtering for unique values, or overwriting each other.

[0066] This invention introduces the concept of root evidence identifiers, assigning a unique root evidence identifier to each original data source, including at least one of the following: document digital object identifier, distributed control system channel identifier, and standard manual number. When multiple conflict items exist under a conflict node, clustering is first performed based on the root evidence identifiers of each conflict item, grouping multiple conflict items sharing the same root identifier into the same evidence cluster. In subsequent confidence propagation, multiple pieces of evidence within the same evidence cluster are not counted repeatedly; only the highest confidence level within the cluster is used as the representative confidence level for cross-cluster aggregation.

[0067] For evidence clusters with different root identifiers but a derivation relationship, the correlation coefficient between the evidence clusters is calculated. The correlation coefficient is calculated by weighting factors such as whether there is a direct citation relationship between the two evidence clusters, whether they reuse the same original dataset, whether they come from the same acquisition device or channel, and whether they belong to the same data production entity. Only when the correlation coefficient is less than a preset correlation threshold will the two evidence clusters participate in subsequent convergence and propagation as independent clusters; otherwise, they will be merged into the same supercluster or participate in the calculation with a lower confidence level.

[0068] The confidence level was initialized into five levels based on the authority of the source type: distributed control system time series data and authoritative standard manuals were at the highest confidence level; high-impact factor academic journals were at high confidence level; mainstream journals and technical reports were at medium-high confidence level; dissertations and internal specifications were at medium confidence level; and conference papers and preprints were at lower confidence level.

[0069] Chain propagation: Propagation occurs along time-series directed edges and the associated links of "process event path, phase, and performance". For a single path, the overall confidence score is the minimum of the confidence scores of the attributes participating in the current propagation and the confidence scores of all edges among all nodes on the path.

[0070] Multi-evidence cluster convergence and propagation: When the same conclusion node is supported by independent paths from multiple different evidence clusters, a convergence rule is adopted. Suppose there are n independent evidence clusters, which, after... If the correlation coefficient satisfies the independence requirement, and the overall confidence level of the i-th evidence cluster is given, then the final confidence level is:

[0071] The physical meaning of this formula is: a conclusion is unreliable only when all independent evidence clusters are unreliable. Multiple paths within the same evidence cluster and different evidence clusters with correlation coefficients exceeding a threshold are not included in duplicate aggregation to prevent evidence from a single source or highly relevant source from being counted repeatedly through multiple expression paths, leading to artificially high confidence levels.

[0072] Step 7: Dynamic incremental update.

[0073] This invention supports incremental updates of knowledge graphs: First, input the newly added multimodal data, perform steps one through five to generate new candidate process events, candidate edges, and candidate knowledge assertions; Secondly, if the newly added process event node belongs to an existing process chain, insert the corresponding time sequence position according to its time interval, trigger the re-solution of the local directed acyclic graph, and only the candidate edges in the newly added node and its neighborhood are used as objects to re-execute the edge selection in step four, while the existing remote nodes and edges are not affected. Secondly, if the newly added data belongs to the same conflict group as the existing data in the conflict node, it is added as a new conflict item, and the confidence ranking of each evidence cluster under the conflict node is recalculated according to step six. Finally, the overall confidence level is recalculated along the path affected by the new data, while the confidence level of the unaffected part remains unchanged.

[0074] Example 2 This embodiment demonstrates how to integrate multimodal data to construct a time-series graph of process events in the alumina calcination process, such as... Figure 3 As shown.

[0075] Input multimodal data: Text data is "Aluminum hydroxide is calcined at 1120℃ in a rotary kiln for about 30 minutes to obtain an alumina product mainly composed of α-"; Image data is a rotary kiln process flow diagram, showing the material entering from the kiln tail, passing through the preheating section, calcining section, cooling section, and then exiting from the kiln head, with each section clearly marked; Time-series data is the rotary kiln temperature curve recorded by the distributed control system: approximately 200℃ in the feeding section, rising to 800℃ in the preheating section (about 20 minutes), holding at 1120℃ for 30 minutes in the calcining section, cooling down to 200℃ in the cooling section (about 25 minutes), and cooling to room temperature in the discharge section, containing a complete temperature-time change sequence.

[0076] Candidate event generation and cross-modal fusion: Time-series data change point detection identified five operation intervals: E1 is the feeding stage at 200℃, E2 is the preheating stage where the temperature rises from 200℃ to 800℃ for 20 minutes, E3 is the calcination stage where the temperature reaches 1120℃ for 30 minutes, E4 is the cooling stage where the temperature drops from 1100℃ to 200℃ for 25 minutes, and E5 is the discharging stage where the temperature drops from 200℃ to room temperature. Five corresponding process nodes were detected in the flowchart. The descriptive sequence "aluminum hydroxide, calcination, α-" was identified in the text.

[0077] like Figure 2 As shown, taking the calcination operation interval E3 as an example, cross-modal matching is performed: its text description "1120℃ high-temperature calcination" has a high similarity to the "calcination section" node name in the flowchart. The time interval can be aligned with the process position in the flowchart, so calculation is possible. The topological position similarity is high, and the final matching score S is equal to 0.87, which is greater than the preset threshold of 0.65, indicating that the three sources point to the same event. The three candidate events form a cluster in the matching graph. Fully linked agglomerative clustering is used. The minimum matching score within the cluster is 0.87, which is greater than the threshold. The cluster is then fused to generate the process event node E3, which carries evidence citations from all three sources.

[0078] Temporal edge generation and directed acyclic graph construction: Based on the time intervals of each event, the end time of E1 is approximately t=0, E2 is approximately t=20min, E3 is approximately t=50min, E4 is approximately t=75min, and E5 is approximately t=90min. Candidate edges are generated based on these time constraints. Adjacency constraints are also checked: E1 to E2 satisfy direct connection in the flowchart; E2 to E3 satisfies both direct connection in the flowchart and text sequence indication; E3 to E4 satisfies direct connection in the flowchart; E4 to E5 satisfies direct connection in the flowchart. Although the cross-level edge E1 to E3 satisfies temporal sequence, there is no direct adjacency relationship in the flowchart or text, thus failing to satisfy the adjacency constraint; therefore, this candidate edge is not generated. Topological sorting is performed to check for acyclicity, confirming that the five-node path is a directed acyclic graph.

[0079] Phase and property mapping: The system detects that the calcination temperature of E3 is 1120℃ ≥ 1100℃, and the duration is 30min ≥ 20min, triggering the phase rule "If the calcination temperature is in the second temperature range and the duration meets the condition, then a phase pointing to α- is generated". The candidate phase assertion was established, and a performance mapping node "purity ≥ 99.5%" was created.

[0080] Output spectrum: Aluminum hydroxide, E1 feed 200℃, E2 preheat 200℃ to 800℃, E3 calcination 1120℃, E4 cooling 1100℃ to 200℃, E5 discharge 200℃ to room temperature, α- Purity ≥ 99.5%. Each node is equipped with reference identifiers for its text, flowchart, and temporal trimodal evidence, supporting complete traceability.

[0081] Furthermore, this embodiment provides a complete numerical calculation for the above-mentioned cross-modal matching process as follows: First, set the weight coefficients for each similarity item. The value of w1 is 0.4, the value of w2 is 0.3, the value of w3 is 0.1, and the value of w4 is 0.2.

[0082] Next, the similarity of each dimension was calculated. Textual semantic similarity s1 equals 0.80. Time interval overlap s2 equals 0.90. Process parameter similarity s4 equals 0.95. Topological location similarity cannot be calculated because the current candidate events come from different nodes in the same flowchart and do not satisfy collinearity; the indicator I3 equals 0.

[0083] Then perform effective modal normalization again.

[0084] molecule = .

[0085] Denominator = .

[0086] The final cross-modal matching score S is 0.78 divided by 0.9, which is approximately 0.87. Since S equals 0.87 and is not lower than the preset threshold of 0.65, the candidate events in this group are determined to be merged into the same process event node.

[0087] The following explains the greedy selection of candidate edges and the process of directed cycle detection.

[0088] Furthermore, this embodiment provides the numerical calculations for the greedy selection of candidate temporal edges and the directed cycle detection process as follows: First, set the weight coefficients for each piece of evidence. α1 is set to 0.4 and corresponds to the time interval constraint. α2 is set to 0.3 and corresponds to the topological constraint. α3 is set to 0.2 and corresponds to the text order constraint. α4 is set to 0.1 and corresponds to the domain knowledge constraint.

[0089] Next, the multimodal evidence score for each candidate edge is calculated. For edge E2→E3, i.e., the preheating to calcination process, the time constraint J1 equals 1 and satisfies the condition that the end time of E2 (20 min) is less than the start time of E3 (50 min). The topological constraint J2 equals 1 and satisfies the condition that there are arrow connections in the flowchart. The text order J3 equals 1 and satisfies the condition that there are order indicators in the text. The domain knowledge J4 equals 1 and satisfies the condition that the two processes are adjacent. The score for this edge is E equals 1.0.

[0090] Noise edge detection is performed again. Noise edge detection: Edge E4→E2, which is the cooling to preheating process, violates the time constraint and is specifically manifested in that the end time of E4 is much later than the start time of E2, with a score of 0.2; if it is temporarily selected, a directed loop E2→E3→E4→E2 is formed. After the directed loop detection is identified, it is marked as "timing conflict" and discarded. The final set of time-series edges, E1 to E2, E2 to E3, E3 to E4, and E4 to E5, forms a directed acyclic graph that fully represents the timing path of the Bayer process for alumina from feeding to packaging.

[0091] Example 3 This embodiment demonstrates how the present invention handles conflicting values ​​of the same process and parameter across different data sources, and how it avoids artificially high confidence levels through evidence clustering. Figure 4 As shown.

[0092] Input multimodal data: The system acquires data from three sources for the alumina dissolution temperature. Source A is historical production data from the distributed control system, with recorded values ​​ranging from 248℃ to 252℃, an average of 250℃, a sampling frequency of 1Hz, and continuous data collection for one year. The evidence root is identified as DCS-Ch01-2024. Source B is the process design manual, with a recorded value of 250℃. The evidence root is identified as Handbook-5thEd-2020. Source C is a single test report, with a recorded value of 260℃. The evidence root is identified as Report-Exp03-2023.

[0093] Conflict node creation and evidence cluster merging: The system determines that there is a numerical conflict among the three factors regarding the dissolution temperature and creates a conflict node. The three conflicting items are stored independently, each labeled with its value, evidence root identifier, modality type, and initial confidence level.

[0094] Suppose there exists a fourth source, D, which is a secondary statistical analysis report of the same batch of distributed control system data as source A. This report also provides a 250°C, and the root of evidence identifier remains DCS-Ch01-2024, sharing the same identifier as source A. The system merges source A and source D into the same evidence cluster, DCS-Ch01-2024. The highest confidence level within this cluster, i.e., the confidence level of source A, is used as the representative confidence level for this cluster. This is counted only once in subsequent aggregation, avoiding the repeated enhancement of confidence levels for the same distributed control system data through both the original record and the statistical analysis report presentation paths.

[0095] Furthermore, suppose there exists a source E, which is an academic paper that reviews distributed control system data from source A, and the paper reports a value of 250°C. The system detects a citation derivation relationship between source E and source A. Based on factors such as whether there is a direct citation relationship and whether the same original dataset is reused, the correlation coefficient is determined to exceed a preset correlation threshold of 0.6. Therefore, the two are considered highly correlated evidence and are not counted as independent evidence clusters in the convergence propagation.

[0096] Regarding confidence level output, the system sorts and outputs data according to the confidence level after merging evidence clusters during the query. The distributed control system cluster corresponds to a temperature of 250℃ and represents a confidence level of 0.95. The manual cluster corresponds to a temperature of 250℃ and represents a confidence level of 0.90. The experimental cluster corresponds to a temperature of 260℃ and represents a confidence level of 0.75. All three are displayed side-by-side, each labeled with an evidence root identifier and modality label.

[0097] This embodiment demonstrates that conflict nodes ensure the complete preservation and traceability of multi-source data, and that the evidence root identification and cluster merging mechanism effectively prevent the same data source from repeatedly enhancing confidence through multiple representation paths. The correlation threshold further prevents different evidence clusters with citation derivation relationships from being counted repeatedly, making the confidence assessment more objective.

[0098] This embodiment provides a complete numerical calculation of the confidence propagation and convergence process as follows: First, the initial confidence levels of each source are determined by its authority. Source A is historical production data of the distributed control system, and C_A equals 0.95. Source B is the recommended value from the standard manual, and C_B equals 0.90. Source C is a single test report, and C_C equals 0.75. Source D is a secondary statistical analysis report of the distributed control system, and C_D equals 0.80. Source E is an academic paper that cites data from Source A, and C_E equals 0.85.

[0099] Next, evidence clusters are merged. The evidence root identifiers from source D and source A both point to the same distributed control system channel, so they are merged into the same evidence cluster A and D, and the highest confidence level representative value within the cluster is taken as 0.95.

[0100] Independence was tested again. The correlation coefficient ρ between source E and the evidence clusters from sources A, B, C, and D was calculated to be 0.85 and greater than the threshold of 0.6. Source E was determined to be non-independent due to a citation relationship and was therefore excluded from the pooling. The remaining independent evidence clusters were A and D with a representative value of 0.95, and the manual cluster B with a representative value of 0.90.

[0101] Finally, a convergence calculation is performed. .

[0102] The comparison is illustrated below. Using a traditional averaging strategy, the average confidence level of the five sources is 0.95 + 0.90 + 0.75 + 0.80 + 0.85, divided by 5, equals 0.85. Furthermore, the failure to eliminate independent sources leads to an inflated confidence level. This invention avoids duplicate counting of the same data source through multiple representation paths by using an evidence root identification merging mechanism, and eliminates highly relevant evidence through a relevance threshold, ensuring objective and accurate confidence level assessment.

[0103] Example 4 This embodiment verifies the quantification effect of the multimodal semantic parsing method of the present invention in constructing an alumina process event map through a comparative experiment with a single-modal baseline method. The dataset uses the full-year production data of an alumina plant in 2024, containing 1000 manually annotated process events and 200 known conflict nodes.

[0104] Table 1. Comparison Results of Multimodal Fusion vs. Single Modality

[0105] As shown in Table 1, under the same dataset and annotation conditions, the multimodal fusion method adopted in this invention significantly outperforms any single-modal baseline in the F1-score. It achieves a 38.8% improvement compared to the pure text method, with values ​​of 95.2% and 68.6%, respectively. Compared to the time-series-only method, it achieves a 30.6% improvement, with values ​​of 95.2% and 72.9%, respectively. Compared to the image-only method, it achieves a 58.9% improvement, with values ​​of 95.2% and 59.9%, respectively.

[0106] The comparative results regarding conflict data processing capabilities indicate the following two points. First, regarding information loss rate, traditional overlay, averaging, and deletion strategies all have a 100% information loss rate, meaning at least one source is retained while the rest are discarded. This invention, through a conflict node mechanism, ensures that all source data is completely preserved and traceable, achieving a 0% information loss rate. Second, regarding the accuracy of confidence assessment, the Spearman rank correlation coefficient between the confidence ranking of traditional methods and expert scores is only 0.42, while the method of this invention achieves a high 0.91.

[0107] In summary, the comparative experiment in this embodiment verifies the beneficial effects of the present invention from two dimensions: event recognition accuracy and conflict data reliability. Multimodal fusion significantly improves the completeness and accuracy of process event recognition, and the conflict node combined with the confidence propagation mechanism achieves zero information loss and high-confidence sorting output of multi-source data.

[0108] This invention also provides a system for constructing an alumina process event graph based on multimodal semantic parsing, including a data acquisition module, a semantic parsing module, an event fusion module, a graph construction module, a knowledge chain generation module, a conflict resolution module, and a confidence propagation module. The data acquisition module performs the processing described in step one above; the semantic parsing module performs the processing described in step two above; the event fusion module performs the processing described in step three above; the graph construction module performs the processing described in step four above; the knowledge chain generation module performs the processing described in step five above; and the conflict resolution module and the confidence propagation module perform the processing described in step six above. The specific implementation methods of each module are the same as those of the corresponding steps in the above method embodiments, and will not be repeated here.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for constructing an alumina process event map based on multimodal semantic parsing, characterized in that, Includes the following steps: Step 1: Acquire alumina process data in at least two different modes and preprocess them. The modes include text data, image data, time-series data, and structured data. Step 2: Perform semantic parsing on each modal data after preprocessing, corresponding to its modal type, to generate a set of candidate process events for each modality; When text data is included, alumina process entities and sequential relationships are identified from the text; when image data is included, process nodes and connection topology are identified from the image; when time-series data is included, candidate process events, their start time, end time, and process parameters are extracted through change point detection. Step 3: Calculate the matching score between cross-modal candidate events. The matching score is a weighted synthesis of the computable similarity items among text semantic similarity, time interval overlap, topological location similarity, and process parameter similarity. When a certain similarity item is not computable, that similarity item is not included in the calculation and its weight is normalized among the remaining computable similarity items. Construct a matching graph with candidate events as vertices and matching scores not lower than a preset threshold as edges. Perform consistency clustering on the matching graph to ensure that the matching scores of any two candidate events within a cluster are not lower than the preset threshold. Each cluster generates a process event node. The process event node includes process type, start time, end time, process parameter set, multimodal evidence reference list, and attribute-level confidence level; Step 4: Determine the time interval relationship between nodes based on the start and end times of each process event node; The generation of candidate time-series edges satisfies both time constraints and adjacency constraints. The time constraint is that the end time of the previous event is earlier than the start time of the next event. The adjacency constraint is that the two events satisfy at least one of the following: direct connection in the flowchart, direct sequential indication in the text, direct material transfer relationship, equipment connection relationship, or adjacent process relationship in the alumina domain. For each candidate temporal edge that meets the conditions, a multimodal evidence score is calculated. Candidate temporal edges are selected one by one according to the multimodal evidence score. After each selection, directed cycle detection is performed to obtain a set of target temporal directed edges that satisfy the acyclic constraint. Together with the process event nodes, they form a directed acyclic graph. The acyclic constraint only applies to the main process event temporal edges within the same production batch or the same material instance. Circular logistics relationships are stored separately as circular logistics edges. Step 5: Map the phase entities corresponding to the alumina material after processing through the time-series process path shown in the directed acyclic graph to the process event nodes, and map the phase entities to the corresponding performance entities to form the "process event path-phase-performance" association link; Step Six: When there are multiple conflicting data from different sources for the same process, the same process parameter, or the same phase, create a conflict node. Each conflicting data item and its corresponding evidence root identifier, source modality, and initial confidence level are stored in parallel under the conflict node. The conflicting items are clustered into evidence clusters based on the evidence root identifier. Multiple pieces of evidence within the same evidence cluster are counted as a single piece of evidence during confidence propagation. The confidence level is propagated along the directed edges of the time sequence; the overall confidence level of a single path is the minimum value of the node attribute confidence level and the edge confidence level participating in the current propagation on that path; when there are multiple independent paths of different evidence clusters pointing to the same node, the final overall confidence level is 1 minus the product of the confidence levels of each path.

2. The method according to claim 1, characterized in that, The matching score S in step three is calculated using the following formula: in, For text semantic similarity, The degree of overlap between time intervals. For topological location similarity, Similarity of process parameters; , , , These are indicators of whether each similarity item can be calculated; a value of 1 indicates that it can be calculated, and a value of 0 indicates that the similarity item cannot be calculated. , , , Preset weighting coefficients and satisfying When the sum of the weights of all similarity items is zero, the matching score of the candidate event pair is not calculated and the candidate event pair is determined to be unmergeable.

3. The method according to claim 1, characterized in that, The consistent clustering in step three adopts a fully linked agglomerative clustering strategy: Initially, each candidate event is treated as a separate cluster. Two clusters are merged sequentially. Merging is performed only when the matching scores of any two candidate events in the merged new cluster are not lower than the preset threshold. Clustering terminates when there are no mergeable cluster pairs that meet the above conditions.

4. The method according to claim 1, characterized in that, The multimodal evidence score E of the candidate temporal edges in step four is calculated using the following formula: in, This is the time interval constraint indicator value. This refers to the topology constraint indicator value of the flowchart. This is a text order constraint indicator value. This is the domain knowledge constraint indicator value. Each indicator function takes a value of 1 when it satisfies the corresponding constraint, and a value of 0 otherwise. , , , The corresponding weight coefficients and satisfying .

5. The method according to claim 1, characterized in that, In step six, the evidence root identifier is a unique code of the original data source, including at least one of the following: document digital object identifier, distributed control system channel identifier, and standard manual number; no weighted average or selective coverage is performed between different conflict items.

6. The method according to claim 1, characterized in that, In step six, for evidence clusters with different root identifiers but with citation or data reuse relationships, the correlation coefficient between evidence clusters is calculated. Only when the correlation coefficient is less than a preset correlation threshold, the two evidence clusters participate as independent clusters in the convergence and propagation in step six.

7. The method according to claim 1, characterized in that, In step five, the mapping between the physical phase and the process event node is based on the following rules: When the calcination temperature is within the first temperature range, it generates a direction pointing to γ- Candidate phase assertion; When the calcination temperature is in the second temperature range, it generates a direction pointing to α- Candidate phase assertion; When the calcination temperature is in the third temperature range and the cooling rate is significantly higher than the conventional cooling rate, α-direction is generated. With γ- Candidate phase assertion for mixed phases; The first temperature range is lower than the second temperature range, and the rule simultaneously records its applicable conditions, source literature, and confidence level; the phase entity includes α- γ- δ- θ- At least one of gibbsite and sodium aluminate; the properties include at least one of purity, true density, specific surface area, pore volume, pore size distribution, thermal stability and thermal shock resistance.

8. The method according to claim 1, characterized in that, It also includes a dynamic incremental update step: Perform steps two through six on the newly added data to generate new candidate process events, candidate edges, and candidate knowledge assertions; If the newly added process event node belongs to an existing process chain, then insert the corresponding time sequence position according to its time interval, and only re-execute the edge selection in step four with the newly added node and its neighboring candidate edges as the objects. If the newly added data belongs to the same conflict group as the existing data in the conflict node, it is added as a new conflict item, and step six is ​​executed for the evidence cluster and propagation path affected by the newly added data; The entire map does not need to be reconstructed.

9. The method according to claim 1, characterized in that, The attribute-level confidence in step three includes event existence confidence, type confidence, time interval confidence, and process parameter confidence, which are calculated independently based on the evidence sources supporting each attribute.

10. A system for constructing an alumina process event map based on multimodal semantic parsing, characterized in that, include: The data acquisition module is used to acquire and preprocess alumina process data in at least two different modes, including text data, image data, time-series data, and structured data. The semantic parsing module is used to perform semantic parsing on each modal data after preprocessing, corresponding to its modal type, and generate a set of candidate process events for each modality. The event fusion module is used to calculate the matching score between cross-modal candidate events. The matching score is a weighted synthesis of computable similarity items among text semantic similarity, time interval overlap, topological position similarity and process parameter similarity. A matching graph is constructed with candidate events as vertices and matching scores not lower than a preset threshold as edges. Consistent clustering is performed on the matching graph, and each cluster generates a process event node. The graph construction module is used to determine time constraints based on the start and end times of each process event node and generate candidate time-series edges by combining them with adjacency constraints. It calculates the multimodal evidence scores of the candidate time-series edges and selects them one by one. After directed cycle detection, it obtains a set of target time-series directed edges that satisfy the acyclic constraints, which together with the process event nodes constitute a directed acyclic graph. The knowledge chain generation module is used to map the phase entities corresponding to the alumina material after processing through the time-series process path shown in the directed acyclic graph to the process event nodes, and to map the phase entities to the corresponding performance entities, forming an association link of "process event path-phase-performance". The conflict resolution module is used to create conflict nodes when there are multiple conflicting data from different sources for the same process, the same process parameter, or the same phase, and to cluster conflicting items into evidence clusters based on evidence root identifiers. The confidence propagation module is used to propagate confidence along the time-series directed edges and converge them when there are multiple independent paths of different evidence clusters pointing to the same node.

Images