A multimodal corpus construction method and system in the field of electronic information manufacturing

Abstract

The application provides a multimodal corpus construction method and system in the field of electronic information manufacturing, and relates to the technical field of corpus construction. The threshold rule is used to filter obvious abnormal values, and the generative adversarial network is used to fill in the missing data, so that the cleaned data retains the nonlinear fluctuation characteristics of the original sensor data and has high integrity. Subsequently, a multi-dimensional associated knowledge graph is constructed by aligning the time and space benchmarks, and the originally discrete supply chain and equipment data are cross-domain fused. Further industrial time sequence logic constraint elimination operation can clean up the edges of the wrong association based on business logic, ensuring the accuracy of the graph structure at the logical level. Finally, through the encapsulation of confidence score and metadata label, the quantitative management of data quality is realized. A high-quality corpus with data integrity, logical consistency and quality traceability is constructed, and the feature expression ability and credibility of the special corpus in the field of electronic manufacturing are improved.

Description

Technical Field

[0001] This application relates to the field of corpus construction technology, and in particular to a method and system for constructing a multimodal corpus in the field of electronic information manufacturing. Background Technology

[0002] Currently, electronic information manufacturing enterprises face challenges in leveraging artificial intelligence (AI) technology to improve production efficiency, including inconsistent data quality, a lack of standardized annotation criteria, and difficulties in integrating heterogeneous data from multiple sources. Particularly in vertical scenarios such as predictive maintenance and business management decision-making, the lack of high-quality, scenario-based, and standardized training corpora often results in AI models with low accuracy and unsatisfactory generalization capabilities.

[0003] The construction of relevant industry corpora typically employs a general IT governance architecture based on statistical rules and relational database integration. In practice, the system first aggregates structured business data from the supply chain management system and unstructured time-series data from production line equipment. During preprocessing, statistical characteristics of the full dataset (such as mean and median) or linear interpolation algorithms are used to numerically impute missing segments of the time-series data, followed by anonymization and formatting. Finally, the system uses the system timestamp of data collection as the unique primary key to align and flatten the processed multimodal data, storing it in the database as a two-dimensional wide table, forming a standardized training dataset.

[0004] However, these technologies struggle to adapt to the high-dimensional, tightly coupled data characteristics of the electronics manufacturing field. In electronics manufacturing, material flow and equipment operation involve objective physical time lags, and there are strict process logic constraints between process parameters. The rigid timestamp alignment methods cannot detect these physical time lags, leading to data misalignment in time and space. Therefore, the corpus quality of these technologies is low, making it difficult to meet the accuracy requirements of the trained AI models. Summary of the Invention

[0005] This application provides a method for constructing a multimodal corpus in the field of electronic information manufacturing, which can be used to improve the quality of the corpus.

[0006] Firstly, this application provides a method for constructing a multimodal corpus in the field of electronic information manufacturing, including: filtering outliers from original multi-source heterogeneous data based on preset threshold rules and imputing missing values ​​based on generative adversarial networks to obtain cleaned data; desensitizing the cleaned data and mapping it into intermediate data in a unified format; performing spatiotemporal benchmark alignment on the intermediate data using production work order number and unified time axis as dual primary keys, wherein vector mapping and entity alignment are performed using supply chain dimension and equipment dimension to obtain a multidimensional association knowledge graph of cross-domain entity relationships; identifying and removing edges in the multidimensional association knowledge graph that do not satisfy logical constraint relationships based on industrial time-series logical constraints; calculating the confidence score of nodes in the multidimensional association knowledge graph based on data integrity indicators; adding metadata tags to nodes and encapsulating them to generate a basic corpus; and extracting the basic corpus to generate a training set based on the demand data of downstream artificial intelligence models.

[0007] By employing the aforementioned technical solutions, threshold rules are used to filter obvious outliers, and generative adversarial networks are used to fill in missing data. This ensures that the cleaned data retains the key information of the nonlinear fluctuation characteristics of the original sensor data while achieving high integrity. Subsequently, a multi-dimensional relational knowledge graph is constructed through spatiotemporal benchmark alignment, enabling cross-domain fusion of originally discrete supply chain and equipment data. Further industrial time-series logical constraint removal operations can clean up erroneous edges based on business logic, ensuring the accuracy of the graph structure at the logical level. Finally, through confidence scoring and metadata tagging, quantitative management of data quality is achieved. This not only solves the problem of difficult multi-source heterogeneous data fusion but also constructs a high-quality corpus with data integrity, logical consistency, and quality traceability, improving the feature expression capabilities and credibility of specialized corpora in the electronics manufacturing field.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of performing a spatiotemporal reference alignment operation on intermediate data using the production work order number and the unified time axis as dual primary keys specifically includes: extracting state change events from the intermediate data and calculating the timing response difference of the state change events in different data streams; determining whether the timing response difference is less than a preset physical transmission hysteresis threshold; if it is less than the preset physical transmission hysteresis threshold, then performing a spatiotemporal reference alignment operation on the intermediate data using the production work order number and the unified time axis as dual primary keys; if it is not less than the preset physical transmission hysteresis threshold, then generating a dynamic compensation time window and using the dynamic compensation time window to perform a translation correction on the unified time axis to obtain a timing correction time axis; and performing a spatiotemporal reference alignment operation on the intermediate data using the production work order number and the timing correction time axis as dual primary keys.

[0009] By employing the aforementioned technical solution, and calculating the temporal response difference of state change events in different data streams, the physical lag caused by logistics buffering between the time of supply chain data generation and the time of physical equipment execution can be quantified. By determining whether the difference exceeds the physical transmission lag threshold, the system can distinguish between real-time synchronization scenarios and physical lag scenarios. For lag scenarios, a dynamic compensation time window is used to adaptively shift and correct the unified time axis, eliminating misalignment problems caused by rigid time alignment. This ensures the authenticity of the correlation between material batches and equipment operating parameters in the graph, avoids false associations where material quality problems from the previous work order are incorrectly attributed to the current equipment, and improves the spatiotemporal alignment accuracy of the corpus in complex production scenarios.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the step of extracting state change events from intermediate state data and calculating the timing response difference of state change events in different data streams specifically includes: performing continuous detection on the equipment operation data stream in the intermediate state data; monitoring the work order register status when the result of the continuous detection is continuous production status; extracting the timestamp when a jump in the work order register value is detected and marking it as a logical switching anchor point; retrieving material error prevention scanning records associated with the value after the jump in the work order register in the supply chain data stream, and extracting the scanning loading anchor point in the material error prevention scanning records; calculating the time difference between the logical switching anchor point and the scanning loading anchor point to obtain the timing response difference.

[0011] By adopting the above technical solution, a feature extraction mechanism was specifically designed for the scenario of material changeover without stopping the system. Continuous detection locks in the continuous production state, and the logic switching anchor point is captured at the instant the work order register changes, representing the system's apparent switching moment. Simultaneously, the scanning and loading anchor point is determined by retrieving material error-proofing scanning records, representing the material's apparent preparation moment. Calculating the time difference between these two points allows for the reconstruction of the implicit physical buffer time experienced by materials from scan confirmation to actual consumption in the continuous production flow. This method enables the system to identify the physical boundary between new and old materials hidden in the continuous waveform without stopping the system or interrupting the data flow, thus providing dynamic correction parameters for subsequent spatiotemporal alignment.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of performing continuous detection on the equipment operation data stream in the intermediate state data, the method further includes: if the result of the continuous detection is a production interruption state, identifying the cold start moment when the equipment state changes from the stop position to the run position; extracting the first transmission arrival signal generated after the cold start moment and the associated work order register value, and marking the timestamp of the transmission arrival signal as a physical anchor point; based on the work order register value, retrieving the corresponding production line material preparation completion record in the supply chain data stream, and extracting the timestamp as a logical ready anchor point; calculating the time difference between the physical anchor point and the logical ready anchor point to obtain the timing response difference.

[0013] By adopting the above technical solution, for scenarios involving production interruptions (shutdowns and line changes), the cold start moment is captured by the signal indicating that the equipment status has changed to the running position. The physical anchor point is determined by identifying the first transmitted signal, marking the true start of the equipment's physical actions. This is compared with the logical ready anchor point in the retrieved production line material preparation completion record, and the calculated time difference directly reflects the non-productive waiting time during the shutdown and line changeover period. This mechanism logically separates the invalid and redundant waiting data generated during downtime from the valid data of formal production. Alignment based on this eliminates the interference of this time gap on the continuity of the corpus's time series, ensuring that the constructed graph only contains associated data from valid production cycles, thus improving the purity and effective information density of the corpus under discontinuous production modes.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the steps of performing spatiotemporal benchmark alignment on intermediate data using production work order number and unified time axis as dual primary keys specifically include: retrieving supply chain material records and equipment operation records with the same work order identifier in the intermediate data based on the production work order number; calculating the intersection-union ratio (IUR) of the material flow time window of the supply chain material record and the processing operation time window of the equipment operation record on the unified time axis; filtering out record pairs with IUR values ​​greater than a preset association threshold, mapping the supply chain material record to a material entity vector, and mapping the equipment operation record to an equipment state vector; calculating the cosine similarity between the material entity vector and the equipment state vector in the vector space, creating edges between vectors whose cosine similarity satisfies the alignment condition, and obtaining a multidimensional association knowledge graph.

[0015] By employing the aforementioned technical solution, a quantified temporal overlap index is introduced by calculating the intersection-union ratio (IUR) of supply chain material records and equipment operation records within a time window. This allows for the rapid initial screening of record pairs that highly match in time, filtering out obviously irrelevant data. Furthermore, by mapping these pairs to entity vectors and calculating cosine similarity, semantic verification of this association is performed in the vector space. Edges are only created in the graph when both temporal overlap (IUR) and content similarity (cosine similarity) conditions are met. Compared to single-timestamp associations, this reduces the probability of erroneous associations caused by temporal coincidences.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the step of identifying and removing edges in a multidimensional association knowledge graph that do not satisfy the logical constraint relationship based on industrial temporal logical constraints specifically includes: parsing a preset industrial process flow matrix to obtain the positive dependency order between process nodes; traversing the edges in the multidimensional association knowledge graph to locate edges whose connection order violates the positive dependency order and does not satisfy the logical constraint relationship; and performing a topology disconnection operation on the edges that do not satisfy the logical constraint relationship.

[0017] By employing the aforementioned technical solution, and utilizing a pre-defined industrial process flow matrix to obtain the positive dependency order between process nodes, traversing the edges in the graph and locating connections that violate this order, it is possible to identify erroneous associations that, while appearing normal in terms of data values, conflict in business logic. Performing a topology disconnection operation on these edges that do not satisfy logical constraints constitutes a deep logical cleansing at the graph level, ensuring that the final generated corpus follows the objective process laws of electronic manufacturing. This enables the AI ​​model trained on the corpus to learn correct causal logic, rather than erroneous noisy associations.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the step of extracting a training set from a basic corpus based on the demand data of downstream artificial intelligence models specifically includes: parsing the model scenario configuration file in the demand data, extracting the required feature dimension list and label field definition; indexing candidate graph nodes containing the feature dimension list and related neighboring edges in the basic corpus; reading the metadata labels carried by the candidate graph nodes, comparing the confidence score with a preset model training quality threshold, filtering and removing nodes with scores lower than the quality threshold to obtain a preferred feature subgraph; extracting the adjacency matrix and flattening the feature tensor on the preferred feature subgraph, aligning the data according to the label field definition, and outputting the training set.

[0019] By adopting the above technical solution, the system can parse configuration files and extract targeted feature dimensions according to the specific needs of downstream AI models, realizing on-demand data customization. By indexing candidate nodes in the graph and reading the confidence scores in their metadata tags, the system can automatically filter out low-quality data nodes based on the model training quality threshold, retaining only the preferred feature subgraphs with high confidence, which facilitates the improvement of AI model training performance.

[0020] Secondly, this application provides a multimodal corpus construction system in the field of electronic information manufacturing. The multimodal corpus construction system in the field of electronic information manufacturing includes: one or more processors and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to cause the multimodal corpus construction system in the field of electronic information manufacturing to perform the method described in the first aspect and any possible implementation of the first aspect.

[0021] Thirdly, this application provides a computer program product containing instructions that, when the computer program product is run on a multimodal corpus construction system in the field of electronic information manufacturing, causes the multimodal corpus construction system in the field of electronic information manufacturing to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on a multimodal corpus construction system in the field of electronic information manufacturing, cause the multimodal corpus construction system in the field of electronic information manufacturing to perform the method described in the first aspect and any possible implementation thereof.

[0023] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0024] 1. By using threshold rules to filter obvious outliers and leveraging generative adversarial networks to fill in missing data, the cleaned data retains the key information of the nonlinear fluctuation characteristics of the original sensor data while achieving high integrity. Subsequently, a multi-dimensional knowledge graph is constructed through spatiotemporal benchmark alignment, enabling cross-domain fusion of previously discrete supply chain and equipment data. Further industrial time-series logical constraint removal operations clean up erroneous associations based on business logic, ensuring the logical accuracy of the graph structure. Finally, confidence scoring and metadata tagging encapsulation enable quantitative management of data quality. This approach not only solves the problem of difficult multi-source heterogeneous data fusion but also constructs a high-quality corpus with data integrity, logical consistency, and quality traceability, enhancing the feature representation capabilities and credibility of specialized corpora in the electronics manufacturing field.

[0025] 2. By calculating the temporal response difference of state change events in different data streams, the physical lag caused by logistics buffering between the time of supply chain data generation and the time of physical execution by equipment can be quantified. By determining whether the difference exceeds the physical transmission lag threshold, the system can distinguish between real-time synchronization scenarios and physical lag scenarios. For lag scenarios, a dynamic compensation time window is used to adaptively shift and correct the unified time axis, eliminating the misalignment problem caused by rigid time alignment. This ensures the authenticity of the correlation between material batches and equipment operating parameters in the graph, avoids false associations where material quality problems in the previous work order are incorrectly attributed to the current equipment, and improves the spatiotemporal alignment accuracy of the corpus in complex production scenarios.

[0026] 3. A feature extraction mechanism was specifically designed for the scenario of material changeover without stopping the system. Continuous detection locks in the ongoing production status, and the logic switching anchor point is captured at the instant the work order register changes, representing the system's apparent switching moment. Simultaneously, the scanning and loading anchor point is determined by retrieving material error-proofing scanning records, representing the material's apparent preparation moment. Calculating the time difference between these two allows the reconstruction of the implicit physical buffer time experienced by the material from scan confirmation to actual consumption in the continuous production flow. This method enables the system to identify the physical boundary between new and old materials hidden in the continuous waveform without stopping the system or interrupting the data flow, thus providing dynamic correction parameters for subsequent spatiotemporal alignment. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating a method for constructing a multimodal corpus in the field of electronic information manufacturing, as described in this application.

[0028] Figure 2 This is another flowchart illustrating the method for constructing a multimodal corpus in the field of electronic information manufacturing, as described in this application.

[0029] Figure 3 yes Figure 2 A flowchart of step S201;

[0030] Figure 4 This is an exemplary hardware structure diagram of a multimodal corpus construction system in the field of electronic information manufacturing, as described in this application. Detailed Implementation

[0031] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0033] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for constructing a multimodal corpus in the field of electronic information manufacturing, as described in this application.

[0034] A method for constructing a multimodal corpus in the field of electronic information manufacturing includes:

[0035] S101. The original multi-source heterogeneous data is filtered for outliers based on preset threshold rules and missing values ​​are filled in based on generative adversarial networks to obtain clean data; the clean data is then de-identified and mapped into intermediate data in a unified format.

[0036] Among them, raw multi-source heterogeneous data refers to the data set collected from different business systems and industrial field equipment within electronic information manufacturing enterprises.

[0037] It should be noted that this embodiment adopts a processing sequence of "outlier filtering first, then missing value imputation". This is because the presence of outliers often skews the statistical characteristics of the data. If they are not processed in advance, they will interfere with the subsequent missing value imputation model. Removing outliers and treating them as missing values ​​can effectively purify the data source.

[0038] In some specific implementations, the mean and standard deviation are first calculated based on the historical full dataset to construct a normal distribution model. For newly collected data points, a multiplier n is set according to a preset accuracy requirement. If a data point value falls outside the range of the mean plus or minus n times the standard deviation, it is identified as an outlier and removed, transforming that position into a missing value to be filled. Subsequently, for both the original missing data and the gaps created by outlier removal, the chain equation multiple imputation method (MICE) is uniformly used for filling. This method utilizes the potential correlations between variables to establish a multiple regression model, obtaining statistically robust imputation results by making multiple predictions and taking the average. No further limitations are specified here.

[0039] In practical applications, missing value imputation based on generative adversarial networks may be statistically realistic, but it can generate false data that does not conform to physical constraints (such as generating negative stress values).

[0040] In other embodiments, a "physical consistency loss" is added to the loss function of the adversarial network. This loss term is set according to physical laws, such as defining "pressure value must be greater than 0" or "energy conservation constraint." When the model backpropagates to update parameters, if the data generated by the generator violates the physical constraints, this loss term will increase sharply, forcing the generator to converge in a direction that conforms to physical laws. Furthermore, at the input of the discriminator, in addition to the original data, physical feature vectors (such as power = voltage x current) can be concatenated to enhance the discriminator's ability to identify pseudo-data that violates physical laws. Through this mechanism, it is ensured that the generated imputed data is not only statistically realistic but also physically legitimate.

[0041] S102. Using the production work order number and unified time axis as dual primary keys, perform spatiotemporal benchmark alignment operation on intermediate data. Among them, vector mapping and entity alignment are performed on the supply chain dimension and equipment dimension to obtain a multi-dimensional association knowledge graph of cross-domain entity relationships.

[0042] Among them, the production work order number is a unique code used in the MES system to identify a specific production task, which is usually associated with the BOM list and process route; the unified time axis is a global standard time base after clock synchronization calibration, used to eliminate errors caused by inconsistencies in the time of servers in different subsystems; vectorization mapping is the process of converting discrete entities (such as material batches, equipment IDs) into low-dimensional dense real number vectors using graph embedding technology; and multidimensional association knowledge graph is a complex network structure based on graph database storage, where nodes represent entities (materials, equipment, processes, etc.) and edges represent spatiotemporal associations or causal relationships between entities.

[0043] Specifically, the system first extracts supply chain data (such as raw material batches) and equipment operation data (such as placement speed and furnace temperature curves) belonging to the same task from a massive data lake, using "production work order number" as the logical link. Then, a "unified time axis" is introduced as the physical link, and a time window sliding algorithm is used to finely match the time periods of material flow with the time periods of equipment processing. Based on this, the system further performs deep fusion: the extracted supply chain entities (such as solder paste batch A) and equipment entities (such as placement machine B) are input into a graph neural network model for vectorization, calculating their similarity in semantic and spatiotemporal spaces. When the similarity meets the conditions, a directed edge is created in the graph indicating "used for" or "processed on," and a weight is assigned to it. The final constructed graph not only contains the explicit "A used B," but also implicitly includes deeper features such as "A and B are closely combined under specific high temperature and humidity environments" through vectors.

[0044] In some specific embodiments, the triples (head entity, relation, tail entity) in the graph are regarded as translation operations in vector space, that is, "head entity vector + relation vector ≈ tail entity vector". The entities in the intermediate data are initialized with random encoding, and the vector representation is continuously optimized by minimizing the energy function to make the distance between the related supply chain entities and equipment entities in the vector space closer, so as to determine and complete the potential alignment relationship based on Euclidean distance.

[0045] In other specific embodiments, a weighted alignment strategy based on attribute similarity is employed. The system predefines an attribute rule base for entity alignment, for example, stipulating that the "outbound time of a material entity" and the "loading time of an equipment entity" must be within a certain error range. The similarity score of the key attributes (timestamp, location code, operator ID) for each pair of candidate entities is calculated, and different weights are assigned to different attributes (e.g., time weight 0.6, location weight 0.4). A weighted total score is calculated; if the total score exceeds a threshold (e.g., 0.85), the two entities are considered successfully aligned, and their attributes are merged and updated in the graph node; this is not limited here.

[0046] In actual use, in multi-process continuous production scenarios, the same production work order number may correspond to the time period of multiple machines, resulting in a vague "one-to-many" alignment, making it difficult to distinguish the specific processing machine.

[0047] In some specific embodiments, step S102 specifically includes:

[0048] S1021. Retrieve supply chain material records and equipment operation records with the same work order identifier from the intermediate state data based on the production work order number.

[0049] Supply chain material records refer to structured data entries generated in ERP or WMS systems that describe the batch number, supplier information, inbound and outbound time and quantity of raw materials (such as PCB boards, chips, solder paste); equipment operation records refer to time-series data collections collected in SCADA or MES systems that describe the process parameter settings, sensor readings and alarm logs of production equipment (such as printing presses, pick-and-place machines, reflow ovens) within a specific time period.

[0050] Specifically, the system establishes a distributed retrieval engine, using the "production order number" from the intermediate data generated from S101 as a globally unique index key. The retrieval engine first traverses the supply chain data partitions, extracting all bills of materials (BOMLists) marked with that production order number and their corresponding actual material issuance records. Simultaneously, it traverses the equipment operation data partitions, extracting all equipment status snapshots and process parameter waveforms belonging to the execution period of that production order number. Through this step, the system aggregates heterogeneous data that was originally scattered across different business subsystems but logically belonged to the same production task into a single context, forming "data packets" to be aligned.

[0051] It should be noted that, since outlier filtering and missing value imputation are performed on the original multi-source heterogeneous data in step S101, the original structure of the data may be changed, resulting in a large number of equipment operation records being unable to directly match the original "production work order number" field.

[0052] In some preferred embodiments, for equipment records with mismatched production work order numbers, this embodiment employs a completion strategy based on time window overlap. The specific execution steps are as follows: First, the system extracts the actual running time period of the abnormal record; then, it queries the Production Execution System (MES) to retrieve all valid work orders and their planned execution times for the specified machine within that time period; next, it calculates the time overlap between the equipment record's running time period and the planned time periods of each valid work order. Finally, based on preset matching rules, the equipment record is assigned to the production work order number that matches other machine IDs and has the highest time overlap rate (e.g., overlap exceeding 90%), thereby achieving accurate repair and association of work order information.

[0053] S1022. Calculate the intersection-combination ratio of the material flow time window of the supply chain material records and the processing operation time window of the equipment operation records on a unified time axis.

[0054] Among them, the material flow time window refers to the time span from material issuance to consumption; the processing operation time window refers to the start and end time span of the equipment executing the work order task; the intersection-union ratio is used to quantify the degree of overlap between two time periods in the time domain.

[0055] Specifically, although S1021 performs a coarse match using the production work order number, since work orders may involve multiple material requisitions and processing at different times, relying solely on the production work order number may lead to incorrect matching of material records from historical dates to equipment records for the current date. The system parses the time attributes of the two sets of records retrieved by S1021, constructing their respective time intervals. Then, on a unified timeline aligned by S102 or S205, the system calculates the intersection duration (overlapping portion) and union duration (total span) of these two intervals, dividing them to obtain the intersection-union ratio (OCR). This OCR directly reflects the synchronicity between the material lifecycle and the equipment operation cycle: the closer the OCR is to 1, the higher the temporal alignment and the more reliable the correlation.

[0056] S1023. Filter out record pairs with an intersection-union ratio greater than a preset association threshold, map supply chain material records to material entity vectors, and map equipment operation records to equipment status vectors.

[0057] The preset association threshold is an empirical value between 0 and 1, used to filter data with time misalignment; mapping refers to using deep learning feature extraction technology to convert the original data records into dense vectors of fixed dimensions.

[0058] Specifically, for the retained high-quality paired data, the system calls a pre-trained graph embedding model. For supply chain records, features such as material ID, supplier, and batch number are extracted and input into the embedding layer to obtain a "material entity vector"; for equipment records, features such as set parameters (recipe), average temperature, and rotational speed are extracted and processed through a fully connected layer or LSTM encoder to obtain an "equipment state vector". This step transforms discrete structured data into a mathematical vector representation that can be understood by a computer.

[0059] S1024. Calculate the cosine similarity between the material entity vector and the equipment state vector in the vector space, create edges between vectors whose cosine similarity satisfies the alignment condition, and obtain a multidimensional association knowledge graph.

[0060] Cosine similarity is an indicator that measures how close two vectors are in a direction; an edge refers to a directed relationship connecting two nodes in a graph database, usually with attributes (such as weight, timestamp).

[0061] As can be seen, by calculating the intersection-union ratio (IUR) of supply chain material records and equipment operation records within a time window, a quantified temporal overlap index is introduced. This allows for the rapid initial screening of record pairs that highly match in time, filtering out obviously irrelevant data. Furthermore, by mapping these pairs to entity vectors and calculating cosine similarity, semantic verification of this association is performed in vector space. Edges are only created in the graph when both temporal overlap (IUR) and content similarity (cosine similarity) conditions are met. Compared to single-timestamp associations, this reduces the probability of erroneous associations caused by temporal coincidences.

[0062] S103. Identify and remove edges in the multidimensional association knowledge graph that do not satisfy the logical constraint relationship based on industrial time sequence logical constraints.

[0063] Among them, industrial temporal logic constraints refer to objective and inviolable sequential or dependency rules in the electronic manufacturing process, such as "solder paste must be printed before chip mounting can be performed"; pruning refers to the operation of deleting edges or nodes that do not meet the conditions in graph theory algorithms; edges are used in knowledge graphs to represent the relationship between two entity nodes, such as "material A flows into process B".

[0064] Specifically, although the preceding steps establish connections through time alignment, incorrect connections may arise due to sensor drift, human error, or system clock fluctuations. The system loads a pre-built rule engine, which embeds the standard process matrix for the SMT production line. The system traverses each edge in the graph, checking the time and type attributes of the nodes connecting the two ends. For example, if an edge is found connecting a "reflow soldering" node to a "printing" node, and the timestamp shows reflow soldering earlier than printing, this clearly violates the logical constraint of "forward flow." In this case, the algorithm determines the edge as a "false association" and performs pruning to physically remove it from the graph. This mechanism-based post-processing is more effective than simple data cleaning in eliminating deeper-level business errors.

[0065] It should be noted that, in actual production, some parts will inevitably need to be reworked due to inspection and adjustment. Therefore, simple time sequence constraints may mistakenly delete these real rework data.

[0066] In some preferred embodiments, when constructing graph nodes, the system scans for "rework" markers or specific error codes in the MES system. When a PCB board ID is detected to be associated with a rework record, its node is tagged with a special label. During logical constraint checks, the rule engine first determines the node's label attribute. For tagged nodes, the system loads a lenient set of "rework logic constraints" (allowing specific process reflows, such as returning to the repair station after AOI inspection). Edges are only removed if they violate both the normal sequence and the rework logic. This ensures both the verification of the normal process and the integrity of the rework samples.

[0067] In some specific embodiments, step S103 specifically includes:

[0068] S1031. Analyze the pre-set industrial process flow matrix to obtain the positive dependency order between process nodes.

[0069] Among them, the industrial process flow matrix refers to the standardized production process definition stored in the form of an adjacency matrix or a directed acyclic graph (DAG), which describes the necessary steps and sequence of a product from raw materials to finished products; the positive dependency sequence refers to the time or logical constraint that process A must be completed before process B.

[0070] Specifically, the system reads the product process BOM and Route files stored in the PLM or MES system. The parsing engine converts these files into a standard numerical matrix or graph structure. In this structure, nodes represent standard processes, and directed edges represent flow directions. The system traverses this structure, establishing a "predecessor-successor" index table that explicitly records the legitimate upstream and downstream processes for each process. This index table serves as the benchmark for logical judgments made by S1032.

[0071] In actual use, due to the existence of multiple process paths and optional procedures, the dependency relationship between procedures is not a single linear chain, and simple sequential comparison can easily lead to misjudgment.

[0072] In some preferred embodiments, a built-in mechanism is incorporated to identify "parallel branches" and "process skipping" logic. When constructing dependencies, the system is not limited to generating a single standard process chain, but rather constructs a "complete set of paths" covering all legal flows through a traversal algorithm. For example, the system allows the path "process A reaches process D via process B" and the path "process A reaches process D via process C" to coexist as valid references. In the subsequent data verification stage, as long as the relationships in the data to be tested conform to any path description in this complete set of paths, it is determined to be valid.

[0073] S1032. Traverse the edges in the multidimensional relational knowledge graph and locate the edges whose connection order violates the positive dependency order and does not satisfy the logical constraint relationship.

[0074] S1033. Perform a topology disconnection operation on edges that do not satisfy logical constraints.

[0075] The topology disconnect operation refers to physically deleting the edge at the graph database level, or marking the edge's attributes as "unavailable / obsolete" so that it will no longer be effective in subsequent graph traversals or queries.

[0076] It is evident that by utilizing a pre-defined industrial process flow matrix to obtain the positive dependency order between process nodes, traversing the edges in the graph, and locating connections that violate this order, it is possible to identify erroneous associations that, while appearing normal in terms of data values, conflict in business logic. Performing a topology disconnection operation on these edges that do not satisfy logical constraints is a deep logical cleansing at the graph level, ensuring that the final generated corpus follows the objective process laws of electronic manufacturing. This enables the AI ​​model trained on the corpus to learn correct causal logic, rather than erroneous noisy associations.

[0077] S104. Calculate the confidence score of the nodes of the multidimensional association knowledge graph based on the data integrity index; and add metadata tags to the nodes to encapsulate and generate a basic corpus.

[0078] Among them, data integrity indicators refer to quantitative standards that measure the completeness of the information contained in a data node, such as the missing rate of key fields and the stability of collection frequency; confidence score is a normalized value (usually 0-100 or 0-1) used to characterize the credibility of the data node; metadata tags refer to descriptive information attached to the graph node.

[0079] Specifically, this step is the process of "quality rating" of the corpus. The system traverses the remaining nodes in the graph after logical cleaning, calls the quality assessment algorithm for each node, calculates a comprehensive score, and finally the entire graph is serialized and packaged to form a basic corpus with a quality specification.

[0080] In some specific embodiments, a multi-factor weighted scoring strategy based on the Analytic Hierarchy Process (AHP) is employed. The system constructs a hierarchical model for data quality assessment, including four dimensions: completeness, timeliness, accuracy, and relevance. Experts are invited to score the relative importance of each dimension, constructing a judgment matrix and calculating weight vectors (e.g., completeness weight 0.4, relevance weight 0.3, etc.). For each graph node, sub-scores for each dimension are calculated, and the final confidence score is obtained through linear weighted summation. Based on the score range (e.g., >0.9 for Grade A, 0.7-0.9 for Grade B), corresponding metadata tags are automatically mapped and added; this is not limited here.

[0081] S105. Based on the demand data of downstream artificial intelligence models, extract the basic corpus to generate a training set.

[0082] Among them, the demand data refers to the configuration file that describes the input feature specifications required by the target AI model (such as yield prediction model, equipment life prediction model), including a specific list of sensor IDs, time window length and label type; the training set refers to the formatted data tensor extracted from the corpus that can be directly consumed by machine learning algorithms.

[0083] Specifically, the system receives requirement configuration files submitted by downstream algorithm engineers. The parsing engine reads the file, performs subgraph queries in the basic corpus, and quickly locates nodes that meet the criteria and their neighbors. At this point, the system automatically filters out low-quality nodes based on the minimum quality threshold specified in the configuration. Finally, the filtered graph data is flattened into matrices or tensors, the labels are aligned, and the training set is packaged and output.

[0084] As can be seen, by using threshold rules to filter obvious outliers and employing generative adversarial networks to fill in missing data, the cleaned data retains the key information of the nonlinear fluctuation characteristics of the original sensor data while achieving high integrity. Subsequently, a multi-dimensional relational knowledge graph is constructed through spatiotemporal benchmark alignment, enabling cross-domain fusion of originally discrete supply chain and equipment data. Further industrial time-series logical constraint removal operations can clean up erroneous associations based on business logic, ensuring the accuracy of the graph structure at the logical level. Finally, through confidence scoring and metadata tagging, quantitative management of data quality is achieved. This not only solves the problem of difficult multi-source heterogeneous data fusion but also constructs a high-quality corpus with data integrity, logical consistency, and quality traceability, improving the feature expression capabilities and credibility of specialized corpora in the electronics manufacturing field.

[0085] In some specific embodiments, step S105 specifically includes:

[0086] S1051. Parse the model scenario configuration file in the requirement data and extract the required feature dimension list and label field definition.

[0087] Among them, the model scenario configuration file refers to a predefined structured document that describes the specific training objectives and input / output specifications of the model; the feature dimension list clarifies the set of specific variable names required for the model input layer; and the label field definition specifies the target variable for prediction and its data source attributes.

[0088] Specifically, when a new training task request is received, the parser reads the corresponding configuration file. First, it identifies the scenario type and loads the corresponding parsing template. Then, it extracts the feature array configuration, mapping the business logic names within it to the specific attribute names of nodes or edges in the knowledge graph. Simultaneously, the parsing system extracts the label definitions, determining the task type (e.g., classification or regression) and the specific source path of the label data. Through this step, the business-level requirement description is transformed into database-level query instructions.

[0089] In some specific embodiments, an automatic mapping strategy based on a meta-model is adopted. Only abstract business concepts need to be specified in the configuration file. The parser automatically retrieves all attribute fields with that semantic meaning in the graph by querying the metadata management system and expands them into a complete feature list. This is not limited here.

[0090] S1052. Index candidate graph nodes and their associated neighboring edges that contain a list of feature dimensions in the basic corpus.

[0091] Candidate graph nodes refer to entity nodes that contain the required feature attributes; neighboring edges refer to topological edges that have a direct or indirect connection with the node, used to introduce associated contextual information.

[0092] Specifically, based on the feature list parsed in the previous step, the system constructs a graph query statement. The query engine performs a retrieval operation in the basic corpus, filtering out all nodes that have the specified label and contain the target attribute. Simultaneously, to enrich the contextual semantics of the features, the query logic typically includes a path expansion clause, retrieving other entity nodes associated with the candidate nodes as well. The system ultimately returns a unique set of identifiers for these nodes and edges, forming a candidate data pool.

[0093] S1053. Read the metadata tags carried by the candidate graph nodes, compare the confidence scores with the preset model training quality threshold, filter and remove nodes with scores lower than the quality threshold to obtain the preferred feature sub-graph.

[0094] Among them, the model training quality threshold is the minimum data quality score requirement defined in the configuration file; the preferred feature subgraph is a subset of graph data that contains only nodes and edges that meet the quality requirements after screening.

[0095] S1054. Extract the adjacency matrix and flatten the feature tensor of the selected feature subgraph, align the data according to the label field definition, and output the training set.

[0096] Among them, adjacency matrix extraction aims to transform the topological structure of the graph into a mathematical matrix representation; feature tensor flattening aims to serialize node attributes into standard vectors or multidimensional arrays; and data label alignment is to associate and match target label values ​​with corresponding feature data.

[0097] Specifically, the system iterates through and optimizes the feature subgraphs. For graph neural network models, the system constructs matrices representing connectivity relationships and node attributes, directly outputting standard graph data format. For traditional neural network models, the system performs a flattening operation, starting from the central node and aggregating the features of its neighboring nodes to form a fixed-dimensional feature vector. Subsequently, based on the previously defined label fields, the system retrieves label values ​​from the associated detection or maintenance records, combines the feature data with the label values, and writes them into a standard format file as the final training set.

[0098] As can be seen, the system can parse configuration files and extract targeted feature dimensions according to the specific needs of downstream AI models, realizing on-demand data customization. By indexing candidate nodes in the graph and reading the confidence scores in their metadata tags, the system can automatically filter out low-quality data nodes based on the model training quality threshold, retaining only the preferred feature subgraphs with high confidence, which facilitates the improvement of AI model training performance.

[0099] In actual use, there is a significant physical transmission lag between the generation of supply chain material data in step S102 (time T1) and the generation of actual equipment consumption data (time T2). Especially under the "non-stop material receiving" process, the old and new material trays are physically connected end-to-end. Although the system records show that the work order or batch has changed, the equipment is still actually consuming the leftover material from the previous batch. In this case, rigidly merging data according to a "unified time axis" will cause a "physical time lag mismatch": the system will incorrectly associate the attributes of the newly received material at time T1 with the parameters of the currently running equipment, resulting in the logical fallacy of "new material based on old material." This alignment error directly leads to the knowledge graph constructing false causal connections, misleading the model to attribute defects caused by old material to the characteristics of new material, causing subsequent fault attribution analysis to completely fail.

[0100] Please see Figure 2 , Figure 2 This is another flowchart illustrating the method for constructing a multimodal corpus in the field of electronic information manufacturing, as described in this application.

[0101] In some preferred embodiments, other implementations of step S102 include:

[0102] S201. Extract state change events from intermediate state data and calculate the timing response difference of state change events in different data streams.

[0103] Intermediate state data refers to a collection of multi-source data that has been cleaned, desensitized, and initially formatted, possessing a unified timestamp format and work order field, but has not yet undergone physical time delay correction; status change events refer to signals that have clear business transition significance in the production process, such as a work order status changing from "issued" to "executed," or an equipment status changing from "stopped" to "running"; timing response difference is used to represent the time deviation between the recording time of the same business action (such as starting production of a work order) in the information flow and the actual execution time in the logistics / equipment flow. This deviation is usually caused by physical logistics transmission, buffer backlog, or network latency.

[0104] Specifically, the system traverses intermediate data to identify paired key events (i.e., identical business actions) that span the supply chain data flow and the equipment operation data flow. For example, the system extracts the time of "Work Order A is issued" from the supply chain data and simultaneously extracts the time of "First piece of Work Order A is produced" from the equipment data. Through a simple time subtraction operation, the difference between the two times is calculated. This difference is not an error, but rather reflects the actual lag in the physical flow of that production batch.

[0105] Please see Figure 3 , Figure 3 yes Figure 2 A flowchart of step S201;

[0106] In some specific embodiments, step S201 specifically includes:

[0107] S2011. Perform continuity detection on the device operation data stream in the intermediate state data.

[0108] Among them, continuity detection refers to the process of determining whether the equipment is in a continuous operation state or an intermittent / stopped state; the equipment operation data stream includes real-time parameters such as the equipment's spindle speed, status code, and heartbeat signal.

[0109] Specifically, this step aims to identify the preconditions for the special scenario of "material change without stopping the machine." The system reads the equipment's operating parameters in real time and sets a sliding window (e.g., 1 minute). Within the window, it checks whether key indicators (such as spindle speed) are always greater than zero, or whether the Run flag of the control system is always True. If the detection results show that the equipment is in a continuous operating state without any shutdown interruption, the system will mark this period as "continuous production state." This is because the hidden timing misalignment only occurs when material is changed while the machine is running; if the machine stops, the processing logic is different (Example 4).

[0110] S2012. If the result of continuous detection is a continuous production status, monitor the status of the work order register.

[0111] The work order register is a memory address in the PLC used to store the number of the currently executing task.

[0112] Specifically, once continuous production is confirmed, the system enters "monitoring mode." The system frequently monitors the work order register address of the equipment. During normal production, this value remains constant. Once this value changes, it indicates that although the machine is still running (physically continuous), the task has changed (logically switched). This step is to capture the moment when "non-stop material change" occurs.

[0113] S2013. When a jump in the value of the work order register is detected, extract the timestamp and mark it as a logic switching anchor point.

[0114] Among them, the logical switching anchor point represents the exact moment of task change at the information system level, and it is a benchmark endpoint for calculating time difference.

[0115] Specifically, once a valid change in the production work order number is confirmed (e.g., from A to B), the system immediately records the time of this event and marks it as the logical switch anchor point. This point in time represents "the system believes a new work order has started," but physically, because there is still old material between the feeder and the nozzle, the new material has not actually begun to be consumed. This anchor point is the minuend used in subsequent calculations of the "lag time."

[0116] S2014. Retrieve material error-proofing scan records in the supply chain data stream that are associated with the values ​​after the work order register changes, and extract the scan loading anchor points in the material error-proofing scan records.

[0117] Among them, the material error prevention scanning record refers to the log of the operator scanning the code to confirm the material being put on the machine; the scanning loading anchor point indicates the moment when the material is ready in the supply chain.

[0118] Specifically, after obtaining the "new production work order number" (e.g., work order B) from S2013, the system uses this as an index to query the error-proofing scan log of the MES system. It finds the earliest material loading record under that work order, extracts its operation time, and marks it as the scan loading anchor point. This time represents the moment the material is installed on the machine. Since the material change is done without stopping the machine, this action usually occurs before the work order switch (the worker receives the material in advance).

[0119] S2015. Calculate the time difference between the logic switching anchor point and the scanning loading anchor point to obtain the timing response difference.

[0120] Here, the timing response difference specifically refers to the buffer time between the material being ready in the supply chain and the system logic accepting it in a continuous production mode without stopping.

[0121] Specifically, the system performs a mathematical subtraction: subtracting the scanning and loading anchor point from the logical switching anchor point. This difference has a clear physical meaning: it represents the time that materials spend "queuing" and waiting at the production line. In a non-stop scenario, this difference is the quantitative manifestation of physical lag. Subsequent steps will use this difference to "postpone" the scanning and loading time or the equipment running time, thereby aligning the two.

[0122] As can be seen, a feature extraction mechanism was specifically designed for the scenario of material changeover without stopping the system. Continuous detection locks in the ongoing production state, and the logic switching anchor point is captured at the instant the work order register changes, representing the system's apparent switching moment. Simultaneously, the scanning and loading anchor point is determined by retrieving material error-proofing scanning records, representing the material's apparent preparation moment. Calculating the time difference between these two allows us to reconstruct the implicit physical buffer time experienced by materials from scan confirmation to actual consumption in the continuous production flow. This method enables the system to identify the physical boundary between new and old materials hidden in the continuous waveform without stopping the system or interrupting the data flow, thus providing dynamic correction parameters for subsequent spatiotemporal alignment.

[0123] In some other embodiments, after step S2011, the method further includes:

[0124] S2016. When the result of continuous detection is a production interruption state, identify the cold start moment when the equipment status changes from the stop position to the run position.

[0125] Among them, production interruption state refers to the state where equipment is stopped and production is suspended; cold start moment refers to the first instant when equipment recovers from a static state to a running state.

[0126] Specifically, when S2011 detects a production continuity interruption, the system enters the "Stop Processing Branch." The system's goal at this point is to find the starting point for the next production cycle. The system continuously monitors the equipment status. When the status changes from "STOP" to "RUN," or when the spindle speed recovers from 0 to its normal value, the system captures this moment and marks it as a cold start. This signifies the end of the stop-and-go period and the start of a new round of production.

[0127] S2017. Extract the first transmission arrival signal generated after the cold start time and the associated work order register value, and mark the timestamp of the transmission arrival signal as the physical anchor point.

[0128] Among them, the transmission arrival signal refers to the signal detected by the sensor that the PCB board has entered the work area; the physical anchor point indicates the physical moment when the first board of the new work order actually begins processing.

[0129] Specifically, machine startup doesn't mean immediate surface mount technology (SMT) placement; there may be initialization steps. The system needs to locate the "first board" at that time. The system listens for board arrival sensor signals. The time of the first sensor signal captured after the cold start is the physical anchor point. Simultaneously, the production work order number at that moment is read to confirm which task has started.

[0130] S2018. Based on the work order register value, retrieve the corresponding production line material preparation completion record in the supply chain data stream and extract the timestamp as the logical ready anchor point.

[0131] Among them, the production line material preparation completion record refers to the record of material issuance from the warehouse and confirmation that there are no errors; the logical ready anchor point indicates the moment when the supply chain is ready.

[0132] Specifically, use the production work order number obtained from S2017 to query the WMS or MES logs. Find the "Material Preparation Completed" or "Verification Passed" record for that work order. This time is usually much earlier than the machine startup time (because the machine is stopped during the changeover). This anchor point represents the logical start.

[0133] S2019. Calculate the time difference between the physical anchor point and the logical ready anchor point to obtain the timing response difference.

[0134] Here, the timing response difference represents the invalid waiting time during the downtime and line changeover.

[0135] Specifically, the system performs a subtraction: physical anchor points minus logically ready anchor points. This difference represents how long the machine was idle before it started running, even though the materials were ready. In subsequent alignment, this difference will be used to eliminate this window period, directly shifting the physical production data to after logical readiness, eliminating the time gap caused by downtime, and maintaining the compactness of the corpus.

[0136] As can be seen, for scenarios involving production interruptions (shutdowns and line changes), the cold start moment is captured by utilizing the signal indicating the equipment status transitions to the running position, and the physical anchor point is determined by identifying the first transmission signal to indicate the actual start of the equipment's physical actions. Comparing this with the logical ready anchor point in the retrieved production line material preparation completion record, the calculated time difference directly reflects the non-productive waiting time during the shutdown and line changeover period. This mechanism logically separates the invalid and redundant waiting data generated during downtime from the valid data of formal production. Aligning with this as a benchmark eliminates the interference of this time gap on the continuity of the corpus's time series, ensuring that the constructed graph only contains associated data from valid production cycles, thus improving the purity and effective information density of the corpus under discontinuous production modes.

[0137] S202. Determine whether the timing response difference is less than the preset physical transmission hysteresis threshold.

[0138] The physical transmission delay threshold is a pre-set empirical time value that represents the normal logistics buffer limit of the production line. If the difference is less than this threshold, it usually means there is no physical backlog, only a system processing delay; if it is greater than this threshold, it means that there is a substantial physical queuing or transmission issue.

[0139] S203. If the data is less than the preset physical transmission hysteresis threshold, then the intermediate data is aligned to a spatiotemporal reference using the production work order number and the unified time axis as dual primary keys.

[0140] S204. If it is not less than the preset physical transmission hysteresis threshold, a dynamic compensation time window is generated. The unified time axis is shifted and corrected using the dynamic compensation time window to obtain the timing correction time axis.

[0141] Among them, the dynamic compensation time window is a parameter that includes a time shift, used to offset physical lag; the time-series correction time axis refers to the new time series generated after mathematical operations, where each time point represents the "logic occurrence time after backtracking".

[0142] Specifically, when a significant physical lag is detected, the system activates its core compensation mechanism. Based on the timing response difference calculated by S201, the system generates a dynamic compensation time window. Then, the system performs a transformation operation on each original timestamp in the equipment's operational data stream: the new timestamp equals the original timestamp minus the difference. The physical meaning of this operation is: although the machine consumed the material at 10:30, because the material was dispatched at 10:00 (a 30-minute lag), the machine's data time is "corrected" by 30 minutes, thus logically forcibly balancing the time difference. The resulting series of new timestamps constitutes the timing correction timeline.

[0143] S205. Using the production work order number and the time-series correction time axis as dual primary keys, perform spatiotemporal reference alignment operation on intermediate state data.

[0144] The spatiotemporal reference alignment operation here is based on the association performed using the modified time reference.

[0145] Specifically, this step represents the alignment endpoint in a "loosely coupled scenario." After the "time correction" in S204, the previously misaligned equipment data has been aligned with the synchronized supply chain data on the timeline. At this point, the system invokes the alignment engine, using the production work order number and the corrected timeline for correlation. Because the physical lag error has been eliminated by the algorithm, the correlation is now highly accurate: the supply chain's outbound records correspond to the (originally lagging) equipment processing parameters. The system merges these two sets of data to generate a logically consistent record.

[0146] It is evident that by calculating the temporal response difference of state change events in different data streams, the physical lag caused by logistics buffering between the time of supply chain data generation and the time of physical execution by equipment can be quantified. By determining whether the difference exceeds the physical transmission lag threshold, the system can distinguish between real-time synchronization scenarios and physical lag scenarios. For lag scenarios, a dynamic compensation time window is used to adaptively shift and correct the unified time axis, eliminating the misalignment problem caused by rigid time alignment. This ensures the authenticity of the correlation between material batches and equipment operating parameters in the graph, avoids false associations where material quality problems in the previous work order are incorrectly attributed to the current equipment, and improves the spatiotemporal alignment accuracy of the corpus in complex production scenarios.

[0147] The following describes an exemplary multimodal corpus construction system 400 in the field of electronic information manufacturing provided by embodiments of this application. Figure 4 This is an exemplary hardware structure diagram of a multimodal corpus construction system 400 in the field of electronic information manufacturing provided in this application embodiment.

[0148] In some embodiments, the multimodal corpus construction system 400 in the field of electronic information manufacturing is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0149] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0151] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0152] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0153] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for constructing a multimodal corpus in the field of electronic information manufacturing, characterized in that, include: Cleaned data is obtained by filtering outliers based on preset threshold rules and imputing missing values ​​based on generative adversarial networks from the original multi-source heterogeneous data. After the cleaned data is de-identified, it is mapped into intermediate data in a unified format; Using production work order number and unified time axis as dual primary keys, spatiotemporal benchmark alignment operation is performed on the intermediate data, wherein vectorized mapping and entity alignment are performed using supply chain dimension and equipment dimension to obtain a multidimensional association knowledge graph of cross-domain entity relationship; Based on industrial time-series logical constraints, identify and remove edges in a multidimensional relational knowledge graph that do not satisfy the logical constraints; Calculate the confidence score of nodes in the multidimensional relational knowledge graph based on data integrity indicators; Metadata tags are added to nodes, and a basic corpus is generated by encapsulating them. Based on the demand data of downstream artificial intelligence models, a training set is generated by extracting the basic corpus.

2. The method according to claim 1, characterized in that, The step of performing spatiotemporal reference alignment on the intermediate data using the production work order number and a unified timeline as dual primary keys specifically includes: Extract state change events from the intermediate state data and calculate the timing response difference of the state change events in different data streams; Determine whether the timing response difference is less than a preset physical transmission hysteresis threshold; If the value is less than the preset physical transmission hysteresis threshold, then the intermediate data is subjected to spatiotemporal reference alignment operation using the production work order number and the unified time axis as dual primary keys. If it is not less than the preset physical transmission hysteresis threshold, a dynamic compensation time window is generated, and the unified time axis is shifted and corrected using the dynamic compensation time window to obtain the timing correction time axis. Using the production work order number and the time-series correction time axis as dual primary keys, a spatiotemporal reference alignment operation is performed on the intermediate state data.

3. The method according to claim 2, characterized in that, The step of extracting state change events from the intermediate state data and calculating the time-series response differences of the state change events in different data streams specifically includes: Continuity detection is performed on the device operation data stream in the intermediate state data; If the continuous monitoring result indicates a continuous production status, monitor the status of the work order register; When a jump in the work order register value is detected, the timestamp is extracted and marked as a logical switching anchor point. Retrieve material error prevention scanning records in the supply chain data stream that are associated with the value of the work order register after a jump, and extract the scanning loading anchor points in the material error prevention scanning records; The timing response difference is obtained by calculating the time difference between the logic switching anchor point and the scanning loading anchor point.

4. The method according to claim 3, characterized in that, After the step of performing continuous detection on the device operation data stream in the intermediate state data, the method further includes: In the case of continuous detection indicating a production interruption, identify the cold start moment when the equipment status changes from the stopped position to the running position; Extract the first transmission arrival signal generated after the cold start time and the associated work order register value, and mark the timestamp of the transmission arrival signal as a physical anchor point; Based on the work order register value, the corresponding production line material preparation completion record is retrieved from the supply chain data stream, and the timestamp is extracted as the logical ready anchor point. The timing response difference is obtained by calculating the time difference between the physical anchor point and the logical ready anchor point.

5. The method according to claim 1 or 2, characterized in that, The step of performing spatiotemporal reference alignment on the intermediate data using the production work order number and a unified timeline as dual primary keys specifically includes: Based on the production work order number, retrieve supply chain material records and equipment operation records with the same work order identifier from the intermediate state data; Calculate the intersection-union ratio (IUU) of the material flow time window of the supply chain material records and the processing operation time window of the equipment operation records on the same time axis; Filter out record pairs with an intersection-union ratio greater than a preset association threshold, map the supply chain material records to material entity vectors, and map the equipment operation records to equipment status vectors; Calculate the cosine similarity between the material entity vector and the equipment state vector in the vector space, create edges between vectors whose cosine similarity satisfies the alignment condition, and obtain the multidimensional association knowledge graph.

6. The method according to claim 1, characterized in that, The step of identifying and removing edges in the multidimensional association knowledge graph that do not satisfy the logical constraints based on industrial time-series logical constraints specifically includes: Analyze the pre-set industrial process flow matrix to obtain the positive dependency order between process nodes; Traverse the edges in the multidimensional association knowledge graph and locate the edges whose connection order violates the positive dependency order and does not satisfy the logical constraint relationship; Perform a topology disconnection operation on edges that do not satisfy logical constraints.

7. The method according to claim 1, characterized in that, The step of extracting the basic corpus to generate a training set based on the demand data of the downstream artificial intelligence model specifically includes: Parse the model scenario configuration file in the requirement data to extract the required feature dimension list and label field definition; The basic corpus indexes candidate graph nodes and their associated neighboring edges that contain the list of feature dimensions; Read the metadata tags carried by the candidate map nodes, compare the confidence scores with the preset model training quality threshold, and filter out nodes with scores lower than the quality threshold to obtain the preferred feature sub-map. The preferred feature subgraph is subjected to adjacency matrix extraction and feature tensor flattening, and data is labeled and aligned according to the label field definition to output the training set.

8. A multimodal corpus construction system in the field of electronic information manufacturing, characterized in that, The multimodal corpus construction system in the field of electronic information manufacturing includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the multimodal corpus construction system in the field of electronic information manufacturing to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on a multimodal corpus construction system in the field of electronic information manufacturing, the multimodal corpus construction system in the field of electronic information manufacturing performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a multimodal corpus construction system in the field of electronic information manufacturing, the multimodal corpus construction system in the field of electronic information manufacturing performs the method as described in any one of claims 1-7.

Images