Collaborative analysis method for optical chip yield based on multi-modal knowledge graph

Abstract

The application discloses a kind of based on multimodal knowledge graph's light chip yield collaborative analysis method, comprising: process parameter record, optical detection image and electrical test signal are respectively structured and handled, and process parameter entity, defect entity and test entity are constructed;With batch number and chip coordinate association each entity, construct multimodal knowledge graph including process sequence, parameter membership, defect generation, test evaluation and cross-modal equivalence relation;Through modal specificity encoder, each entity is embedded, and graph attention is enhanced based on intra-modal subgraph;According to cross-modal equivalence relation, calculate semantic consistency and determine fusion weight, generate multimodal fusion embedding by gate fusion;Rule, path and probability reasoning are executed and conflict resolution, and output yield prediction value and influence path.The application realizes the deep correlation of cross-modal data and explainable reasoning, improves the accuracy and root cause positioning ability of yield analysis.

Description

Technical Field

[0001] This invention relates to the field of optical chip technology, and specifically to a collaborative analysis method for optical chip yield based on multimodal knowledge graphs. Background Technology

[0002] Optical chip manufacturing involves multiple precision processes such as epitaxy, photolithography, and etching. Its final yield is influenced by a combination of factors, including process parameters, defect morphology, and electrical performance. In existing yield analysis workflows, data from different modalities—such as process parameter records, optical inspection images, and electrical test signals—are typically collected and stored independently by different systems and analyzed separately by engineers in their respective fields. This results in fragmented information across different stages, making it difficult to form a complete cognitive chain from process deviations to defect generation and performance degradation. When yield anomalies occur, repeated cross-departmental communication and trial-and-error are often required to pinpoint the possible causes. While existing technologies have attempted to integrate multimodal data analysis methods, these typically involve simply concatenating or weighting the feature vectors of different modalities. This fails to effectively distinguish the information differences and complementary relationships between different modalities when describing the same object, resulting in limited information representation capabilities after fusion. Summary of the Invention

[0003] In view of the above problems, the present invention provides a collaborative analysis method for optical chip yield based on multimodal knowledge graph. By constructing a knowledge graph that associates data from different modalities and adaptively fusing information from each modality, it achieves deep collaboration across modalities and root cause localization of yield.

[0004] To achieve the above objectives, this application provides a collaborative analysis method for optical chip yield based on multimodal knowledge graphs, including: Acquire multimodal raw data during the optical chip manufacturing process. The multimodal raw data includes process parameter records, optical detection images, and electrical test signals. The process parameter records are structured to construct process parameter entities; Defect regions are segmented and feature vectors are extracted from optical inspection images to construct defect entities; Feature point detection and time-series feature vector extraction are performed on electrical test signals to construct test entities; A global anchor point identifier is formed by using the batch number and chip coordinates to associate the process parameter entity, defect entity and test entity with the same chip object; Based on process parameter entities, defect entities, and test entities, a multimodal knowledge graph is constructed. The multimodal knowledge graph includes process entities, process parameter entities, defect entities, and test entities, as well as process sequence relationships, parameter membership relationships, defect generation relationships, test evaluation relationships, and cross-modal equivalence relationships. A modality-specific encoder is used to embed process parameter entities, defect entities, and test entities respectively, generating embedded representations of text modality, image modality, and time sequence modality; Based on the embedding representation, an intramodal subgraph is constructed for each modality, and a graph attention network is used for message passing to generate an enhanced representation. For entity groups with cross-modal equivalence relations, the semantic consistency between the enhanced representations of different modalities is calculated, the fusion weight of each modality is determined based on the semantic consistency, and a gating fusion mechanism is used to generate multimodal fusion embedded representations. Perform yield reasoning on a multimodal knowledge graph. Yield reasoning includes rule reasoning, path reasoning, and probabilistic reasoning to obtain the reasoning results. The inference results are conflict-resolved, credibility weights are assigned based on historical verification accuracy, and yield predictions and impact paths are generated through weighted fusion.

[0005] Furthermore, the process parameter records are structured to construct process parameter entities, including: Extract process parameter records, which include epitaxial growth temperature curves, photolithography exposure dose values, etching gas flow rate and RF power, deposition chamber pressure and temperature, and annealing temperature curves. The process parameter records are digitized to generate numerical attribute values ​​for each parameter. Construct parameter membership relationships by using process name as the header entity and parameter name as the tail entity, and process parameter entity as the attribute. Adding parameter membership relationships to the multimodal knowledge graph enables process parameter entities to establish associations with process entities in the multimodal knowledge graph through parameter membership relationships; The entity identifier is composed of the process name and the parameter name. The process parameter entity is constructed with numerical attribute values. Add process parameter entities to the multimodal knowledge graph, and establish cross-modal equivalence relationships between process parameter entities, defect entities, and test entities through global anchor point identifiers.

[0006] Furthermore, defect region segmentation and feature vector extraction are performed on the optical inspection image to construct the defect entity, including: Acquire optical inspection images, perform noise reduction and illumination correction preprocessing on the optical inspection images, and generate preprocessed images; Calculate the gray-level histogram of the preprocessed image, identify the valley positions in the gray-level histogram, and use the gray-level value corresponding to the valley position as the segmentation threshold to divide the preprocessed image into high gray-level regions and low gray-level regions. Gradient magnitude is calculated in the preprocessed image, and pixels with gradient magnitude exceeding a preset gradient threshold are identified as edge pixels. The edge pixels are then connected to form a closed contour. Perform region intersection operation between high grayscale regions and closed contours, and extract regions that simultaneously meet the high grayscale condition and the closed contour condition as candidate defect regions. Obtain the contour pixel set of the candidate defect region, fit the contour pixel set with the minimum bounding rectangle, extract the length and width of the minimum bounding rectangle, and calculate the ratio of the length to the width as the aspect ratio. The total number of pixels contained in the statistical contour pixel set is used as the area of ​​the defect region; Calculate the diameter of the circle with the same area as the defect region as the equivalent circle diameter; The standard deviation of the distance from each contour pixel in the contour pixel set to the fitted contour curve is calculated as the edge roughness index. The area of ​​the defect region, its aspect ratio, the diameter of the equivalent circle, and the edge roughness index are concatenated to generate a geometric feature vector; Obtain the set of pixel gray values ​​within the candidate defect region, construct the gray-level co-occurrence matrix of the pixel gray value set, and calculate the contrast statistics, correlation statistics, energy statistics, and uniformity statistics of the gray-level co-occurrence matrix. Contrast statistics, correlation statistics, energy statistics, and uniformity statistics are concatenated to generate a texture feature vector. The geometric feature vector and the texture feature vector are concatenated to generate the attribute vector of the defective entity. The pixel coordinates of the candidate defect region in the optical inspection image are converted into the two-dimensional coordinate position of the chip on the wafer. The two-dimensional coordinate positions are used to form entity identifiers to construct defect entities. The attributes of the defect entities are attribute vectors. Add defect entities to the multimodal knowledge graph and establish cross-modal equivalence relationships between defect entities, process parameter entities, and test entities through global anchor point identification.

[0007] Furthermore, feature point detection and time-series feature vector extraction are performed on the electrical test signal to construct the test entity, including: Acquire electrical test signals, perform baseline drift correction and noise filtering on the electrical test signals, and generate preprocessed signals; The IV characteristic curve in the preprocessed signal is obtained, the first derivative of the IV characteristic curve is calculated, and the voltage position where the first derivative first reaches the preset positive threshold is identified as the threshold voltage point. The second derivative of the IV characteristic curve is calculated, and the voltage position where the second derivative reaches its maximum value is identified as the peak point of the differential resistance. Optical power-current mapping is performed on the IV characteristic curve to identify the current position where the optical power reaches the saturation inflection point as the saturation current point. The threshold voltage point, the peak point of the differential resistance, and the saturation current point are taken as the set of IV characteristic points; The spectral response curve in the preprocessed signal is obtained, and peak detection is performed on the spectral response curve to identify the wavelength position where the spectral intensity reaches the maximum value as the peak wavelength point. Search outwards from the peak wavelength point to both sides, and identify the left and right wavelength positions where the spectral intensity drops to half the peak intensity as the endpoints of the full width at half maximum (FWHM) interval. Calculate the spectral intensity difference between the peak wavelength point and the adjacent second-peak wavelength point, and identify the wavelength position where the spectral intensity difference reaches the maximum value as the extreme point of the side-mode suppression ratio; The peak wavelength point, the endpoint of the full width at half maximum (FWHM) interval, and the extreme point of the side mode suppression ratio are taken as the set of spectral feature points; The IV feature point set and the spectral feature point set are merged to generate a feature point set; the x-coordinate and y-coordinate values ​​of each feature point in the feature point set are extracted. Piecewise linear fitting is performed on adjacent feature points in the feature point set, and the slope value of each fitted line is calculated. The horizontal axis value, vertical axis value, and slope value are concatenated to generate the temporal feature vector of the test entity; The unique chip number is used to form an entity identifier, and a test entity is constructed. The attributes of the test entity are time-series feature vectors. Add the test entity to the multimodal knowledge graph and establish cross-modal equivalence relationships between the test entity, process parameter entity, and defect entity through global anchor point identification.

[0008] Furthermore, a global anchor point identifier is formed using the batch number and chip coordinates to associate process parameter entities, defect entities, and test entities with the same chip object, including: Obtain the wafer batch number corresponding to the process parameter entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the defect entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the test entity and the two-dimensional coordinate position of the chip on the wafer. The wafer batch number and the two-dimensional coordinate position are concatenated to generate a global anchor point identifier. The global anchor point identifier is a combination of the wafer batch number string and the two-dimensional coordinate position string. Assign the global anchor point identifier to the associated attribute of the process parameter entity, assign the global anchor point identifier to the associated attribute of the defect entity, and assign the global anchor point identifier to the associated attribute of the test entity. Traverse all entities in the multimodal knowledge graph, identify process parameter entities, defect entities, and test entities with the same global anchor identifier, establish cross-modal equivalence relations between process parameter entities and defect entities, between process parameter entities and test entities, and between defect entities and test entities. The cross-modal equivalence relations use the global anchor identifier as the relation attribute.

[0009] Furthermore, based on process parameter entities, defect entities, and test entities, a multimodal knowledge graph is constructed, including: Obtain process parameter entities, defect entities, and test entities; obtain process entities in the optical chip manufacturing process. Define the process sequence relationship, which is used to connect adjacent process entities, with the order in which the processes are executed as the direction of the relationship. Define parameter membership relationships, which are used to connect process entities and process parameter entities, with the process entity as the head entity and the process parameter entity as the tail entity. Define the defect generation relationship, which is used to connect the process entity and the defect entity, with the process entity as the head entity and the defect entity as the tail entity. Define a test evaluation relationship, which is used to connect the test entity and the yield level entity, with the test entity as the head entity and the yield level entity as the tail entity. Define cross-modal equivalence relations, which are used to connect process parameter entities, defect entities, and test entities that have the same global anchor point identifier; Add process entities, process parameter entities, defect entities, and test entities as entity nodes to the knowledge graph; Obtain the process timing dependencies between process entities, and establish process sequence relationship edges between process entities with dependencies; Obtain the parameter membership relationship between the process entity and the process parameter entity, and establish parameter membership relationship edges between the process entity and the process parameter entity; Obtain the defect generation relationship between the process entity and the defect entity, and establish a defect generation relationship edge between the process entity and the defect entity; Obtain the test evaluation relationship between the test entity and the yield level entity, and establish test evaluation relationship edges between the test entity and the yield level entity; obtain process parameter entities, defect entities and test entities with the same global anchor point identifier, establish cross-modal equivalence relationship edges between process parameter entities and defect entities, establish cross-modal equivalence relationship edges between process parameter entities and test entities, establish cross-modal equivalence relationship edges between defect entities and test entities, and generate a multimodal knowledge graph.

[0010] Furthermore, a modality-specific encoder is used to embed process parameter entities, defect entities, and test entities respectively, generating embedded representations of text modality, image modality, and time-series modality, including: Obtain the numerical attribute vector of the process parameter entity. The numerical attribute vector contains the splicing result of temperature statistical features, dose feature values, etching features and deposition features. The numerical attribute vector is input into a fully connected neural network. The fully connected neural network contains an input layer, multiple hidden layers and an output layer connected in sequence. The number of neurons in the input layer is equal to the dimension of the numerical attribute vector. The hidden layer performs a non-linear transformation on the numerical attribute vector. The output layer outputs a vector of the first preset dimension. The output of the output layer is used as a text modality embedding representation. Obtain the attribute vector of the defective entity. The attribute vector contains the concatenation result of the geometric feature vector and the texture feature vector. The attribute vector is input into the residual network, which contains an initial convolutional layer, multiple residual blocks, and a global average pooling layer connected in sequence. The initial convolutional layer performs a convolution operation on the attribute vector to generate an initial feature map. Each residual block performs an identity mapping and a convolution mapping superposition operation on the initial feature map to generate a residual feature map. The global average pooling layer performs a global average pooling operation on the residual feature map output by the last residual block to generate a vector of the first preset dimension. The output of the global average pooling layer is used as the image modality embedding representation. Obtain the temporal feature vector of the test entity. The temporal feature vector contains the concatenation result of the abscissa value, ordinate value and slope value of the feature point and the adjacent point. The temporal feature vector is input into a gated recurrent network. The gated recurrent network contains an input layer, multiple gated recurrent units, and a fully connected mapping layer connected in sequence. The input layer expands the temporal feature vector by time step. Each gated recurrent unit calculates the hidden state of the current time step based on the input of the current time step and the hidden state of the previous time step. The fully connected mapping layer maps the hidden state of the last time step to a vector of the first preset dimension. The output of the fully connected mapping layer is used as the temporal modality embedding representation.

[0011] Furthermore, based on the embedding representation, an intramodal subgraph is constructed for each modality, and a graph attention network is used for message passing to generate enhanced representations, including: Obtain text modality embedding representations, image modality embedding representations, and temporal modality embedding representations; Obtain the process sequence relationship and parameter membership relationship in the multimodal knowledge graph. Use the text modal embedding representation corresponding to the process entity with process sequence relationship as text subgraph node, and use the text modal embedding representation corresponding to the process entity and process parameter entity with parameter membership relationship as text subgraph node. Establish text subgraph edges between text subgraph nodes with process sequence relationship and between text subgraph nodes with parameter membership relationship to generate text modal subgraph. Obtain cross-modal equivalence relations in the multimodal knowledge graph, use the image modality embedding representation corresponding to the defect entity with cross-modal equivalence relations as the image subgraph node, obtain the two-dimensional coordinate position of the defect entity on the wafer, calculate the Euclidean distance between the two-dimensional coordinate positions corresponding to any two image subgraph nodes, establish image subgraph edges between image subgraph nodes whose Euclidean distance is less than a preset distance threshold, and generate image modality subgraph. Obtain cross-modal equivalence relations in the multimodal knowledge graph. Use the temporal modality embedding representation corresponding to the test entity with cross-modal equivalence relations as the temporal subgraph node. Obtain the chip batch number and the row index of the chip on the wafer corresponding to the test entity. Select the temporal subgraph nodes with the same batch number and adjacent row indices as candidate node pairs. Establish temporal subgraph edges between candidate node pairs to generate a temporal modality subgraph. The text modality subgraph, image modality subgraph, and temporal modality subgraph are input into the graph attention network. The graph attention network calculates the attention coefficient between the target node embedding representation and the embedding representation of each neighboring node for each target node. The neighboring node embedding representations are weighted and summed according to the attention coefficients. The weighted summation result is fused with the target node embedding representation to generate the enhanced representation of each target node. The enhanced representation includes text modality enhanced representation, image modality enhanced representation, and temporal modality enhanced representation.

[0012] Furthermore, for entity groups with cross-modal equivalence relations, the semantic consistency degree between the enhanced representations of different modalities is calculated. The fusion weights of each modality are determined based on the semantic consistency degree, and a gating fusion mechanism is used to generate a multimodal fusion embedding representation, including: Obtain text modal augmentation representations, image modal augmentation representations, and temporal modal augmentation representations; obtain cross-modal equivalence relations in multimodal knowledge graphs; and identify text modal augmentation representations, image modal augmentation representations, and temporal modal augmentation representations with cross-modal equivalence relations as entity groups. The cosine similarity between the text modal augmentation representation and the image modal augmentation representation in the entity group is calculated as the first consistency score; the cosine similarity between the text modal augmentation representation and the temporal modal augmentation representation is calculated as the second consistency score; and the cosine similarity between the image modal augmentation representation and the temporal modal augmentation representation is calculated as the third consistency score. The vector entropy value of the text modality augmentation representation is calculated as the text modality information entropy; the vector entropy value of the image modality augmentation representation is calculated as the image modality information entropy; and the vector entropy value of the temporal modality augmentation representation is calculated as the temporal modality information entropy. The first consistency score, the second consistency score, and the third consistency score are weighted and averaged to generate a cross-modal consistency score. The text modal information entropy is divided by the sum of the text modal information entropy, the image modal information entropy, and the temporal modal information entropy to generate the text modal gating parameters; The image modal information entropy is divided by the sum of the text modal information entropy, the image modal information entropy, and the temporal modal information entropy to generate the image modal gating parameters; The temporal modality entropy is divided by the sum of the text modality entropy, the image modality entropy, and the temporal modality entropy to generate the temporal modality gating parameters; The cross-modal consistency score is multiplied by the text modality gating parameter to generate the text modality fusion coefficient; The cross-modal consistency score is multiplied by the image modality gating parameters to generate image modality fusion coefficients; The cross-modal consistency score is multiplied by the temporal modality gating parameter to generate the temporal modality fusion coefficient; The text modality enhancement representation is multiplied by the text modality fusion coefficient to generate a text modality weighted representation; The image modality enhancement representation is multiplied by the image modality fusion coefficients to generate the image modality weighted representation; The temporal modality augmentation representation is multiplied by the temporal modality fusion coefficients to generate a temporal modality weighted representation; The text modality weighted representation, image modality weighted representation, and temporal modality weighted representation are added element-wise to generate a multimodal fusion embedding representation.

[0013] Furthermore, yield reasoning is performed on the multimodal knowledge graph. Yield reasoning includes rule-based reasoning, path-based reasoning, and probabilistic reasoning, yielding reasoning results, including: Obtain the process parameter entity attributes and defect entity attributes from the multimodal knowledge graph, and obtain the rule base. The rule base contains multiple process rules stored in the form of triples. Each process rule includes the premise, reasoning conclusion and confidence level. The matching degree of the process parameter entity attributes and defect entity attributes is calculated with the preconditions of each process rule. When the matching degree exceeds the preset matching threshold, the process rule is triggered. The reasoning conclusion of the triggered process rule is extracted as the rule reasoning conclusion, and the confidence degree of the triggered process rule is extracted as the rule reasoning confidence degree. Obtain the yield level entity from the multimodal knowledge graph. Starting from the yield level entity, perform a breadth-first search in the reverse direction of the defect generation relation edge, parameter membership relation edge and process sequence relation edge. During the search, set the upper limit of path length and the path semantic relevance threshold. Only retain paths whose path length does not exceed the upper limit of path length and whose path semantic relevance is not lower than the path semantic relevance threshold as candidate impact paths. Calculate the weight product of each relation edge on each candidate impact path as the path semantic strength. Take the candidate impact path whose path semantic strength exceeds the preset strength threshold as the path reasoning conclusion and use the path semantic strength as the path reasoning confidence. Obtain the set of entity types and the set of relation types in the multimodal knowledge graph. Construct a conditional probability graphical model with entity type as node type and relation type as edge type. Obtain the joint distribution frequency between entity attributes and yield level in historical manufacturing data. Train the conditional probability table of each type of edge based on the joint distribution frequency. The entity attributes corresponding to the multimodal fusion embedding representation of the current batch are used as evidence nodes to input the conditional probability graph model. The posterior probability distribution of each yield level is calculated through the belief propagation algorithm. The yield level with the highest posterior probability is extracted as the probabilistic inference conclusion, and the posterior probability value corresponding to the yield level with the highest posterior probability is extracted as the probabilistic inference confidence. The conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are combined to generate a reasoning result. The inference results are conflict-resolved, and credibility weights are assigned based on historical verification accuracy. A weighted fusion is then used to generate predicted yield values ​​and impact paths, including: Obtain the conclusion and confidence level of rule-based reasoning, obtain the conclusion and confidence level of path-based reasoning, and obtain the conclusion and confidence level of probability-based reasoning. Determine whether the conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are consistent. If they are consistent, the conclusion of rule-based reasoning is used as the predicted yield, and the candidate influencing paths in the path-based reasoning conclusion are used as the influencing paths. If the three are inconsistent, proceed with the next steps; Obtain the historical verification dataset, which contains multiple historical inference records. Each historical inference record includes an inference layer identifier, inference conclusion, actual result, and verification accuracy. The historical verification accuracy corresponding to the rule reasoning layer is extracted from the historical verification dataset as the rule reasoning credibility weight, the historical verification accuracy corresponding to the path reasoning layer is extracted as the path reasoning credibility weight, and the historical verification accuracy corresponding to the probability reasoning layer is extracted as the probability reasoning credibility weight. Multiply the rule-based reasoning conclusion by the rule-based reasoning credibility weight to generate a weighted rule-based reasoning conclusion; Multiply the path reasoning conclusion by the path reasoning credibility weight to generate a weighted path reasoning conclusion; Multiply the probabilistic reasoning conclusion by the probabilistic reasoning credibility weight to generate a weighted probabilistic reasoning conclusion; The weighted conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are added element by element to generate a predicted yield value. Obtain the set of candidate impact paths in the path reasoning conclusion. Each candidate impact path contains the entity sequence on the path and the path semantic strength. The path semantic strength of each candidate influence path is multiplied by the path reasoning credibility weight to generate the weighted path semantic strength of each candidate influence path. The candidate impact path set is sorted from high to low according to the weighted path semantic strength, and the top preset number of candidate impact paths in the sorting results are extracted as impact paths.

[0014] Unlike existing technologies, the above technical solution provides a collaborative analysis method for optical chip yield based on a multimodal knowledge graph. This method includes: structuring process parameter records, optical inspection images, and electrical test signals to construct process parameter entities, defect entities, and test entities; associating each entity with batch number and chip coordinates to construct a multimodal knowledge graph containing process sequence, parameter membership, defect generation, test evaluation, and cross-modal equivalence relations; embedding each entity using a modality-specific encoder and performing graph attention enhancement based on intramodal subgraphs; calculating semantic consistency and determining fusion weights based on cross-modal equivalence relations, and generating multimodal fusion embeddings through gated fusion; executing rule, path, and probabilistic reasoning and conflict resolution, and outputting the predicted yield value and influencing paths. This invention achieves deep association and interpretable reasoning of cross-modal data, improving the accuracy of yield analysis and root cause localization capabilities.

[0015] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0016] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of the present invention and other related contents, and should not be considered as limitations on this application.

[0017] In the accompanying drawings of the instruction manual: Figure 1 This is a schematic diagram illustrating steps S101 to S111 of the method described in the specific implementation embodiment; Figure 2 This is a schematic diagram illustrating steps S201 to S206 of the method described in a specific implementation. Figure 3 This is a schematic diagram illustrating steps S301 to S308 of the method described in a specific implementation. Figure 4 This is a schematic diagram illustrating steps S401 to S406 of the method described in a specific embodiment; Figure 5 The diagram illustrates steps S501 to S505 of the method described in the specific implementation. Detailed Implementation

[0018] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0019] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0020] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0021] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0022] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0023] Without further limitations, the use of terms such as “comprising,” “including,” “having,” or other similar open-ended expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0024] The processor described in the embodiments of this application can be implemented by hardware, firmware, software, or a combination thereof. It can be a circuit, one or more of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or a microprocessor. It also includes other physical, biological, or chemical structures that can implement the same or equivalent functions as the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in the various embodiments of this application, or any combination of the steps mentioned therein.

[0025] The computer program involved in the embodiments can be stored in a computer device readable storage medium, which includes, but is not limited to, disks, magnetic tapes, magnetic cards, floppy disks, flash memory, optical disks, optical cards, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM), and electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical, or chemical structures that can achieve the same or equivalent functions as the storage media listed above, such as DNA, RNA, proteins, and other units with information storage capabilities. In specific embodiments, the storage medium involved can be one of the above-mentioned media types, or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiments can be centrally stored in a single medium, or distributed and stored in multiple media. The memory containing the computer device readable storage medium can be non-volatile memory or random access memory. These computer device readable storage media can be built into the device, or can be connected to the device involved in the embodiments as an external device or part of an external device. In some embodiments, the memory having a computer device readable storage medium is deployed locally; in other embodiments, the memory may be deployed remotely from the processor, for example, as a network-attached memory accessed via RF circuitry or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), or a suitable combination thereof, as long as computer device access to the memory is enabled. Furthermore, the computer program involved in the embodiments may be stored in plaintext / ciphertext form, or it may be designed as training data, integrated and recombined through model training and implicitly stored in the parameter states of a deep neural network or other machine learning model.

[0026] Please see Figure 1 This embodiment provides a collaborative analysis method for optical chip yield based on multimodal knowledge graphs, including: S101. Acquire multimodal raw data during the optical chip manufacturing process. The multimodal raw data includes process parameter records, optical detection images, and electrical test signals. S102. Perform structured processing on the process parameter records to construct process parameter entities; S103. Perform defect region segmentation on the optical inspection image and extract feature vectors to construct defect entities; S104. Perform feature point detection on the electrical test signal and extract the time-series feature vector to construct the test entity; S105. Use batch number and chip coordinates to form a global anchor point identifier, and associate the process parameter entity, defect entity and test entity with the same chip object. S106. Based on process parameter entities, defect entities, and test entities, construct a multimodal knowledge graph. The multimodal knowledge graph includes process entities, process parameter entities, defect entities, and test entities, as well as process sequence relationships, parameter membership relationships, defect generation relationships, test evaluation relationships, and cross-modal equivalence relationships. S107. A modality-specific encoder is used to embed the process parameter entity, defect entity, and test entity respectively, generating embedded representations of text modality, image modality, and time sequence modality. S108. Construct intramodal subgraphs for each modality based on embedded representations, and use graph attention networks for message passing to generate enhanced representations; S109. For entity groups with cross-modal equivalence relations, calculate the semantic consistency between the enhanced representations of different modalities, determine the fusion weight of each modality based on the semantic consistency, and use a gating fusion mechanism to generate multimodal fusion embedded representations. S110. Perform yield reasoning on a multimodal knowledge graph. Yield reasoning includes rule reasoning, path reasoning, and probability reasoning to obtain the reasoning result. S111. Conflict resolution is performed on the reasoning results. Credibility weights are assigned based on historical verification accuracy. The yield prediction value and influence path are generated through weighted fusion.

[0027] In step S101, process parameter records can be extracted from the historical database of the manufacturing execution system, recording the equipment settings and measured values ​​for each process; optical inspection images are obtained by scanning the wafer surface with an automated optical inspection device at a specific wavelength and resolution; electrical test signals are generated by applying a bias voltage to the chip through a probe station and collecting the response current.

[0028] In step S102, the structured processing transforms unstructured text or tabular records into a set of numerical attributes with a unified format by parsing the field labels and values ​​in the process parameter records. The process parameter entity is a graph node carrying this set of numerical attributes, and its identifier consists of both the process name and the parameter name.

[0029] In step S103, defect region segmentation is based on the grayscale distribution and gradient change features of the optical inspection image. Through a joint operation of thresholding and edge detection, regions that may contain defects are separated from the background. Feature vectors are extracted from the segmented regions, encompassing quantitative indicators describing the region's geometry and surface texture. The defect entity is identified by this feature vector as an attribute and by the region's location information on the wafer.

[0030] In step S104, feature point detection identifies local extrema, inflection points, or abrupt slope changes in the characteristic curve of the electrical test signal, extracting inflection points or extrema with clear physical significance. The timing feature vector is formed by concatenating the coordinates of these feature points and the rates of change between their adjacent points. The test entity uses this timing feature vector as an attribute and is identified by a unique chip number.

[0031] In step S105, the global anchor identifier is generated by concatenating the wafer batch number and the chip's location information on the wafer into a string, and is a unique identification code that identifies a chip object. By assigning this identifier as an association attribute to each entity, different modal entities with the same identifier can be quickly retrieved and associated in subsequent steps, achieving cross-modal data alignment.

[0032] In step S106, the multimodal knowledge graph uses process entities, process parameter entities, defect entities, and test entities as nodes. Process sequence relationships establish directed edges between process entities based on process timing; parameter membership relationships point process parameter entities to their respective process entities; defect generation relationships point defect entities to their generating process entities; test evaluation relationships point test entities to yield grade entities; and cross-modal equivalence relationships establish bidirectional edges between different modal entities with the same global anchor identifier.

[0033] In step S107, the modality-specific encoder employs different architectures for the data characteristics of the three modalities. The text modality embedding representation is generated by a fully connected network that processes numerical attribute vectors; the image modality embedding representation is generated by a residual network that processes defect attribute vectors; and the temporal modality embedding representation is generated by a gated recurrent network that processes temporal feature vectors. All three embedding representations are mapped to the same preset dimension, making the data from different modalities comparable in a unified semantic space.

[0034] In step S108, the intramodal subgraph is constructed independently for each modality, with its nodes consisting of the embedded representation of that modality, and edges established based on relationships or spatial proximity between entities in the knowledge graph. The graph attention network calculates the attention coefficients between the target node and its neighboring nodes, weightedly aggregating neighborhood information, so that the embedded representation of each node is integrated into its local structural context, generating a context-aware enhanced representation.

[0035] In step S109, for entity groups associated through cross-modal equivalence relations, the semantic consistency degree is quantified by calculating the vector similarity between different modal augmented representations, reflecting the alignment degree of each modality in describing the same object. The gated fusion mechanism dynamically calculates the fusion weight of each modality based on this semantic consistency degree and the information richness of each modal augmented representation, performing weighted aggregation of the augmented representations of each modality to generate a multimodal fusion embedding representation.

[0036] In step S110, rule-based reasoning triggers corresponding rules and outputs reasoning conclusions by matching entity attributes with preconditions in a preset rule base. The preset rule base can be written by domain experts based on the physical mechanisms and experience of optical chip manufacturing processes. Path reasoning performs a breadth-first search along the relation edges in the knowledge graph to uncover causal paths from yield level to potential root causes. Probabilistic reasoning calculates the posterior probability of each yield level in the current state based on a conditional probability model trained on historical data.

[0037] In step S111, conflict resolution is used to handle situations where the three inference conclusions are inconsistent. The historical verification accuracy is obtained statistically from the inference records of past batches, reflecting the reliability of each inference method. This accuracy is used as a confidence weight to perform a weighted fusion of the three inference conclusions, and finally outputs the predicted yield value. At the same time, the candidate path with the highest weighted semantic strength is selected from the path inference results as the influencing path.

[0038] This embodiment constructs a multimodal knowledge graph containing processes, process parameters, defects, and test entities, unifying the scattered heterogeneous data in the optical chip manufacturing process into a structured knowledge representation. It uses a modality-specific encoder and graph attention network to generate enhanced representations for each modality, and achieves adaptive fusion through a gating fusion mechanism based on cross-modal equivalence relations. Finally, through the collaborative execution and conflict resolution of rules, paths, and probabilistic reasoning, it outputs the predicted yield value and the influencing path, realizing interpretable collaborative analysis and root cause localization of optical chip yield.

[0039] Please see Figure 2 In some embodiments, the process parameter records are structured to construct process parameter entities, including: S201. Extract process parameter records, which include epitaxial growth temperature curves, photolithography exposure dose values, etching gas flow rate and RF power, deposition chamber pressure and temperature, and annealing temperature curves. S202. Perform numerical processing on the process parameter records to generate numerical attribute values ​​corresponding to each parameter. S203. Construct parameter membership relationships by using process name as the head entity and parameter name as the tail entity. S204. Add the parameter membership relationship to the multimodal knowledge graph so that the process parameter entity can establish a relationship with the process entity in the multimodal knowledge graph through the parameter membership relationship; S205. Construct a process parameter entity by combining the process name and parameter name as the entity identifier. The attributes of the process parameter entity are numerical attribute values. S206. Add the process parameter entity to the multimodal knowledge graph, and establish cross-modal equivalence relationships between the process parameter entity, the defect entity, and the test entity through global anchor point identification.

[0040] In step S201, the epitaxial growth temperature curve and annealing temperature curve can be obtained from the continuous monitoring log of the equipment temperature sensor, recording the temperature change trajectory throughout the entire process cycle; the photolithography exposure dose value is read from the process recipe file of the photolithography machine; the etching gas flow rate and RF power are extracted from the real-time monitoring system of the etching equipment; and the deposition chamber pressure and temperature are obtained from the process parameter records of the deposition equipment.

[0041] In step S202, the numerical processing adopts different methods for different types of parameters: for continuous time-series data such as temperature curves, the mean, variance and first-order difference mean of the window can be calculated after sampling at a fixed time window, and then concatenated into numerical attribute values; for discrete numerical parameters such as photolithography exposure dose value, etching gas flow rate and RF power, deposition chamber pressure and temperature, their set values ​​and measured values ​​can be directly extracted as numerical attribute values.

[0042] In steps S203 to S204, parameter membership relationships are used to express the semantics of "a certain process has a certain parameter," and the process parameter entity carries the specific numerical information of that parameter. The parameter membership relationships are added to the multimodal knowledge graph in the form of edges. Process parameter entities establish associations with corresponding process entities through these edges, enabling rapid retrieval of all subordinate process parameter entities and their numerical attributes from the process entity, thus providing graph structure support for root cause analysis at the parameter level.

[0043] In steps S205 and S206, entity identifiers are generated by concatenating process names and parameter names into strings, ensuring that each process parameter entity is unique across the entire graph. The attributes of this entity are assigned as numerical attribute value vectors. The global anchor identifiers carried by the process parameter entity, defect entity, and test entity are all generated by combining wafer batch number and chip coordinates. When the global anchor identifiers carried by the three entities are consistent, cross-modal equivalence relationships are established between the process parameter entity and the defect entity, between the process parameter entity and the test entity, and between the defect entity and the test entity, respectively, so that process parameters, defect morphology, and electrical test results are associated with the same physical chip.

[0044] This embodiment generates numerical attribute values ​​for each parameter by quantifying multiple types of process parameter records; constructs process parameter entities by combining process names and parameter names as entity identifiers, and associates them with corresponding process entities through parameter membership relationships; finally, establishes cross-modal equivalence relationships between process parameter entities, defect entities, and test entities through global anchor point identifiers, thereby realizing the structured expression and cross-modal association of process parameter information in the optical chip manufacturing process.

[0045] Please see Figure 3 In some embodiments, defect region segmentation and feature vector extraction are performed on the optical inspection image to construct the defect entity, including: S301. Acquire the optical detection image, perform noise reduction and illumination correction preprocessing on the optical detection image, and generate a preprocessed image. S302. Calculate the gray-level histogram of the preprocessed image, identify the valley positions in the gray-level histogram, and use the gray-level value corresponding to the valley position as the segmentation threshold to divide the preprocessed image into high gray-level regions and low gray-level regions. Gradient magnitude is calculated in the preprocessed image, and pixels with gradient magnitude exceeding a preset gradient threshold are identified as edge pixels. The edge pixels are then connected to form a closed contour. S303. Perform region intersection operation between the high grayscale region and the closed contour, and extract the region that simultaneously satisfies the high grayscale condition and the closed contour condition as the candidate defect region. S304. Obtain the contour pixel set of the candidate defect region, fit the contour pixel set with the minimum bounding rectangle, extract the length and width of the minimum bounding rectangle, and calculate the ratio of the length to the width as the aspect ratio. The total number of pixels contained in the statistical contour pixel set is used as the area of ​​the defect region; Calculate the diameter of the circle with the same area as the defect region as the equivalent circle diameter; The standard deviation of the distance from each contour pixel in the contour pixel set to the fitted contour curve is calculated as the edge roughness index. The area of ​​the defect region, its aspect ratio, the diameter of the equivalent circle, and the edge roughness index are concatenated to generate a geometric feature vector; S305. Obtain the set of pixel gray values ​​inside the candidate defect region, construct the gray-level co-occurrence matrix of the pixel gray value set, and calculate the contrast statistics, correlation statistics, energy statistics and uniformity statistics of the gray-level co-occurrence matrix. Contrast statistics, correlation statistics, energy statistics, and uniformity statistics are concatenated to generate a texture feature vector. S306. Concatenate the geometric feature vector and the texture feature vector to generate the attribute vector of the defective entity. S307. Convert the pixel coordinates of the candidate defect region in the optical inspection image into the two-dimensional coordinate position of the chip on the wafer, form an entity identifier with the two-dimensional coordinate position, construct the defect entity, and the attributes of the defect entity are attribute vectors. S308. Add the defect entity to the multimodal knowledge graph and establish cross-modal equivalence relationships between the defect entity, process parameter entity, and test entity through global anchor point identification.

[0046] In step S301, the denoising process can employ Gaussian filtering. The kernel size is determined by a noise estimation algorithm based on the scale characteristics of the image noise. Specifically, the gray-level variance of local image regions can be calculated, and the median of each local variance can be used as an estimate of the noise level. The size of the filter kernel is then deduced from this estimate. The illumination correction preprocessing can employ homomorphic filtering. Its cutoff frequency is set based on the spatial frequency distribution of uneven illumination. For example, the cutoff frequency can be determined by analyzing the energy concentration range of low-frequency components after performing a Fourier transform on the image. This is used to eliminate gray-level deviations caused by uneven illumination and generate a preprocessed image.

[0047] In step S302, the trough position is located by finding the zero-crossing point where the first derivative of the gray-level histogram curve changes from positive to negative. The gray value corresponding to this position is used as the segmentation threshold. The preset gradient threshold is determined based on the cumulative distribution of the gradient magnitude histogram. Specifically, the gradient magnitude of the entire image can be calculated first and its histogram can be statistically analyzed. Then, the magnitude corresponding to the cumulative distribution reaching the preset percentile is taken as the threshold. Pixels exceeding this threshold are identified as edge pixels, and then adjacent edge pixels are connected to form a closed contour through an edge connection algorithm.

[0048] In step S303, the region intersection operation first generates a binary mask for the high grayscale region and an internal filling mask for the closed contour. Then, a pixel-by-pixel logical AND operation is performed on the two masks, and only the set of pixels that simultaneously meet the two conditions is retained. The extracted region is used as a candidate defect region.

[0049] In step S304, the minimum bounding rectangle fitting can be obtained by using a rotating caliper algorithm to obtain the minimum area rectangle that can enclose the set of contour pixels. The fitted contour curve in the edge roughness index is obtained by smoothing the contour pixels through polynomial fitting or spline interpolation. The order of the polynomial or the number of nodes of the spline can be adaptively set according to the number of contour pixels. Then, the standard deviation of the Euclidean distance from each contour pixel to the fitted curve is calculated.

[0050] In step S305, the pixel spacing of the gray-level co-occurrence matrix is ​​set to 1 pixel based on the image resolution. Matrices are constructed in four directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. The average of the contrast, correlation, energy, and uniformity statistics calculated for each direction is then used as the final statistics. The contrast statistics reflect the clarity of the texture; the correlation statistics measure the similarity of textures in row or column directions; the energy statistics characterize the uniformity and regularity of the texture; and the uniformity statistics measure the local variation amplitude of the texture.

[0051] In step S306, before splicing, the geometric feature vector and the texture feature vector are normalized to their maximum and minimum values ​​respectively, so that each feature component is in the same numerical range. Then, the two normalized vectors are spliced ​​together end to end in the order of geometric feature first and texture feature last to generate the attribute vector of the defective entity.

[0052] In steps S307 to S308, the conversion from pixel coordinates to the chip's two-dimensional coordinate position on the wafer is calculated based on the resolution parameters of the optical inspection image, i.e., the actual physical size corresponding to each pixel, and the pixel coordinate position of the wafer origin in the image, using a linear mapping formula. Specifically, the wafer coordinates are equal to the product of the pixel coordinates and the resolution parameters plus the origin offset. The converted two-dimensional coordinate positions are used to generate entity identifiers through string formatting, constructing a defect entity. The attributes of this entity are assigned to the attribute vector generated in step S306. The defect entity is added to the multimodal knowledge graph. By matching the global anchor identifier carried by the defect entity with the global anchor identifiers of the process parameter entity and the test entity, a cross-modal equivalence relationship is established when the three are consistent.

[0053] In this embodiment, after denoising and illumination correction preprocessing of the optical inspection image, region segmentation and edge detection are performed based on the valley position of the grayscale histogram and the gradient amplitude threshold, respectively. Candidate defect regions are extracted through region intersection operation. Geometric feature vectors and texture feature vectors are extracted from the candidate defect regions and concatenated into attribute vectors. Defect entities are constructed using the wafer two-dimensional coordinate position obtained by converting pixel coordinates as identifiers. Finally, cross-modal equivalence relationships are established between the defect entities, process parameter entities, and test entities through global anchor point identifiers, realizing the structured extraction and cross-modal association of defect information in the optical chip optical inspection image.

[0054] In some embodiments, feature point detection and time-series feature vector extraction are performed on the electrical test signal to construct a test entity, including: Acquire electrical test signals, perform baseline drift correction and noise filtering on the electrical test signals, and generate preprocessed signals; The IV characteristic curve in the preprocessed signal is obtained, the first derivative of the IV characteristic curve is calculated, and the voltage position where the first derivative first reaches the preset positive threshold is identified as the threshold voltage point. The second derivative of the IV characteristic curve is calculated, and the voltage position where the second derivative reaches its maximum value is identified as the peak point of the differential resistance. Optical power-current mapping is performed on the IV characteristic curve to identify the current position where the optical power reaches the saturation inflection point as the saturation current point. The threshold voltage point, the peak point of the differential resistance, and the saturation current point are taken as the set of IV characteristic points; The spectral response curve in the preprocessed signal is obtained, and peak detection is performed on the spectral response curve to identify the wavelength position where the spectral intensity reaches the maximum value as the peak wavelength point. Search outwards from the peak wavelength point to both sides, and identify the left and right wavelength positions where the spectral intensity drops to half the peak intensity as the endpoints of the full width at half maximum (FWHM) interval. Calculate the spectral intensity difference between the peak wavelength point and the adjacent second-peak wavelength point, and identify the wavelength position where the spectral intensity difference reaches the maximum value as the extreme point of the side-mode suppression ratio; The peak wavelength point, the endpoint of the full width at half maximum (FWHM) interval, and the extreme point of the side mode suppression ratio are taken as the set of spectral feature points; The IV feature point set and the spectral feature point set are merged to generate a feature point set; the x-coordinate and y-coordinate values ​​of each feature point in the feature point set are extracted. Piecewise linear fitting is performed on adjacent feature points in the feature point set, and the slope value of each fitted line is calculated. The horizontal axis value, vertical axis value, and slope value are concatenated to generate the temporal feature vector of the test entity; The unique chip number is used to form an entity identifier, and a test entity is constructed. The attributes of the test entity are time-series feature vectors. Add the test entity to the multimodal knowledge graph and establish cross-modal equivalence relationships between the test entity, process parameter entity, and defect entity through global anchor point identification.

[0055] In this embodiment, baseline drift correction can be performed by estimating the overall trend of the electrical test signal through polynomial fitting and then subtracting it from the original signal. The order of the polynomial is set according to the complexity of the signal drift. Noise filtering can be performed by wavelet denoising, with the wavelet basis function selected according to the signal waveform characteristics. The number of decomposition layers is determined according to the signal sampling rate and the noise frequency band distribution to generate a preprocessed signal.

[0056] The preset positive threshold is determined based on the noise level of the IV characteristic curve in the low-voltage region. Specifically, the standard deviation of the first derivative in the low-voltage region can be calculated, and a multiple of this standard deviation can be taken as the threshold. The voltage position where the first derivative first reaches this threshold is taken as the threshold voltage point. For optical power-current mapping, after the photodetector converts the optical signal output by the chip into an electrical signal, a correspondence table between optical power and injected current is established. This table is then interpolated or fitted to obtain a continuous mapping function. The saturation inflection point where the optical power tends to stabilize with increasing current is found on this mapping function and taken as the saturation current point.

[0057] When performing peak detection on the spectral response curve, each peak is located by finding the point where the first derivative of the curve crosses zero and the second derivative is negative. The peak with the highest spectral intensity is taken as the peak wavelength point. Starting from the peak wavelength point, the process iterates point by point to the left and right, stopping when the spectral intensity first falls below half of the peak intensity, and recording this position as the endpoint of the full width at half maximum (FWHM). Adjacent secondary peak wavelength points are obtained by finding the local maxima of spectral intensity near the main peak (excluding the main peak). The difference in spectral intensity between the peak wavelength point and the secondary peak wavelength point is calculated, and peak detection is then performed on this difference sequence. The wavelength position where the difference reaches a maximum value is identified as the extreme point of the sidemode suppression ratio (SMRR).

[0058] After merging the IV feature point set and the spectral feature point set, the feature points are sorted in ascending order of their x-coordinate values. Two adjacent feature points are then used as a segment for piecewise linear fitting, and the slope of each fitted line is calculated using the least squares method. The x-coordinate value, y-coordinate value, and slope value of each segment are then concatenated in a fixed order to generate the temporal feature vector of the test entity.

[0059] The unique chip identifier can be obtained from the batch record or chip package number during the wafer manufacturing process. Test entities are added to the multimodal knowledge graph. By matching the global anchor identifier carried by the test entity with the global anchor identifiers of the process parameter entity and the defect entity, a cross-modal equivalence relationship is established when all three match.

[0060] This embodiment identifies the threshold voltage point, differential resistance peak point, and saturation current point from the IV characteristic curve after baseline drift correction and noise filtering of the electrical test signal, and identifies the peak wavelength point, full width at half maximum (FWHM) endpoint, and side-mode suppression ratio (SMR) extreme point from the spectral response curve. The two types of feature points are merged and a time-series feature vector is generated by piecewise linear fitting. The test entity is constructed with the chip's unique number as the identifier, and a cross-modal equivalence relationship is established between the test entity, the process parameter entity, and the defect entity through global anchor point identification. This realizes the structured extraction and cross-modal correlation of key features in the electrical test signal of the optical chip.

[0061] In some embodiments, a global anchor point identifier is formed by batch number and chip coordinates, which associates process parameter entities, defect entities, and test entities with the same chip object, including: Obtain the wafer batch number corresponding to the process parameter entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the defect entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the test entity and the two-dimensional coordinate position of the chip on the wafer. The wafer batch number and the two-dimensional coordinate position are concatenated to generate a global anchor point identifier. The global anchor point identifier is a combination of the wafer batch number string and the two-dimensional coordinate position string. Assign the global anchor point identifier to the associated attribute of the process parameter entity, assign the global anchor point identifier to the associated attribute of the defect entity, and assign the global anchor point identifier to the associated attribute of the test entity. Traverse all entities in the multimodal knowledge graph, identify process parameter entities, defect entities, and test entities with the same global anchor identifier, establish cross-modal equivalence relations between process parameter entities and defect entities, between process parameter entities and test entities, and between defect entities and test entities. The cross-modal equivalence relations use the global anchor identifier as the relation attribute.

[0062] In this embodiment, the wafer batch number can be obtained from the batch record of the manufacturing execution system, and the two-dimensional coordinate position of the chip on the wafer can be obtained from the layout design file of the photolithography process or the coordinate record of the probe test. When the process parameter entity, defect entity, and test entity are constructed, their corresponding wafer batch numbers and the two-dimensional coordinate positions of the chip on the wafer have been extracted from their respective data sources and stored as entity attributes. This step can directly read these attributes.

[0063] When concatenating wafer batch numbers with two-dimensional coordinate positions, underscores can be used as delimiters. Before concatenation, the horizontal and vertical coordinates should be formatted, for example, by converting them into strings of a fixed length. If the length is insufficient, zeros should be added to the high bits to ensure that the generated global anchor identifiers have a uniform length and format, which facilitates subsequent string matching and parsing.

[0064] When assigning the global anchor identifier as a relational attribute to each entity, an attribute field named "Global Anchor Identifier" is added to the entity node, and the generated composite string is directly assigned to this field. This assignment operation is performed immediately after each entity is built, ensuring that each entity carries complete relational information before being added to the multimodal knowledge graph.

[0065] During the traversal and matching, all entities in the multimodal knowledge graph are first grouped according to the value of the global anchor identifier, with entities having the same global anchor identifier grouped together. For entities belonging to the three types—process parameter entities, defect entities, and test entities—within each group, cross-modal equivalence relations are established between each pair of entities; that is, a relation edge is established between process parameter entities and defect entities, between process parameter entities and test entities, and between defect entities and test entities. Each cross-modal equivalence relation edge uses the global anchor identifier as a relation attribute; preferably, a timestamp information for establishing the relation can also be attached. This operation can be performed uniformly after all entities are added to the graph, or it can be triggered each time a new entity is added to maintain the real-time consistency of the graph.

[0066] This embodiment obtains the wafer batch number and the two-dimensional coordinate position of the chip on the wafer from the manufacturing execution system and layout design file, splices them in a fixed format to generate a global anchor point identifier and assigns it as the association attribute of each entity. Then, by traversing and matching groups, cross-modal equivalence relationships are established between process parameter entities, defect entities and test entities with the same global anchor point identifier, thereby achieving accurate alignment and unified association of multimodal data in the optical chip manufacturing process.

[0067] In some embodiments, a multimodal knowledge graph is constructed based on process parameter entities, defect entities, and test entities, including: Obtain process parameter entities, defect entities, and test entities; obtain process entities in the optical chip manufacturing process. Define the process sequence relationship, which is used to connect adjacent process entities, with the order in which the processes are executed as the direction of the relationship. Define parameter membership relationships, which are used to connect process entities and process parameter entities, with the process entity as the head entity and the process parameter entity as the tail entity. Define the defect generation relationship, which is used to connect the process entity and the defect entity, with the process entity as the head entity and the defect entity as the tail entity. Define a test evaluation relationship, which is used to connect the test entity and the yield level entity, with the test entity as the head entity and the yield level entity as the tail entity. Define cross-modal equivalence relations, which are used to connect process parameter entities, defect entities, and test entities that have the same global anchor point identifier; Add process entities, process parameter entities, defect entities, and test entities as entity nodes to the knowledge graph; Obtain the process timing dependencies between process entities, and establish process sequence relationship edges between process entities with dependencies; Obtain the parameter membership relationship between the process entity and the process parameter entity, and establish parameter membership relationship edges between the process entity and the process parameter entity; Obtain the defect generation relationship between the process entity and the defect entity, and establish a defect generation relationship edge between the process entity and the defect entity; Obtain the test evaluation relationship between the test entity and the yield level entity, and establish test evaluation relationship edges between the test entity and the yield level entity; obtain process parameter entities, defect entities and test entities with the same global anchor point identifier, establish cross-modal equivalence relationship edges between process parameter entities and defect entities, establish cross-modal equivalence relationship edges between process parameter entities and test entities, establish cross-modal equivalence relationship edges between defect entities and test entities, and generate a multimodal knowledge graph.

[0068] In this embodiment, the process entity can be extracted from the standard process flow file of optical chip manufacturing or read from the process definition table of the manufacturing execution system. The attributes of each process entity include at least the process name, process number and its execution sequence number in the process flow.

[0069] The temporal order in the process sequence relationship is determined based on the arrangement order of each process in the standard process flow document or the process execution timestamp recorded in the manufacturing execution system. The process entity executed first is used as the head entity of the relationship, and the process entity executed later is used as the tail entity of the relationship.

[0070] The yield grade entities are classified according to the preset yield judgment criteria in the chip test results. For example, chips that meet the performance index requirements in the test results can be marked as qualified products, and those that do not meet the requirements can be marked as unqualified products. This serves as the basis for classifying the yield grade entities.

[0071] When adding process entities, process parameter entities, defect entities, and test entities as entity nodes to the knowledge graph, a node is created for each entity using the node creation statement of the graph database, and the attribute fields of the entity are mapped to the attribute values ​​of the node.

[0072] The establishment of all relationship edges can be performed uniformly after all entity nodes have been added. Process sequence dependencies are obtained by parsing the relationships between the preceding and succeeding processes of each process from the standard process flow file, and process sequence relationship edges are established between process entities with dependencies. Parameter membership relationship edges are established based on the correspondence between parameters and processes determined during the construction of the process parameter entity. Defect generation relationship edges are established based on the defect generation process information recorded in the defect entity, which has been extracted from its corresponding process context during defect entity construction. Test evaluation relationship edges are established based on the correspondence between the test entity and the yield grade entity, which has been determined during yield grade classification. Cross-modal equivalence relationship edges are established based on the matching results of global anchor point identifiers, connecting process parameter entities, defect entities, and test entities pairwise. Preferably, the establishment of related edges can be triggered each time a new entity is added to maintain real-time consistency of the graph.

[0073] This embodiment obtains process entities from standard process flow documents and manufacturing execution systems, and defines process sequence relationships, parameter membership relationships, defect generation relationships, test evaluation relationships, and cross-modal equivalence relationships. Process parameter entities, defect entities, and test entities are added as nodes to the knowledge graph, and corresponding relationship edges are established according to the correspondence between each entity. Finally, a complete multimodal knowledge graph containing process, process parameter, defect, and test information is generated, providing a structured knowledge representation foundation for subsequent multimodal fusion and yield reasoning.

[0074] Please see Figure 4 In some embodiments, a modality-specific encoder is used to embed process parameter entities, defect entities, and test entities respectively, generating embedded representations of text modality, image modality, and time-series modality, including: S401. Obtain the numerical attribute vector of the process parameter entity. The numerical attribute vector contains the splicing result of temperature statistical features, dose feature values, etching features and deposition features. S402. Input the numerical attribute vector into a fully connected neural network. The fully connected neural network includes an input layer, multiple hidden layers and an output layer connected in sequence. The number of neurons in the input layer is equal to the dimension of the numerical attribute vector. The hidden layer performs a non-linear transformation on the numerical attribute vector. The output layer outputs a vector of the first preset dimension. The output of the output layer is used as a text modality embedding representation. S403. Obtain the attribute vector of the defective entity. The attribute vector contains the concatenation result of the geometric feature vector and the texture feature vector. S404. Input the attribute vector into the residual network. The residual network contains an initial convolutional layer, multiple residual blocks, and a global average pooling layer connected in sequence. The initial convolutional layer performs a convolution operation on the attribute vector to generate an initial feature map. Each residual block performs an identity mapping and a convolution mapping superposition operation on the initial feature map to generate a residual feature map. The global average pooling layer performs a global average pooling operation on the residual feature map output by the last residual block to generate a vector of the first preset dimension. The output of the global average pooling layer is used as the image modality embedding representation. S405. Obtain the temporal feature vector of the test entity. The temporal feature vector contains the concatenation result of the abscissa value, ordinate value and slope value of the feature point and the adjacent point. S406. Input the temporal feature vector into a gated recurrent network. The gated recurrent network contains an input layer, multiple gated recurrent units, and a fully connected mapping layer connected in sequence. The input layer expands the temporal feature vector by time step. Each gated recurrent unit calculates the hidden state of the current time step based on the input of the current time step and the hidden state of the previous time step. The fully connected mapping layer maps the hidden state of the last time step to a vector of the first preset dimension. The output of the fully connected mapping layer is used as the temporal modality embedding representation.

[0075] In step S401, the temperature statistical features in the numerical attribute vector are derived from the numerical processing results of the epitaxial growth temperature curve and the annealing temperature curve, the dose feature value is derived from the numerical processing results of the photolithography exposure dose value, the etching feature is derived from the numerical processing results of the etching gas flow rate and the radio frequency power, and the deposition feature is derived from the numerical processing results of the deposition chamber pressure and temperature.

[0076] In step S402, the number of hidden layers and the number of neurons per layer in the fully connected neural network can be set according to the dimension of the numerical attribute vector and the amount of training data. Preferably, the optimal configuration can be determined after evaluating the performance of different network structures on the validation set through cross-validation. The first preset dimension refers to the vector dimension jointly output by the output layer of the fully connected neural network, the fully connected layer after the global average pooling layer of the residual network, and the fully connected mapping layer of the gated recurrent network. This dimension needs to be consistent with the input dimension of the subsequent graph attention network to ensure that the embedding representations of different modalities are comparable in a unified space. The specific value of the first preset dimension can be determined comprehensively based on the dimensional range of the input vectors of each modality and the complexity of the downstream task. Before model training, the dimension value that performs best on the validation set can be selected from the preset candidate value set through grid search. When the hidden layer performs nonlinear transformation on the numerical attribute vector, an activation function is connected after each hidden layer to introduce nonlinear expressive power.

[0077] In step S403, the attribute vector of the defective entity is formed by concatenating the geometric feature vector and the texture feature vector in the order of geometric feature first and texture feature last, and its dimension is the sum of the dimensions of the geometric feature vector and the texture feature vector.

[0078] In step S404, the initial convolutional layer of the residual network sets the kernel size according to the local correlation range of the attribute vectors, and the stride is usually set to 1 to maintain the spatial size of the feature map. The number of residual blocks is set according to the complexity of the attribute vectors and the scale of the training data. It can be initialized using residual network parameters pre-trained on a large image dataset, and then fine-tuned on optical chip defect data to accelerate convergence and improve feature extraction capability. The global average pooling layer calculates the average value of all spatial locations for each feature map output by the last residual block, compressing the three-dimensional feature map into a one-dimensional vector. The dimension of this vector is equal to the number of channels of the feature map. It is then mapped to a vector of the first preset dimension through a fully connected layer as an image modality embedding representation.

[0079] In step S405, the temporal feature vector of the test entity is formed by concatenating the abscissa value, ordinate value and slope value of the feature points in the order of the feature points. Its dimension is equal to the product of the number of feature points and the number of values ​​carried by each feature point.

[0080] In step S406, the number of gated recurrent units is set according to the time step length of the temporal feature vector and the complexity of temporal dependencies. Preferably, it can be set to be equal to or appropriately reduced to decrease model complexity. Each gated recurrent unit controls the flow of information by updating and resetting the gates, enabling the network to learn long-term dependencies in the temporal data. The fully connected mapping layer maps the hidden state of the last time step to a vector of the first preset dimension as a temporal modality embedding representation. The gated recurrent network is trained end-to-end using backpropagation algorithm and temporal backpropagation. The preset range in the gradient clipping technique is the upper limit of the gradient norm. This upper limit can be determined according to the statistical distribution of the gradient norm during network training. For example, the sum of the mean and standard deviation of the gradient norm in several iterations in the early stages of training can be taken as the upper limit. When the gradient norm exceeds this upper limit, it is scaled down to the upper limit to prevent gradient explosion.

[0081] This embodiment uses a fully connected neural network to embed the numerical attribute vectors of process parameter entities to generate text modal embedding representations, a residual network to embed the attribute vectors of defect entities to generate image modal embedding representations, and a gated recurrent network to embed the temporal feature vectors of test entities to generate temporal modal embedding representations. All three embedding representations are mapped to the same preset dimension, making data from different modalities comparable in a unified semantic space, thus providing a foundation for subsequent construction of intramodal subgraphs and cross-modal fusion.

[0082] Please see Figure 5 In some embodiments, an intramodal subgraph is constructed for each modality based on the embedded representation, and a graph attention network is used for message passing to generate an enhanced representation, including: S501, Obtain text modality embedding representation, image modality embedding representation, and temporal modality embedding representation; S502. Obtain the process sequence relationship and parameter membership relationship in the multimodal knowledge graph. Use the text modal embedding representation corresponding to the process entity with process sequence relationship as the text subgraph node. Use the text modal embedding representation corresponding to the process entity and process parameter entity with parameter membership relationship as the text subgraph node. Establish text subgraph edges between text subgraph nodes with process sequence relationship and between text subgraph nodes with parameter membership relationship to generate a text modal subgraph. S503. Obtain cross-modal equivalence relations in the multimodal knowledge graph. Use the image modality embedding representation corresponding to the defect entity with cross-modal equivalence relations as the image subgraph node. Obtain the two-dimensional coordinate position of the defect entity on the wafer. Calculate the Euclidean distance between the two-dimensional coordinate positions corresponding to any two image subgraph nodes. Establish image subgraph edges between image subgraph nodes with Euclidean distance less than a preset distance threshold to generate an image modality subgraph. S504. Obtain cross-modal equivalence relations in the multimodal knowledge graph. Use the temporal modality embedding representation corresponding to the test entity with cross-modal equivalence relations as the temporal subgraph node. Obtain the chip batch number and the row index of the chip on the wafer corresponding to the test entity. Use the temporal subgraph nodes with the same batch number and adjacent row indices as candidate node pairs. Establish temporal subgraph edges between candidate node pairs to generate a temporal modality subgraph. S505. Input the text modality subgraph, image modality subgraph, and temporal modality subgraph into the graph attention network. The graph attention network calculates the attention coefficient between the target node embedding representation and the embedding representation of each neighboring node for each target node. The neighboring node embedding representations are weighted and summed according to the attention coefficients. The weighted summation result is fused with the target node embedding representation to generate the enhanced representation of each target node. The enhanced representation includes text modality enhanced representation, image modality enhanced representation, and temporal modality enhanced representation.

[0083] In step S502, the process sequence relationship and parameter membership relationship are obtained by traversing the corresponding type of relationship edges in the multimodal knowledge graph. Each relationship edge carries the identification information of the entities at both ends. Based on the entity identification, the corresponding embedding vector is searched in the generated text modality embedding representation as a text subgraph node. Text subgraph edges are established between nodes with corresponding relationships to generate a text modality subgraph.

[0084] In step S503, the preset distance threshold is set based on the physical spacing of the chips on the wafer. Specifically, it can be calculated by statistically analyzing the average distance between the center points of adjacent chips on the wafer, and taking a multiple of this average distance as the threshold. This ensures that only spatially adjacent defect entities are connected, while allowing for minor deviations caused by uneven chip arrangement. The specific value of the multiple can be determined by cross-validation before model training, based on the regularity of the chip arrangement on the wafer.

[0085] In step S504, the row index of the chip on the wafer is obtained by parsing the two-dimensional coordinate position corresponding to the test entity. The row number can be obtained by dividing the coordinate by the chip row spacing and rounding it. Two timing subgraph nodes with the same batch number and row indexes that differ by 1 are selected as candidate node pairs. Timing subgraph edges are established between the candidate node pairs to generate a timing mode subgraph.

[0086] In step S505, for each target node, the graph attention network first calculates the attention coefficients between the target node embedding representation and the embedding representations of each neighboring node. Specifically, this can be achieved by concatenating the two embedding representations and performing a dot product operation with a learnable weight vector, followed by LeakyReLU activation and Softmax normalization to obtain the final attention coefficients. The neighboring node embedding representations are then weighted and summed based on the attention coefficients. The weighted sum is then element-wise added to the target node embedding representation, and a nonlinear transformation is performed through a fully connected layer to generate the enhanced representation for each target node.

[0087] This embodiment constructs text modality subgraphs, image modality subgraphs, and temporal modality subgraphs respectively, enabling the embedding representation of each modality to obtain contextual information from neighboring nodes in its corresponding subgraph. Through the adaptive weighted aggregation mechanism of the graph attention network, the enhanced representation of each node can be selectively fused according to the importance differences of neighboring nodes, providing richer feature representations for subsequent cross-modal semantic consistency calculation and gating fusion.

[0088] In some embodiments, for a group of entities with cross-modal equivalence relations, the semantic consistency degree between the enhanced representations of different modalities is calculated, the fusion weights of each modality are determined based on the semantic consistency degree, and a gating fusion mechanism is used to generate a multimodal fusion embedded representation, including: Obtain text modal augmentation representations, image modal augmentation representations, and temporal modal augmentation representations; obtain cross-modal equivalence relations in multimodal knowledge graphs; and identify text modal augmentation representations, image modal augmentation representations, and temporal modal augmentation representations with cross-modal equivalence relations as entity groups. The cosine similarity between the text modal augmentation representation and the image modal augmentation representation in the entity group is calculated as the first consistency score; the cosine similarity between the text modal augmentation representation and the temporal modal augmentation representation is calculated as the second consistency score; and the cosine similarity between the image modal augmentation representation and the temporal modal augmentation representation is calculated as the third consistency score. The vector entropy value of the text modality augmentation representation is calculated as the text modality information entropy; the vector entropy value of the image modality augmentation representation is calculated as the image modality information entropy; and the vector entropy value of the temporal modality augmentation representation is calculated as the temporal modality information entropy. The first consistency score, the second consistency score, and the third consistency score are weighted and averaged to generate a cross-modal consistency score. The text modal information entropy is divided by the sum of the text modal information entropy, the image modal information entropy, and the temporal modal information entropy to generate the text modal gating parameters; The image modal information entropy is divided by the sum of the text modal information entropy, the image modal information entropy, and the temporal modal information entropy to generate the image modal gating parameters; The temporal modality entropy is divided by the sum of the text modality entropy, the image modality entropy, and the temporal modality entropy to generate the temporal modality gating parameters; The cross-modal consistency score is multiplied by the text modality gating parameter to generate the text modality fusion coefficient; The cross-modal consistency score is multiplied by the image modality gating parameters to generate image modality fusion coefficients; The cross-modal consistency score is multiplied by the temporal modality gating parameter to generate the temporal modality fusion coefficient; The text modality enhancement representation is multiplied by the text modality fusion coefficient to generate a text modality weighted representation; The image modality enhancement representation is multiplied by the image modality fusion coefficients to generate the image modality weighted representation; The temporal modality augmentation representation is multiplied by the temporal modality fusion coefficients to generate a temporal modality weighted representation; The text modality weighted representation, image modality weighted representation, and temporal modality weighted representation are added element-wise to generate a multimodal fusion embedding representation.

[0089] In this embodiment, before calculating the cosine similarity, the text modality augmentation representation, image modality augmentation representation, and temporal modality augmentation representation are first subjected to L2 normalization so that the magnitude of each augmentation representation is 1. Then, the dot product between the normalized vectors is calculated as the cosine similarity to ensure that the similarity value is not affected by the difference in the magnitude of each modality vector.

[0090] When calculating the vector entropy value, the probability distribution is first obtained by Softmax normalization of the values ​​of each dimension of each augmentation representation. The temperature parameter of Softmax can be set to 1, or it can be adaptively adjusted according to the numerical distribution range of each modal augmentation representation. Then, the Shannon entropy of the probability distribution is calculated as the information entropy of each modality, which is used to quantify the information richness of each modal augmentation representation.

[0091] When generating cross-modal consistency scores, the weight coefficients of the first consistency score, the second consistency score, and the third consistency score can be set to equal weights, or they can be allocated according to the historical performance of each modality on the validation set. Preferably, the entropy weight method can be used to automatically calculate the weights based on the dispersion of each consistency score, and the scores with greater dispersion are assigned higher weights.

[0092] After element-wise addition of the text modality weighted representation, image modality weighted representation, and temporal modality weighted representation, the resulting vector is L2 normalized to make the modulus of the multimodal fusion embedding representation 1, so as to maintain consistency with the input distribution of the subsequent inference network layers.

[0093] This embodiment calculates the cosine similarity between three pairs of augmented representations as a semantic consistency score, calculates the vector entropy value of each augmented representation as an information richness index, and generates the fusion coefficient of each modality by weighted combination of consistency score and information entropy. Finally, it generates a multimodal fusion embedding representation by weighted summation and normalization, thereby realizing the adaptive fusion of information from different modalities at the semantic level.

[0094] In some embodiments, yield reasoning is performed on a multimodal knowledge graph. Yield reasoning includes rule-based reasoning, path-based reasoning, and probabilistic reasoning to obtain reasoning results, including: Obtain the process parameter entity attributes and defect entity attributes from the multimodal knowledge graph, and obtain the rule base. The rule base contains multiple process rules stored in the form of triples. Each process rule includes the premise, reasoning conclusion and confidence level. The matching degree of the process parameter entity attributes and defect entity attributes is calculated with the preconditions of each process rule. When the matching degree exceeds the preset matching threshold, the process rule is triggered. The reasoning conclusion of the triggered process rule is extracted as the rule reasoning conclusion, and the confidence degree of the triggered process rule is extracted as the rule reasoning confidence degree. Obtain the yield level entity from the multimodal knowledge graph. Starting from the yield level entity, perform a breadth-first search in the reverse direction of the defect generation relation edge, parameter membership relation edge and process sequence relation edge. During the search, set the upper limit of path length and the path semantic relevance threshold. Only retain paths whose path length does not exceed the upper limit of path length and whose path semantic relevance is not lower than the path semantic relevance threshold as candidate impact paths. Calculate the weight product of each relation edge on each candidate impact path as the path semantic strength. Take the candidate impact path whose path semantic strength exceeds the preset strength threshold as the path reasoning conclusion and use the path semantic strength as the path reasoning confidence. Obtain the set of entity types and the set of relation types in the multimodal knowledge graph. Construct a conditional probability graphical model with entity type as node type and relation type as edge type. Obtain the joint distribution frequency between entity attributes and yield level in historical manufacturing data. Train the conditional probability table of each type of edge based on the joint distribution frequency. The entity attributes corresponding to the multimodal fusion embedding representation of the current batch are used as evidence nodes to input the conditional probability graph model. The posterior probability distribution of each yield level is calculated through the belief propagation algorithm. The yield level with the highest posterior probability is extracted as the probabilistic inference conclusion, and the posterior probability value corresponding to the yield level with the highest posterior probability is extracted as the probabilistic inference confidence. The conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are combined to generate a reasoning result. The inference results are conflict-resolved, and credibility weights are assigned based on historical verification accuracy. A weighted fusion is then used to generate predicted yield values ​​and impact paths, including: Obtain the conclusion and confidence level of rule-based reasoning, obtain the conclusion and confidence level of path-based reasoning, and obtain the conclusion and confidence level of probability-based reasoning. Determine whether the conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are consistent. If they are consistent, the conclusion of rule-based reasoning is used as the predicted yield, and the candidate influencing paths in the path-based reasoning conclusion are used as the influencing paths. If the three are inconsistent, proceed with the next steps; Obtain the historical verification dataset, which contains multiple historical inference records. Each historical inference record includes an inference layer identifier, inference conclusion, actual result, and verification accuracy. The historical verification accuracy corresponding to the rule reasoning layer is extracted from the historical verification dataset as the rule reasoning credibility weight, the historical verification accuracy corresponding to the path reasoning layer is extracted as the path reasoning credibility weight, and the historical verification accuracy corresponding to the probability reasoning layer is extracted as the probability reasoning credibility weight. Multiply the rule-based reasoning conclusion by the rule-based reasoning credibility weight to generate a weighted rule-based reasoning conclusion; Multiply the path reasoning conclusion by the path reasoning credibility weight to generate a weighted path reasoning conclusion; Multiply the probabilistic reasoning conclusion by the probabilistic reasoning credibility weight to generate a weighted probabilistic reasoning conclusion; The weighted conclusions of rule-based reasoning, path-based reasoning, and probability-based reasoning are added element by element to generate a predicted yield value. Obtain the set of candidate impact paths in the path reasoning conclusion. Each candidate impact path contains the entity sequence on the path and the path semantic strength. The path semantic strength of each candidate influence path is multiplied by the path reasoning credibility weight to generate the weighted path semantic strength of each candidate influence path. The candidate impact path set is sorted from high to low according to the weighted path semantic strength, and the top preset number of candidate impact paths in the sorting results are extracted as impact paths.

[0095] In this embodiment, the confidence level of each process rule in the rule base can be directly assigned by domain experts based on the reliability of their process experience, or obtained by statistically analyzing the frequency of the simultaneous occurrence of the rule's preconditions and inference conclusions from historical data; the higher the frequency, the higher the confidence level. The preset matching threshold is determined on the validation set using a grid search method based on the number and precision requirements of the rules in the rule base, minimizing the false trigger rate while ensuring recall.

[0096] In path reasoning, the semantic relevance of a path is obtained by averaging the cosine similarity between the entity embeddings at both ends of each relation edge along the path. The upper limit of the path length is set based on the maximum possible number of hops from the yield level entity to the process parameter entity in the knowledge graph. The preset strength threshold is determined based on the semantic strength distribution of historical paths, taking the value corresponding to the first certain percentile after sorting all historical path semantic strength values ​​as the threshold.

[0097] In probabilistic reasoning, the conditional probability table is obtained by statistically analyzing the frequency of each state of the child nodes under each combination of parent node states in historical manufacturing data, and then dividing by the total frequency of that combination of parent node states. The joint distribution frequency is obtained by statistically analyzing the number of times each entity attribute value and yield level occur simultaneously in historical batches. When performing the statistics, continuous entity attribute values ​​can be discretized into several intervals before frequency counting.

[0098] The historical validation dataset is constructed by dividing historical batch data into training and validation sets. After training each inference model on the training set, the inference is run on the validation set and the inference conclusions are compared with the actual results. The number of correct inferences divided by the total number of inferences is the historical validation accuracy of that inference layer.

[0099] When the inference conclusion is a category label, the category label is first converted into a one-hot encoded vector, then multiplied by the confidence weight and then weighted and summed. The category with the largest weighted sum is taken as the predicted yield value.

[0100] The preset number of influence paths can be set to a fixed value according to the actual analysis needs of the process engineer, or dynamically determined according to the distribution of the weighted path semantic intensity. The path whose weighted path semantic intensity exceeds the sum of the mean and standard deviation of all path intensities is taken as the final output influence path.

[0101] This embodiment uses three complementary approaches—rule-based reasoning, path-based reasoning, and probabilistic reasoning—to perform yield inference from three perspectives: knowledge-driven, structure-driven, and data-driven. It also uses a conflict resolution mechanism to weight and fuse inconsistent reasoning conclusions based on historical verification accuracy, ultimately outputting a yield prediction value and an explainable impact path, thus achieving reliable prediction and root cause localization of optical chip yield.

[0102] By adopting the above technical solutions, this invention differs from existing technologies and has the following beneficial effects: It associates process parameters, defect morphology, and electrical test results with the same chip object through global anchor point identification, constructing a multimodal knowledge graph that includes process sequence, parameter membership, defect generation, test evaluation, and cross-modal equivalence relations, organizing heterogeneous data into a unified structured knowledge representation; it uses a modality-specific encoder to map each modal entity to a unified semantic space, generates context-aware enhanced representations through intramodal subgraphs and graph attention networks, calculates semantic consistency based on cross-modal equivalence relations, and uses a gating fusion mechanism to achieve adaptive weighted fusion; finally, through the collaborative execution of rule-based reasoning, path-based reasoning, and probabilistic reasoning, and conflict resolution based on historical verification accuracy, it outputs a predicted yield value and an interpretable impact path, achieving accurate prediction and root cause localization of optical chip yield, significantly improving the fusion quality of multimodal data and the credibility of reasoning results.

[0103] Finally, it should be noted that although the above embodiments have been described in the text and drawings of this application, this should not limit the scope of patent protection of this application. Any technical solutions that are based on the essential concept of this application and utilize the content described in the text and drawings of this application, resulting in equivalent structural or procedural substitutions or modifications, as well as the direct or indirect application of the technical solutions of the above embodiments to other related technical fields, are all included within the scope of patent protection of this application.

Claims

1. A collaborative analysis method for optical chip yield based on multimodal knowledge graph, characterized in that, include: Acquire multimodal raw data during the optical chip manufacturing process. The multimodal raw data includes process parameter records, optical detection images, and electrical test signals. The process parameter records are structured to construct process parameter entities; Defect regions are segmented and feature vectors are extracted from optical inspection images to construct defect entities; Feature point detection and time-series feature vector extraction are performed on electrical test signals to construct test entities; A global anchor point identifier is formed by using the batch number and chip coordinates to associate the process parameter entity, defect entity and test entity with the same chip object; Based on process parameter entities, defect entities, and test entities, a multimodal knowledge graph is constructed. The multimodal knowledge graph includes process entities, process parameter entities, defect entities, and test entities, as well as process sequence relationships, parameter membership relationships, defect generation relationships, test evaluation relationships, and cross-modal equivalence relationships. A modality-specific encoder is used to embed process parameter entities, defect entities, and test entities respectively, generating embedded representations of text modality, image modality, and time sequence modality; Based on the embedding representation, an intramodal subgraph is constructed for each modality, and a graph attention network is used for message passing to generate an enhanced representation. For entity groups with cross-modal equivalence relations, the semantic consistency between the enhanced representations of different modalities is calculated, the fusion weight of each modality is determined based on the semantic consistency, and a gating fusion mechanism is used to generate multimodal fusion embedded representations. Perform yield reasoning on a multimodal knowledge graph. Yield reasoning includes rule reasoning, path reasoning, and probabilistic reasoning to obtain the reasoning results. The inference results are conflict-resolved, credibility weights are assigned based on historical verification accuracy, and yield predictions and impact paths are generated through weighted fusion.

2. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, The process parameter records are structured to construct process parameter entities, including: Extract the process parameter record, which includes epitaxial growth temperature curve, photolithography exposure dose value, etching gas flow rate and RF power, deposition chamber pressure and temperature, and annealing temperature curve; The process parameter records are numerically processed to generate numerical attribute values ​​for each parameter. Parameter membership relationships are constructed by using the process name as the head entity, the parameter name as the tail entity, and the process parameter entity as the attribute. The parameter membership relationship is added to the multimodal knowledge graph, so that the process parameter entity is associated with the process entity in the multimodal knowledge graph through the parameter membership relationship; The process parameter entity is constructed by combining the process name and parameter name to form an entity identifier, and the attribute of the process parameter entity is the numerical attribute value. The process parameter entity is added to the multimodal knowledge graph, and cross-modal equivalence relationships are established between the process parameter entity, the defect entity, and the test entity through the global anchor point identifier.

3. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, Defect region segmentation and feature vector extraction are performed on optical inspection images to construct defect entities, including: The optical detection image is acquired, and the optical detection image is preprocessed with noise reduction and illumination correction to generate a preprocessed image. Calculate the grayscale histogram of the preprocessed image, identify the valley positions in the grayscale histogram, and use the grayscale value corresponding to the valley position as a segmentation threshold to divide the preprocessed image into high grayscale regions and low grayscale regions. The gradient magnitude is calculated on the preprocessed image, and pixels with gradient magnitudes exceeding a preset gradient threshold are identified as edge pixels. The edge pixels are then connected to form a closed contour. Perform a region intersection operation between the high grayscale region and the closed contour, and extract the region that simultaneously satisfies the high grayscale condition and the closed contour condition as the candidate defect region. Obtain the contour pixel set of the candidate defect region, perform minimum bounding rectangle fitting on the contour pixel set, extract the length and width of the minimum bounding rectangle, and calculate the ratio of the length to the width as the aspect ratio. The total number of pixels contained in the outline pixel set is used as the area of ​​the defect region; Calculate the diameter of a circle with the same area as the defect region as the equivalent circle diameter; The standard deviation of the distance from each contour pixel in the contour pixel set to the fitted contour curve is calculated as the edge roughness index. The area of ​​the defect region, the aspect ratio, the equivalent circle diameter, and the edge roughness index are concatenated to generate a geometric feature vector; Obtain the set of pixel gray values ​​within the candidate defect region, construct the gray-level co-occurrence matrix of the pixel gray value set, and calculate the contrast statistics, correlation statistics, energy statistics, and uniformity statistics of the gray-level co-occurrence matrix. The contrast statistic, the correlation statistic, the energy statistic, and the uniformity statistic are concatenated to generate a texture feature vector; The geometric feature vector and the texture feature vector are concatenated to generate the attribute vector of the defective entity; The pixel coordinates of the candidate defect region in the optical inspection image are converted into two-dimensional coordinates of the chip on the wafer. The two-dimensional coordinates are used to form an entity identifier to construct the defect entity. The attributes of the defect entity are the attribute vector. The defect entity is added to the multimodal knowledge graph, and cross-modal equivalence relationships are established between the defect entity, the process parameter entity, and the test entity through the global anchor point identifier.

4. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, Feature point detection and time-series feature vector extraction are performed on electrical test signals to construct test entities, including: The electrical test signal is acquired, and baseline drift correction and noise filtering are performed on the electrical test signal to generate a preprocessed signal; The IV characteristic curve in the preprocessed signal is obtained, the first derivative of the IV characteristic curve is calculated, and the voltage position where the first derivative first reaches a preset positive threshold is identified as the threshold voltage point. The second derivative of the IV characteristic curve is calculated, and the voltage position where the second derivative reaches its maximum value is identified as the peak point of the differential resistance. The optical power-current mapping is performed on the IV characteristic curve, and the current position where the optical power reaches the saturation inflection point is identified as the saturation current point. The threshold voltage point, the peak point of the differential resistance, and the saturation current point are taken as the IV feature point set; The spectral response curve in the preprocessed signal is obtained, and peak detection is performed on the spectral response curve to identify the wavelength position where the spectral intensity reaches the maximum value as the peak wavelength point. Searching outwards from the peak wavelength point, the left and right wavelength positions where the spectral intensity drops to half the peak intensity are identified as the endpoints of the full width at half maximum (FWHM) interval. Calculate the spectral intensity difference between the peak wavelength point and the adjacent second peak wavelength point, and identify the wavelength position where the spectral intensity difference reaches the maximum value as the extreme point of the side-mode suppression ratio; The peak wavelength point, the endpoint of the full width at half maximum (FWHM) interval, and the extreme point of the side mode suppression ratio are taken as the set of spectral feature points; The IV feature point set and the spectral feature point set are merged to generate a feature point set; the abscissa and ordinate values ​​of each feature point in the feature point set are extracted. Piecewise linear fitting is performed on adjacent feature points in the feature point set, and the slope value of each fitted line is calculated. The horizontal coordinate value, the vertical coordinate value, and the slope value are concatenated to generate the temporal feature vector of the test entity; The test entity is constructed by using the chip's unique serial number as the entity identifier, and the attribute of the test entity is the timing feature vector; The test entity is added to the multimodal knowledge graph, and cross-modal equivalence relationships are established between the test entity, the process parameter entity, and the defect entity through the global anchor point identifier.

5. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, A global anchor point identifier is formed using the batch number and chip coordinates, which associates process parameter entities, defect entities, and test entities with the same chip object, including: Obtain the wafer batch number corresponding to the process parameter entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the defect entity and the two-dimensional coordinate position of the chip on the wafer; obtain the wafer batch number corresponding to the test entity and the two-dimensional coordinate position of the chip on the wafer. The wafer batch number and the two-dimensional coordinate position are concatenated to generate a global anchor point identifier, which is a combination of the wafer batch number string and the two-dimensional coordinate position string. The global anchor point identifier is assigned to the associated attribute of the process parameter entity, the global anchor point identifier is assigned to the associated attribute of the defect entity, and the global anchor point identifier is assigned to the associated attribute of the test entity. Traverse all entities in the multimodal knowledge graph, identify process parameter entities, defect entities, and test entities with the same global anchor point identifier, establish cross-modal equivalence relations between the process parameter entities and the defect entities, establish cross-modal equivalence relations between the process parameter entities and the test entities, and establish cross-modal equivalence relations between the defect entities and the test entities. The cross-modal equivalence relations use the global anchor point identifier as the relation attribute.

6. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, Based on process parameter entities, defect entities, and test entities, a multimodal knowledge graph is constructed, including: Obtain the process parameter entity, the defect entity, and the test entity to obtain the process entity in the optical chip manufacturing process; Define the process sequence relationship, which is used to connect adjacent process entities, with the order of process execution time as the direction of the relationship; Define parameter membership relationships, which are used to connect process entities and process parameter entities, with process entities as the head entity and process parameter entities as the tail entity. Define a defect generation relationship, which is used to connect process entities and defect entities, with the process entity as the head entity of the relationship and the defect entity as the tail entity of the relationship; Define a test evaluation relationship, which is used to connect the test entity and the yield level entity, with the test entity as the head entity and the yield level entity as the tail entity. Define a cross-modal equivalence relation, which is used to connect process parameter entities, defect entities, and test entities that have the same global anchor point identifier; The process entity, the process parameter entity, the defect entity, and the test entity are added as entity nodes to the knowledge graph. Obtain the process timing dependencies between the process entities, and establish process sequence relationship edges between the process entities with dependencies; Obtain the parameter membership relationship between the process entity and the process parameter entity, and establish parameter membership relationship edges between the process entity and the process parameter entity; Obtain the defect generation relationship between the process entity and the defect entity, and establish a defect generation relationship edge between the process entity and the defect entity; Obtain the test evaluation relationship between the test entity and the yield level entity, and establish a test evaluation relationship edge between the test entity and the yield level entity; obtain process parameter entities, defect entities and test entities with the same global anchor point identifier, establish cross-modal equivalence relationship edges between the process parameter entities and the defect entities, establish cross-modal equivalence relationship edges between the process parameter entities and the test entities, establish cross-modal equivalence relationship edges between the defect entities and the test entities, and generate the multimodal knowledge graph.

7. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, A modality-specific encoder is used to embed process parameter entities, defect entities, and test entities respectively, generating embedded representations of text modality, image modality, and time-series modality, including: Obtain the numerical attribute vector of the process parameter entity, wherein the numerical attribute vector contains the splicing result of temperature statistical features, dose feature values, etching features and deposition features; The numerical attribute vector is input into a fully connected neural network, which includes an input layer, multiple hidden layers and an output layer connected in sequence. The number of neurons in the input layer is equal to the dimension of the numerical attribute vector. The hidden layer performs a nonlinear transformation on the numerical attribute vector. The output layer outputs a vector of a first preset dimension. The output of the output layer is used as a text modality embedding representation. Obtain the attribute vector of the defective entity, wherein the attribute vector contains the concatenation result of the geometric feature vector and the texture feature vector; The attribute vector is input into a residual network, which includes an initial convolutional layer, multiple residual blocks, and a global average pooling layer connected in sequence. The initial convolutional layer performs a convolution operation on the attribute vector to generate an initial feature map. Each residual block performs an identity mapping and a convolution mapping superposition operation on the initial feature map to generate a residual feature map. The global average pooling layer performs a global average pooling operation on the residual feature map output by the last residual block to generate a vector of a first preset dimension. The output of the global average pooling layer is used as an image modality embedding representation. Obtain the temporal feature vector of the test entity, wherein the temporal feature vector contains the concatenation result of the abscissa value, ordinate value and slope value of the feature point and the adjacent point; The temporal feature vector is input into a gated recurrent network, which includes an input layer, multiple gated recurrent units, and a fully connected mapping layer connected in sequence. The input layer expands the temporal feature vector by time step. Each gated recurrent unit calculates the hidden state of the current time step based on the input of the current time step and the hidden state of the previous time step. The fully connected mapping layer maps the hidden state of the last time step to a vector of a first preset dimension. The output of the fully connected mapping layer is used as the temporal modality embedding representation.

8. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, Based on the embedding representation, an intramodal subgraph is constructed for each modality. A graph attention network is used for message passing to generate an enhanced representation, including: Obtain the text modality embedding representation, the image modality embedding representation, and the temporal modality embedding representation; Obtain the process sequence relationship and parameter membership relationship in the multimodal knowledge graph. Use the text modal embedding representation corresponding to the process entity with process sequence relationship as the text subgraph node. Use the text modal embedding representation corresponding to the process entity and process parameter entity with parameter membership relationship as the text subgraph node. Establish text subgraph edges between text subgraph nodes with process sequence relationship and between text subgraph nodes with parameter membership relationship to generate a text modal subgraph. Obtain cross-modal equivalence relations in the multimodal knowledge graph, use the image modality embedding representation corresponding to the defect entity with cross-modal equivalence relations as the image subgraph node, obtain the two-dimensional coordinate position of the defect entity on the wafer, calculate the Euclidean distance between the two-dimensional coordinate positions corresponding to any two image subgraph nodes, establish image subgraph edges between image subgraph nodes whose Euclidean distance is less than a preset distance threshold, and generate an image modality subgraph. Obtain cross-modal equivalence relations in the multimodal knowledge graph, use the temporal modality embedding representation corresponding to the test entity with cross-modal equivalence relations as temporal subgraph nodes, obtain the chip batch number and the row index of the chip on the wafer corresponding to the test entity, take the temporal subgraph nodes with the same batch number and adjacent row indices as candidate node pairs, establish temporal subgraph edges between the candidate node pairs, and generate a temporal modality subgraph; The text modality subgraph, the image modality subgraph, and the temporal modality subgraph are respectively input into a graph attention network. The graph attention network calculates the attention coefficient between the target node embedding representation and the embedding representation of each neighboring node for each target node. The neighboring node embedding representations are weighted and summed according to the attention coefficients. The weighted summation result is fused with the target node embedding representation to generate an enhanced representation for each target node. The enhanced representation includes text modality enhanced representation, image modality enhanced representation, and temporal modality enhanced representation.

9. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, For entity groups with cross-modal equivalence relations, the semantic consistency degree between the enhanced representations of different modalities is calculated. The fusion weights of each modality are determined based on the semantic consistency degree. A gated fusion mechanism is then used to generate a multimodal fusion embedding representation, including: Obtain the text modality augmented representation, the image modality augmented representation, and the temporal modality augmented representation; obtain the cross-modal equivalence relation in the multimodal knowledge graph; and identify the text modality augmented representation, image modality augmented representation, and temporal modality augmented representation with the cross-modal equivalence relation as entity groups. The cosine similarity between the text modal augmentation representation and the image modal augmentation representation in the entity group is calculated as the first consistency score; the cosine similarity between the text modal augmentation representation and the temporal modal augmentation representation is calculated as the second consistency score; and the cosine similarity between the image modal augmentation representation and the temporal modal augmentation representation is calculated as the third consistency score. The vector entropy value of the text modality enhancement representation is calculated as the text modality information entropy; the vector entropy value of the image modality enhancement representation is calculated as the image modality information entropy; and the vector entropy value of the temporal modality enhancement representation is calculated as the temporal modality information entropy. The first consistency score, the second consistency score, and the third consistency score are weighted and averaged to generate a cross-modal consistency score. The text modality entropy is divided by the sum of the text modality entropy, the image modality entropy, and the temporal modality entropy to generate text modality gating parameters; The image modal information entropy is divided by the sum of the text modal information entropy, the image modal information entropy, and the temporal modal information entropy to generate image modal gating parameters; Divide the temporal modality information entropy by the sum of the text modality information entropy, the image modality information entropy, and the temporal modality information entropy to generate temporal modality gating parameters; The cross-modal consistency score is multiplied by the text modality gating parameter to generate the text modality fusion coefficient; The cross-modal consistency score is multiplied by the image modality gating parameter to generate the image modality fusion coefficient; The cross-modal consistency score is multiplied by the temporal modality gating parameter to generate the temporal modality fusion coefficient; The text modality enhancement representation is multiplied by the text modality fusion coefficient to generate a text modality weighted representation; The image modality enhancement representation is multiplied by the image modality fusion coefficient to generate the image modality weighted representation; The temporal modality enhancement representation is multiplied by the temporal modality fusion coefficients to generate a temporal modality weighted representation; The text modality weighted representation, the image modality weighted representation, and the temporal modality weighted representation are added element-wise to generate a multimodal fusion embedding representation.

10. The method for collaborative analysis of optical chip yield based on multimodal knowledge graphs according to claim 1, characterized in that, Perform yield reasoning on a multimodal knowledge graph. Yield reasoning includes rule-based reasoning, path-based reasoning, and probabilistic reasoning to obtain reasoning results, including: Obtain the process parameter entity attributes and defect entity attributes from the multimodal knowledge graph, and obtain the rule base. The rule base contains multiple process rules stored in the form of triples. Each process rule includes the preconditions, reasoning conclusions and confidence levels. The matching degree of the process parameter entity attribute and the defect entity attribute with the precondition of each process rule is calculated. When the matching degree exceeds the preset matching threshold, the process rule is triggered. The reasoning conclusion of the triggered process rule is extracted as the rule reasoning conclusion, and the confidence degree of the triggered process rule is extracted as the rule reasoning confidence degree. Obtain the yield level entity in the multimodal knowledge graph. Starting from the yield level entity, perform a breadth-first search in the reverse direction of the defect generation relationship edge, parameter membership relationship edge, and process sequence relationship edge. During the search, set an upper limit for path length and a path semantic relevance threshold. Only retain paths whose path length does not exceed the upper limit for path length and whose path semantic relevance is not lower than the path semantic relevance threshold as candidate influence paths. Calculate the weight product of each relationship edge on each candidate influence path as the path semantic strength. Take the candidate influence path whose path semantic strength exceeds the preset strength threshold as the path inference conclusion. Use the path semantic strength as the path inference confidence. Obtain the entity type set and relation type set in the multimodal knowledge graph, construct a conditional probability graph model with entity type as node type and relation type as edge type, obtain the joint distribution frequency between entity attributes and yield level in historical manufacturing data, and train the conditional probability table of each type of edge based on the joint distribution frequency. The entity attributes corresponding to the multimodal fusion embedding representation of the current batch are used as evidence nodes and input into the conditional probability graph model. The posterior probability distribution of each yield level is calculated through the belief propagation algorithm. The yield level with the highest posterior probability is extracted as the probabilistic inference conclusion, and the posterior probability value corresponding to the yield level with the highest posterior probability is extracted as the probabilistic inference confidence. The rule-based reasoning conclusion, the path-based reasoning conclusion, and the probability-based reasoning conclusion are combined to generate a reasoning result; The inference results are conflict-resolved, and credibility weights are assigned based on historical verification accuracy. A weighted fusion is then used to generate predicted yield values ​​and impact paths, including: Obtain the rule-based reasoning conclusion and the rule-based reasoning confidence level, obtain the path-based reasoning conclusion and the path-based reasoning confidence level, and obtain the probability-based reasoning conclusion and the probability-based reasoning confidence level; Determine whether the rule-based reasoning conclusion, the path-based reasoning conclusion, and the probability-based reasoning conclusion are consistent. If they are consistent, then the rule-based reasoning conclusion is used as the predicted yield value, and the candidate influence path in the path-based reasoning conclusion is used as the influence path. If the three are inconsistent, proceed with the next steps; Obtain a historical verification dataset, which contains multiple historical inference records. Each historical inference record includes an inference layer identifier, inference conclusion, actual result, and verification accuracy. The historical verification accuracy corresponding to the rule reasoning layer is extracted from the historical verification dataset as the rule reasoning credibility weight, the historical verification accuracy corresponding to the path reasoning layer is extracted as the path reasoning credibility weight, and the historical verification accuracy corresponding to the probability reasoning layer is extracted as the probability reasoning credibility weight. Multiply the rule-based reasoning conclusion by the rule-based reasoning credibility weight to generate a weighted rule-based reasoning conclusion; Multiply the path reasoning conclusion by the path reasoning credibility weight to generate a weighted path reasoning conclusion; Multiply the probabilistic reasoning conclusion by the probabilistic reasoning credibility weight to generate a weighted probabilistic reasoning conclusion; The weighted conclusions of the rule-based reasoning, the weighted conclusions of the path-based reasoning, and the weighted conclusions of the probability-based reasoning are added element by element to generate a predicted yield value. Obtain the set of candidate impact paths in the path reasoning conclusion, where each candidate impact path contains the entity sequence on the path and the path semantic strength; The path semantic strength of each candidate influence path is multiplied by the path reasoning credibility weight to generate the weighted path semantic strength of each candidate influence path. The candidate influence path set is sorted from high to low according to the weighted path semantic strength, and the top preset number of candidate influence paths in the sorting results are extracted as influence paths.

Images