Semiconductor device detection and evaluation method and system based on multi-modal data

Abstract

The invention relates to the technical field of semiconductor device intelligent detection, in particular to a semiconductor device detection evaluation method and system based on multi-modal data, and the method comprises the steps: constructing a detection equipment registration model and a data filtering model, and adjusting a data detection module through the registration model, obtaining multi-modal monitoring data of the semiconductor device by combining the adjusted data detection module and the data filtering model of the semiconductor device; establishing a semiconductor device feature extraction and fusion mechanism, and obtaining a multi-modal feature fusion result of the semiconductor device based on the multi-modal monitoring data of the semiconductor device; setting a semiconductor device defect analysis model, and obtaining a defect quantitative analysis result of the semiconductor device in combination with the semiconductor device fault mode information and the multi-mode feature fusion result; and monitoring and evaluating the defect type and the quality condition of the semiconductor device based on the embedded clustering analysis model and the defect quantitative analysis result. According to the invention, defect detection and quality monitoring of the semiconductor device are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection of semiconductor devices, and in particular to a semiconductor device detection and evaluation method and system based on multimodal data. Background Art

[0002] The semiconductor manufacturing process is highly complex and precise, covering several key processes such as wafer preparation, photolithography, etching, thin film deposition, ion implantation, and metallization. Each link is tightly coupled, and slight fluctuations in process parameters may lead to a significant drop in product yield. In order to ensure the quality of semiconductor products, improve production efficiency, and reduce production costs, it is necessary to conduct problem detection and quality control on semiconductor devices.

[0003] The current inspection technology system faces numerous challenges. At the execution level, it relies on subjective operator experience, making it difficult to reliably identify nanoscale defects, resulting in high rates of missed detections and false positives. At the algorithmic level, traditional rule-matching and machine learning algorithms suffer from weak feature extraction capabilities and poor generalization, making them unable to adapt to defect morphologies and inspection requirements during the manufacturing process. This results in lagging inspection efficiency for semiconductor devices.

[0004] On the other hand, the quality inspection process of semiconductor devices involves multimodal data such as optical microscopy images, SEM morphology data, XRD crystal structure information, IV / CV electrical parameters, thermal distribution maps, etc. However, existing inspection methods mostly adopt the data island mode and independently analyze single modal data, ignoring the coupling effect between different physical fields, making it difficult to ensure the comprehensiveness and effectiveness of device quality inspection results.

[0005] In order to adapt to the development direction of intelligence, automation and refinement in the semiconductor industry, it is necessary to optimize defect identification results and quality inspection technologies, further integrate deep learning and intelligent model algorithms, and upgrade existing inspection methods. This will help meet the quality control needs of the production process, improve the accuracy of defect identification results, and provide technical support for the effective combination of semiconductor inspection technology and production processes. Summary of the Invention

[0006] In response to the shortcomings of existing methods and the needs of practical applications, in order to achieve effective mining and maximum utilization of detection data, deep integration and information interaction of detection data, enhance the recognition ability and generalization of semiconductor device defect identification mechanisms, improve the confidence and accuracy of product quality inspection results, and provide technical support for the intelligent, automated and refined development of the semiconductor industry. On the one hand, the present invention provides a semiconductor device inspection and evaluation method based on multimodal data, the method comprising: constructing an inspection equipment alignment model and a data filtering model, adjusting a semiconductor device data inspection module using the inspection equipment alignment model to obtain an adjusted semiconductor device data inspection module, and combining the adjusted semiconductor device data inspection module and the data filtering model to obtain semiconductor device multimodal monitoring data; establishing a semiconductor device feature extraction and fusion mechanism, and obtaining a multimodal feature fusion result of the semiconductor device based on the semiconductor device multimodal monitoring data and the semiconductor device feature extraction and fusion mechanism; setting a semiconductor device defect analysis model, and obtaining a defect quantitative analysis result of the semiconductor device based on semiconductor device failure mode information, the semiconductor device defect analysis model, and the multimodal feature fusion result; and monitoring and evaluating the defect type and quality status of the semiconductor device based on the embedded cluster analysis model and the defect quantitative analysis result to achieve defect detection and quality monitoring of the semiconductor device. The present invention designs inspection equipment, data processing, defect analysis, and quality monitoring links, which are conducive to quickly and accurately detecting and identifying defects in semiconductor devices, timely discovering problems in the production process, reducing the production of unqualified products, and improving the inspection speed and operation efficiency of the method.

[0007] Optionally, constructing a detection device alignment model and a data filtering model, and adjusting the semiconductor device data detection module using the detection device alignment model to obtain the adjusted semiconductor device data detection module includes: obtaining the detection device alignment model based on a weighted average method and weight information of each detection device; and regulating the operating frequency of each detection device in the semiconductor device data detection module using the detection device alignment model to obtain a unified operating frequency for the semiconductor device data detection module. The present invention utilizes a unified device operating frequency, making data collected from different batches and production lines comparable and consistent, thereby helping to ensure the stability and practicality of the semiconductor device quality monitoring method.

[0008] Optionally, the construction of the detection device alignment model and the data filtering model, and the use of the detection device alignment model to adjust the semiconductor device data detection module to obtain the adjusted semiconductor device data detection module include: adding a time node correction model to the detection device alignment model based on the adaptive filtering method and the unified operating frequency; analyzing the time nodes of each detection device in the semiconductor device data detection module using the time node correction model to obtain a time offset of the semiconductor device data detection module; adjusting the synchronization time node of the semiconductor device data detection module based on the time offset; and obtaining the adjusted semiconductor device data detection module based on the synchronization time node. The time node correction model of the present invention can achieve time synchronization of each device, reduce device failures and data processing anomalies caused by time errors, further reduce system maintenance and management costs, and improve the efficiency of multi-dimensional data analysis and processing.

[0009] Optionally, the construction of the detection equipment alignment model and the data filtering model, the use of the detection equipment alignment model to adjust the semiconductor device data detection module to obtain an adjusted semiconductor device data detection module, and the combination of the adjusted semiconductor device data detection module and the data filtering model to obtain semiconductor device multimodal monitoring data includes: obtaining initial monitoring data of the semiconductor device based on the adjusted semiconductor device data detection module; establishing a data filtering model based on the information fractional order transformation expression and the initial monitoring data; and using the data filtering model to filter the initial monitoring data to obtain semiconductor device multimodal monitoring data. The multimodal monitoring data of the present invention has better representativeness and consistency, can better reflect the actual distribution characteristics of semiconductor devices, and enables the present invention to adapt to device detection tasks in different batches and under different production conditions.

[0010] Optionally, the establishment of a semiconductor device feature extraction and fusion mechanism, and obtaining a multimodal feature fusion result of a semiconductor device based on the semiconductor device multimodal monitoring data and the semiconductor device feature extraction and fusion mechanism, includes: using the semiconductor device feature extraction and fusion mechanism to perform data screening on the semiconductor device multimodal monitoring data to obtain an electrical signal dataset, a thermal signal dataset, and a surface signal dataset of the semiconductor device; establishing a semiconductor device electrical signal feature analysis function, a semiconductor device thermal signal feature analysis function, and a semiconductor device surface signal feature analysis function in the semiconductor device feature extraction and fusion mechanism; performing feature extraction and analysis on the electrical signal dataset, the thermal signal dataset, and the surface signal dataset using the semiconductor device electrical signal feature analysis function, the semiconductor device thermal signal feature analysis function, and the semiconductor device surface signal feature analysis function, and obtaining an electrical signal feature analysis result, a thermal signal feature analysis result, and a surface signal feature analysis result of the semiconductor device. The present invention screens the multimodal monitoring data through a feature extraction and fusion mechanism, dividing it into an electrical signal dataset, a thermal signal dataset, and a surface signal dataset, which can accurately focus on data related to different physical characteristics, help to deeply understand the performance of the device in different physical dimensions, and avoid confusion and interference between different types of data.

[0011] Optionally, establishing a semiconductor device electrical signal feature analysis function, a semiconductor device thermal signal feature analysis function and a semiconductor device surface signal feature analysis function in the semiconductor device feature extraction and fusion mechanism includes: introducing a three-dimensional convolutional neural network model, and establishing a semiconductor device electrical signal feature analysis function based on the three-dimensional convolutional neural network model and the electrical signal data set; analyzing the electrical signal data through the semiconductor device electrical signal feature analysis function to obtain an electrical signal feature analysis result; introducing a self-attention mechanism model, and establishing a semiconductor device thermal signal feature analysis function based on the self-attention mechanism model and the thermal signal data set; analyzing the thermal signal data set using the semiconductor device thermal signal feature analysis function to obtain a thermal signal feature analysis result; introducing a deformation similarity calculation expression, and establishing a semiconductor device surface signal feature analysis function based on the deformation similarity calculation expression and the surface signal data set; analyzing the surface signal data set through the semiconductor device surface signal feature analysis function to obtain a surface signal feature analysis result.

[0012] The present invention extracts features from electrical signal data based on a three-dimensional convolution kernel function, making full use of the distribution characteristics of electrical signals in three-dimensional space; analyzes thermal signal features based on a self-attention mechanism, automatically focusing on key feature information in thermal signals; and the surface signal feature analysis function can consider the deformation similarity and learnable weights between different time nodes, dynamically analyzing the changing characteristics of surface signals in time series.

[0013] Optionally, the method of setting up a semiconductor device defect analysis model and obtaining a semiconductor device defect quantitative analysis result based on semiconductor device failure mode information, the semiconductor device defect analysis model, and the multimodal feature fusion result includes: adding a cosine similarity quantization function to the semiconductor device defect analysis model based on the semiconductor device's failure feature vector information and a vector cosine value analysis method; configuring a Euclidean distance quantization function in the semiconductor device defect analysis model based on the semiconductor device's failure feature vector information and information index structure; and extracting surface brightness data, surface contrast data, and surface structure data from a surface signal data set, and establishing a structural similarity quantization function in the semiconductor device defect analysis model based on the surface brightness data, the surface contrast data, and the surface structure data. The present invention quantitatively analyzes semiconductor device defects from different angles to obtain multiple quantitative results, enabling a more comprehensive understanding of the device's defect characteristics and improving the accuracy of fault diagnosis results.

[0014] Optionally, the defect quantification analysis result of the semiconductor device obtained based on the semiconductor device failure mode information, the semiconductor device defect analysis model and the multimodal feature fusion result includes: analyzing the cosine similarity between the semiconductor device and the failure mode based on the cosine similarity quantization function in the semiconductor device failure mode information and the semiconductor device defect analysis model; analyzing the Euclidean distance between the semiconductor device and the failure mode based on the Euclidean distance quantization function in the semiconductor device failure mode information and the semiconductor device defect analysis model; analyzing the structural similarity between the semiconductor device and the failure mode based on the structural similarity quantization function in the semiconductor device failure mode information and the semiconductor device defect analysis model; and obtaining the defect quantification analysis result of the semiconductor device by combining the cosine similarity, the Euclidean distance and the structural similarity.

[0015] This method evaluates the consistency of semiconductor device topography, texture, and other structural aspects with failure modes. By comprehensively utilizing three quantization functions to comprehensively inspect devices from different perspectives, it avoids the omissions and false detections that can occur with a single quantization method, further improving the accuracy of defect identification.

[0016] Optionally, the monitoring and evaluation of the defect type and quality status of the semiconductor device based on the embedded clustering analysis model and the defect quantification analysis results to achieve defect detection and quality monitoring of the semiconductor device includes: using the embedded clustering analysis model to perform embedded clustering analysis on the cosine similarity and the Euclidean distance, and obtaining embedded clustering analysis results at different positions of the semiconductor device; dynamically monitoring and evaluating the defect type and quality status of the semiconductor device in combination with the embedded clustering analysis results and the structural similarity to obtain defect prediction results and quality status evaluation results of the semiconductor device; adjusting and optimizing the process parameters and manufacturing system of the semiconductor device based on the defect prediction results and the quality status evaluation results to achieve defect detection and quality monitoring of the semiconductor device.

[0017] The embedded cluster analysis model of the present invention performs a fusion analysis on the cosine similarity and the Euclidean distance, and can classify devices with similar feature change trends into one category, thereby identifying potential defect types and improving the sensitivity of the defect detection method.

[0018] Secondly, to efficiently implement the semiconductor device inspection and evaluation method based on multimodal data provided by the present invention, the present invention also provides a semiconductor device inspection and evaluation system based on multimodal data, the system comprising an input device, a processor, an output device, and a memory, wherein the input device, processor, output device, and memory are interconnected, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions. The semiconductor device inspection and evaluation system based on multimodal data provided by the present invention has a compact structure, strong applicability, and greatly improves operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a semiconductor device detection and evaluation method based on multimodal data according to the present invention; Figure 2 This is a structural diagram of the semiconductor device detection and evaluation system based on multimodal data of the present invention. DETAILED DESCRIPTION

[0020] Specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the present invention. In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that these specific details are not necessarily required to practice the present invention. In other instances, well-known circuits, software, or methods are not specifically described to avoid obscuring the present invention.

[0021] Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "in one embodiment," "in an embodiment," "an example," or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in one or more embodiments or examples. Furthermore, those of ordinary skill in the art will appreciate that the figures provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0022] See Figure 1 In order to improve the accuracy of semiconductor device inspection data, achieve efficient integration and interaction between data, optimize the comprehensiveness and adaptability of defect identification mechanisms, enhance the reliability and accuracy of product quality inspection results, and provide technical support for the semiconductor industry, the present invention provides a semiconductor device inspection and evaluation method based on multimodal data, the semiconductor device inspection and evaluation method comprising the following steps: S1. Construct a detection equipment registration model and a data filtering model. Use the detection equipment registration model to adjust the semiconductor device data detection module to obtain an adjusted semiconductor device data detection module. Combine the adjusted semiconductor device data detection module and the data filtering model to obtain semiconductor device multimodal monitoring data. The specific setting steps and implementation contents are as follows: First, the operating frequency of each detection device in the semiconductor device data detection module is regulated by using the detection device registration model to obtain a unified operating frequency of the semiconductor device data detection module.

[0023] In order to achieve consistency in the operating frequencies of various detection devices in the semiconductor device data detection module, and thus ensure the synchronization and accuracy of data acquisition and processing, the embodiment uses a detection device alignment model to regulate the operating frequencies of various detection devices. The semiconductor device data detection module contains a variety of sensors, detectors and data detectors. The device operating frequency is an important influencing factor of data detection.

[0024] During the data detection process of semiconductor devices, the difference in the operating frequency of each detection equipment will lead to asynchronous signal acquisition, which in turn affects the accuracy of subsequent data analysis and defect identification. To solve the related problems, the detection equipment registration model is used to adjust the operating frequency of each detection equipment in the data detection module. During the equipment registration process, all sensor devices need to be equivalent to a single data acquisition unit, and then the sampling frequency of all devices in the module is calculated; then the weighted average method is introduced, and based on the different weight information of different device performance, the unified operating frequency of the semiconductor device data detection module is finally obtained, which helps to ensure the coordination and effectiveness of each sensor device in the data acquisition process.

[0025] In this embodiment, the steps and specific implementation contents of regulating the operating frequency of the semiconductor device data detection module are as follows: Based on the operating data of each detection device in the semiconductor device data detection module, the average sampling frequency of all detection devices is calculated, and a total of There are detection devices or apparatuses, and the sampling frequency values ​​of different detection devices are marked as , so the average sampling frequency of each detection device in the module satisfies the following relationship:

[0026] in, Indicates the average sampling frequency of each detection device in the semiconductor device data detection module, Indicates the number of detection devices in the semiconductor device data detection module, Indicates the actual sampling frequency value of different detection equipment.

[0027] In the semiconductor device data detection module, the performance of the detection equipment directly determines its influence weight in the detection process and the quality and validity of the collected data. In other words, the performance of the detection equipment is the key factor affecting its weight. To ensure the scientificity and accuracy of the data detection results, the weights of different detection equipment can be determined based on the weight information, and the following relationship must be satisfied:

[0028] in, Represents the weights of different detection devices in the semiconductor device data detection module, Indicates the actual performance index of different testing equipment, Indicates the number of detection devices in the semiconductor device data detection module.

[0029] Based on the average sampling frequency and weights of different detection devices in the module, a corresponding detection device registration model is set up to determine the unified operating frequency of each detection device in the semiconductor device data detection module. Using the detection device registration model, a weighted value is assigned to the average sampling frequency of different detection devices in the module to obtain the unified operating frequency in the embodiment, which satisfies the following relationship:

[0030] in, Indicates the average sampling frequency of each detection device in the semiconductor device data detection module, Indicates the number of detection devices in the semiconductor device data detection module, Represents the weights of different detection devices, Indicates the actual sampling frequency value of different detection equipment.

[0031] During the semiconductor device data detection process, the difference in operating frequencies of different detection equipment will cause signal acquisition to be asynchronous, thereby affecting the quality of device detection data. In the embodiment, the detection equipment alignment model is used to adjust the operating frequency of each detection equipment to obtain a unified operating frequency, and all devices are equivalent to a single data acquisition unit, avoiding the signal acquisition asynchrony problem caused by frequency differences, further ensuring that the semiconductor device data detection module can effectively collect semiconductor device data, and laying the foundation for subsequent semiconductor device detection and evaluation.

[0032] Then, a time node correction model was added to the detection equipment alignment model based on the adaptive filtering method and unified operating frequency; at the same time, the time nodes of each detection device in the semiconductor device data detection module were analyzed through the above-mentioned time node correction model, and the time offset of the semiconductor device data detection module was obtained; and then the synchronization time node of the semiconductor device data detection module was adjusted according to the time offset.

[0033] In this embodiment, a time-node mechanism is constructed based on the frequency alignment results (unified operating frequency) of the semiconductor device data detection module to ensure scientific synchronization of various detection devices in the time dimension. Adaptive filtering technology is used to estimate the drift rate of the variable frequency oscillator in real time to address the impact of environmental factors (such as temperature fluctuations and humidity changes) on the semiconductor device data detection module. Furthermore, a random term of signal noise is introduced to construct a time-node correction model in the detection device alignment model.

[0034] The above time node correction model satisfies the following relationship:

[0035] in, Indicates the start time To current time The time offset, Indicates the start time node, Indicates the current time node, Indicates the average sampling frequency of each detection device in the semiconductor device data detection module, represents the drift rate of the variable frequency oscillator, represents the signal noise random term.

[0036] Time offset refers to the difference between the actual time of the detection device and the ideal synchronization time, which quantifies the degree of deviation of the detection device in the time dimension caused by various factors.

[0037] In the semiconductor device data detection module, each detection device needs to work under a unified time base to ensure the synchronization of data acquisition and processing. In actual operation, due to factors such as frequency differences, oscillator drift and signal noise, the device time will gradually deviate from the ideal time. Time offset The calculation provides a reference basis and adjustment benchmark for subsequent time correction.

[0038] It is the starting reference point for time offset calculation, that is, the starting moment of the correction process. It is a fixed time marker in the time synchronization process. All subsequent time offset calculations are based on this point to effectively track and calculate subsequent time changes.

[0039] Refers to the current moment when the time offset is calculated, and is a time variable that is continuously updated as the system runs. Together they constitute the time interval ,This interval is an important time span for calculating the time offset, and can reflect the cumulative time deviation of the detection equipment over a period of time.

[0040] In the time node correction model, sampling frequency is closely related to time. Changes in sampling frequency can affect the device's perception of time and the pace of data collection. In this embodiment, the average sampling frequency is used as a parameter to represent the average data acquisition rate of the detection module. The time offset calculation reflects the time deviation caused by the difference between the sampling frequency and the ideal frequency.

[0041] Oscillators are key components in generating stable clock signals in testing equipment. The drift rate describes the degree to which the oscillator's output frequency varies over time. In practical applications, oscillator frequency drift can be affected by factors such as temperature, voltage, and aging, leading to inaccurate clock signals in testing equipment and causing time offset. The drift rate of a variable-frequency oscillator, as described above, reflects the cumulative impact of oscillator frequency drift on time offset.

[0042] The signal noise random term covers the impact of various random interference factors on the time offset during the operation of the detection equipment. The above interference factors include but are not limited to electromagnetic interference and thermal noise. The random term reflects the contribution or influence of uncertain interference factors on the time offset, which can make the model closer to the actual situation and improve the accuracy of time correction.

[0043] The time node correction model is used to dynamically adjust the time nodes of each detection device in the semiconductor device data detection module, and finally the time nodes of each detection device in the module are synchronized. Before the time node correction is performed, the maximum time difference between the detection devices can reach ; After correction, the time difference is reduced to It fully meets the detection requirements for semiconductor device data in practical applications and provides a guarantee for the efficient collection of semiconductor device data detection in the Internet of Things environment.

[0044] In an optional embodiment, the semiconductor device data detection module is adjusted based on the synchronization time node to obtain an adjusted semiconductor device data detection module.

[0045] In the first step, the detection equipment registration model deeply analyzes the sampling frequency average and weight characteristics of each detection equipment in the semiconductor device data detection module, and determines the unified operating frequency of each detection equipment in the module through mathematical operations and logical deduction, laying the foundation for subsequent time dimension regulation.

[0046] In the second step, based on the determined unified operating frequency, the detection equipment registration model calls the time node correction model to perform time node control and adjustment operations on each detection equipment. The above process fully considers the potential influences such as environmental factors and equipment performance differences, and further ensures the coordination and synchronization of each detection equipment in the time node (dimension).

[0047] After the above process is completed, the adjusted semiconductor device data detection module can be obtained, in which each detection device is synchronized in operating frequency and time nodes, which can provide guarantees for accurate detection and efficient analysis of semiconductor devices.

[0048] Next, initial monitoring data of the semiconductor device is obtained according to the adjusted semiconductor device data detection module; and a data filtering model is established based on the information fractional order transformation expression and the initial monitoring data; In the embodiment, initial monitoring data of the semiconductor device is obtained based on the adjusted semiconductor device data detection module.

[0049] Based on the optimized and adjusted semiconductor device data detection module architecture, various detection devices can synchronously and parallelly collect data information such as the electrical signal parameters, temperature distribution curves, and optical morphology characteristics of semiconductor devices in response to the multi-dimensional characteristics of semiconductor devices, thereby obtaining initial monitoring data.

[0050] In order to ensure the consistency of the initial monitoring data of the semiconductor device in the spatial dimension, a spatial calibration method is used in the embodiment, and an advanced spatial feature point matching algorithm is used to perform position alignment operations on the data collection points of the semiconductor device.

[0051] The above-mentioned spatial calibration method includes: selecting key structures on the device as spatial reference points based on the design drawings and functional requirements of the semiconductor device. The above-mentioned reference points should have clear geometric characteristics and stable spatial positions, such as the edge corners of the device and the key nodes of the internal circuit. Furthermore, a three-dimensional measurement environment is constructed based on the design drawings and functional requirements of the semiconductor device, which requires a three-dimensional laser scanner, an optical positioning system, and stable temperature and humidity control equipment. The three-dimensional laser scanner is used to obtain accurate three-dimensional geometric information of the semiconductor device; the optical positioning system is used to monitor the spatial position changes of the data collection points in real time; the temperature and humidity control equipment ensures stable environmental conditions to avoid deformation of the device due to environmental factors, which affects the calibration results. Based on this, the semiconductor device is firmly fixed on the measurement platform to ensure that the device will not move or deform during the measurement process, thereby ensuring the spatial accuracy of the initial monitoring data of the semiconductor device.

[0052] The aforementioned spatial feature point matching algorithms include extracting semiconductor device feature points based on semiconductor device design drawings and geometric features. These algorithms can utilize methods such as the Harris corner detection algorithm and the SIFT (Scale Invariant Feature Transform) feature point extraction algorithm. The Harris corner detection algorithm detects corners by calculating grayscale changes in local regions of semiconductor devices, offering advantages such as simple computation and good real-time performance. The SIFT feature point extraction algorithm, based on semiconductor device scale space theory, can extract feature points that are both scale-invariant and rotation-invariant. Using the SIFT (Scale Invariant Feature Transform) algorithm to perform local feature detection on captured semiconductor device images, the algorithm can detect stable and distinguishable feature points across different scale spaces.

[0053] In addition to feature point extraction methods based on geometric features, special feature points can also be extracted by combining the material and functional characteristics of semiconductor devices. In one alternative embodiment, for semiconductor devices composed of different materials, feature points can be extracted at material interfaces; for circuit regions with specific functions, feature points can be extracted at circuit nodes.

[0054] At the same time, the calibrated and aligned position information is mapped to the three-dimensional coordinate space of the stereo depth data, further realizing the organic combination of the initial monitoring data and the device spatial dimension, and finally generating the initial monitoring data after dual calibration in time and space.

[0055] The collection content of the initial monitoring data of the above-mentioned semiconductor devices is as follows; Electrical signal acquisition: Using the probe station and online monitoring module, the current, voltage, power consumption and other key electrical parameters of semiconductor devices are collected in real time and continuously. The current accuracy can be as high as level, the voltage accuracy can reach level, and thus can accurately capture subtle electrical changes in the device during operation, providing data support for device performance evaluation and fault diagnosis.

[0056] Thermal imaging acquisition: Using a highly sensitive infrared thermal imager, the temperature distribution of semiconductor devices under working conditions is recorded in real time, generating a continuous and dynamic temperature field curve with better spatial resolution than It can clearly present the temperature gradient changes on the surface and inside of semiconductor devices, providing intuitive and accurate data basis for studying the thermal characteristics, heat dissipation performance and potential thermal failure problems of devices.

[0057] Optical morphology acquisition: Advanced optical equipment such as optical microscopes or laser confocal scanning microscopes (LSCMs) are used to perform high-resolution imaging of semiconductor device surface morphology. Image acquisition and analysis can accurately identify physical defects such as cracks and voids on the device surface, providing critical morphological information for manufacturing process evaluation, reliability analysis, and failure mode research. Furthermore, multi-angle ray tracing technology, combined with structured light scanning methods, enables comprehensive, high-precision three-dimensional morphological data acquisition of semiconductor device surfaces, ultimately achieving submicron depth resolution. This allows for precise depiction of the device's surface microscopic undulations and three-dimensional structure, providing information support for research such as microstructural analysis, surface roughness assessment, and three-dimensional modeling of semiconductor devices.

[0058] Using the adjusted semiconductor device data detection module, the electrical signal parameters, thermal signal information, temperature distribution curve, surface signal data and optical morphology characteristics of the semiconductor device are comprehensively collected, and the initial monitoring data of the semiconductor device is finally obtained.

[0059] Furthermore, a data filtering model was established based on the information fractional-order transformation expression and the above initial monitoring data.

[0060] Based on the initial monitoring data of semiconductor devices, The sending timestamp of each data and the sending timestamp sequence information of the related monitoring data need to satisfy the following relationship:

[0061] in, Indicates the sending timestamp sequence information of the initial monitoring data. Indicates the first sending timestamp of the initial monitoring data. Indicates the second sending timestamp of the initial monitoring data. Indicates the initial monitoring data Send timestamp.

[0062] During the operation of the semiconductor device data detection module, the collected data information can be filtered and analyzed based on the initial monitoring data of the semiconductor device and the corresponding sending timestamp information, following the preset network data filtering conditions. In this process, based on the sending timestamp sequence information Record the receiving timestamp information of the initial monitoring data in sequence, and ensure that the receiving timestamp sequence information satisfies the following mathematical relationship:

[0063] in, Indicates the receiving timestamp sequence information of the initial monitoring data, Indicates the first receiving timestamp of the initial monitoring data. Indicates the second receiving timestamp of the initial monitoring data. Indicates the initial monitoring data The module can also store the timestamp sequence information in an array structure for subsequent monitoring and tracing of monitoring data.

[0064] The above preset network data filtering conditions, that is, the settings of the data filtering model are as follows: In order to achieve the filtering optimization of the redundant data of the initial monitoring data, the redundant data filtering technology of the fractional Fourier transform feature compression is introduced in the embodiment. Represents the redundant data feature vector corresponding to different receiving timestamps, and uses Represents the distribution probability of redundant data in the initial monitoring data.

[0065] The semiconductor device inspection and evaluation method is to deeply analyze the intrinsic characteristics and dynamic change patterns of the initial monitoring data. The fractional-order Fourier transform technology is used to capture the time-frequency characteristics of the monitoring data in a non-stationary state. The fractional-order Fourier transform expression is also the information fractional-order transform expression, which satisfies the following relationship:

[0066] in, The fractional Fourier transform expression representing the data acquisition device information is: represents the amplitude of information flow, represents the imaginary unit, Indicates the rotation angle of the control time-frequency plane, Indicates association with the time axis sampling point. Indicates association with the frequency axis sampling point, represents the frequency resolution parameter in the fractional domain, represents the sampling time interval in the time domain, represents the total number of sampling points during discretization, and Sgn represents the sign function.

[0067] The fractional Fourier transform expression of data acquisition device information can describe the distribution characteristics of data signals in the joint time-frequency domain (fractional domain).

[0068] By rotating the time-frequency plane (angle ), which can reveal the local time-frequency correlation that cannot be directly observed by traditional Fourier transform, and is applicable to non-stationary signals (such as transient noise or pulse signals in semiconductor devices).

[0069] The amplitude of the information flow reflects the energy intensity of the output signal of the data acquisition device. In the semiconductor detection process, the amplitude is directly related to the current, voltage or electromagnetic radiation intensity of the device, and is a key indicator for evaluating device performance.

[0070] The imaginary unit satisfies the following relationship: , which can be used to represent the phase change of the data signal. Introducing the phase term in the transformation can enable the signal to achieve time-frequency rotation in the fractional domain.

[0071] represents the rotation angle of the control time-frequency plane, where Satisfies the following relationship θ= ,in Indicates the transformation order, which can be used to further analyze the rotation angle.

[0072] When , it degenerates into a time domain signal; When , it degenerates into the traditional Fourier transform (frequency domain); When is other values, the data signal is rotated to the fractional order domain in the time-frequency plane, which can reveal the local time-frequency characteristics.

[0073] Adjustment can be used to optimize the detection sensitivity of semiconductor device transient signals (such as switching noise and threshold voltage fluctuation).

[0074] The discretized time-frequency index corresponds to the grid point of the discrete fractional Fourier transform (DFrFT). Associated with the time axis sampling point; It is associated with the frequency axis sampling point, based on which the continuous signal is mapped to the discrete fractional order domain, which helps the subsequent detection signal processing.

[0075] Represents the frequency resolution parameter in the fractional domain (related to the sampling interval or scale transformation).

[0076] Indicates the sampling interval (or time resolution) in the time domain.

[0077] and The combination of terms together describes the dispersion effect of the signal in the fractional domain, that is, the degree of diffusion of different frequency components in the time-frequency plane. In the process of semiconductor quality inspection, the relevant parameters are directly related to the response time of the detector or the sampling rate of the data acquisition card.

[0078] Sgn represents the sign function, when At that time , otherwise , symbolic function Determines the direction of rotation of the fractional transform (clockwise or counterclockwise).

[0079] above It further ensures that the transformation satisfies the periodic boundary conditions and avoids spectrum leakage. The phase factor in the discrete Fourier transform can be used to realize the rotation of the time-frequency grid and the total number of sampling points (or signal length) during discretization.

[0080] In the present embodiment, during the semiconductor device monitoring data processing process, a redundant data filtering model, i.e., a data filtering model, is constructed for the initial monitoring data. This model uses the distribution probability of redundant data in the time-frequency space as a key analysis factor. This probability can be obtained based on statistical analysis methods or probabilistic modeling methods, further reflecting the possibility of data redundancy at specific time-frequency points.

[0081] Based on the distribution probability of redundant data in time-frequency space , fractional Fourier transform expression and redundant data filtering threshold , redundant data filtering is performed on the initial monitoring data of the semiconductor device.

[0082] The information fractional Fourier transform expression is introduced to convert the original monitoring data from the time domain or frequency domain to the time-frequency domain to obtain the transformation coefficient , which can more comprehensively capture the characteristics of the signal in the time-frequency joint space. On this basis, a redundant data filtering threshold is set ,This threshold can be adjusted and optimized according to actual needs, such as data storage capacity limitations, ,processing efficiency and other requirements.

[0083] The above data filtering model satisfies the following relationship:

[0084] in, Represents the fractional Fourier transform coefficient after filtering redundant data, Represents a sampling point associated with the time axis, represents the sampling points associated with the frequency axis, represents the information fractional order transformation expression, express The corresponding probability weighting factor is, represents the distribution probability of redundant data in the initial monitoring data, represents the decision threshold for redundant data filtering, Indicates the processing result when the condition is not met.

[0085] It represents the fractional Fourier transform coefficient after redundant data filtering, that is, the filtered time-frequency domain signal.

[0086] The original fractional Fourier transform expression of the data acquisition device information maps the time domain signal to the time-frequency joint domain through the fractional Fourier transform (FrFT), where the rotation angle is It reveals local time-frequency characteristics that cannot be observed by traditional Fourier transform.

[0087] is the fractional order.

[0088] is the index of the discrete time-frequency grid, corresponding to the transformed time-frequency coordinate.

[0089] express The corresponding probability weighting factor.

[0090] Finally, the initial monitoring data is filtered using the above data filtering model to obtain multimodal monitoring data of semiconductor devices.

[0091] The distribution probability of redundant data in the time-frequency space can be obtained by statistical methods (such as histogram analysis) or model estimation (such as Gaussian mixture model) , based on which each time-frequency component is reflected Due to the possibility of redundant data, if a certain area A higher value indicates that the time-frequency components in this area are highly repetitive or the information density is low, which serves as the basis for threshold filtering to distinguish valid signals from redundant information.

[0092] The decision threshold for redundant data filtering can be obtained based on experience or optimization algorithms (such as maximizing the signal-to-noise ratio). A threshold that is too high may lead to the loss of key features, while a threshold that is too low cannot effectively remove redundancy and the boundary conditions for dividing important time-frequency components and redundant components.

[0093] It means that when the data condition does not meet the decision threshold, When the time-frequency component Forced to zero, it means that the data point information is judged to be redundant data.

[0094] Finally, the initial monitoring data of the adjusted semiconductor device data detection module is filtered using the above data filtering model to obtain multimodal monitoring data of the semiconductor device.

[0095] Accurate and efficient multimodal monitoring data is a key element in device performance evaluation and fault diagnosis. During data filtering, the data filtering model uses the distribution probability of redundant data and the decision threshold as core references, combined with the fractional Fourier transform formula, to process the initial monitoring data from the adjusted semiconductor device data detection module. This implemented model data filtering model utilizes a more sophisticated probability-weighted filtering mechanism, abandoning the traditional simple binarization and zeroing method. Instead, it introduces a probability weighting factor, further achieving precise filtering of redundant data through continuous attenuation driven by redundant probability.

[0096] The data filtering model satisfies the following relationship:

[0097] This data filtering model can effectively remove redundant information from the initial data. During the semiconductor device monitoring process, weak effective features and redundant information overlap with each other. For example, weak fault signals can easily be submerged in redundant data. The above data filtering model can closely combine the statistical characteristics of the signal and accurately distinguish effective information from redundant information. While ensuring the integrity of key information, it removes redundant components to the greatest extent, which is conducive to obtaining high-quality multimodal monitoring data for semiconductor devices, providing solid and reliable data support for the research and development, production and maintenance of devices, and significantly improving the implementation effect and application value of semiconductor device detection and evaluation methods.

[0098] Furthermore, the method for acquiring multimodal monitoring data of semiconductor devices in this embodiment is only an optional condition of the present invention. In one or some other embodiments, the method for acquiring multimodal monitoring data can be adjusted according to the composition of the initial monitoring data and the information optimization requirements, which can better adapt to the changing characteristics of the semiconductor device monitoring signal and obtain high-quality, low-redundancy device monitoring data, providing accurate and reliable data basis for subsequent device performance analysis, fault diagnosis, etc.

[0099] S2. Establish a semiconductor device feature extraction and fusion mechanism, and obtain the multimodal feature fusion results of semiconductor devices based on the above-mentioned semiconductor device multimodal monitoring data and the semiconductor device feature extraction and fusion mechanism. The specific steps and implementation contents are as follows: The operation of semiconductor devices involves multi-physical field data such as electricity, heat, and morphology. However, data from different physical fields are often heterogeneous and difficult to be directly used in collaboration for device performance evaluation and fault diagnosis. To this end, deep learning technology is introduced in the embodiment to analyze the multi-physical field coupling of semiconductor devices. A deep learning model is designed and trained to classify, integrate and mine multi-physical field data such as electricity, heat, and surface morphology in the multi-modal monitoring data of semiconductor devices. The model can automatically learn the intrinsic correlation and complex mapping relationship between different physical field data, and map the originally complex, dispersed and heterogeneous multi-modal monitoring data into a unified three-dimensional coordinate system to generate a representative multi-dimensional dynamic working condition mapping matrix. .

[0100] In an optional embodiment, in order to realize the classification, integration and mining of multimodal monitoring data of semiconductor devices (multi-physical field data such as electrical, thermal, surface morphology, etc.), the data is mapped to a unified three-dimensional coordinate system to generate a multi-dimensional dynamic working condition mapping matrix.

[0101] Using a one-dimensional convolutional neural network (1D-CNN) to extract features from electrical signal data in multimodal monitoring data of semiconductor devices can effectively capture local features and temporal patterns in electrical signals; using a two-dimensional convolutional neural network (2D-CNN) to extract features from thermal imaging images and surface topography images respectively can extract thermal signal features and surface topography geometric features; electrical signal features, thermal signal features and surface topography geometric features are spliced ​​together, and weights are assigned to each feature through an attention mechanism. Finally, the fused features are input into a fully connected network to output three-dimensional coordinates. The three-dimensional coordinate system is divided into multiple grid cells, and the data points are assigned to the corresponding grid cells according to the output coordinate values, thereby constructing a multi-dimensional dynamic working condition mapping matrix.

[0102] The above-mentioned mapping matrix is ​​not a simple data stacking and integration, but is based on a deep learning model to analyze the interaction and mutual information of multiple physical fields of semiconductor devices under various complex working conditions. It can then consider the dynamic changes and actual conditions of multiple dimensions such as electricity, heat, and surface morphology. It helps to analyze the synergy and potential conflicts between different physical fields, and is conducive to the subsequent optimization of semiconductor device design and manufacturing processes.

[0103] In the field of semiconductor device performance evaluation and fault diagnosis, efficient screening and classification of multimodal monitoring data is a key step in obtaining key feature information and revealing the intrinsic physical mechanism of the device. The embodiment is based on the multimodal monitoring data of semiconductor devices and uses a scientific and reasonable data processing process to systematically classify the feature information signals closely related to the device operating status.

[0104] Firstly, the semiconductor device feature extraction and fusion mechanism is used to screen the multimodal monitoring data of semiconductor devices to obtain the electrical signal dataset, thermal signal dataset and surface signal dataset of semiconductor devices.

[0105] During the operation of semiconductor devices, multimodal monitoring data covers multiple physical field information such as electricity, heat, and surface topography. To efficiently and accurately extract valuable information, it is necessary to use the semiconductor device feature extraction and fusion mechanism to filter the multimodal monitoring data to obtain electrical signal datasets, thermal signal datasets, and surface signal datasets. The following is a specific data filtering method: The semiconductor device feature extraction and fusion mechanism is set up to screen relevant data based on the data source and acquisition device. For electrical signal data screening, it is necessary to clarify the acquisition device and channel, and determine the device used to collect electrical signals. Different acquisition channels correspond to different circuit nodes or signal types of semiconductor devices. According to the electrical signal data, the data obtained by the acquisition channel related to the signal type is selected and included in the electrical signal data set, thereby obtaining the semiconductor device electrical signal matrix. For thermal signal data screening, it is necessary to confirm the thermal imaging equipment and the measurement area, determine the measurement area covered by the thermal imaging equipment, select the thermal imaging data related to the heating area of ​​the semiconductor device, and then obtain the semiconductor device thermal signal matrix For surface signal data screening, it is necessary to determine the surface morphology acquisition equipment and scanning range, clarify the scanning range and resolution of the equipment, ensure that it can meet the needs of semiconductor device surface morphology analysis, select the data obtained with appropriate scanning range and resolution, incorporate it into the surface signal data set, and finally obtain the semiconductor device surface data signal matrix .

[0106] In an alternative embodiment, an electrical signal dataset of a semiconductor device is screened.

[0107] The electrical signal data set of semiconductor devices is extracted from the multimodal monitoring data through the data screening mechanism. The electrical signal matrix of semiconductor devices is constructed. , which realizes the effective description of electrical signals in the spatial dimension, Respectively represent spatial coordinates, which are used to locate the specific position of the electrical signal inside the device. The above-mentioned semiconductor device electrical signal matrix records the electrical parameters such as the intensity information, current density, voltage distribution, etc. of the semiconductor device electrical signal, such as current ,Voltage , power consumption ,capacitance , threshold voltage The current density information can be marked as And voltage distribution information can be marked with color The above electrical signal matrix intuitively reflects the distribution of the electric field and the flow of current inside the device, providing a comprehensive and accurate data basis for the analysis of the device's electrical characteristics.

[0108] In an alternative embodiment, a thermal signal dataset of a semiconductor device is screened.

[0109] Extract the thermal signal dataset of semiconductor devices from multimodal monitoring data in spatial coordinates Constructed a semiconductor device thermal signal matrix for a three-dimensional architecture , realizing the spatial distribution description of the device thermal signal. The above matrix covers the temperature field, heat flux density, temperature distribution , heat flux , thermal resistance And other key thermal parameters, where the temperature field information is marked as To intuitively present the spatial distribution of the temperature inside the device, the heat flux density is marked as It reflects the direction and intensity of heat flow inside the device.

[0110] Due to the complex thermal physical processes within semiconductor devices, the embodiments focus on screening the intrinsic correlation between the current path and heat diffusion in the thermal signal. The current path serves as the channel for energy transmission in the device, and its distribution and changes will directly affect the generation and diffusion of heat. The heat diffusion process will also have a feedback effect on the electrical properties around the current path. By establishing a thermal signal matrix, the relevant dependencies can be analyzed from both global and local levels, providing key clues for revealing the multi-physical field coupling mechanism within the device.

[0111] In an alternative embodiment, a surface signal dataset of a semiconductor device is screened.

[0112] A surface signal dataset of semiconductor devices is extracted from multimodal monitoring data. This dataset covers multiple key parameters that characterize the microscopic and macroscopic properties of the device surface, including but not limited to surface roughness. , which reflects the length and width of the surface micro-geometric lines and is an intuitive indicator of the surface structural integrity of semiconductor devices. It can reveal the stress damage or fatigue cracks that may occur in the device during manufacturing and use; the thickness of the oxide layer It is not only related to the electrical insulation performance of the device, but also affects the chemical stability and reliability of the device; spectral reflectivity It is an important representation of optical properties and can be used to evaluate the composition, purity and surface treatment effects of device surface materials.

[0113] In addition, the surface height map in the surface signal dataset is obtained with a resolution of Drawing further depicts the fluctuation of the device surface in the vertical direction, providing an intuitive basis for the microscopic morphology of the device surface; the curvature radius Warpage reflects the degree of local curvature of the surface and plays an important role in understanding the deformation of the device under packaging, thermal stress, etc. It describes the overall planarity change of the device from a macroscopic perspective and is a key indicator for evaluating the structural stability of the device during manufacturing and use.

[0114] In order to construct a comprehensive and accurate surface signal set, the spatial coordinates The semiconductor device surface data signal matrix was obtained as a benchmark , to achieve accurate positioning and quantitative description of characteristic information such as surface roughness, deformation and displacement in three-dimensional space. Among them, the surface roughness information can be expressed as And the deformation displacement information can be expressed as It can intuitively display the fluctuations of the device surface micromorphology and the deformation at different spatial positions, providing key data support for a deep understanding of the relationship between device surface characteristics and performance.

[0115] In order to fully reveal the multi-physical field coupling mechanism and performance evolution law of semiconductor devices under different working conditions, the electrical signal matrix, thermal signal matrix and surface data signal matrix of the semiconductor device at the same time node (moment) are obtained in the embodiment, providing a more comprehensive and accurate data basis for device quality detection, reliability assessment and fault diagnosis.

[0116] Then, the semiconductor device electrical signal feature analysis function, the semiconductor device thermal signal feature analysis function and the semiconductor device surface signal feature analysis function were established in the above-mentioned semiconductor device feature extraction and fusion mechanism.

[0117] A three-dimensional convolutional neural network model was introduced, and based on the three-dimensional convolutional neural network model and the electrical signal dataset, a semiconductor device electrical signal feature analysis function was established in the semiconductor device feature extraction and fusion mechanism.

[0118] In order to analyze the complex characteristics of electrical signals in time and space dimensions and obtain more representative feature information, the embodiment introduces a three-dimensional convolutional neural network (3DCNN) technology to analyze the semiconductor device electrical signal matrix. For feature analysis, the above-mentioned three-dimensional convolutional neural network technology can simultaneously capture and analyze features in both time and space dimensions, and can effectively mine the correlation characteristics in the electrical signal matrix.

[0119] During the actual operation of semiconductor devices, the phenomenon of electrical and thermal conduction and the corresponding gradient changes reflect the internal information of the device. The three-dimensional convolutional neural network technology can accurately extract local spatiotemporal characteristics such as the gradient changes of electrical and thermal conduction. The above-mentioned characteristic information does not exist in isolation, but is closely related to the energy transmission and conversion process inside the device. For example, when current passes through the device, the change in resistance in the local area will cause the change in the electrical and thermal conduction gradient, which in turn reflects the material properties, contact status and other information of the area. At the same time, the relevant characteristics are also important clues to reveal the potential failure mechanism. Continuous monitoring and analysis of the above-mentioned characteristic information can accurately detect possible defects or anomalies inside the device, providing strong support for device reliability assessment and fault warning.

[0120] In the specific implementation process, the 3DCNN operation uses a three-dimensional convolution kernel As the basic unit of feature extraction, the dynamic window is used to extract the electrical signal matrix of the semiconductor device. Sliding scan is performed on the convolution kernel. When the convolution kernel slides to a local area of ​​the input matrix When , the data in the local area will be deeply processed, and the convolution kernel weight can be calculated Corresponding input value The above process is to perform weighted summation on the spatial and temporal distribution of the electrical signal characteristics in the local area, thereby generating an electrical signal characteristic result that can characterize the characteristics of the local area.

[0121] The above-mentioned electrical signal characteristic analysis process, i.e., the semiconductor device electrical signal characteristic analysis function, needs to satisfy the following relationship:

[0122] in, Indicates the electrical signal characteristic analysis results of semiconductor devices, Indicates that the convolution kernel is The radius of the direction, Indicates that the convolution kernel is The radius of the direction, Indicates that the convolution kernel is The radius of the direction, represents the three-dimensional convolution kernel function, Represented in three-dimensional space The weight matrix on Represents the coordinate index in three-dimensional space, Indicates the position of the convolution kernel in the input electrical signal data The coordinates of the corresponding input data points; in, The electrical signal feature analysis result of the semiconductor device is a comprehensive feature representation obtained after 3DCNN processing of the input electrical signal matrix.

[0123] Respectively represent the convolution kernel in The radii in the three directions jointly determine the size of the convolution kernel, that is, the local area that can be covered during each processing.

[0124] The weight matrices at different coordinates in three-dimensional space can be continuously optimized through the training process to better extract effective features from electrical signals.

[0125] The coordinate index in three-dimensional space can be used to accurately locate the position of the convolution kernel in the local area.

[0126] The coordinates of the input data point corresponding to the current position of the convolution kernel in the input electrical signal data contain information such as the electrical signal strength and change trend at different positions. The above formula can efficiently and accurately extract the local spatiotemporal characteristics of the semiconductor device electrical signal, laying a solid foundation for subsequent feature fusion and device performance analysis.

[0127] A self-attention mechanism model is introduced, and based on the above self-attention mechanism model and thermal signal dataset, a semiconductor device thermal signal feature analysis function is established in the semiconductor device feature extraction and fusion mechanism.

[0128] In order to understand the internal heat transfer mechanism of the device and evaluate the risk of thermal failure, and further analyze subtle abnormal phenomena such as local overheating, the semiconductor device thermal signal matrix In-depth characteristic analysis of relevant thermal parameters in the device will help to fully understand the thermal behavior of semiconductor devices and provide a scientific theoretical guidance for the optimized design and performance improvement of the device.

[0129] There is a complex long-range dependency between the current path and heat diffusion in semiconductor devices. This correlation plays an important role in understanding the thermal properties of the device. To effectively capture this long-range dependency, a Transformer model based on the self-attention mechanism is introduced in the embodiment to perform feature analysis on the thermal signals of semiconductor devices.

[0130] The self-attention mechanism in the above Transformer model can be expressed by the following formula:

[0131] in, represents the weighted aggregation result of the thermal signal features, represents the self-attention mechanism, Indicates the location of the heat signal, Indicates the query vector calculation similarity, Indicates the actual feature information, Represents the scaling factor.

[0132] The weighted aggregation result integrates the correlation information between thermal signals at different locations.

[0133] The core activation function in the self-attention mechanism can convert the similarity matrix into a probability distribution, thereby achieving a quantitative evaluation of the importance of features.

[0134] The query vector represents the current heat signal location that needs attention and reflects the model's query requirements for specific location information.

[0135] It can also represent a key vector for use with the query vector Calculate similarity to measure the degree of correlation between thermal signals at different locations.

[0136] It can also indicate that the value vector contains actual feature information, which is the basis for the model to extract and aggregate features; The scaling factor can scale the dot product result to prevent the gradient vanishing problem caused by the dot product result being too large, and ensure the stability of model training.

[0137] The above self-attention mechanism, query vector, key vector and value vector are all transformed from the heat signal matrix through linear transformation The specific formula is as follows:

[0138] in, Indicates the location of the heat signal, Represents the semiconductor device thermal signal matrix, express The corresponding learnable weight matrix;

[0139] in, Indicates the query vector calculation similarity, Represents the semiconductor device thermal signal matrix, express The corresponding learnable weight matrix;

[0140] in, Indicates the actual feature information, Represents the semiconductor device thermal signal matrix, express The corresponding learnable weight matrix; The above-mentioned learnable weight matrix can be continuously optimized through the back-propagation algorithm during the model training process to adapt to the thermal signal feature extraction requirements of different semiconductor devices and different working conditions.

[0141] Based on the above self-attention mechanism, linear transformation expression and thermal signal matrix, the thermal signal feature analysis function in the semiconductor device feature extraction and fusion mechanism is constructed, and the following relationship is satisfied:

[0142] in, Indicates the thermal signal characteristic analysis results of semiconductor devices, represents the activation function in the self-attention mechanism, Indicates the query location The heat signal feature vector, express Key feature information of the location, represents the matrix transpose operation, represents the scaling factor of the characteristic analysis function, Represents the actual characteristic information in the thermal signal; The thermal signature analysis function for semiconductor devices integrates the correlation between thermal signals at different locations in the thermal signal matrix, comprehensively and accurately reflecting the thermal characteristics of semiconductor devices. This function provides a deep understanding of the internal heat transfer process and local heat distribution within the device, providing a key basis for device reliability assessment, fault diagnosis, and design optimization.

[0143] The activation function in the self-attention mechanism converts the input similarity matrix into a probability distribution, and at the same time performs an exponential operation on each element in the similarity matrix and normalizes it so that the value range of all elements is within , and their sum is 1. Based on this, each element can represent a probability value, reflecting the contribution weight of the heat signal at the corresponding position to the current query position.

[0144] The heat signal feature vectors of different query positions represent the feature information of the heat signal position that currently needs to be paid attention to in the self-attention mechanism. When processing the heat signal matrix, a corresponding query vector is generated for each position, which is used to calculate the similarity with the key vectors of all positions to determine the relative importance of heat signals at different positions to the query position.

[0145] The key feature information at different locations, known as the key vector, can be used to calculate similarity with the query vector. The key vector contains the key feature information for each location in the heat signature matrix. By performing a dot product operation with the query vector, the degree of correlation between heat signatures at different locations can be measured. The higher the similarity, the more correlated the heat signatures at two locations are, and the greater the contribution of the heat signature at that location to the query location during feature aggregation.

[0146] When computing the dot product of the query vector and the key vector, a matrix transposition operation is required to adjust the dimensions because the query vector and the key vector may not match. The matrix transposition operation enables the dot product operation to proceed smoothly. By transposing the key vector, it changes it from a row vector to a column vector (or vice versa), thereby satisfying the matrix multiplication rules of the dot product operation.

[0147] The scaling factor of the characteristic analysis function can prevent the dot product result from being too large, which will cause the gradient to disappear, thereby ensuring the stability and convergence of the thermal signal characteristic analysis function.

[0148] The actual feature information in the thermal signal is the value vector, which contains the actual feature information at different positions in the thermal signal matrix. It is the target object for feature aggregation by the self-attention mechanism. After obtaining the normalized weight through the softmax function, the value vector is weighted and summed using the relevant weights to obtain the final thermal signal feature analysis result.

[0149] The deformation similarity calculation expression is introduced, and based on the deformation similarity calculation expression and the surface signal dataset, a semiconductor device surface signal feature analysis function is established in the semiconductor device feature extraction and fusion mechanism.

[0150] The dynamic characteristics of the surface morphology of semiconductor devices are key indicators for characterizing device reliability. The dynamic features of the surface morphology are extracted and analyzed. Specifically, spatiotemporal topological modeling is performed, a dynamic graph structure is constructed through deformation similarity, and the spatiotemporal correlation between surface roughness and deformation is quantified. Multi-scale feature aggregation is performed based on graph convolution and attention mechanisms to extract the local morphology and global dynamic evolution characteristics of semiconductor devices. The specific implementation content is as follows: First, the dynamic graph of the semiconductor device is constructed and the deformation similarity is calculated.

[0151] Surface data signal matrix Converted to a surface data view and meeting the following conditions , where the nodes For each spatial point For a node, Indicates the construction of spatiotemporal adjacency based on deformation similarity; It indicates that the adjacency matrix is ​​composed of deformation similarities, which can quantify the spatiotemporal correlation between different nodes.

[0152] The above deformation similarity calculation expression satisfies the following relationship:

[0153] in, Indicates the deformation similarity of different time nodes, Represents the surface data signal of the semiconductor device at time node i, represents the surface data signal of the semiconductor device at time node j, Represents the similarity bandwidth parameter.

[0154] The deformation similarity at different time nodes quantifies the similarity of surface morphology between different time nodes, thus reflecting the coupling relationship between deformation and roughness of semiconductor devices. Where time node i and time node j are adjacent time nodes and , the deformation similarity (edge ​​weight) between node i and node j is used to quantify the correlation between surface morphology and deformation. The larger the deformation similarity value, the higher the similarity.

[0155] in Used to describe the surface roughness information and deformation displacement information of semiconductor devices at different time points in space, including but not limited to information such as surface roughness, spatial point deformation direction and amplitude.

[0156] The bandwidth parameter controls the rate at which similarity decays. The smaller the value of the parameter, the more similar deformation-roughness combinations are retained (sparse graph); the larger the value of the parameter, the more extensive deformation differences are included (dense graph). symbol It indicates that the Euclidean distance further quantifies the spatiotemporal differences and deformation of the surface morphology of semiconductor devices.

[0157] Then, based on the above expressions and deformation similarity analysis results, the surface dynamic features of semiconductor devices are extracted in combination with graph neural network technology.

[0158] Based on the spatiotemporal graph convolution layer and deformation similarity, spatiotemporal graph convolution operation is performed to extract surface dynamic features. The surface feature vectors of semiconductor devices at different time nodes satisfy the following relationship:

[0159] in, Represents the surface feature vectors of semiconductor devices at different time nodes, represents the activation function, Indicates the total number of time nodes, Represents any time node, express The adjacent time nodes of a time node, express Time nodes and Deformation similarity of time nodes, express The learnable weights of time nodes, express The offset corresponding to the time node; Through multi-layer spatiotemporal graph convolution, we can aggregate the surface feature vectors of different time nodes, and then obtain the dynamic evolution feature set of the surface state, which satisfies the following relationship:

[0160] in, represents the surface feature set of the semiconductor device, represents the surface feature vector at the first time node, represents the surface feature vector at the second time node, Represents the surface feature vector at the nth time node.

[0161] Next, feature extraction and analysis are performed on the electrical signal dataset, thermal signal dataset and surface signal dataset through the semiconductor device electrical signal feature analysis function, semiconductor device thermal signal feature analysis function and semiconductor device surface signal feature analysis function, and the electrical signal feature analysis results, thermal signal feature analysis results and surface signal feature analysis results of the semiconductor device are obtained.

[0162] The electrical signal characteristic analysis function of the semiconductor device is used to perform characteristic analysis on the electrical signal data set to obtain the electrical signal characteristic analysis result of the semiconductor device.

[0163] The electrical signal characteristic analysis results output by the semiconductor device electrical signal characteristic analysis function satisfy the following relationship:

[0164] in, Indicates the electrical signal characteristic analysis results of semiconductor devices, Indicates that the convolution kernel is The radius of the direction, Indicates that the convolution kernel is The radius of the direction, Indicates that the convolution kernel is The radius of the direction, represents the three-dimensional convolution kernel function, Represented in three-dimensional space The weight matrix on Represents the coordinate index in three-dimensional space, Indicates the position of the convolution kernel in the input electrical signal data The coordinates of the corresponding input data points; Based on the semiconductor device thermal signal feature analysis function, feature extraction and analysis of the thermal signal data set are performed to obtain the thermal signal feature analysis results of the semiconductor device.

[0165] The thermal signal characteristic analysis results output by the semiconductor device thermal signal characteristic analysis function satisfy the following relationship:

[0166] in, Indicates the thermal signal characteristic analysis results of semiconductor devices, represents the activation function in the self-attention mechanism, Indicates the query location The heat signal feature vector, express Key feature information of the location, represents the matrix transpose operation, represents the scaling factor of the characteristic analysis function, Represents the actual characteristic information in the thermal signal; The surface signal feature analysis function of the semiconductor device is used to perform feature extraction and analysis on the surface signal data set to obtain the surface signal feature analysis result of the semiconductor device.

[0167] The surface signal characteristic analysis results output by the semiconductor device surface signal characteristic analysis function satisfy the following relationship:

[0168] in, Represents the surface feature vectors of semiconductor devices at different time nodes, represents the activation function, Indicates the total number of time nodes, Represents any time node, express The adjacent time nodes of a time node, express Time nodes and Deformation similarity of time nodes, express The learnable weights of time nodes, express The offset corresponding to the time node;

[0169] in, represents the surface feature set of the semiconductor device, represents the surface feature vector at the first time node, represents the surface feature vector at the second time node, Represents the surface feature vector at the nth time node.

[0170] In this embodiment, in order to effectively couple the multimodal data of semiconductor devices with the physical field, the electrical signal characteristic analysis results of semiconductor devices are combined based on the three-dimensional geometric registration algorithm. , Thermal signal characteristic analysis results , surface feature analysis results Perform unified integration processing to eliminate multi-sensor spatial sampling differences.

[0171] Among them, dynamic super-resolution reconstruction technology is used to perform scale-adaptive enhancement on low-resolution feature fields to ensure consistency in temporal and spatial resolution: quantile normalization is used to eliminate the skewness of the distribution of features of different physical fields, enhance feature comparability, and help obtain multi-physical field coupling feature results of semiconductor devices.

[0172] At the same time, a weighted graph embedding algorithm is used to perform topological modeling on the multi-physical field coupling feature results, construct a multi-dimensional topological feature matrix, perform feature clustering on the multi-dimensional topological feature matrix through non-negative matrix decomposition, and extract the main correlation features to generate a correlation matrix, thereby effectively obtaining the multimodal feature fusion results of semiconductor devices.

[0173] Furthermore, the feature information extraction method of the semiconductor device in this embodiment is only an optional condition of the present invention. In one or some other embodiments, the feature information extraction method of the semiconductor device can be optimized according to the detection and evaluation requirements of the semiconductor device and the device characteristics. Different feature extraction methods are configured for different semiconductor devices, which can improve the applicability of the semiconductor device detection and evaluation method.

[0174] S3. Setting a semiconductor device defect analysis model, and obtaining a semiconductor device defect quantitative analysis result based on the semiconductor device failure mode information, the semiconductor device defect analysis model, and the multimodal feature fusion result. The specific implementation content is as follows: Firstly, the cosine similarity quantization function, Euclidean distance quantization function and structural similarity quantization function were added to the semiconductor device defect analysis model.

[0175] Based on the fault feature vector information of semiconductor devices and the vector cosine value analysis method, a cosine similarity quantization function is added to the semiconductor device defect analysis model.

[0176] Cosine similarity is the cosine value of the angle between two vectors in the feature space. It eliminates the dimension effect through normalization and only retains the direction information. In semiconductor device detection, It can characterize the multi-physics field feature vector information under dynamic conditions. It is the characteristic basic vector information of the preset fault mode.

[0177] The cosine similarity quantization function satisfies the following relationship:

[0178] in, Indicates semiconductor device dynamic monitoring information With preset failure mode library The cosine similarity between Indicates semiconductor device dynamic monitoring information Middle Row, No. Column element information, Represents a preset failure mode library Middle Row, No. Column element information, Represents the information row and column index structure.

[0179] The above multi-physics field feature vector information The characteristic basis vector information of the preset fault mode The dimensions are ,in and The number of sampling points in the spatial or temporal dimension respectively.

[0180] Specifically represents the matrix Middle Row, No. The elements of the column, represented at position The physical quantity information value at .

[0181] Representation matrix Middle Row, No. The element of the column indicates that the preset fault template is at position The information value at .

[0182] is the row and column index of the matrix, traversing all positions The global similarity can be calculated by calculating the product of the corresponding elements of the two matrices, which initially reflects their collaborative change trend in spatial distribution.

[0183] The denominator part of the calculation represents its overall energy or intensity, and the similarity is normalized to Interval: 1 means the two matrices are in exactly the same direction (same distribution pattern); 0 means the two matrices are orthogonal (no correlation); -1 means the two matrices are in completely opposite directions (opposite distribution patterns).

[0184] By comparing multi-physics field eigenvector information The characteristic basis vector information of the preset fault mode The cosine similarity of the two images can effectively identify the most matching defect type.

[0185] Configuring a Euclidean distance quantization function in a semiconductor device defect analysis model based on the fault feature vector information and information index structure of the semiconductor device; The above Euclidean distance quantization function satisfies the following relationship:

[0186] in, Indicates semiconductor device dynamic monitoring information With preset failure mode library The Euclidean distance between Indicates semiconductor device dynamic monitoring information Middle Row, No. Column element information, Represents a preset failure mode library Middle Row, No. Column element information, Represents the information row and column index structure; The above quantization function reflects the cumulative difference between the two matrices at all space-time positions. Record the temperature distribution under dynamic conditions, and If it is a crack fault template, a larger Euclidean distance indicates that the current working condition is significantly different from the crack pattern.

[0187] Surface brightness data, surface contrast data and surface structure data are extracted from the surface signal data set, and then a structural similarity quantization function is established in the semiconductor device defect analysis model based on the surface brightness data, surface contrast data and surface structure data.

[0188] The above structural similarity quantization function satisfies the following relationship:

[0189] in, Indicates semiconductor device dynamic monitoring information With preset failure mode library The structural similarity between Indicates semiconductor device dynamic monitoring information The mean value of brightness information in Represents a preset failure mode library The mean value of brightness information in Represents the stability constant corresponding to the brightness information, Indicates semiconductor device dynamic monitoring information The standard deviation of contrast in , Represents a preset failure mode library The standard deviation of contrast in , represents the stability constant corresponding to the contrast standard deviation, Indicates semiconductor device dynamic monitoring information With preset failure mode library The covariance of represents the stability constant corresponding to the standard deviation of structural differences, Indicates semiconductor device dynamic monitoring information The standard deviation of the structural differences in Represents a preset failure mode library The standard deviation of the structural differences in .

[0190] The value range of the structural similarity quantification function satisfies , 1 means completely similar, -1 means completely opposite, and the similarity is decomposed into three independent components: brightness, contrast, and structure, which is convenient for fault tracing. Each stability constant can prevent the denominator from being zero.

[0191] Then, the defect quantitative analysis results of the semiconductor device are obtained by using the cosine similarity quantization function, the Euclidean distance quantization function and the structural similarity quantization function in the semiconductor device defect analysis model.

[0192] The cosine similarity between semiconductor devices and failure modes is analyzed based on the cosine similarity quantization function in the semiconductor device failure mode information and the semiconductor device defect analysis model. The cosine similarity quantization (directional consistency measurement) can quantify the monitoring data. Failure Mode Library The directional consistency can detect whether the propagation direction of small cracks matches the historical failure mode, avoiding misjudgment due to absolute temperature differences.

[0193] Based on the semiconductor device failure mode information and the Euclidean distance quantization function in the semiconductor device defect analysis model, the Euclidean distance between the semiconductor device and the failure mode is analyzed and the monitoring data is calculated. Failure Mode Library Norm distance quantifies the degree of amplitude deviation and helps identify faults with significantly abnormal amplitudes, such as short circuits.

[0194] The structural similarity quantization function in the semiconductor device failure mode information and the semiconductor device defect analysis model is used to analyze the structural similarity between the semiconductor device and the failure mode. Based on the product of the three SSIM factors (brightness, contrast, and structure), the spatial distribution similarity between dynamic data and the fault template is quantified, which is conducive to identifying spatially sensitive faults such as surface cracks.

[0195] Combining cosine similarity, Euclidean distance, and structural similarity can yield quantitative analysis results of semiconductor device defects. In this embodiment, multi-dimensional quantification of cosine similarity (directional consistency), Euclidean distance (amplitude deviation), and structural similarity (spatial distribution), combined with dynamic weight allocation and hierarchical decision-making, facilitates high-precision detection and traceability analysis of semiconductor device defects.

[0196] In the embodiment, feature fusion is performed on the defect quantification analysis results of the semiconductor device, and fused feature information is generated in the following manner:

[0197] Electrical modal weight It can be dynamically adjusted according to the current density through support vector machine (SVM).

[0198] Thermal mode weight Dynamic calibration can be performed based on the thermal impedance model, and the corresponding weights can be increased under high temperature conditions.

[0199] Optical modal weight Triggered by surface roughness threshold.

[0200] Furthermore, the analysis method of the defect quantification analysis results in this embodiment is only an optional condition of the present invention. In one or some other embodiments, the analysis method of the defect quantification analysis results can be adjusted according to the actual environment of the device.

[0201] Furthermore, the analysis method of the defect quantification analysis results in this embodiment is only an optional condition of the present invention. In one or some other embodiments, the analysis method of the defect quantification analysis results can be adjusted according to the actual environment of the device.

[0202] S4. Monitor and evaluate the defect types and quality status of semiconductor devices based on the embedded cluster analysis model and defect quantification analysis results to achieve defect detection and quality monitoring of semiconductor devices. The specific implementation content is as follows: The defect type identification principle in this embodiment is as follows: Cosine similarity measures the directional similarity of two vectors, meaning that semiconductor device monitoring can reflect the directional consistency of data features at different locations or under different operating conditions. In one optional embodiment, if the cosine similarity of the electrical signal feature vectors at two locations is high, it indicates that their signal change trends are similar.

[0203] Euclidean distance, on the other hand, focuses on measuring the absolute distance between vectors, reflecting the actual degree of difference between semiconductor data points in space. In thermal signal analysis, Euclidean distance can reflect the difference in temperature distribution between different regions of a semiconductor device.

[0204] Firstly, the embedding clustering analysis model is used to perform embedding clustering analysis on cosine similarity and Euclidean distance, and the embedding clustering analysis results of different positions of semiconductor devices are obtained.

[0205] The cosine similarity and Euclidean distance are used to generate an embedding vector using Node2Vec. Combining the defect localization method of graph embedding (Node2Vec) and cluster analysis (K-Means) achieves cross-modal alignment of dynamic operating condition data and static fault template libraries, solving the defect localization difficulties faced by traditional methods under noise interference and pattern drift, and helping to identify the defect type and location of the device.

[0206] The embedding vector generated by Node2Vec is a graph embedding algorithm that maps nodes in a graph structure (corresponding to different locations or data features of semiconductor devices) into a low-dimensional vector space. By incorporating cosine similarity and Euclidean distance into the graph structure, Node2Vec can capture the complex relationships and topological structures between data. In semiconductor device structures, the edge weights between nodes can be determined based on cosine similarity and Euclidean distance. The embedding vector generated by Node2Vec incorporates this relationship information, allowing the originally complex data relationships to be preserved and reflected in the low-dimensional space.

[0207] Map the matrix in the fault template library to a low-dimensional space ( ∈ ), Represents the embedding dimension d: usually 2 or 3, preserving the topological relationship: The embedding vector { Clustering is divided into x groups, each group corresponds to a type of semiconductor device fault defect. The expression of the clustering analysis function is as follows:

[0208] in, represents the cluster analysis function, represents the cluster analysis model, represents the embedding vector of the fault template, Indicates the total number of templates.

[0209] The defect types and quality status of semiconductor devices are further explained based on the semiconductor historical database: Based on the semiconductor historical database, the common defect types and characteristics are as follows; Electrical defects primarily include shorts and shorts. A short circuit, in electrical signal data, manifests as an abnormal direct connection between two supposedly independent circuit nodes, resulting in an abnormal increase in current and an abnormal decrease in voltage. Through embedding clustering analysis, the embedding vectors for short circuit faults are significantly different from those for other normal states or different types of faults, forming a separate group in the clustering results.

[0210] A circuit break occurs when the electrical signal is interrupted somewhere in the circuit, resulting in zero or near-zero current and an abnormally high voltage. Its embedding vector features are also different from other states and will be distinguished during clustering.

[0211] Thermal defects primarily include localized overheating and uneven temperature distribution. Localized overheating refers to areas of thermal imaging data that exhibit significantly higher temperatures than surrounding areas. By converting thermal signal data into embedded vectors and clustering them, the embedded vectors of localized overheating faults can be distinguished from those of other normal thermal distributions or different thermal fault types, thereby determining the defect type.

[0212] Uneven temperature distribution refers to the situation where the surface temperature of a semiconductor device is unevenly distributed and does not conform to the temperature distribution pattern during normal operation. This may indicate poor heat dissipation or other thermal problems. The above-mentioned embedded clustering analysis can identify this abnormal temperature distribution pattern and classify it as the corresponding thermal defect type.

[0213] Surface topographic defects primarily include scratches and cracks. Scratches appear as sudden changes in surface height in surface topography data. Through image processing and embedding vector generation, the embedding vectors for scratch defects are distinguished from other normal surfaces or vectors of different surface defect types in clustering. Cracks, on the other hand, damage the surface structure and form unique features in the embedding vector space, distinguishing them from other defect types.

[0214] In the embodiment, the defect types and quality status of semiconductor devices are illustrated based on the semiconductor historical database, which can provide a more comprehensive understanding of the defect detection and quality monitoring method based on the embedded clustering analysis model and defect quantification analysis results, and provide more effective support and reference basis for the production and quality control of semiconductor devices.

[0215] Then, the defect types and quality status of semiconductor devices are dynamically monitored and evaluated by combining the embedded cluster analysis results and the structural similarity to obtain the defect prediction results and quality status evaluation results of semiconductor devices.

[0216] In the embodiment, the defect type decoupling of semiconductor devices is achieved through embedded spatial clustering, and the degree of defect evolution is quantified by combining structural similarity (SSIM) to construct a three-dimensional dynamic monitoring model of defect type-evolution degree-quality grade, which helps to achieve closed-loop analysis from defect detection to quality degradation prediction.

[0217] Finally, based on the above defect prediction results and quality status assessment results, the process parameters and manufacturing system of the semiconductor device are adjusted and optimized to achieve defect detection and quality monitoring of the semiconductor device.

[0218] Based on the defect prediction and quality assessment results obtained through the above process, we fine-tune the process parameters of semiconductor devices and implement targeted optimization of the manufacturing system. This series of measures enables precise detection of semiconductor device defects and comprehensive quality monitoring, ensuring that product performance and quality meet high standards.

[0219] In an optional embodiment, an iterative optimization algorithm is used to dynamically adjust semiconductor device manufacturing process parameters such as doping concentration, annealing temperature, and system configurations such as detection equipment sampling frequency and signal gain based on defect prediction results and quality assessment feedback, forming a closed-loop control link of defect prediction, quality assessment, and process optimization, further improving the defect detection rate and reducing quality fluctuation rate.

[0220] Based on the defect prediction results and quality status assessment, a multi-objective optimization model can be constructed and combined with the particle swarm optimization algorithm to solve the optimal parameter combination of semiconductor device process parameters, system configuration and weight coefficients to further improve defect detection accuracy and manufacturing stability.

[0221] See Figure 2 In an optional embodiment, to efficiently execute the semiconductor device inspection and evaluation method based on multimodal data provided by the present invention, the present invention also provides a semiconductor device inspection and evaluation system based on multimodal data. The system includes a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the specific steps of the semiconductor device inspection and evaluation method based on multimodal data and related embodiments provided by the present invention. The semiconductor device inspection and evaluation system based on multimodal data provided by the present invention is structurally complete, objective, and stable.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A semiconductor device detection and evaluation method based on multimodal data, characterized in that: The method comprises: Constructing a detection equipment registration model and a data filtering model, adjusting a semiconductor device data detection module using the detection equipment registration model to obtain an adjusted semiconductor device data detection module, and combining the adjusted semiconductor device data detection module with the data filtering model to obtain semiconductor device multimodal monitoring data; Establishing a semiconductor device feature extraction and fusion mechanism, and obtaining a multimodal feature fusion result of the semiconductor device based on the semiconductor device multimodal monitoring data and the semiconductor device feature extraction and fusion mechanism; Setting a semiconductor device defect analysis model, and obtaining a semiconductor device defect quantitative analysis result based on semiconductor device failure mode information, the semiconductor device defect analysis model, and the multimodal feature fusion result; Based on the embedded cluster analysis model and the defect quantification analysis results, the defect type and quality status of the semiconductor device are monitored and evaluated to achieve defect detection and quality monitoring of the semiconductor device.

2. The semiconductor device detection and evaluation method based on multimodal data according to claim 1, characterized in that: The construction of the detection equipment registration model and the data filtering model, and the use of the detection equipment registration model to adjust the semiconductor device data detection module to obtain the adjusted semiconductor device data detection module include: A detection device registration model is obtained based on a weighted average method and weight information of each detection device; The detection device registration model is used to regulate the operating frequency of each detection device in the semiconductor device data detection module to obtain a unified operating frequency of the semiconductor device data detection module.

3. The semiconductor device detection and evaluation method based on multimodal data according to claim 2, characterized in that: The construction of the detection equipment registration model and the data filtering model, and the use of the detection equipment registration model to adjust the semiconductor device data detection module to obtain the adjusted semiconductor device data detection module include: Adding a time node correction model to the detection device registration model based on the adaptive filtering method and the unified operating frequency; Analyzing the time nodes of each detection device in the semiconductor device data detection module by using the time node correction model, and obtaining the time offset of the semiconductor device data detection module; Adjusting the synchronization time node of the semiconductor device data detection module according to the time offset; An adjusted semiconductor device data detection module is obtained based on the synchronization time node.

4. The semiconductor device detection and evaluation method based on multimodal data according to claim 1, characterized in that: The step of constructing a detection device registration model and a data filtering model, adjusting a semiconductor device data detection module using the detection device registration model to obtain an adjusted semiconductor device data detection module, and combining the adjusted semiconductor device data detection module with the data filtering model to obtain semiconductor device multimodal monitoring data includes: Obtaining initial monitoring data of the semiconductor device according to the adjusted semiconductor device data detection module; Establishing a data filtering model based on the information fractional order transformation expression and the initial monitoring data; The initial monitoring data is filtered using the data filtering model to obtain multimodal monitoring data of the semiconductor device.

5. The semiconductor device detection and evaluation method based on multimodal data according to claim 1, characterized in that: The step of establishing a semiconductor device feature extraction and fusion mechanism and obtaining a multimodal feature fusion result of the semiconductor device based on the semiconductor device multimodal monitoring data and the semiconductor device feature extraction and fusion mechanism includes: Using the semiconductor device feature extraction and fusion mechanism to perform data screening on the semiconductor device multimodal monitoring data to obtain an electrical signal dataset, a thermal signal dataset, and a surface signal dataset of the semiconductor device; Establishing a semiconductor device electrical signal feature analysis function, a semiconductor device thermal signal feature analysis function, and a semiconductor device surface signal feature analysis function in the semiconductor device feature extraction and fusion mechanism; The electrical signal data set, the thermal signal data set and the surface signal data set are subjected to feature extraction and analysis through the semiconductor device electrical signal feature analysis function, the semiconductor device thermal signal feature analysis function and the semiconductor device surface signal feature analysis function, and the electrical signal feature analysis results, thermal signal feature analysis results and surface signal feature analysis results of the semiconductor device are obtained.

6. The semiconductor device detection and evaluation method based on multimodal data according to claim 5, characterized in that: The establishment of a semiconductor device electrical signal feature analysis function, a semiconductor device thermal signal feature analysis function, and a semiconductor device surface signal feature analysis function in the semiconductor device feature extraction and fusion mechanism includes: Introducing a three-dimensional convolutional neural network model, and establishing a semiconductor device electrical signal feature analysis function based on the three-dimensional convolutional neural network model and the electrical signal dataset; Analyzing the electrical signal data using the semiconductor device electrical signal characteristic analysis function to obtain an electrical signal characteristic analysis result; Introducing a self-attention mechanism model, and establishing a semiconductor device thermal signal feature analysis function based on the self-attention mechanism model and the thermal signal dataset; Analyzing the thermal signal data set using the semiconductor device thermal signal feature analysis function to obtain a thermal signal feature analysis result; Introducing a deformation similarity calculation expression, and establishing a semiconductor device surface signal feature analysis function based on the deformation similarity calculation expression and the surface signal data set; The surface signal data set is analyzed using the semiconductor device surface signal feature analysis function to obtain a surface signal feature analysis result.

7. The semiconductor device detection and evaluation method based on multimodal data according to claim 1, characterized in that: The step of setting a semiconductor device defect analysis model and obtaining a defect quantitative analysis result of the semiconductor device based on the semiconductor device failure mode information, the semiconductor device defect analysis model, and the multimodal feature fusion result includes: Based on the fault feature vector information of semiconductor devices and the vector cosine value analysis method, a cosine similarity quantization function is added to the semiconductor device defect analysis model; Configuring a Euclidean distance quantization function in a semiconductor device defect analysis model based on the fault feature vector information and information index structure of the semiconductor device; Surface brightness data, surface contrast data and surface structure data are extracted according to the surface signal data set, and a structural similarity quantization function is established in a semiconductor device defect analysis model based on the surface brightness data, the surface contrast data and the surface structure data.

8. The semiconductor device detection and evaluation method based on multimodal data according to claim 7, characterized in that: The method of obtaining a defect quantitative analysis result of a semiconductor device based on the semiconductor device failure mode information, the semiconductor device defect analysis model and the multimodal feature fusion result includes: Analyzing the cosine similarity between semiconductor devices and failure modes based on semiconductor device failure mode information and a cosine similarity quantization function in a semiconductor device defect analysis model; Analyzing the Euclidean distance between semiconductor devices and failure modes based on semiconductor device failure mode information and a Euclidean distance quantization function in a semiconductor device defect analysis model; Analyzing the structural similarity between the semiconductor device and the failure mode based on the semiconductor device failure mode information and the structural similarity quantization function in the semiconductor device defect analysis model; The defect quantification analysis result of the semiconductor device is obtained by combining the cosine similarity, the Euclidean distance and the structural similarity.

9. The semiconductor device detection and evaluation method based on multimodal data according to claim 8, characterized in that: The monitoring and evaluation of the defect type and quality status of the semiconductor device based on the embedded cluster analysis model and the defect quantification analysis result to achieve defect detection and quality monitoring of the semiconductor device includes: Performing an embedded cluster analysis on the cosine similarity and the Euclidean distance using the embedded cluster analysis model, and obtaining embedded cluster analysis results at different positions of the semiconductor device; Dynamically monitoring and evaluating the defect type and quality status of the semiconductor device by combining the embedded cluster analysis result and the structural similarity to obtain a defect prediction result and a quality status evaluation result of the semiconductor device; Based on the defect prediction results and the quality status assessment results, the process parameters and manufacturing system of the semiconductor device are adjusted and optimized to achieve defect detection and quality monitoring of the semiconductor device.

10. A semiconductor device detection and evaluation system based on multimodal data, characterized in that: The system includes a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the semiconductor device detection and evaluation method based on multimodal data as described in any one of claims 1 to 9.

Images