Method for detecting defects in a semiconductor device casting based on visual recognition

Abstract

The application discloses a semiconductor equipment casting defect detection method based on visual recognition and belongs to the field of casting detection methods. The method comprises the following steps: a finite element model is established to simulate the cooling process of a perfect casting and a damaged casting, a first feature parameter set is extracted based on surface temperature difference, and a defect detection model is trained; a plurality of detection positions are arranged along a cooling line to collect casting infrared thermal image sequences, surface areas deviating from a standard cooling process are marked as defect characterization areas, and a second feature parameter set is extracted; model input is performed on the results of each detection position to obtain a plurality of predictions and calculate a confidence degree; when the confidence degree is high, a fusion result is taken as a defect mark; when the confidence degree is low, physical inversion is performed based on the infrared sequence to obtain a defect mark; after surface defects are screened out in combination with visible light images, the casting is comprehensively scored according to the defect mark, and is divided into qualified, scrapped or sent to CT for final judgment.

Description

Technical Field

[0001] This invention relates to the field of casting inspection methods. More specifically, this invention relates to a visual recognition-based method for detecting defects in semiconductor equipment castings. Background Technology

[0002] Semiconductor equipment includes various types of structural castings, such as process cavities, wafer transport cavities, equipment racks, and bases. Currently, non-destructive testing of internal defects in castings mainly relies on industrial computed tomography (CT) technology. Industrial CT can perform three-dimensional scanning and tomographic reconstruction of castings, accurately acquiring the location, size, and morphological information of internal defects, and is currently recognized as one of the most reliable methods for detecting internal defects in castings. In the actual production inspection process of semiconductor equipment castings, a combination of "full surface inspection + internal CT sampling" is usually adopted: all castings are visually inspected and their surface dimensions are measured. However, for time-consuming and labor-intensive industrial CT internal inspection, due to factors such as long scanning time per piece, high equipment purchase and operating costs, and a shortage of professional interpreters, sampling inspection can generally only be carried out at a rate of 5% to 10%.

[0003] Because the occurrence of internal defects in castings within the same batch is often highly random, sampling inspection cannot guarantee the internal quality of castings not sampled. When castings with internal defects flow into subsequent processing and assembly stages, the defects may only be discovered during the finishing stage, by which time a significant amount of processing time and tooling costs have already been wasted.

[0004] Under existing offline inspection systems, castings undergo a natural cooling process of tens of minutes from casting to complete cooling. From arrival at the collection area to manual sampling and transfer to the testing laboratory, the entire process often takes tens of minutes or even hours. Resources during this time are rarely utilized in existing technical solutions. After casting, the casting undergoes a natural or forced cooling stage. During this cooling process, the casting itself becomes a heat source continuously radiating heat. Over the several minutes or tens of minutes it travels along the cooling line, this continuous cooling process provides a natural thermal excitation condition for infrared thermal imaging inspection without additional external stimulation. Due to differences in thermal conduction between the casting's interior and surface, and between defective and intact areas, a temperature distribution characteristic that can be captured by an infrared thermal imager will form on the surface. When defects such as shrinkage cavities, porosity, gas holes, or cracks exist inside the casting, the thermal conductivity of the defective area is significantly lower than that of dense metal, resulting in a temperature difference between the surface above the defect and the intact area during cooling.

[0005] To address the aforementioned issues, it is necessary to propose an online inspection method capable of rapid pre-screening of castings on the production line. This method would allow castings to be categorized into three types during the cooling stage: "clearly qualified," "clearly scrapped," and "questionable, awaiting final CT inspection." Clearly qualified castings can be released directly, clearly scrapped castings can be removed, and only questionable castings will be sent to CT for final confirmation. This would significantly reduce the pressure on CT sampling while effectively intercepting castings with internal defects. Summary of the Invention

[0006] One objective of this invention is to provide a method for detecting defects in semiconductor equipment castings based on visual recognition. This method utilizes the thermal radiation of the casting itself during the cooling process as a natural thermal excitation, combines infrared thermal imaging and visible light images, and uses a dual mechanism of AI model inference and physical inversion to screen internal defects in the casting online. Based on the scoring results, the casting is categorized as qualified, scrapped, or sent for final CT inspection, thereby reducing reliance on full inspection by industrial CT and improving detection efficiency.

[0007] To achieve these objectives and other advantages according to the present invention, the present invention provides a method for detecting defects in semiconductor device castings based on visual recognition, comprising the following steps: S1. Establish a finite element model to simulate the standard cooling process of the intact casting model and obtain the cooling process temperature data of each surface of the intact casting model. S2. Inject internal defects of different sizes and depths into the finite element model to simulate the cooling process of the damaged casting model and obtain the cooling temperature data of each surface of the damaged casting model. S3. Based on the difference in temperature data of the cooling process of each surface of the intact casting model and the damaged casting model, determine the surface temperature anomaly region corresponding to each internal defect, and extract the first feature parameter set representing the temperature and shape of each surface temperature anomaly region. S4. Using the first feature parameter set as input and the size and depth of the corresponding internal defects as labels, construct a training sample set and train the defect detection model. S5. Multiple sets of detection positions are set at intervals along the cooling line. Each detection position collects a set of infrared thermal image sequences of each detectable surface of the cooled casting at different temperatures. S6. For each detectable surface infrared thermal image sequence, based on the mapping relationship between the infrared thermal image pixel coordinates and the physical position of the casting surface, the casting surface area that deviates from the standard cooling process is marked as the defect characterization area, and the second feature parameter set characterizing its temperature and shape is extracted. S7. For internal defects in the same defect characterization area, input the second feature parameter set extracted from each detection position into the defect detection model to obtain multiple prediction results for the same internal defect. S8. Calculate the confidence level of multiple prediction results. When the confidence level calculation meets the preset conditions, the prediction result is used as the defect label of the internal defect. When the confidence level does not meet the preset conditions, perform physical inversion on the internal defect based on multiple infrared thermal image sequences, and use the physical inversion result as the defect label of the internal defect. S9. Based on the defect markings, the castings are comprehensively scored, and the castings are judged as qualified, scrapped, or sent to CT for final determination according to the scoring results.

[0008] Preferably, step S3 includes the following steps: S31. Divide the cooling process temperature data of the intact casting model and the damaged casting model into multiple time segments with the same duration as the infrared thermal image sequence. S32. Compare the temperature deviation of the same node in the intact casting model and the damaged casting model frame by frame within the time segment. Nodes with deviations exceeding the preset temperature threshold in multiple consecutive frames are identified as thermal deviation nodes. Adjacent thermal deviation nodes are aggregated into the first candidate defect area. S33. Count the frequency of each first candidate defect region in all time segments, and determine the candidate regions with a frequency higher than the preset proportion threshold as surface temperature abnormal regions. S34. Extract the shape and temperature feature parameters of the surface temperature anomaly region to form the first feature parameter set of the surface temperature anomaly region.

[0009] Preferably, step S6 includes: S61. For a detectable surface, extract the actual temperature change curve of each pixel in the infrared thermal image sequence, and use the pre-calibrated pixel-surface node mapping relationship to extract the standard temperature change curve of the same surface node in the corresponding detection position temperature range from the intact casting model. S62. Compare the actual temperature change curve of each pixel with the standard temperature change curve frame by frame. If the temperature deviation at multiple consecutive times exceeds the preset threshold, the corresponding surface node is marked as a thermal deviation node, and adjacent thermal deviation nodes are aggregated into a second candidate defect area. S63. When the same physical location on the detectable surface is marked as the second candidate defect region at multiple detection positions, it is taken as the defect characterization region. S64. Extract the shape and temperature feature parameters of each defect characterization region as a second feature parameter set.

[0010] Preferably, the defect detection model is a BP neural network model optimized based on particle swarm optimization algorithm, and adopts a three-layer feedforward network structure, including an input layer, at least one hidden layer and an output layer; The input layer has multiple input neuron nodes, each corresponding to a feature parameter in the first feature parameter set; the hidden layer has multiple hidden neuron nodes, each input neuron node is connected to each hidden neuron node, and each hidden neuron node is connected to the output layer; the output layer has three output neuron nodes, each corresponding to the length, width, and depth of the internal defect. The construction and training process of the defect detection model includes: using the first feature parameter set of each sample in the training sample set as the input value of the input neuron node, and using the size parameter and depth information of the corresponding defect as the training target; using the particle swarm optimization algorithm to globally optimize the initial weights and thresholds of the BP neural network model, and using the prediction mean square error of the BP neural network model on the training sample set as the fitness function of each particle in the particle swarm; assigning the optimal initial weights and thresholds obtained by optimization to the BP neural network model, and performing iterative training through the error backpropagation algorithm; when the error index of the output result reaches the preset requirement, the training is completed, and the defect detection model is obtained.

[0011] Preferably, the confidence level is calculated using the following formula: in, Q 0 For confidence level, L、W、D These represent the three prediction dimensions of internal defects: length, width, and depth. m j This represents the average value of the same internal defect across multiple detection locations. s j To obtain the standard deviation of the same internal defect at multiple detection locations. e This is a preset regularization constant.

[0012] Preferably, the physical inversion includes the following process: A1. Using the location of the internal defect predicted by the defect detection model as the center, cut out a local sub-model from the global finite element model of the intact casting model. The local sub-model contains at least one detectable surface and several undetectable surfaces, and contains undetermined defects inside. A2. For each group of detection positions, perform the following operations on the local sub-model: The temperature of each node on the undetectable surface is fixed as the temperature change sequence of the intact casting model at the corresponding spatial position and time segment; The initial temperature of each node on the detectable surface is assigned to the initial temperature of the corresponding pixel in the infrared thermal image sequence, and a heat dissipation boundary condition is applied to the detectable surface. The initial temperature of the remaining nodes within the local sub-model is estimated by the initial temperature distribution of the detectable and undetectable surfaces; A3. Perform cooling simulations for each local sub-model and record the simulated temperature change sequence of the detectable surface within the simulation time segment; A4. Calculate the similarity between the simulated temperature change sequence and the measured temperature change sequence of the detectable surface in each local sub-model, and calculate the joint similarity evaluation value of the current undetermined defect under multiple detection positions. A5. With the goal of achieving the optimal joint similarity evaluation value, the boundary shape function and depth of the undetermined defect are updated using an iterative optimization algorithm. A2 to A4 are repeated until convergence or the preset number of iterations is reached. The converged boundary shape function and depth are used as the final defect label.

[0013] Preferably, the heat conduction process within the local sub-model is described by a three-dimensional unsteady heat conduction equation, and the corresponding temperature characteristic parameters are extracted from the simulated temperature change sequence and the measured temperature change sequence, respectively. The formula for calculating the joint similarity score is: in, G To detect the number of bit groups, c g For the first g Group weighting coefficients M The types of characteristic parameters for temperature characteristics. w j For the first j The weights of each feature parameter; For the first g Group 1 in the simulated temperature change sequence j A dimensionless feature parameter For the first g Group 1 in the measured temperature change sequence j A dimensionless characteristic parameter; With the objective of minimizing the joint similarity evaluation value, the gradient descent method is used to simultaneously evaluate the defect boundary shape function. f (i) and defect depth D Perform iterative updates: in, n This represents the current iteration number. for J right f(θ) The variational derivative, for J right D Partial derivatives, step size and Determined through line search.

[0014] Preferably, it also includes: On the cooling line, visible light images of each detectable surface of the casting are simultaneously acquired by an industrial camera. A target detection model is used to identify surface defects in the visible light images to obtain the surface defects present on the cooled casting. The location of the surface defect is compared with the spatial location of the defect characterization area obtained in step S6, and defect characterization areas that have a positional correspondence with the surface defect are filtered out.

[0015] Preferably, the comprehensive score includes: in, α This is a weighting coefficient for the appearance defect score, with a value ranging from 0 to 1. S surf and S int Each external and internal defect is scored independently based on its shape and size. P This represents the total number of external defects. Q This represents the total number of internal defects. when DI It is considered qualified when it is less than the first threshold. DI If the value exceeds the second threshold, it is considered scrapped. DI If the value is between the first threshold and the second threshold, it is determined that the document should be sent to CT for final determination.

[0016] Preferably, in step S5, the cooling casting is subjected to a standard cooling process at each detection point.

[0017] The present invention has at least the following beneficial effects: First, this invention moves the inspection process from offline sampling to the casting cooling production line. By setting multiple sets of infrared thermal imaging detection positions at intervals on the cooling line, the cooling process of each casting is monitored in real time. Utilizing the thermal radiation during the casting's own cooling process as a natural thermal excitation source, it captures the temperature anomalies caused by internal defects on the surface, achieving online preliminary screening of all castings. Only castings with questionable quality are sent to industrial CT for final confirmation, allowing industrial CT inspection resources to be concentrated on high-risk castings, reducing reliance on full inspection by industrial CT. This effectively improves inspection efficiency while ensuring accuracy, and solves the risk of opaque internal quality of castings in the same batch and defective castings flowing into subsequent processes under the existing sampling inspection model.

[0018] Secondly, this invention constructs a dual detection mechanism that combines model inference and physical inversion, balancing detection efficiency and accuracy. Confidence is calculated for multiple sets of predicted results for the same internal defect. When the confidence level meets preset conditions, the AI ​​model's prediction is directly adopted as the defect label. Only when the confidence level is low or the prediction results are divergent is the physical inversion mechanism activated. This mechanism embeds the defect's geometric model into a local sub-model, iteratively optimizing the defect boundary shape function and depth to invert the defect's length, width, area, and depth. This ensures rapid identification of high-confidence defects and accurate inversion of low-confidence defects, enabling detection to achieve the required accuracy for engineering practice within production line cycle time requirements.

[0019] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a method flow diagram of one technical solution of the present invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0022] It should be noted that, unless otherwise specified, the experimental methods described in the following embodiments are all conventional methods, and the reagents and materials described are all commercially available unless otherwise specified. In the description of this invention, the orientation or positional relationship indicated by the terms is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing this invention and simplifying the description. It does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0023] like Figure 1 As shown, this invention provides a visual recognition-based method for detecting defects in semiconductor equipment castings, which is generally divided into two parts: an offline preparation stage and an online detection stage. The offline preparation stage includes steps S1-S4, mainly completing the establishment of the finite element model, the generation of simulation data, and the training of the defect detection model. The online detection stage includes steps S5-S9, which involves real-time acquisition, identification, inversion, and scoring of castings during the cooling process at the production site, including the following steps: S1. Establish a finite element model to simulate the standard cooling process of the intact casting model and obtain the cooling temperature data of each surface of the intact casting model.

[0024] Specifically, based on the mold design dimensions of the equipment casting, a three-dimensional geometric model of the casting is established in commercial finite element analysis software. Commercial finite element analysis software can employ general-purpose finite element platforms with transient thermal analysis capabilities, such as ANSYS, ABAQUS, and COMSOL Multiphysics. The finite element model reflects the true geometry of the casting, including all external surfaces, wall thickness distribution, and internal structural features. After establishing the geometric model, corresponding material thermophysical properties, obtainable from material handbooks, are assigned to each region of the model, including but not limited to material density, specific heat capacity, and thermal conductivity.

[0025] The initial and boundary conditions for the cooling process are set on the complete casting model. The initial temperature of each node of the casting model can be set to the measured temperature after the casting is completed and demolded. For example, for aluminum alloy castings, the initial temperature can be set to a typical value between 350℃ and 500℃. Regarding boundary conditions, heat dissipation boundaries corresponding to the cooling line environment are applied to all outer surfaces of the casting model. These heat dissipation boundaries include, but are not limited to, natural convection heat transfer and radiation heat transfer. The heat transfer coefficient for natural convection heat transfer can be determined based on empirical correlations using the on-site wind speed and temperature difference; the emissivity of the casting surface in radiation heat transfer can be set based on the surface oxidation degree and roughness.

[0026] After setting the material parameters, initial conditions, and boundary conditions, a finite element transient thermal analysis is performed to simulate the cooling process of a intact casting from its initial temperature under a standard cooling environment. This yields temperature distribution data for each surface of the intact casting model at each time step during the cooling process. The standard cooling process refers to the cooling process experienced by the casting as it passes normally along the cooling line under preset ambient temperature, air velocity, and heat transfer conditions. In actual implementation, the simulation parameters can be calibrated and corrected by collecting measured cooling curves from multiple qualified castings during normal and stable production stages, ensuring that the standard cooling process output by the finite element model approximates the actual temperature evolution of a qualified casting on the cooling line as closely as possible.

[0027] S2. Incorporate internal defects of different sizes and depths into the finite element model to simulate the cooling process of the damaged casting model and obtain the cooling temperature data of each surface of the damaged casting model.

[0028] Specifically, within the internal geometry of a complete casting model, holes or low-density regions of corresponding shapes are generated according to pre-defined defect parameters, and the thermal properties of the defect material are individually assigned to these regions. Since internal defects are typically filled with air or low-density oxides, their thermal conductivity is much lower than that of the dense metal base material. Therefore, the thermal conductivity of the defect region can be set to be on the order of air thermal conductivity. The shape of the defect can approximate an ideal geometry, such as a sphere, ellipsoid, cylinder, or flat disk, or it can be implanted after reverse engineering based on the actual three-dimensional morphology of the defect obtained from actual industrial CT scans.

[0029] The parameters for implanted defects include the spatial location of the defect within the casting, the maximum planar dimension of the defect, the dimension along the casting wall thickness, and the geometry of the defect. By varying the combination of these parameters, various defect scenarios can be covered, ranging from near the inner surface to near the outer surface, from small to large sizes, and from regular to irregular shapes. For each combination of defect parameters, transient thermal analysis is performed after setting the same initial and boundary conditions as in step S1.

[0030] S3. Based on the difference in temperature data of the cooling process of each surface of the intact casting model and the damaged casting model, determine the surface temperature anomaly region corresponding to each internal defect, and extract the first feature parameter set representing the temperature and shape of each surface temperature anomaly region.

[0031] Specifically, the surface temperature of the damaged casting model is compared point-by-point with that of the intact casting model to calculate the temperature deviation. When the temperature deviation of a surface node exceeds a preset temperature deviation threshold at multiple consecutive sampling times, the node is marked as a thermal deviation node. The preset temperature deviation threshold is determined based on the temperature measurement accuracy of the infrared thermal imager in that temperature range and the on-site noise level, and can be 1.5 to 3 times the measurement accuracy range. After marking, spatial connectivity analysis is performed on the thermal deviation nodes using the topological adjacency relationship of the finite element mesh on the casting surface. Spatial connectivity analysis can be implemented using breadth-first search or depth-first search algorithms based on the mesh node adjacency list, or using disjoint-set data structure algorithms or region growing algorithms from the field of image processing. Topologically adjacent thermal deviation nodes are aggregated into a candidate surface temperature anomaly region. Each candidate surface temperature anomaly region corresponds to the thermal response projection of an internal defect on the casting surface.

[0032] Feature parameters are extracted from each marked surface temperature anomaly region to form a first feature parameter set. The first feature parameter set includes two main categories: shape feature parameters and temperature feature parameters. Shape feature parameters are extracted based on the physical geometry of the surface temperature anomaly region on the casting surface, and may include the region's physical length, physical width, and physical area. The region's physical length and width can be taken as the long and short sides of the smallest bounding rectangle projected onto the surface, and the region's physical area is calculated as the integral area of ​​the region on the surface grid. Temperature feature parameters are extracted based on the amplitude and distribution characteristics of temperature deviations at each node within the surface temperature anomaly region, including the maximum temperature deviation, average temperature deviation, and standard deviation of temperature deviation at the nodes within the region. The first feature parameter set extracted from each surface temperature anomaly region is then matched with the defect size (length, width) and defect depth set when the damaged casting model was implanted in step S2, thus forming a labeled sample.

[0033] S4. Using the first set of feature parameters as input and the size and depth of the corresponding internal defects as labels, construct a training sample set and train the defect detection model. The defect detection model can employ artificial neural networks and their variants, or regression models with continuous value output capabilities such as support vector regression, random forests, and gradient boosting trees. During training, the training sample set is divided into training and validation subsets, and a cross-validation strategy is used to evaluate the model's generalization performance under different hyperparameters to prevent overfitting to the training data. The trained defect detection model has the ability to predict the length, width, and depth of corresponding internal defects based on surface temperature anomaly feature parameters. After model training is complete, its model structure and parameters are solidified and deployed to the production line's industrial control computer or edge computing device.

[0034] S5. Multiple detection positions are set at intervals along the cooling line. Each detection position collects a sequence of infrared thermal images of each detectable surface of the cooled casting at different temperatures.

[0035] Specifically, multiple detection positions are spaced apart along the conveying direction of the cooling line. The distance between adjacent detection positions is determined based on the running speed of the casting conveyor belt and the cooling rate of the casting at that stage, ensuring that the surface temperature of the casting corresponding to different detection positions is within different temperature ranges. For example, a detection position can be set at locations with casting surface temperatures of approximately 450℃, 350℃, and 250℃. Each detection position is equipped with at least one infrared thermal imager fixed on a dedicated bracket, with its field of view covering one or more detectable surfaces of the casting at that position. Multiple infrared thermal imagers can work together to cover multiple visible surfaces of the same casting.

[0036] When the casting passes a certain detection position on the cooling line, the infrared thermal imager at that detection position continuously acquires temperature distribution images of the corresponding detectable surface of the casting over a period of time, forming a series of infrared thermal images. Each series of infrared thermal images consists of several frames arranged in chronological order, and each frame records the temperature distribution information of the detectable surface at the corresponding moment.

[0037] Because the ambient temperature, air velocity, and local airflow conditions at the cooling line site may differ from the set standard cooling conditions due to factors such as season, workshop ventilation, and heat dissipation from adjacent equipment, the overall cooling curve of the casting measured at the production site may not directly align with the standard cooling curve simulated by finite element analysis in terms of absolute values ​​and cooling rate. However, under normal circumstances, the internal heat conduction rate of the casting is much greater than the rate of heat dissipation from the surface to the environment. The dominant mechanism for temperature reduction at various points in the casting is uniform heat dissipation to the outer surface. Therefore, at any given moment, the temperature distribution on each surface of the casting is relatively uniform, and the surface temperature differences mainly stem from the influence of internal defects on local heat flow. When comparing the measured temperature sequence with the standard cooling temperature sequence, the initial measured temperature of the infrared thermal image sequence at the current detection location can be used to retrieve the time segment from the standard cooling process temperature data of the intact casting model that has the closest surface temperature level. The temperature change sequence of this segment is then used as the comparison benchmark for the current detection location. In this way, the comparison is conducted under conditions where the surface temperature is closest, which can effectively eliminate the overall temperature deviation caused by macroscopic environmental fluctuations in the cooling line and accurately extract the local temperature anomaly characteristics caused by internal defects.

[0038] S6. For each detectable surface infrared thermal image sequence, based on the mapping relationship between the infrared thermal image pixel coordinates and the physical position of the casting surface, the casting surface area that deviates from the standard cooling process is marked as a defect characterization area, and a second feature parameter set characterizing its temperature and shape is extracted.

[0039] Specifically, the mapping relationship between the pixel coordinates of the infrared thermal image and the physical position of the casting surface can be obtained by spatially calibrating the infrared thermal imager. The calibration steps include determining the spatial position and attitude parameters of the infrared thermal imager relative to the casting transport plane, as well as the intrinsic parameter matrix of the infrared thermal imager. After calibration, any pixel in the infrared thermal image can be mapped to the corresponding node or corresponding spatial coordinate on the finite element mesh of the casting surface.

[0040] Based on the mapping relationship, for each pixel in a sequence of infrared thermal images, the actual temperature value of that pixel is extracted frame by frame in the sequence to obtain the actual temperature change curve corresponding to that pixel. Simultaneously, from the temperature data of the cooling process of the intact casting model obtained in step S1, the standard cooling temperature change curve of the time segment with the same surface node as the mapped position of the pixel and whose surface temperature level is closest to the measured initial temperature of the current detection position is extracted as a comparison benchmark. The actual temperature change curve and the standard temperature change curve are compared at each sampling time, and the temperature deviation at each time point is calculated. When the temperature deviation of a pixel exceeds a preset deviation threshold at multiple consecutive sampling times, the casting surface node corresponding to that pixel is marked as a thermal deviation node. Adjacent and physically connected thermal deviation nodes are aggregated into a defect characterization region. Each defect characterization region is a projection area that indirectly reflects internal defects through abnormal surface thermal response.

[0041] For each marked defect characterization region, extract shape feature parameters and temperature feature parameters of the same type as in step S3 to form a second feature parameter set.

[0042] S7. For internal defects in the same defect characterization area, the second feature parameter sets extracted from each detection position are input into the defect detection model to obtain multiple prediction results for the same internal defect. Since the same casting passes through multiple detection positions sequentially on the cooling line, the same internal defect will cause surface temperature anomalies in the infrared thermal image sequences acquired at different detection positions, and will be marked as the corresponding defect characterization area in step S6, from which multiple sets of second feature parameter sets are extracted. The multiple sets of second feature parameter sets corresponding to the same internal defect are input into the defect detection model trained in step S4 to obtain multiple prediction results for the internal defect at multiple detection positions. Each prediction result includes predicted values ​​for the length, width, and depth of the internal defect.

[0043] S8. Calculate the confidence level of multiple prediction results. When the confidence level meets the preset conditions, the prediction result is used as the defect label of the internal defect. When the confidence level does not meet the preset conditions, perform physical inversion on the internal defect based on multiple infrared thermal image sequences, and use the physical inversion result as the defect label of the internal defect.

[0044] The defect detection model provides multiple sets of predictions for the same internal defect at various detection locations, constituting multiple independent assessments of the defect under different temperature conditions. This provides a practical basis for quantifying the consistency of these multiple predictions. If the predictions under different temperature conditions are close to each other, it indicates that the thermal response characteristics of the defect are stable across multiple temperature ranges, and the model's predictions are reliable. However, when the predictions show significant discrepancies between different detection locations, it indicates that the model's judgment of the defect has considerable uncertainty, and directly adopting the predictions carries risks.

[0045] When the confidence level meets the preset conditions, multiple prediction results are fused, and the fused result is used as the defect label for the internal defect. When the confidence level does not meet the preset conditions, multiple infrared thermal image sequences are used as input, and the geometric parameters of the internal defect are iteratively inverted using a finite element model. The physical inversion result is used as the defect label for the internal defect.

[0046] S9. Based on the defect markings, the castings are comprehensively scored, and the castings are judged as qualified, scrapped, or sent to CT for final determination according to the scoring results.

[0047] Specifically, for each casting that has undergone collection and analysis at all inspection points, all detected defect markers are compiled. Each defect marker contains information on the length, width, and depth of the defect. Based on a pre-defined weighted scoring model, the dimensional information of each defect marker is weighted and calculated, and combined with the number of defects, a comprehensive score for the casting is generated. The weighting coefficients in the scoring model reflect the relative importance of different dimensional dimensions to the casting quality and can be specifically configured according to the service conditions and stress characteristics of the casting being inspected.

[0048] After obtaining the comprehensive score, the score result is compared with a preset first threshold and a second threshold for classification. When the comprehensive score is lower than the first threshold, it indicates that the internal quality of the casting is good, and it is judged as qualified and can be directly released to the subsequent processing steps. When the comprehensive score is higher than the second threshold, it indicates that the casting has internal defects that affect its use, and it is judged as scrap and directly removed from the production line. When the comprehensive score is between the first threshold and the second threshold, it indicates that the quality of the casting is in a gray area, and online detection methods are insufficient to make a definitive conclusion. It is judged as sending it for final CT evaluation, and the casting is transferred to industrial CT equipment for further tomographic scanning and precise evaluation.

[0049] Through the complete S1-S9 process, online screening and triage of all internal defects in castings are achieved. Qualified castings are released directly, while scrapped castings are promptly removed. Only castings with intermediate scores that are questionable are sent to industrial CT for final assessment. This concentrates industrial CT's inspection resources on castings that truly require high-precision judgment, improving overall inspection efficiency and reducing production costs while ensuring inspection accuracy. The entire acquisition process is completed simultaneously with the natural cooling of the casting. Subsequent calculation tasks, including feature extraction, model inference, physical inversion, and comprehensive scoring, can be completed offline on the field industrial control computer, edge server, or cloud platform within tens of minutes to several hours before the casting enters the CT scanner.

[0050] In another technical solution, step S3 includes the following steps: S31. Divide the cooling process temperature data of the intact casting model and the damaged casting model into multiple time segments with the same duration as the infrared thermal image sequence.

[0051] Specifically, the intact casting model and the damaged casting model were respectively simulated to complete the cooling process from the initial temperature to the offline temperature of the cooling line. The resulting cooling process temperature data covered the entire cooling process in the time dimension. However, in the actual online inspection stage, the acquisition of infrared thermal image sequences corresponds to a continuous acquisition period when the casting passes through a certain detection position. The duration of this period is determined by the field of view of the detection position and the running speed of the casting conveyor belt, and is usually several seconds to tens of seconds. In order to maintain consistency with the time scale in the online inspection stage, the simulated cooling process temperature data needs to be divided into multiple continuous and non-overlapping time segments with the same duration as the infrared thermal image sequence, starting from the initial moment. Each time segment after division is exactly the same as a single set of infrared thermal image sequences in terms of duration. Taking ANSYS as the finite element analysis software as an example, the division of time segments can be achieved by setting the output interval of the solution step in the transient thermal analysis post-processing of ANSYS Workbench, or by exporting the simulated temperature data and completing the time segmentation in the MATLAB or Python environment using time index slicing.

[0052] S32. Within a time segment, compare the temperature deviation of the same nodes in the intact casting model and the damaged casting model frame by frame. Nodes with deviations exceeding a preset temperature threshold in multiple consecutive frames are identified as thermal deviation nodes, and adjacent thermal deviation nodes are aggregated into the first candidate defect region.

[0053] Within each predefined time segment, for surface nodes located at the same spatial coordinates in both the intact and damaged casting models, the temperature values ​​at each sampling moment within that time segment are compared frame by frame to calculate the temperature deviation. The nodes compared here strictly correspond in spatial location to ensure that the comparison represents the temperature response differences of the same physical location in different models. To suppress the interference of simulation numerical noise or instantaneous temperature fluctuations caused by mesh discretization on the marking results, when the temperature deviation of a surface node exceeds a preset temperature threshold in multiple consecutive sampling frames of the current time segment, that node is marked as a thermal deviation node within that time segment. This frame-by-frame comparison and deviation determination can be implemented in MATLAB using matrix operations, or in Python using array operations from the NumPy library. After marking the thermal deviation nodes within a single time segment, spatial connectivity analysis is performed using the topological adjacency relationships of the finite element mesh on the casting surface: On the finite element mesh, connectivity aggregation can be completed by traversing the adjacency list of mesh nodes using Python and performing a breadth-first search using collections.deque; on the pixel domain of the infrared thermal image, the connected componentsWithStats function of OpenCV can be used to directly mark the connectivity in a single call, while returning the bounding rectangle, area, and centroid coordinates of each connectivity. Topologically adjacent thermal deviation nodes are aggregated into several regions, and each region serves as a first candidate defect region within that time segment.

[0054] S33. Count the frequency of each first candidate defect region in all time segments. Candidate regions with a frequency exceeding a preset proportion threshold are identified as surface temperature anomaly regions. Since real internal defects continuously cause identifiable surface temperature deviations at multiple stages of the cooling process, the corresponding candidate regions should recur in a sufficient number of time segments. Frequency statistics can be performed on the occurrence frequency of each first candidate defect region in all time segments, calculating the proportion of its occurrences to the total number of time segments. Frequency statistics can be implemented in MATLAB or Python by traversing all time segments and their corresponding candidate region lists, and establishing a mapping dictionary between region identifiers and occurrence frequencies. When this proportion exceeds a preset proportion threshold, the candidate region is identified as a surface temperature anomaly region.

[0055] S34. Extract shape and temperature feature parameters of the surface temperature anomaly region to form the first feature parameter set of the region. The shape feature parameters are extracted based on the physical geometry of the region on the casting surface, including at least the length and width of the minimum bounding rectangle projected onto the casting surface, and the physical area of ​​the region. The temperature feature parameters are extracted based on the amplitude and distribution characteristics of temperature deviations at each node within the region during cooling, including the average temperature of the nodes within the region, the maximum temperature difference, and the time to reach the peak temperature difference. The shape and temperature feature parameters together constitute the first feature parameter set of the surface temperature anomaly region, and their parameter types and extraction methods are completely consistent with those described in step S3. The first feature parameter set will be used in step S4 to construct a training sample set and train the defect detection model.

[0056] In another technical solution, step S6 includes: S61. For a detectable surface, extract the actual temperature change curve of each pixel in the infrared thermal image sequence, and use the pre-calibrated pixel-surface node mapping relationship to extract the standard temperature change curve of the same surface node in the corresponding detection position temperature range from the intact casting model.

[0057] Specifically, for a set of infrared thermal images acquired at a certain detection location, this sequence covers all frames of a detectable surface of the casting during the corresponding cooling period. For each pixel in the infrared thermal image of this detectable surface, its frame-by-frame temperature value is extracted along the time axis, forming the actual temperature change curve corresponding to that pixel. This curve reflects the temperature history of the surface location under measured conditions as it cools. The temperature sequence extraction can be performed by directly opening the sequence file using professional software provided by the infrared thermal imager manufacturer (such as FLIR Research Studio or FOTRIC Analyzer), selecting any pixel of interest to export its temperature-time series data, or by using the VideoReader function in MATLAB to read the sequence file and extract the temperature value of each pixel frame-by-frame for batch processing.

[0058] While extracting the measured temperature curve, it is necessary to obtain the corresponding standard temperature change curve as a comparison benchmark. Using the pixel-surface node mapping relationship established beforehand through spatial calibration of the infrared thermal imager, the physical node on the casting surface corresponding to each pixel is determined. Spatial calibration can employ the Zhang Zhengyou calibration method, using a checkerboard calibration plate of known size as the calibration object. Calibration images of different poses are captured by the infrared thermal imager at the casting transport plane position. The intrinsic parameter matrix and distortion coefficients of the infrared thermal imager are obtained using the OpenCV calibrateCamera function. Then, the extrinsic parameter matrix of the infrared thermal imager relative to the casting transport plane coordinate system is solved using a direct linear transformation algorithm. Finally, a one-to-one mapping relationship is established between the pixel coordinates in the infrared thermal image and the coordinates of the finite element mesh nodes on the casting surface. After calibration, the standard temperature change curve of the same surface node within the time period corresponding to the measured temperature range at the current detection position is extracted from the temperature data of the intact casting model during the cooling process obtained in step S1. The standard temperature change curve is the theoretical cooling trajectory that a perfect casting model should exhibit in the same spatial location and similar surface temperature range under defect-free conditions.

[0059] S62. Compare the actual temperature change curve of each pixel with the standard temperature change curve frame by frame. If the temperature deviation at multiple consecutive times exceeds the preset threshold, the corresponding surface node is marked as a thermal deviation node, and adjacent thermal deviation nodes are aggregated into a second candidate defect region.

[0060] Specifically, after obtaining the actual temperature change curve and the standard temperature change curve for each pixel, the two curves are compared at each sampling time, and the temperature deviation between them is calculated at each frame. Frame-by-frame comparison and deviation calculation can be implemented in MATLAB using matrix operations; the same operation can also be performed in Python using array operations from the NumPy library.

[0061] When the temperature deviation of a pixel exceeds the preset temperature deviation threshold at multiple consecutive sampling times, it indicates that the surface position of the casting corresponding to the pixel has undergone a temperature evolution that is inconsistent with the standard cooling process during the actual cooling process. This continuous deviation cannot be explained by random noise or instantaneous environmental disturbance, but points to the possible existence of internal defects below the surface node. Therefore, the surface node of the casting corresponding to the pixel is marked as a thermal deviation node.

[0062] After thermal deviation node labeling is completed, spatial connectivity analysis is performed. In the pixel domain of the infrared thermal image, pixels with temperature deviations exceeding the threshold can be set to a foreground value of 1, and the rest can be set to 0 to generate a binary image. Then, the connected components labeling is completed in one call using the OpenCV connectedComponentsWithStats function. This function also returns statistical information such as the bounding rectangle, area, and centroid coordinates of each connected component.

[0063] S63. When the same physical location on the detectable surface is marked as a second candidate defect region at multiple detection positions, it is considered a defect characterization region. Specifically, the pixel-surface node mapping relationship of the infrared thermal imager established in step S61 can be used to uniformly map the second candidate defect regions marked at different detection positions to the same spatial coordinate system of the casting's three-dimensional geometric model, and spatial position comparison can be performed within this unified coordinate system. When the same physical location on the detectable surface is independently marked as a second candidate defect region at multiple detection positions, it indicates that this location continuously exhibits an abnormal thermal response deviating from the standard cooling process under multiple cooling stages and various surface temperature conditions. This abnormality is repeatable both temporally and spatially; therefore, this region is identified as a defect characterization region.

[0064] An indirect but reliable correspondence is established between the defect characterization area and the internal defect. This is the combined identification result of the projected areas of the surface thermal anomalies caused by the internal defect at different detection positions. If a second candidate defect area appears only at a few detection positions and is not confirmed at the same physical location at other detection positions, the area may be caused by factors such as local environmental disturbances, non-uniform surface emissivity, or field of view obstruction, and will not be included in the subsequent processing as a defect characterization area.

[0065] S64. Extract the shape and temperature feature parameters of each defect characterization region as the second feature parameter set. Parameter extraction can be performed in MATLAB using the `regionprops` function to batch calculate the geometric properties of the region. Temperature deviation statistics can be calculated directly from the temperature deviation values ​​of nodes within each region using built-in functions such as `mean`, `max`, and `std`. In Python, the minimum bounding rectangle can be obtained using OpenCV's `cv2.minAreaRect` function, and the temperature deviation statistics can be calculated using statistical functions from the SciPy library. The types, definitions, and extraction calculation methods of the parameters in the second feature parameter set are consistent with those of the first feature parameter set.

[0066] In another technical solution, the defect detection model is a BP neural network model optimized based on particle swarm optimization algorithm, which adopts a three-layer feedforward network structure, including an input layer, at least one hidden layer and an output layer. The input layer has multiple input neuron nodes, each corresponding to a feature parameter in the first feature parameter set; the hidden layer has multiple hidden neuron nodes, each input neuron node is connected to each of the hidden neuron nodes, and each hidden neuron node is connected to the output layer; the output layer has three output neuron nodes, each corresponding to the length, width, and depth of the internal defect. The construction and training process of the defect detection model includes the following steps: B1. Construction and Preprocessing of Training Sample Set: In step S3, the first feature parameter set of the surface temperature anomaly region was extracted for the defective casting model, and it was correlated with the size and depth of the implanted defect to form labeled samples. A training sample set for model learning and evaluation was constructed based on all labeled samples. Each training sample consists of an input feature vector and an output target vector. The input feature vector is composed of the feature parameters in the first feature parameter set arranged in a fixed order, including shape feature parameters and temperature feature parameters; the output target vector is the length, width, and depth of the corresponding internal defect, which are three real numbers. To eliminate the training bias that may be caused by the different dimensions of the feature parameters, each dimension of the input feature vector is subjected to min-max normalization, which linearly maps the feature values ​​to the [0,1] interval; the output target vector is subjected to the same normalization process according to the preset reasonable range of the defect size. After randomly shuffling all samples, they are divided into training set and validation set in a ratio of 7:3 or 8:2, which are used for model parameter learning and generalization performance monitoring, respectively.

[0067] B2. Construct a three-layer feedforward backpropagation (BP) neural network consisting of an input layer, a single hidden layer, and an output layer. The number of nodes in the input layer is equal to the total dimension of the first feature parameter set, i.e., the sum of the number of shape and temperature feature parameters. The number of nodes in the hidden layer is set according to Kolmogorov's theorem or related empirical formulas, and fine-tuned during actual training based on validation performance. The sigmoid function is chosen as the activation function for the hidden layer to provide sufficient nonlinear expressive power. The output layer has 3 nodes, corresponding to the normalized predicted values ​​of defect length, width, and depth, respectively. A linear activation function is used in the output layer to ensure that the predicted value range is not compressed. All weights and thresholds in the neural network are randomly initialized in a uniform distribution within the interval [-0.5, 0.5], and all weights and thresholds to be optimized in the network are concatenated sequentially into a real-valued vector, forming the position vector of each particle in the particle swarm optimization algorithm. The dimension of this vector is completely determined by the network structure and is the sum of the number of weights from the input layer to the hidden layer, the number of weights from the hidden layer to the output layer, and the number of thresholds from the hidden layer to the output layer. The corresponding particle velocity vectors are also initialized to zero or tiny random values ​​in the same dimensional space.

[0068] The hyperparameters of the particle swarm optimization algorithm are set as follows: the population size is typically 30 particles, the individual learning factor and the social learning factor are both set to 2.0, and the inertia weight adopts a dynamic decay strategy, with an initial value of 0.9, which decreases linearly to 0.4 as the iteration progresses. This design enables the algorithm to have strong global exploration capabilities in the early stages of the search and strong local fine-grained search capabilities in the later stages.

[0069] B3. Perform iterative search using the particle swarm optimization algorithm to obtain the optimal initial values ​​for the neural network weights and thresholds. In each iteration, first decode the position vector of each particle into the corresponding BP neural network weights and thresholds, then perform forward propagation using training set samples to calculate the network's predicted defect length, width, and depth for each sample.

[0070] The mean squared error between the predicted values ​​and the true label values ​​across all training samples is used as the fitness evaluation metric for the particle. Since particle swarm optimization typically seeks to maximize fitness, the reciprocal of the mean squared error is taken as the fitness value; the smaller the mean squared error, the higher the fitness. After fitness evaluation, the particle state is adjusted according to the following standard velocity-position update formula: in and They represent the first k In the next iteration, the particle i In the d Dimensional velocity and position, r 1 and r 2 A random number in the range [0,1]. c 1 Individual learning factors c 2 It is a social learning factor.

[0071] The iteration process continues until the iteration convergence condition is met. The iteration termination condition can be set to meet one of the following convergence conditions: reaching the preset maximum number of iterations or the improvement of the global optimal fitness of the population within 20 consecutive generations is less than a threshold.

[0072] When the iteration terminates, the globally optimal position vector is output. As the optimal initial parameter set for a neural network.

[0073] B4. Decode the globally optimal position vector obtained in step B3 and assign it to the BP neural network as the starting point for training weights and thresholds. Supervised iterative training is performed on the training set using the backpropagation algorithm. In each iteration, the normalized input feature vector is fed into the network. After weighted summation in the input layer, sigmoid activation in the hidden layer, and linear output in the output layer, predicted values ​​for defect length, width, and depth are obtained. The mean squared error loss function is used to calculate the deviation between the predicted values ​​and the true label values. The chain rule is used to propagate the error signal backward from the output layer layer by layer, calculating the gradient for each weight and threshold. Parameter updates use gradient descent with momentum, the learning rate controls the update step size, and the momentum term is used to accelerate convergence and suppress oscillations. During training, the validation error is monitored on an independent validation set. When the validation error no longer decreases in multiple consecutive iterations, an early stopping mechanism is triggered to terminate training, and the network parameters with the best validation performance at this point are saved as the completed defect detection model.

[0074] B5. Evaluate the performance of the trained defect detection model to ensure it meets the accuracy requirements for engineering applications. Input validation set or independent test set samples into the model and calculate statistical indicators such as mean absolute error, root mean square error, and coefficient of determination between the predicted defect size and the actual defect size. Generally, the relative errors in defect length and width predictions should be within acceptable ranges, while depth predictions can be appropriately relaxed due to the influence of thermal conduction delay effects. If the evaluation indicators meet the preset accuracy requirements, solidify the model structure and parameters and deploy it to the production line's industrial control computer or edge computing device for online use in step S7; otherwise, adjust the network structure, hyperparameters, or expand the training sample set and repeat steps B1-B4 for optimization until the requirements are met.

[0075] In another technical solution, the reliability calculation formula is: in, Q 0 For confidence level, L、W、D These represent the three prediction dimensions of internal defects: length, width, and depth. m j This represents the average value of the same internal defect across multiple detection locations. s j To obtain the standard deviation of the same internal defect at multiple detection locations. e This is a preset regularization constant to avoid division by zero errors. Confidence level. Q The closer the value is to 1, the higher the consistency of the prediction results of the defect by different detection positions and the more reliable the prediction results of the defect detection model. A lower confidence level indicates that the prediction results are more divergent.

[0076] In another technical approach, physical inversion includes the following process: A1. Centered on the predicted internal defect location by the defect detection model, a local sub-model is cut from the global finite element model of the intact casting model. This local sub-model contains at least one detectable surface and several undetectable surfaces, and contains an undetermined defect. Taking ANSYS software as an example, the cutting operation can be implemented in ANSYS Workbench using the geometric cutting function of SpaceClaim or DesignModeler: Centered on the internal defect location, create a local geometry enclosing the region. Use Boolean operations to cut the corresponding region from the global model, preserving the node distribution of the boundary surfaces. The cut local sub-model contains at least one detectable surface (i.e., the outer surface of the casting that can be captured by an infrared thermal imager during the cooling process) and several undetectable surfaces (i.e., the internal cross-section generated by the cutting and the outer surface of the casting not covered by the infrared thermal imager). Its interior contains an undetermined defect region with an initial range determined by the predicted dimensions.

[0077] A2. For each group of detection positions, perform the following operations on the local sub-model: The temperature of each node on the undetectable surface in the local sub-model is fixed as a time-varying sequence of the temperature of the intact casting model in step S1 at the corresponding spatial location and time segment. This temperature sequence can be retrieved from the ANSYS transient thermal analysis result file by node number and time step, and applied as the first type of boundary condition to the corresponding surface of the sub-model.

[0078] The initial temperature of each node on the detectable surface is assigned to the measured temperature of the corresponding pixel position in the starting frame of the infrared thermal image sequence for the corresponding detection location in step S5. The temperature data of the starting frame of the infrared thermal image sequence can be exported as a matrix of temperature values ​​for each pixel using infrared thermal imager software such as FLIRResearch Studio or FOTRIC Analyzer. Then, the temperature values ​​of each pixel are assigned to the corresponding surface node through the pixel-surface node mapping relationship pre-calibrated in step S6.

[0079] The initial temperature field of the remaining elements within the local sub-model, excluding the region of undetermined defects, is directly estimated using a spatial interpolation algorithm based on the temperature distributions of the detectable and undetectable surface nodes, which have already been assigned temperatures. In MATLAB, the `scatteredInterpolant` function can be used, with nodes of known temperatures on both types of surfaces as interpolation source points, employing linear interpolation or natural neighborhood interpolation methods to directly calculate the initial temperature values ​​of the remaining nodes within the sub-model. Alternatively, the same interpolation estimation operation can be performed in Python using the `griddata` function in SciPy. This interpolation estimation method eliminates the need for additional heat conduction calculations, resulting in high computational efficiency and ensuring a smooth transition of internal node temperatures between known surface temperatures.

[0080] A3. Perform cooling simulations for each local sub-model, recording the predicted temperature change sequence of the detectable surface within the simulation time segment. Perform cooling simulations with defects for each local sub-model under the corresponding conditions at each detection location, recording the predicted temperature change sequence of the detectable surface within the simulation time segment. The cooling simulation with defects is completed in the TransientThermal module of ANSYS Workbench. The simulation duration is set to be consistent with the acquisition duration of the infrared thermal image sequence for the corresponding detection location, and the solution step size is consistent with the frame interval of the infrared thermal image sequence to ensure that the temporal resolution of the simulation output matches the measured data. Before the simulation begins, the thermal conductivity of the defect region to be determined is set to a preset value lower than the thermal conductivity of normal metals to simulate air or low-density oxide filling. The current geometric boundary of the defect region to be determined is described by the defect boundary shape function. After the simulation is completed, the temperature change sequence of each node of the detectable surface within the entire simulation time segment is extracted from the simulation results. This sequence is the predicted temperature change sequence corresponding to the current detection location. If multiple detection locations are involved, the batch processing interface of ANSYS Workbench can be called by MATLAB or Python scripts to automatically drive the sub-models corresponding to each detection location to run sequentially.

[0081] A4. Calculate the similarity between the simulated temperature change sequence and the measured temperature change sequence of the detectable surface in each local sub-model. Calculate the joint similarity evaluation value of the current undetermined defect at multiple detection positions. The similarity evaluation can use metrics such as the sum of squared temperature deviations, correlation coefficients, or dynamic time warping distance. In practice, the predicted temperature sequence and the measured temperature sequence can be imported into MATLAB respectively. Matrix operations are used to calculate the squares of the temperature deviations at corresponding nodes at each time step and sum them to obtain the similarity evaluation value at the current detection position. The similarity evaluation values ​​of each detection position are then weighted and summed according to preset detection position weight coefficients to obtain the joint similarity evaluation value of the current undetermined defect model at multiple detection positions.

[0082] A5. With the goal of achieving the optimal joint similarity evaluation value, an iterative optimization algorithm is used to update the boundary shape function and depth of the undetermined defect. A2 to A4 are repeated until convergence or the preset number of iterations is reached. The converged boundary shape function and depth are used as the final defect label. The iterative optimization framework can be implemented using the `fminunc` or `fmincon` functions in MATLAB. In each iteration, the MATLAB main control script updates the input parameter file of ANSYS Workbench based on the current defect parameters. A batch processing command is used to call ANSYS to perform a cooling simulation with defects. After the simulation is completed, the result file is automatically read and the joint similarity evaluation value is calculated. The optimization algorithm updates the defect parameters based on the current evaluation value and gradient information, and then proceeds to the next iteration. Gradient information can be obtained through the finite difference method or by directly calculating the gradient of the objective function with respect to the design variables using the adjoint solver of ANSYS Fluent. The iterative convergence condition can be set as the objective function value falling below a preset convergence threshold, or the relative change in several consecutive iterations being less than a preset proportion. The converged defect boundary shape function and depth directly provide the length, width, and depth of the defect, which are used as the final defect label for that internal defect.

[0083] In this technical solution, the entire physical inversion process is automatically executed by software and calculation programs, eliminating the need for manual step-by-step operation. The local sub-model cutting, boundary condition application, flawed cooling simulation, temperature sequence extraction, similarity evaluation value calculation, and iterative updates of defect boundary shape functions and depths involved in A1~A5 are all uniformly scheduled through a master control script written in MATLAB or Python. The master control script automatically calls the ANSYS Workbench batch processing interface to execute finite element simulations according to a preset process. After the simulation is completed, it automatically reads the result files and performs data post-processing and optimization algorithm updates, forming a closed-loop automated iterative process. The final defect label is automatically output after the convergence condition is met. The entire inversion process can be completed unattended.

[0084] Regarding computational requirements, since physical inversion is triggered only on a few questionable internal defects with insufficient confidence, rather than being performed on the entire surface area of ​​every casting, the overall computational load is manageable. The local sub-model covers only a limited area around the predicted defect location, with a mesh size much smaller than the global casting model. A single simulation of cooling with defects typically takes several to tens of seconds. Iterative optimization usually converges after dozens of iterations, and a complete inversion of a single defect can be completed within minutes. These computational tasks can run on industrial control computers or edge computing devices deployed on the production line. To further improve processing efficiency, simulation tasks can be distributed to workshop-level servers or cloud computing nodes. The multi-core processors in current mainstream industrial control computers or edge computing devices are sufficient to support the aforementioned computational overhead, meeting production line cycle time requirements without relying on large computing clusters.

[0085] In another technical solution, the heat conduction process within the local sub-model is described by a three-dimensional unsteady-state heat conduction equation. Corresponding temperature characteristic parameters are extracted from both the simulated and measured temperature change sequences. These temperature characteristic parameters include the regional average temperature, the regional maximum temperature difference, and the arrival time of the temperature difference peak. The three-dimensional unsteady-state heat conduction equation is automatically discretized and numerically solved in the Transient Thermal module of ANSYS Workbench without the need for manually writing solver code.

[0086] After each simulated cooling process with damage, the temperature change sequence of each node on the detectable surface is extracted from the simulation results over time. Simultaneously, the measured temperature change sequence of each node at the same surface location is extracted from the infrared thermal image sequence of the corresponding detection site acquired in step S5. Corresponding temperature feature parameters are extracted from both the simulated and measured temperature change sequences. These temperature feature parameters may include the regional average temperature, the regional maximum temperature difference, and the time to reach the peak temperature difference.

[0087] After each simulation of cooling with defects, temperature distribution data of the detectable surface is extracted from the simulation results. To make a meaningful comparison with measured data, it is first necessary to identify temperature anomaly regions caused by internal defects from the temperature data. In the simulation results, the temperature change sequence of each node on the detectable surface over time is extracted and compared with the standard cooling temperature sequence of the intact casting model at the corresponding surface location in step S1. Nodes whose temperature deviation continuously exceeds a preset threshold are marked as simulated abnormal nodes. Adjacent simulated abnormal nodes are aggregated into simulated abnormal regions through spatial connectivity analysis. On the measured data side, the defect characterization region has already been marked from the infrared thermal image sequence in step S6; this defect characterization region is the measured abnormal region. Thus, the simulated abnormal region and the measured abnormal region correspond to each other in spatial location, both pointing to the thermal response projection range of the internal defect on the casting surface.

[0088] Temperature characteristic parameters were extracted from both the simulated and measured anomaly regions. These extracted parameters included the region's average temperature, maximum temperature difference, and peak temperature arrival time. The region's average temperature was defined as the arithmetic mean of the temperatures of all nodes within the anomaly region over the entire time segment, reflecting the overall temperature level of the defective region during the cooling process. The maximum temperature difference was defined as the maximum temperature difference between each node in the anomaly region and its corresponding node in the intact region over the entire time segment, reflecting the maximum disturbance intensity of the internal defect on the surface temperature field. The peak temperature arrival time was defined as the moment when the temperature difference in the anomaly region first rose to its maximum, reflecting the influence of the internal defect depth on the heat conduction delay.

[0089] Another possible approach for comparison is frame-by-frame, pixel-by-pixel comparison. This involves calculating the deviation between the simulated and measured temperatures for each node on the detectable surface at each sampling time, and then summing the squared deviations of all nodes and all time points as the similarity evaluation value. This approach has the highest information utilization rate, but its computational cost is significant. This technical solution, while retaining core temperature anomaly discrimination information, significantly reduces the computational load of a single similarity evaluation, better meeting the real-time requirements of online production line inspection. It also exhibits strong resistance to interference factors such as inter-frame noise from infrared thermal imagers and uneven local surface emissivity, which is beneficial for maintaining the stability and convergence consistency of the similarity evaluation during multi-detection-position joint iterative inversion.

[0090] The formula for calculating the joint similarity score is: in, G To detect the number of bit groups, c g For the first g Group weighting coefficients M The types of characteristic parameters for temperature characteristics. w j For the first j The weights of each feature parameter; For the first g Group 1 in the simulated temperature change sequence j A dimensionless feature parameter For the first g Group 1 in the measured temperature change sequence j Dimensionless feature parameters, of which the number of detection bit groups G The weighting coefficient of each detection bit is determined by the number of detection bits actually deployed and from which valid data was collected. c g The settings can be differentiated based on factors such as the temperature measurement accuracy, field-of-view coverage integrity, or signal-to-noise ratio of the infrared thermal imager at the detection location. In the absence of prior knowledge, all settings can be set to 1; the weights of each feature parameter... w j Sensitivity analysis can be used to determine the sensitivity of each feature parameter to changes in defect size and depth through simulation experiments. Feature parameters with higher sensitivity are assigned greater weights, and the sum of all weights can be normalized to 1. In the initial implementation, they can be evenly distributed.

[0091] With the objective of minimizing the joint similarity evaluation value, the gradient descent method is used to simultaneously evaluate the defect boundary shape function. f (i) and defect depth D Perform iterative updates: in, n This represents the current iteration number. for J right f(θ) The variational derivative, for J right D Partial derivatives, step size and Determined through line search.

[0092] To minimize the joint similarity evaluation value J To achieve the objective, the gradient descent method is used to simultaneously analyze the defect boundary shape function. f (i) and defect depth D Perform iterative updates. The meaning and value assignment of each symbol in the updated formula are as follows: n This is the current iteration number, starting from 0. f n (i) For the first n The defect boundary shape function at each iteration is stored in the computer as a coordinate vector of a set of discrete control points.

[0093] Specifically, a local polar coordinate system is established in a plane parallel to the casting surface, with the geometric center of the defect region as the origin. The boundary profile of the defect in this plane is represented as a function of the polar angle θ and discretized into N control points, each control point corresponding to a fixed polar angle. i i and its extreme diameter value r i , r 1 ~ r N constitute f(θ) The parameterized vector. N can be 8 to 20 to balance the expressive power of the boundary shape and the efficiency of iterative computation. In ANSYS Workbench's SpaceClaim or DesignModeler, the defect boundary contour curve is generated using the control point coordinates, and then stretched or rotated along the wall thickness direction to generate the three-dimensional geometry of the defect. The boundary of this geometry is determined by the current iteration step. f(θ) The parameter vector uniquely determines this. During iterative updates... f(θ) In this case, you only need to modify the extreme radius values ​​of each control point and regenerate the geometry. D n For the first n The defect depth value at the next iteration is represented by a length scalar in the wall thickness direction. , They represent JRates of change along the direction of shape perturbation and the direction of depth perturbation.

[0094] In the update formula for J right f(θ) The variational derivative, in f(θ) When parameterized as a control point radial quantity, this variational derivative is transformed into J For each control point polar radius r i The gradient vector formed by the partial derivatives of . Update the formula for J The partial derivative with respect to the defect depth D.

[0095] Gradient information can be obtained in at least two ways, the first of which is the finite difference method: for each component in the current parameter vector r i Apply small positive perturbations in sequence Δr , Δr It can be taken as 1% to 5% of the current extreme diameter value or a fixed value; for depth D Applying a small perturbation individually ΔD , ΔD Use 1% to 5% of the current depth value or a fixed value; regenerate the defect geometry for the parameters after the disturbance and perform a cooling simulation with the defect, calculating the values ​​before and after the disturbance. J The ratio of the value difference to the perturbation is used as the approximate gradient.

[0096] The second method is the adjoint method: it directly calculates the gradient of the objective function with respect to the design variables using the adjoint method based on the finite element model, and directly solves the objective function after completing the positive simulation of cooling with damage. J Sensitivity to finite element mesh node coordinates. Establishing control point parameters. r i The mapping relationship between the coordinates of each mesh node on the defect boundary. For any mesh node on the defect boundary, based on its polar angle in the local polar coordinate system, the coordinates of the nodes are mapped to the coordinates of the nodes at two adjacent control points. (i i , r i ) and (i i+1 , r i+1 ) Linear interpolation or spline interpolation is performed between the nodes to obtain the extreme radius value of that node. r(θ) This allows us to determine its spatial coordinates. After obtaining this mapping relationship, the chain rule is applied. The node coordinate sensitivity is transformed into a gradient with respect to the control point parameters, where the sensitivity is traversed along the defect boundary. ri All affected mesh nodes, x k For the first k The spatial coordinate vector of each node For the output of the solver J Sensitivity to node coordinates For node coordinates to control point parameters r i The partial derivatives are analytically obtained using the interpolation mapping relationship described above. This method requires only one forward simulation and one adjoint solution to obtain all gradient components, resulting in high computational efficiency.

[0097] The method of obtaining the result is similar to that described above: In the finite difference method, for D Rerun the simulation and calculate after applying the disturbance separately. J The difference; in the adjoint method, it can be obtained by performing sensitivity analysis and chain rule transformation on the grid node coordinates in the wall thickness direction of the defect region.

[0098] Step length and The update magnitudes of shape and depth in each iteration can be controlled separately using Armijo conditions or a backtracking search method: starting with a unit step size, the step size is gradually reduced until the decrease in J reaches a preset proportion. In MATLAB, this can be achieved by writing a backtracking search subfunction. The iteration terminates when the value of J falls below a preset convergence threshold, or when the relative change in J is less than a preset proportion in several consecutive iterations, or when the preset maximum number of iterations is reached. (The last sentence appears to be incomplete and possibly refers to a concluding statement about convergence.) f(θ) Directly provide the length and width of the defect, and the converged value. D The defect depth is directly given and used as the final defect marker for that internal defect. The entire iterative optimization process is automatically scheduled and executed by a MATLAB or Python main control script, including calling the ANSYS Workbench batch processing interface to run simulations, read results, calculate objective functions and gradients, and update defect parameters, without manual intervention.

[0099] Another technical approach also includes: On the cooling line, visible light images of each detectable surface of the casting are acquired synchronously by an industrial camera. A target detection model is used to identify surface defects in the visible light images to obtain the surface defects present on the cooled casting. The location of the surface defects is compared with the defect characterization area obtained in step S6, and defect characterization areas that correspond to the location of the surface defects are screened out.

[0100] Industrial cameras can be CMOS area array cameras with GigE Vision interface, with a resolution no lower than that of infrared thermal image sequences. The frame rate should be consistent with or an integer multiple of the acquisition frame rate of the infrared thermal imager. The acquisition and processing of visible light images can be achieved through the OpenCV library to realize image acquisition and trigger synchronization control.

[0101] Visible light images acquired at each detection location are input into a pre-trained target detection model to automatically identify surface defects present on the detectable surface of the casting in the visible light images. The target detection model can employ deep learning-based target detection networks such as YOLOv8 or Faster R-CNN. The identified surface defects refer to casting defects present on the outer surface of the casting, such as surface cracks, cold shuts, surface porosity, and inclusions. These surface defects have identifiable visual features in visible light images, but they can also cause abnormal surface temperatures in infrared thermal images.

[0102] To ensure spatial correspondence, it is necessary to compare the spatial locations of surface defects identified in the visible light image with the marked defect characterization areas in the infrared thermal image. The comparison is based on the pre-established mapping relationship between the pixel coordinates of the visible light image, the pixel coordinates of the infrared thermal image, and the physical location of the casting surface. The bounding box of the surface defect in the visible light image coordinate system is uniformly transformed to the physical coordinate system of the casting surface.

[0103] If the spatial location of a certain external defect coincides with or has an overlap exceeding a preset threshold with the defect characterization area marked in step S6, then it is determined that the surface temperature anomaly of the defect characterization area is caused by the external defect, rather than the thermal response projection of an internal defect on the surface. This defect characterization area is then removed from the list of candidate areas for predicting internal defects, and no further feature parameter extraction or defect detection model inference is performed on it.

[0104] In another technical solution, the comprehensive score includes: in, α The weighting coefficient for surface defect scoring ranges from 0 to 1. Its specific value can be configured according to the relative sensitivity of surface quality and internal quality of the casting under inspection in terms of its service conditions.

[0105] S surf and S int Each external and internal defect is scored independently based on its shape and size. P This represents the total number of external defects. Q This represents the total number of internal defects. Ssurf and S int Based on the length, width, and depth of the defect, the defect's geometric parameters can be converted into a score between 0 and 100 according to a preset scoring mapping function. When DI It is considered qualified when it is less than the first threshold. DI If the value exceeds the second threshold, it is considered scrapped. DI When the value falls between the first threshold and the second threshold, it is determined to be sent for final CT evaluation. Through the above-mentioned diversion mechanism, CT evaluation resources are concentrated only on questionable castings whose scores fall in the middle gray area, thereby achieving a balance between detection efficiency and detection accuracy.

[0106] In another technical solution, in step S5, the cooling casting undergoes a standard cooling process at each detection point. Since the standard cooling process simulated by the intact casting model in step S1 is based on preset ambient temperature, airflow velocity, and heat transfer conditions, to ensure that the actual cooling behavior of the casting during the online acquisition phase approximates this simulation benchmark as closely as possible, environmental control measures can be implemented at each detection point to ensure the casting is under standard cooling conditions when passing through the detection point. For example, closed or semi-closed isolation covers or baffles can be installed at each detection point to shield against random airflow disturbances caused by equipment operation, ventilation, and personnel movement within the workshop, thereby maintaining the relative stability of the airflow field within the detection point. Simultaneously, temperature sensors can be installed inside the isolation covers to monitor the ambient temperature in real time, feeding the measured ambient temperature back to the parameter correction in subsequent comparison stages. For each detectable surface of the casting, ensure that its heat exchange with the outside world is primarily through natural convection and surface radiation, avoiding interference from non-uniform heat dissipation methods such as localized forced cooling or spraying on the surface temperature distribution.

[0107] It should be noted that although the steps are described in a specific order above, this does not mean that they must be performed in that order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required functionality is achieved. The number of devices and processing scale described herein are for simplification of the invention; applications, modifications, and variations of this invention will be readily apparent to those skilled in the art.

[0108] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A method for detecting defects in semiconductor equipment castings based on visual recognition, characterized in that, Includes the following steps: S1. Establish a finite element model to simulate the standard cooling process of the intact casting model and obtain the cooling process temperature data of each surface of the intact casting model. S2. Inject internal defects of different sizes and depths into the finite element model to simulate the cooling process of the damaged casting model and obtain the cooling temperature data of each surface of the damaged casting model. S3. Based on the difference in temperature data of the cooling process of each surface of the intact casting model and the damaged casting model, determine the surface temperature anomaly region corresponding to each internal defect, and extract the first feature parameter set representing the temperature and shape of each surface temperature anomaly region. S4. Using the first feature parameter set as input and the size and depth of the corresponding internal defects as labels, construct a training sample set and train the defect detection model. S5. Multiple sets of detection positions are set at intervals along the cooling line. Each detection position collects a set of infrared thermal image sequences of each detectable surface of the cooled casting at different temperatures. S6. For each detectable surface infrared thermal image sequence, based on the mapping relationship between the infrared thermal image pixel coordinates and the physical position of the casting surface, the casting surface area that deviates from the standard cooling process is marked as the defect characterization area, and the second feature parameter set characterizing its temperature and shape is extracted. S7. For internal defects in the same defect characterization area, input the second feature parameter set extracted from each detection position into the defect detection model to obtain multiple prediction results for the same internal defect. S8. Calculate the confidence level of multiple prediction results. When the confidence level calculation meets the preset conditions, the prediction result is used as the defect label of the internal defect. When the confidence level does not meet the preset conditions, perform physical inversion on the internal defect based on multiple infrared thermal image sequences, and use the physical inversion result as the defect label of the internal defect. S9. Based on the defect markings, the castings are comprehensively scored, and the castings are judged as qualified, scrapped or sent to CT for final determination according to the scoring results. Physical inversion includes the following processes: A1. Using the location of the internal defect predicted by the defect detection model as the center, cut out a local sub-model from the global finite element model of the intact casting model. The local sub-model contains at least one detectable surface and several undetectable surfaces, and contains undetermined defects inside. A2. For each local sub-model corresponding to a detection position, perform the following operations: Fix the temperature of each node on the undetectable surface to the temperature change sequence of the intact casting model at the corresponding spatial position and time segment; Assign the initial temperature of each node on the detectable surface to the initial temperature of the corresponding pixel in the infrared thermal image sequence, and apply heat dissipation boundary conditions to the detectable surface; The initial temperature of the remaining nodes in the local sub-model is estimated from the initial temperature distribution of the detectable and undetectable surfaces. A3. Perform cooling simulations for each local sub-model and record the simulated temperature change sequence of the detectable surface within the simulation time segment; A4. Calculate the similarity between the simulated temperature change sequence and the measured temperature change sequence of the detectable surface in each local sub-model, and calculate the joint similarity evaluation value of the current undetermined defect under multiple detection positions. A5. With the goal of achieving the optimal joint similarity evaluation value, the boundary shape function and depth of the undetermined defect are updated using an iterative optimization algorithm. A2 to A4 are repeated until convergence or the preset number of iterations is reached. The converged boundary shape function and depth are used as the final defect label.

2. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, Step S3 includes the following steps: S31. Divide the cooling process temperature data of the intact casting model and the damaged casting model into multiple time segments with the same duration as the infrared thermal image sequence. S32. Compare the temperature deviation of the same node in the intact casting model and the damaged casting model frame by frame within the time segment. Nodes with deviations exceeding the preset temperature threshold in multiple consecutive frames are identified as thermal deviation nodes. Adjacent thermal deviation nodes are aggregated into the first candidate defect area. S33. Count the frequency of each first candidate defect region in all time segments, and determine the candidate regions with a frequency higher than the preset proportion threshold as surface temperature abnormal regions. S34. Extract the shape and temperature feature parameters of the surface temperature anomaly region to form the first feature parameter set of the surface temperature anomaly region.

3. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, Step S6 includes: S61. For a detectable surface, extract the actual temperature change curve of each pixel in the infrared thermal image sequence, and use the pre-calibrated pixel-surface node mapping relationship to extract the standard temperature change curve of the same surface node in the corresponding detection position temperature range from the intact casting model. S62. Compare the actual temperature change curve of each pixel with the standard temperature change curve frame by frame. If the temperature deviation at multiple consecutive times exceeds the preset threshold, the corresponding surface node is marked as a thermal deviation node, and adjacent thermal deviation nodes are aggregated into a second candidate defect area. S63. When the same physical location on the detectable surface is marked as the second candidate defect region at multiple detection positions, it is taken as the defect characterization region. S64. Extract the shape and temperature feature parameters of each defect characterization region as a second feature parameter set.

4. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, The defect detection model is a BP neural network model optimized based on particle swarm optimization algorithm, and adopts a three-layer feedforward network structure, including an input layer, at least one hidden layer and an output layer. The input layer has multiple input neuron nodes, each corresponding to a feature parameter in the first feature parameter set; the hidden layer has multiple hidden neuron nodes, each input neuron node is connected to each hidden neuron node, and each hidden neuron node is connected to the output layer; the output layer has three output neuron nodes, each corresponding to the length, width, and depth of the internal defect. The construction and training process of the defect detection model includes: using the first feature parameter set of each sample in the training sample set as the input value of the input neuron node, and using the size parameter and depth information of the corresponding defect as the training target; using the particle swarm optimization algorithm to globally optimize the initial weights and thresholds of the BP neural network model, and using the prediction mean square error of the BP neural network model on the training sample set as the fitness function of each particle in the particle swarm; assigning the optimal initial weights and thresholds obtained by optimization to the BP neural network model, and performing iterative training through the error backpropagation algorithm; when the error index of the output result reaches the preset requirement, the training is completed, and the defect detection model is obtained.

5. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, The confidence level is calculated using the following formula: ； in, Q 0 For confidence level, L, W, D These represent the three prediction dimensions of internal defects: length, width, and depth. μ j This represents the average value of the same internal defect across multiple detection locations. σ j To obtain the standard deviation of the same internal defect at multiple detection locations. ε This is a preset regularization constant.

6. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, The heat conduction process within the local sub-model is described by a three-dimensional unsteady heat conduction equation, and the corresponding temperature characteristic parameters are extracted from the simulated temperature change sequence and the measured temperature change sequence, respectively. The formula for calculating the joint similarity score is: ； in, G To detect the number of bit groups, γ g For the first g Group weighting coefficients M The types of characteristic parameters for temperature characteristics. w j For the first j The weights of each feature parameter; For the first g Group 1 in the simulated temperature change sequence j A dimensionless feature parameter For the first g Group 1 in the measured temperature change sequence j A dimensionless characteristic parameter; With the objective of minimizing the joint similarity evaluation value, the gradient descent method is used to simultaneously evaluate the defect boundary shape function. f(θ) and defect depth D Perform iterative updates: ； in, n This represents the current iteration number. for J right f(θ) The variational derivative, for J right D Partial derivatives, step size and Determined through line search.

7. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, Also includes: On the cooling line, visible light images of each detectable surface of the casting are simultaneously acquired by an industrial camera. A target detection model is used to identify surface defects in the visible light images to obtain the surface defects present on the cooled casting. The location of the surface defect is compared with the spatial location of the defect characterization area obtained in step S6, and defect characterization areas that have a positional correspondence with the surface defect are filtered out.

8. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 7, characterized in that, The overall score includes: ； in, α This is a weighting coefficient for the external defect scoring, with a value ranging from 0 to 1. S surf and S int Each external and internal defect is scored independently based on its shape and size. P This represents the total number of external defects. Q This represents the total number of internal defects. when DI It is considered qualified when it is less than the first threshold. DI If the value exceeds the second threshold, it is considered scrapped. DI If the value is between the first threshold and the second threshold, it is determined that the document should be sent to CT for final determination.

9. The method for detecting defects in semiconductor equipment castings based on visual recognition as described in claim 1, characterized in that, In step S5, the cooling casting is subjected to a standard cooling process at each detection point.

Images