Intelligent detection system and method for die casting machining fused with visual identification

Abstract

The invention provides a die casting machining intelligent detection system and method fused with visual identification, and the method comprises the steps: carrying out the combined preprocessing of the surface visual image data of different process nodes and the physical signal of a die casting material in the machining process of a target die casting, and obtaining the surface effective data and physical effective data related to the defects of the die casting; extracting surface defect features of the target die casting in the surface effective data, extracting physical defect features of the target die casting material in the physical effective data, and fusing the surface defect features and the physical defect features into associated fusion defect features of the target die casting; and obtaining space-time coordinate information of the sensor network, and determining defect characterization information including the defect type and the defect position of the target die casting machining defect according to the associated fusion defect features and the space-time coordinate information. By adopting the scheme of the invention, defect detection can be carried out on the processing of the die casting based on the collaborative association between the surface and physical defect characteristics.

Description

Technical Field

[0001] This application relates to the field of processing and inspection technology, and more specifically, to an intelligent inspection system and method for die-casting parts processing that integrates visual recognition. Background Technology

[0002] Processing inspection refers to the process of inspecting and measuring the dimensions, shape, position, and surface quality of machined parts using precision measuring equipment and techniques. It is a core link in modern manufacturing systems to ensure product quality. Through real-time or post-processing inspection, it verifies whether the processed parts meet the design drawings and process requirements.

[0003] Die castings are widely used in high-end manufacturing fields such as automobiles and aerospace due to their high forming efficiency and strong adaptability of mechanical properties. The surface and internal quality of die castings directly determine the reliability of end products. Efficient and accurate processing and inspection have become core requirements of the industry. Traditional inspection methods have significant limitations: manual visual inspection relies on experience, is inefficient and prone to missing minute defects; single visual sensors can only capture surface morphological anomalies and cannot identify internal physical defects such as material porosity and stress concentration; single physical parameter sensors (such as temperature and stress sensors) can monitor material characteristics, but it is difficult to correlate them with the specific shape and location of surface defects. At the same time, existing technologies do not make full use of historical defect data to guide inspection, the effective data extraction rate in the preprocessing stage is low, and surface and physical defect features are treated in isolation, losing the information on their synergistic relationship. Therefore, how to perform defect detection on die castings based on the synergistic relationship between surface and physical defect features has become a problem faced by the industry. Summary of the Invention

[0004] This application provides an intelligent inspection system and method for die casting processing that integrates visual recognition, which can detect defects in die castings based on the synergistic correlation between surface and physical defect features.

[0005] In a first aspect, this application provides an intelligent inspection method for die-casting part processing that integrates visual recognition, wherein a sensor network including visual sensors and physical parameter sensors is pre-deployed on the target die-casting part, and the method includes the following steps: The surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting are collected through a sensor network. Based on the historical defect features corresponding to the target die casting, the surface visual image data and the physical signal of the die casting material are jointly preprocessed to obtain effective surface data and effective physical data related to the die casting defects. Surface defect features of the target die casting are extracted from the effective surface data, and physical defect features of the target die casting material are extracted from the effective physical data. The surface defect features and the physical defect features are correlated and fused to obtain the correlated and fused defect features of the target die casting. The spatiotemporal coordinate information of the sensor network is obtained, and the defect characterization information, including the defect type and defect location of the target die casting processing defect, is determined based on the associated fusion defect features and the spatiotemporal coordinate information.

[0006] In some embodiments, the physical signals of the die-casting material include the processing temperature, internal stress, material density, and surface hardness of the die-casting.

[0007] In some embodiments, joint preprocessing of the surface visual image data and the physical signal of the die casting material based on the historical defect features corresponding to the target die casting to obtain effective surface data and effective physical data related to die casting defects specifically includes: Obtain the historical defect features corresponding to the target die casting; Anomaly processing is performed on the surface visual image data and the physical signal of the die casting material to obtain anomaly-processed surface visual image data and physical signal of the die casting material. Based on the correlation rules between surface defects and physical defects in the historical defect features, cross-validation is performed on the surface visual image data after anomaly processing and the physical signals of the die casting material to obtain effective surface data and effective physical data related to die casting defects.

[0008] In some embodiments, extracting surface defect features of the target die casting from the effective surface data specifically includes: The suspected defect areas of the target die casting are extracted from the effective surface data; The geometric, textural, and contour features of the target die casting are determined based on the suspected defect areas. The surface defect features of the target die casting are selected from geometric features, texture features, and contour features based on the historical defect features corresponding to the target die casting.

[0009] In some embodiments, extracting the physical defect features of the target die-casting material from the physical effective data specifically includes: Extract data features from each dimension of the physical valid data; Based on the historical defect features corresponding to the target die casting, the physical defect features of the target die casting material are extracted from all data features.

[0010] In some embodiments, the correlation and fusion of the surface defect features and the physical defect features to obtain the correlation and fused defect features of the target die casting specifically includes: The surface defect features and the physical defect features are time-series aligned to construct a feature time-series correlation matrix; Obtain the correlation rules between surface defects and physical defects in the historical defect features; Based on the aforementioned correlation rules, the feature temporal correlation matrix is ​​fused in layers to obtain the correlation fusion defect features of the target die casting.

[0011] In some embodiments, determining defect characterization information, including the defect type and location of the target die-casting machining defect, based on the associated fusion defect features and the spatiotemporal coordinate information specifically includes: Pre-build a defect type-association fusion feature mapping library; The defect type of the target die casting is obtained by comparing the correlation fusion defect features with the defect type-correlation fusion feature mapping library. The actual coordinate information of the surface defect of the target die casting is located based on the spatiotemporal coordinate information transformation and the associated fusion defect features. Based on the defect type and the actual coordinate information, defect characterization information, including the defect type and defect location of the target die casting machining defect, is determined.

[0012] Secondly, this application provides an intelligent inspection system for die-casting parts processing that integrates visual recognition, comprising: The acquisition module is used to acquire surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting part through a sensor network; The processing module is used to perform joint preprocessing on the surface visual image data and the physical signal of the die casting material based on the historical defect features corresponding to the target die casting, so as to obtain effective surface data and effective physical data related to the die casting defects. The processing module is also used to extract surface defect features of the target die casting from the surface effective data, extract physical defect features of the target die casting material from the physical effective data, and correlate and fuse the surface defect features and the physical defect features to obtain the correlated and fused defect features of the target die casting. The execution module is used to acquire the spatiotemporal coordinate information of the sensor network, and determine the defect characterization information, including the defect type and defect location of the target die casting processing defect, based on the associated fusion defect features and the spatiotemporal coordinate information.

[0013] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described intelligent detection method for die-casting parts processing that integrates visual recognition.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intelligent detection method for die-casting parts processing incorporating visual recognition.

[0015] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects: The intelligent inspection system and method for die-casting parts processing based on fused visual recognition provided in this application firstly acquires visual images and physical signals by deploying a sensor network at each processing node, ensuring comprehensive data acquisition from both appearance and internal material properties, providing a foundation for collaborative defect analysis. Secondly, joint preprocessing based on historical defect features effectively filters out surface and physical data related to defects, reducing noise interference. Then, by extracting and associating fused surface defect features and physical defect features, complementary information on appearance and internal material properties is achieved, enhancing the reliability of defect identification and avoiding the shortcomings of a single data source. Finally, by combining the spatiotemporal coordinate information of the sensor network, the defect type and location are accurately determined, making the detection results more operable. Thus, overall, the efficiency and accuracy of die-casting part processing defect detection are improved through the collaborative association of surface and physical features. Using the scheme of this application, defect detection of die-casting parts processing can be performed based on the collaborative association between surface and physical defect features. Attached Figure Description

[0016] Figure 1 This is an exemplary flowchart of an intelligent inspection method for die-casting parts processing that integrates visual recognition, as shown in some embodiments of this application. Figure 2 This is an exemplary flowchart illustrating the determination of surface defect features according to some embodiments of this application; Figure 3 This is an exemplary flowchart illustrating the determination of defect characterization information according to some embodiments of this application; Figure 4 This is a schematic diagram of the structure of an intelligent inspection system for die casting processing that integrates visual recognition, as shown in some embodiments of this application; Figure 5 This is a schematic diagram of the structure of a computer device for implementing an intelligent inspection method for die-casting part processing that integrates visual recognition, according to some embodiments of this application. Detailed Implementation

[0017] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] refer to Figure 1The figure is an exemplary flowchart of an intelligent inspection method for die casting processing that integrates visual recognition, according to some embodiments of this application. This intelligent inspection method for die casting processing that integrates visual recognition mainly includes the following steps: The pre-deployment of a sensor network including vision sensors and physical parameter sensors around the target die casting can be achieved in the following way: according to each process station of die casting, deploy the sensor network around the die casting. The vision sensors use 20-megapixel industrial CCD cameras, with 3-4 units evenly distributed around the periphery at each station, and the lens is 50-80cm away from the surface of the die casting. The physical parameter sensors include temperature and stress sensors, as well as density detectors and hardness testers, which are installed at the mold cavity exit, tool contact end, transfer fixture and finished product inspection table, respectively.

[0019] In step 101, surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting are collected through a sensor network.

[0020] It should be noted that the surface visual image data in this application represents the surface image information of each process node in the processing of the target die casting, reflecting the surface morphology, appearance integrity and surface defect related status of the die casting; the die casting material physical signal represents the physical quantity data related to the inherent properties of the die casting material and the material state during processing, reflecting the core material characteristics such as the uniformity of the die casting material, the stability of the internal structure and mechanical properties, and thermal properties. The die casting material physical signal includes physical parameters such as the processing temperature, internal stress, material density and surface hardness of the die casting.

[0021] In step 102, the surface visual image data and the physical signal of the die casting material are jointly preprocessed based on the historical defect features corresponding to the target die casting to obtain effective surface data and effective physical data related to the die casting defects.

[0022] In some embodiments, the joint preprocessing of the surface visual image data and the physical signal of the die casting material based on the historical defect features corresponding to the target die casting to obtain effective surface data and effective physical data related to die casting defects can be achieved through the following steps: Obtain the historical defect features corresponding to the target die casting; Anomaly processing is performed on the surface visual image data and the physical signal of the die casting material to obtain anomaly-processed surface visual image data and physical signal of the die casting material. Based on the correlation rules between surface defects and physical defects in the historical defect features, cross-validation is performed on the surface visual image data after anomaly processing and the physical signals of the die casting material to obtain effective surface data and effective physical data related to die casting defects.

[0023] It should be noted that the historical defect features in this application represent a standardized set of information formed by characterizing and refining various defects found in the past processing of the same model and production process conditions of the target die casting. This reflects the types of defects that are prone to occur in the same production scenario for this type of die casting, the correlation between defects and processing parameters, and the key quantitative basis for defect judgment. Historical defect features include two parts: historical surface defect features and historical material physical defect features. Historical surface defect features cover the visual quantitative parameters of typical defects such as surface cracks, porosity, dents, scratches, and burrs. For example, geometric thresholds for crack width ≥ 0.1 mm and porosity diameter ≥ 0.3 mm, grayscale ranges where the defect area is 15-30 grayscale levels lower than the normal area, and contrast parameters with an edge gradient ≥ 20. Historical material physical defect features include temperature, internal stress, material density, and surface hardness during the die casting processing. The system includes the normal fluctuation range and abnormal fluctuation threshold of core physical parameters such as temperature; it also includes the correlation rules between surface defects and material physical defects. All features are derived from statistical analysis of historical defect detection data and stored in quantitative form. Among them, the correlation rules between surface defects and physical defects in historical defect features include the combination relationship of feature parameters of surface defects and physical defects, the probability threshold of synergistic occurrence, the allocation of feature contribution weights, and the validity judgment conditions. The correlation rules represent the statistically verified standardized mapping relationship between abnormal surface morphology and abnormal material physical properties of this type of die casting under the same production process conditions. It reflects the inherent causal relationship or synergistic evolution law between changes in the physical state of the material and the generation of surface defects during the die casting process, directly revealing the root cause of surface defects and the degree of influence of abnormal physical parameters on surface quality, and providing a quantitative basis for the correlation and fusion of the two types of defect features.

[0024] In specific implementation, the surface visual image data and the physical signal of the die-casting material are subjected to anomaly processing to obtain the anomaly-processed surface visual image data and the physical signal of the die-casting material. This can be achieved in the following way: Anomaly processing is performed on the surface visual image data. First, a 3×3 Gaussian filter kernel is used for noise reduction. Then, image noise is removed by weighted averaging of neighboring pixels. Next, the color image is converted to a grayscale image using a weighted averaging method. Finally, the Sobel operator is used to calculate the pixel gradient along the x-axis and y-axis respectively to enhance the edge contrast between the defect area and the normal area, thus obtaining the anomaly-processed surface visual image data. Anomaly processing is performed on the physical signal of the die-casting material. A mean filter is performed using 10 consecutive data points as a sliding window to smooth signal fluctuations. The mean μ and standard deviation σ of all data for this parameter are calculated. Outliers <μ-3σ or >μ+3σ are removed to eliminate sudden interference data. Then, the min-max normalization formula is used to normalize each physical parameter to the [0,1] interval, thus obtaining the anomaly-processed physical signal of the die-casting material. In some embodiments, other methods can also be used to determine this, which are not limited here.

[0025] In addition, in specific implementation, based on the correlation rules between surface defects and physical defects in the historical defect features, cross-validation is performed on the surface visual image data after anomaly processing and the physical signals of the die-casting material to obtain the effective surface data and effective physical data related to die-casting defects. This can be achieved in the following way: First, for the surface visual image data after anomaly processing, the suspected defect area is screened out using a threshold segmentation method with the historical surface defect gray value threshold as the benchmark. That is, using the preset surface defect gray value threshold in the historical defect features as the benchmark, a global binarization threshold segmentation algorithm is used to mark the pixels in the image whose gray value falls within the defect gray value threshold range as foreground pixels, and the remaining pixels as background pixels, to obtain a binarized image containing the suspected defect area. The geometric parameters of this area are extracted and compared with the historical surface defect features. The extracted geometric parameters are then compared with the corresponding defect data in the historical surface defect features. The parameter threshold ranges for each defect type are compared one by one. The number of feature items that the actual parameter falls within the historical threshold range is counted, and the matching degree is calculated, i.e., matching degree = number of actual parameters within the historical threshold range / total number of feature items. For the physical signal after anomaly processing, based on the set of physical parameters associated with each suspected surface defect type clearly defined by the correlation rules, such as temperature and stress parameters associated with suspected crack areas, and density parameters associated with suspected porosity areas, the corresponding normal range thresholds in the historical defect features are retrieved according to the parameter type. The real-time acquired data of each physical parameter is judged for the range. If a parameter data is > the upper limit of the range or < the lower limit of the range, it is considered an abnormal type of the feature physical parameter corresponding to different surface defect types based on the correlation rules and the collaborative judgment threshold. For example, cracks correspond to temperatures exceeding the threshold by ≥30℃, stress exceeding the threshold by ≥50MPa, and porosity corresponds to densities below the threshold by ≥0.1g / cm³. 3 If the surface image data is found to have a match rate of ≥85% with the suspected defect area and at least one abnormal parameter exists in the corresponding physical signal (e.g., surface crack suspected area match rate ≥85%, and processing temperature exceeds the normal range by ≥30℃), then the surface image data is considered valid surface data and the corresponding physical signal is considered valid physical data. If the single-dimensional data matches historical defect features by ≥90% (e.g., pore area match rate ≥90% in the surface image), or the density in the physical signal is consistently below 2.6 g / cm³, then the surface image data is considered valid surface data and the corresponding physical signal is considered valid physical data. 3 If the matching degree is ≥90%, it is directly determined as the corresponding valid data. Finally, all surface valid data and physical valid data that meet the above verification rules are output. In some embodiments, other verification methods can also be used, which are not limited here.

[0026] It should be noted that the surface effective data in this application refers to the set of visual features directly related to surface defects of die castings, reflecting whether there are surface defects such as cracks, pores, dents, and scratches on the surface of die castings, as well as the geometric shape and visual contrast of the defects. The physical effective data refers to the set of physical quantity data closely related to material defects of die castings, reflecting the material uniformity, internal structural stability, and abnormal fluctuations of physical parameters such as temperature, stress, density, and hardness during the processing of die castings. It is directly related to the existence and severity of material defects such as material porosity, stress concentration, and substandard mechanical properties.

[0027] In step 103, the surface defect features of the target die casting are extracted from the surface effective data, the physical defect features of the target die casting material are extracted from the physical effective data, and the surface defect features and the physical defect features are correlated and fused to obtain the correlated and fused defect features of the target die casting.

[0028] In some embodiments, reference Figure 2 As shown, this figure is an exemplary flowchart for determining surface defect features in some embodiments of this application. In this embodiment, the extraction of surface defect features of the target die-casting part from the effective surface data can be achieved by the following steps: In step 1031, suspected defect areas of the target die casting are extracted from the effective surface data; In step 1032, the geometric features, texture features, and contour features of the target die casting are determined based on the suspected defect area; In step 1033, surface defect features of the target die casting are selected from geometric features, texture features, and contour features based on the historical defect features corresponding to the target die casting.

[0029] In specific implementation, the extraction of suspected defect areas of the target die casting from the effective surface data can be achieved in the following way: an adaptive threshold segmentation algorithm is used to extract suspected defect areas from the effective surface data. The segmentation window size is set to 15×15. A dynamic threshold is determined by calculating the local mean and standard deviation of pixels within each window, i.e., dynamic threshold = local mean - 0.8 × local standard deviation. Areas with pixel gray values ​​lower than the dynamic threshold are marked as foreground (suspected defects), and areas with gray values ​​higher than the threshold are marked as background, resulting in a preliminary binarized image. Subsequently, a morphological opening operation of a 3×3 structuring element is performed on the binarized image. First, the image is eroded with a 3×3 square structuring element to remove small noise points with an area ≤ 8 pixels. Then, the image is expanded with the same structuring element to fill in the small holes in the defect area. An eight-neighbor connected region labeling algorithm is used to traverse the image, and areas with ≥ 10 connected pixels are identified as suspected defect areas. Other extraction methods can also be used in other embodiments, which are not limited here.

[0030] In addition, in specific implementation, determining the geometric features, texture features, and contour features of the target die-casting part based on the suspected defect areas can be achieved in the following way: calculate the geometric features for each suspected defect area, and then calculate the number of pixels within the area by combining the pixel-physical area conversion factor calibrated by the camera (i.e., 1 pixel corresponds to 0.01 mm). 2 ) Obtain the defect area S; traverse the pixels along the region contour and accumulate the Euclidean distance between adjacent pixels to obtain the defect perimeter L; calculate the major axis length a and minor axis length b of the circumscribed rectangle to obtain the aspect ratio a / b; use the formula roundness = 4πS / L 2 Roundness was calculated, with circular defects having a roundness close to 1 and linear defects having a roundness close to 0. The maximum extension length and maximum width of crack-like defects were also measured. Texture features were extracted using the gray-level co-occurrence matrix (GLCM) method. A 32×32 pixel analysis window was selected centered on the suspected defect area, with gray levels set to 256, a pixel step size of 1, and calculation directions of 0°, 45°, 90°, and 135°. The contrast, correlation, energy, and entropy of the GLCM in each of the four directions were calculated, and the average of the four directions was taken as the texture feature. Contrast reflects the degree of difference in pixel gray levels, and the formula is ΣΣ(ij). 2 P(i,j) represents the probability of matrix elements. Correlation reflects the linear correlation of gray-level distributions, and its formula is ΣΣ(ijP(i,j)-μxμy) / (σxσy), where μ is the mean and σ is the standard deviation. Energy reflects texture uniformity, and its formula is ΣΣP(i,j). 2 Entropy reflects texture complexity, and its formula is -ΣΣP(i,j)log2P(i,j). Contour features are then extracted using chain code encoding: 4-connected chain codes are used to encode the contour of suspected defective regions, recording the direction (up, down, left, right) of each pixel relative to the previous pixel. By traversing the chain code sequence, points where the direction change between adjacent chain codes is ≥90° are calculated as inflection points, and the number of inflection points is counted. Simultaneously, the curvature of each point on the contour is calculated, and characterized by the interior angle value of the triangle formed by that point and two adjacent contour points. An interior angle ≤120° is considered a high curvature point, thus obtaining the contour features. In other embodiments, other methods can also be used to determine this, which are not limited here.

[0031] In addition, in specific implementation, the surface defect features of the target die casting can be screened from geometric features, texture features and contour features based on the historical defect features corresponding to the target die casting. This can be achieved in the following way: retrieve the historical defect features corresponding to the target die casting, extract the geometric feature threshold, texture feature threshold and contour feature threshold corresponding to various surface defects in the features, compare the currently extracted geometric features, texture features and contour features with the above thresholds one by one, screen out all feature parameters that fall within the historical defect feature threshold range, and combine these feature parameters to determine the surface defect features of the target die casting. Other methods can be used for screening in other embodiments, which are not limited here.

[0032] It should be noted that, in this application, the suspected defect area represents the spatial location and boundary of potential defects on the surface of the die casting, reflecting the preliminary distribution of potential defects on the surface of the die casting; geometric features are the quantitative attributes of the spatial morphology of defects in the suspected defect area, reflecting the physical dimensional characteristics of the size and shape of the defects, and directly related to the intuitive morphological appearance of the defects; texture features represent the spatial distribution pattern and variation characteristics of the grayscale of the defect area, reflecting the differences between the defect area and the normal surface in terms of material uniformity, surface roughness, and grayscale gradation rules; contour features represent the geometric shape and variation rules of the defect edge, reflecting the irregularity of the defect edge, the density of inflection point distribution, the curvature change trend, and other detailed features; surface defect features represent the core quantitative characterization of defects on the surface of the die casting, reflecting the actual types and severity of defects on the surface of the die casting.

[0033] In some embodiments, extracting the physical defect features of the target die-casting material from the physical effective data can be achieved using the following steps: Extract data features from each dimension of the physical valid data; Based on the historical defect features corresponding to the target die casting, the physical defect features of the target die casting material are extracted from all data features.

[0034] In specific implementation, extracting data features from the physical valid data in each dimension can be achieved as follows: The physical valid data is decomposed into dimensions based on its parameters. For each dimension's time-series data, a sliding window analysis method is used to extract statistical features. The window size is set to 12 data points, corresponding to a data acquisition time interval of 2 seconds. The window covers 24 seconds of data. The mean, variance, valley, and range of the data are calculated window by window. Simultaneously, the first-order difference method is used to calculate the rate of change of adjacent data points. The average rate of change, maximum rate of change, and the number of times the rate of change exceeds a preset threshold are statistically analyzed for each dimension. The preset thresholds are set as follows: temperature change rate ≥ 5℃ / s, stress change rate ≥ 20MPa / s, and density change rate ≥ 0.01g / (cm³). 3·s) Hardness change rate ≥ 5HBW / s; The 3-standard deviation rule is used to identify mutation points for each dimension of data. If the difference between a data point and the overall mean of that dimension is ≥ 3 × the overall standard deviation, it is determined to be a mutation point. The value of the mutation point, the time of occurrence, and the magnitude of change before and after the mutation are recorded to form the dynamic change characteristics of each dimension. The above statistical characteristics and dynamic change characteristics together constitute the data characteristics of each dimension. In other embodiments, the gas method can also be used to determine the data, which is not limited here.

[0035] In addition, in specific implementation, extracting the physical defect features of the target die-casting material from all data features based on the historical defect features corresponding to the target die-casting can be achieved in the following way: retrieve the historical defect features corresponding to the target die-casting, and extract the physical parameter feature thresholds corresponding to various material defects in the physical characteristics, such as the average density ≤ 2.6g / cm³ corresponding to material porosity. 3 Density variance ≥ 0.02g 2 / cm 6 Stress concentration corresponds to a peak stress ≥350MPa, abrupt change points ≥3, and a maximum stress change rate ≥30MPa / s. Hardness failure corresponds to an average hardness <180HBW or >220HBW and a hardness range ≥40HBW. The data features extracted from each dimension are compared with the above thresholds one by one, and all feature parameters falling within the historical defect feature threshold range are selected. The selected feature parameters are determined as the physical defect features of the target die casting material. In other embodiments, gas can also be used to determine the defects, but this is not limited here.

[0036] It should be noted that the data features in this application represent the static distribution and dynamic change patterns of each physical parameter during the processing, reflecting the basic fluctuation characteristics of the physical properties of the material during the die casting process; the physical defect features represent the abnormal characteristics of the physical properties of the die casting material, reflecting the degree of abnormality in the uniformity of the die casting material, the stability of the internal structure, and the core material characteristics such as mechanical and thermal properties, directly corresponding to the existence and severity of specific material defects such as material porosity, stress concentration, and substandard hardness.

[0037] In some embodiments, the correlation and fusion of the surface defect features and the physical defect features to obtain the correlated and fused defect features of the target die casting can be achieved by the following steps: The surface defect features and the physical defect features are time-series aligned to construct a feature time-series correlation matrix; Obtain the correlation rules between surface defects and physical defects in the historical defect features; Based on the aforementioned correlation rules, the feature temporal correlation matrix is ​​fused in layers to obtain the correlation fusion defect features of the target die casting.

[0038] In specific implementation, the surface defect features and physical defect features are time-series aligned to construct a feature time-series correlation matrix. This can be achieved as follows: extract the acquisition timestamps corresponding to the surface defect features and physical defect features respectively, use a timestamp alignment algorithm for time-series alignment, set a time matching error threshold of ±0.1 seconds, and group surface defect features and physical defect features with the same timestamp or errors within the threshold into the same time-series group; divide the time window according to the processing procedure, with each window corresponding to 10 time-series groups, covering a processing time of 20 seconds, and construct a feature time-series correlation matrix. The row index of the matrix is ​​the time window number, the column index is the specific parameter name of the two types of features, and the matrix elements are the quantized values ​​of the corresponding feature parameters, thus realizing the spatiotemporal correlation mapping of the two types of features. Other alignment methods can also be used in other embodiments, which are not limited here.

[0039] Furthermore, in specific implementation, the layered fusion of the feature time-series correlation matrix based on the aforementioned correlation rules to obtain the correlated fusion defect features of the target die-casting can be achieved in the following way: Secondly, the frequency of co-occurrence of surface defects and physical defects in the correlation rules is statistically analyzed. Feature combinations with a co-occurrence probability ≥ 85% are defined as strong correlation rules, and those with a co-occurrence probability of 60%-84% are defined as weak correlation rules. At the same time, the feature weights corresponding to each rule are recorded. Based on the co-occurrence probability setting, the weight of each feature in the strong correlation rule is equal to the contribution ratio of that feature in the rule. For example, in the crack-related strong correlation rule mentioned above, the crack length weight is 0.4, the stress peak weight is 0.35, and the temperature rise weight is 0.25; the weights of each feature in the weak correlation rule are all 0.5. The first layer is the fusion of strong correlation features. The feature combinations in each time window of the feature time-series correlation matrix are traversed and matched with the strong correlation rules one by one. For the successfully matched feature combinations, a weighted summation formula is used for fusion. The first layer is strong association fusion feature value, calculated as Σ (feature parameter value × corresponding weight). Simultaneously, the confidence level is calculated using the formula: strong association confidence = matching rule's probability of collaboration × feature parameter compliance rate, where the feature parameter compliance rate = number of features meeting historical thresholds / total number of rule features. The second layer is weak association feature fusion. For feature combinations that do not match strong association rules but meet weak association rules, a logical AND operation is used to determine validity; that is, if all feature parameters meet historical thresholds, it is considered valid. The weak association fusion feature value is calculated using a weighted average method with the rule's probability of collaboration as the weight. The third layer is feature integration. Strong association fusion results with confidence < 0.7 and invalid weak association fusion results are removed. The remaining valid fusion feature values ​​are combined with the corresponding association rule identifier, confidence level, and time-series window information to form an association fusion defect feature containing defect collaboration feature quantification value, association strength, and spatiotemporal attributes. Other fusion methods can be used in other embodiments, which are not limited here.

[0040] It should be noted that the feature temporal correlation matrix in this application represents the spatiotemporal correlation mapping relationship between surface defect features and physical defect features at different time nodes during the die casting process, reflecting the synergistic occurrence pattern of the two types of defect features in the time dimension and the correspondence of parameter quantification values; the correlation fusion defect features represent the synergistic characterization information of die casting surface defects and material physical defects and the quantification result of correlation strength, reflecting the defect type attributes, severity quantification value, spatiotemporal correlation attributes and feature correlation confidence, realizing a comprehensive and accurate integrated characterization of die casting defects.

[0041] In step 104, the spatiotemporal coordinate information of the sensor network is obtained, and defect characterization information, including the defect type and defect location of the target die casting processing defect, is determined based on the associated fusion defect features and the spatiotemporal coordinate information.

[0042] It should be noted that the spatiotemporal coordinate information in this application represents a standardized representation of the spatial deployment location and data acquisition time dimension information of each visual sensor and physical parameter sensor in the sensor network. It reflects the spatial distribution relationship of each sensor in the die casting processing scenario, the temporal correlation of data acquisition, and the mapping relationship of corresponding processing steps / stations. This includes the three-dimensional spatial coordinates of each sensor calibrated by laser, the installation station number, the name of the corresponding processing step, the data acquisition timestamp, the unique identifier of the sensor device, and the data acquisition triggering conditions.

[0043] In some embodiments, reference Figure 3 As shown, this figure is an exemplary flowchart for determining defect characterization information in some embodiments of this application. In this embodiment, the defect characterization information, including the defect type and defect location of the target die-casting part machining defect, is determined based on the associated fused defect features and the spatiotemporal coordinate information, which can be achieved by the following steps: In step 1041, a defect type-association fusion feature mapping library is pre-constructed; In step 1042, the associated fusion defect features are compared with the defect type-associated fusion feature mapping library to obtain the defect type of the target die casting processing defect; In step 1043, the actual coordinate information of the surface defect of the target die casting is located based on the spatiotemporal coordinate information transformation and the associated fusion defect features; In step 1044, defect characterization information, including the defect type and defect location of the target die casting machining defect, is determined based on the defect type and the actual coordinate information.

[0044] In practical implementation, the pre-construction of a defect type-association fusion feature mapping library can be achieved in the following way: Based on statistical analysis of over 5000 sets of historical defect data and associated fusion features of the same model and process as the target die casting, the library is constructed and stored according to typical processing defect types of die castings. Each defect type corresponds to a set of standardized associated fusion feature templates. Each template includes the surface-physical co-feature quantification threshold, feature weight ratio, and minimum matching confidence threshold corresponding to the defect. All template parameters are determined through statistical frequency analysis and weighted average methods. That is, the feature parameters for each type of defect are... The frequency of occurrence was counted using statistical frequency analysis. Features with a frequency ≥ 80% were selected as core features, and those with a frequency between 50% and 79% were selected as auxiliary features. Weights were assigned according to the probability of co-occurrence of each feature and defect (i.e., the number of times a feature and defect occur simultaneously / the total number of defects). The weights of core features were summed to 0.6 based on their co-occurrence probability, and the weights of auxiliary features were summed to 0.4. The weighted average method was used to calculate the historical data mean of the feature parameters, and the quantization threshold was determined by combining it with three times the standard deviation. After fine-tuning with 1000 sets of validation data, the template parameters were obtained. Finally, all the templates were used as a defect type-association fusion feature mapping library.

[0045] In addition, in specific implementation, the defect type of the target die casting machining defect can be obtained by comparing the correlation fusion defect features with the defect type-correlation fusion feature mapping library through similarity comparison. This can be achieved by using the Euclidean distance similarity comparison algorithm to compare the currently obtained correlation fusion defect features with the feature templates of each defect type in the defect type-correlation fusion feature mapping library one by one, calculating the Euclidean distance between the current feature and each template, filtering out the feature template with the smallest Euclidean distance and a matching confidence level ≥ the lowest threshold of the corresponding defect type, and determining the defect type corresponding to the template as the defect type of the target die casting machining defect.

[0046] In addition, in specific implementation, the actual coordinate information of the surface defect of the target die-casting part, based on the spatiotemporal coordinate information conversion and the associated fusion defect feature localization, can be achieved in the following way: Extract the unique identifier of the sensor device that collected the associated fusion defect feature, the pre-calibrated three-dimensional spatial calibration parameters via laser, the installation station coordinates, and the data acquisition timestamp from the spatiotemporal coordinate information of the sensor network. Convert the pixel coordinates (u,v) of the visual area of ​​the surface defect corresponding to the associated fusion defect feature into three-dimensional physical coordinates (X1,Y1,Z1) in the local coordinate system of the die-casting part. The conversion formula is [X1,Y1,Z1]. T =K -1 [u,v,1] T×s, K is the intrinsic parameter matrix, s is the scale factor, and combined with the parameters of the overall reference coordinate system of the die casting in the spatiotemporal coordinate information, with the center of the positioning hole of the die casting as the origin, through coordinate translation (ΔX=X0-X reference, ΔY=Y0-Y reference, ΔZ=Z0-Z reference) and rotation operation (based on the station installation angle correction), the local physical coordinates are converted into the actual three-dimensional coordinates (X,Y,Z) in the global coordinate system of the die casting.

[0047] In addition, in specific implementation, the defect characterization information, including the defect type and defect location of the target die casting machining defect, can be determined according to the defect type and the actual coordinate information in the following way: integrate the defect type, the actual three-dimensional coordinates of the defect in the global coordinate system, and associate them with the machining process name and workstation number in the spatiotemporal coordinate information. At the same time, calculate the defect severity level based on the quantitative value of the associated fused defect features. That is, firstly, based on the historical defect data of the target die casting and the corresponding failure loss degree, determine the comprehensive quantitative value threshold of Level 1 (severe), Level 2 (medium), and Level 3 (minor); extract the quantitative values ​​of the core collaborative features and auxiliary collaborative features in the associated fused defect features, and sum them according to the preset weights (core 0.6, auxiliary 0.4) to obtain the comprehensive quantitative value; compare the comprehensive quantitative value with the threshold, and determine the level by matching the corresponding interval. For example, a fused feature value ≥ 0.9 is a Level 1 severe defect, 0.7-0.9 is a Level 2 defect, and 0.5-0.7 is a Level 3 defect. Finally, a structured defect characterization information containing defect type name, actual three-dimensional physical coordinates, corresponding machining process and workstation, acquisition time, and severity level is formed.

[0048] It should be noted that the defect type-association fusion feature mapping library in this application represents a one-to-one mapping relationship between different processing defect types and corresponding association fusion feature templates, reflecting the surface-physical synergistic feature patterns of various defects and providing a quantitative comparison basis for defect type determination; the defect type represents the essential attribute of the defect, reflecting the specific quality anomaly category caused by factors such as process and material during the die casting process; the actual coordinate information represents the precise spatial location of the defect on the die casting, reflecting the specific distribution orientation of the defect and its relative positional relationship with the reference feature; the defect characterization information represents the characterization information of the complete attributes of the die casting processing defect, reflecting the defect type, precise location, generation scenario and impact degree, providing a comprehensive and accurate technical basis for defect tracing, rectification and quality control.

[0049] Furthermore, in another aspect of this application, in some embodiments, this application provides an intelligent inspection system for die-casting part processing that integrates visual recognition, as referenced. Figure 4The figure is a schematic diagram of the structure of an intelligent inspection system for die casting processing that integrates visual recognition, according to some embodiments of this application. The intelligent inspection system 400 for die casting processing that integrates visual recognition includes: a data acquisition module 401, a processing module 402, and an execution module 403, which are described below: The acquisition module 401 in this application is mainly used to acquire surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting through a sensor network; Processing module 402, in this application, is used to perform joint preprocessing on the surface visual image data and the physical signal of the die casting material based on the historical defect features corresponding to the target die casting, so as to obtain effective surface data and effective physical data related to the die casting defects; It should be noted that the processing module 402 in this application is also used to extract the surface defect features of the target die casting from the surface effective data, extract the physical defect features of the target die casting material from the physical effective data, and correlate and fuse the surface defect features and the physical defect features to obtain the correlated and fused defect features of the target die casting. The execution module 403 in this application is mainly used to acquire the spatiotemporal coordinate information of the sensor network, and determine the defect characterization information, including the defect type and defect location of the target die casting processing defect, based on the associated fusion defect features and the spatiotemporal coordinate information.

[0050] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described intelligent detection method for die casting processing that integrates visual recognition.

[0051] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing an intelligent inspection method for die-casting part processing that integrates visual recognition, according to some embodiments of this application. The intelligent inspection method for die-casting part processing that integrates visual recognition in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0052] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0053] The communication bus 502 can be used to transmit information between the aforementioned components.

[0054] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CDROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0055] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method used in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0056] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0057] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0058] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0059] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intelligent detection method for die-casting parts processing that integrates visual recognition.

[0060] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0061] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An intelligent inspection method for die-casting parts processing that integrates visual recognition, wherein, A sensor network comprising visual sensors and physical parameter sensors is pre-deployed on the attachment of a target die-cast part, characterized in that the method includes the following steps: The surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting are collected through a sensor network. Based on the historical defect features corresponding to the target die casting, the surface visual image data and the physical signal of the die casting material are jointly preprocessed to obtain effective surface data and effective physical data related to the die casting defects. Surface defect features of the target die casting are extracted from the effective surface data, and physical defect features of the target die casting material are extracted from the effective physical data. The surface defect features and the physical defect features are correlated and fused to obtain the correlated and fused defect features of the target die casting. The spatiotemporal coordinate information of the sensor network is obtained, and the defect characterization information, including the defect type and defect location of the target die casting processing defect, is determined based on the associated fusion defect features and the spatiotemporal coordinate information.

2. The method as described in claim 1, characterized in that, The physical signals of the die-casting material include the processing temperature, internal stress, material density, and surface hardness of the die-casting.

3. The method as described in claim 1, characterized in that, Based on the historical defect features corresponding to the target die casting, the surface visual image data and the physical signal of the die casting material are jointly preprocessed to obtain effective surface data and effective physical data related to the die casting defects, specifically including: Obtain the historical defect features corresponding to the target die casting; Anomaly processing is performed on the surface visual image data and the physical signal of the die casting material to obtain anomaly-processed surface visual image data and physical signal of the die casting material. Based on the correlation rules between surface defects and physical defects in the historical defect features, cross-validation is performed on the surface visual image data after anomaly processing and the physical signals of the die casting material to obtain effective surface data and effective physical data related to die casting defects.

4. The method as described in claim 1, characterized in that, Extracting the surface defect features of the target die casting from the effective surface data specifically includes: The suspected defect areas of the target die casting are extracted from the effective surface data; The geometric, textural, and contour features of the target die casting are determined based on the suspected defect areas. The surface defect features of the target die casting are selected from geometric features, texture features, and contour features based on the historical defect features corresponding to the target die casting.

5. The method as described in claim 1, characterized in that, Extracting the physical defect features of the target die-casting material from the effective physical data specifically includes: Extract data features from each dimension of the physical valid data; Based on the historical defect features corresponding to the target die casting, the physical defect features of the target die casting material are extracted from all data features.

6. The method as described in claim 1, characterized in that, The correlation and fusion of the surface defect features and the physical defect features to obtain the correlation and fusion defect features of the target die casting specifically include: The surface defect features and the physical defect features are time-series aligned to construct a feature time-series correlation matrix; Obtain the correlation rules between surface defects and physical defects in the historical defect features; Based on the aforementioned correlation rules, the feature temporal correlation matrix is ​​fused in layers to obtain the correlation fusion defect features of the target die casting.

7. The method as described in claim 1, characterized in that, Based on the associated fusion defect features and the spatiotemporal coordinate information, the defect characterization information, including the defect type and defect location of the target die-casting part machining defects, is determined, specifically including: Pre-build a defect type-association fusion feature mapping library; The defect type of the target die casting is obtained by comparing the correlation fusion defect features with the defect type-correlation fusion feature mapping library. The actual coordinate information of the surface defect of the target die casting is located based on the spatiotemporal coordinate information transformation and the associated fusion defect features. Based on the defect type and the actual coordinate information, defect characterization information, including the defect type and defect location of the target die casting machining defect, is determined.

8. An intelligent inspection system for die-casting parts processing integrating visual recognition, characterized in that, include: The acquisition module is used to acquire surface visual image data and physical signals of the die casting material at different process nodes during the processing of the target die casting part through a sensor network; The processing module is used to perform joint preprocessing on the surface visual image data and the physical signal of the die casting material based on the historical defect features corresponding to the target die casting, so as to obtain effective surface data and effective physical data related to the die casting defects. The processing module is also used to extract surface defect features of the target die casting from the surface effective data, extract physical defect features of the target die casting material from the physical effective data, and correlate and fuse the surface defect features and the physical defect features to obtain the correlated and fused defect features of the target die casting. The execution module is used to acquire the spatiotemporal coordinate information of the sensor network, and determine the defect characterization information, including the defect type and defect location of the target die casting processing defect, based on the associated fusion defect features and the spatiotemporal coordinate information.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the intelligent inspection method for die casting processing fused with visual recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent detection method for die casting machining that integrates visual recognition as described in any one of claims 1 to 7.

Images