Method and device for inspecting surface treatment effect of casting based on machine vision

Abstract

This invention belongs to the field of casting quality inspection technology, and provides a method and device for quality inspection of casting surface treatment effects based on machine vision. The method includes: extracting the real-time light intensity of a non-fully open area of ​​the casting from a first high-definition image; predicting an illumination parameter combination that reduces the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area; capturing a second high-definition image under the illumination intensity; identifying and removing artifact areas in the second high-definition image to obtain a third high-definition image; extracting quality inspection parameters from the third high-definition image; and evaluating the surface treatment effect of the casting based on the quality inspection parameters. This invention effectively overcomes the artifact interference problem in traditional casting quality inspection methods, achieving high-precision and high-efficiency detection of casting surface treatment effects.

Description

Technical Field

[0001] This invention relates to the field of casting quality inspection technology, and more specifically, to a quality inspection method and device for the surface treatment effect of castings based on machine vision. Background Technology

[0002] In modern manufacturing, castings are an important basic component of mechanical products. The surface treatment quality of castings directly affects the corrosion resistance, wear resistance, and appearance quality of the products, and thus relates to the overall performance and service life of the mechanical products.

[0003] Traditional sensor-based inspection methods (such as eddy current testing and ultrasonic testing) can detect internal and surface defects in castings to a certain extent, but they have limitations in detecting surface treatment effects (such as coating uniformity, roughness variation, and surface color consistency), and cannot directly obtain surface texture features and subtle defect information. While machine vision-based inspection technologies, which have emerged in recent years, have partially addressed the shortcomings of traditional methods, existing technologies still have many problems. These mainly manifest in the following ways: the surface area of ​​castings is divided into fully open and non-fully open areas. When inspecting the surface treatment effect of fully open areas, conventional feature extraction algorithms can effectively cope with ambient light interference, thus ensuring high inspection accuracy. However, for non-fully open areas such as holes and deep grooves, existing machine vision-based inspection methods usually require the use of lighting (e.g., integrated with a camera) to illuminate and capture images before analysis and inspection. Due to the complex internal structure of holes, the artifacts produced by multiple reflections of light on the inner wall of the hole are easily misjudged as real defects, greatly affecting the accuracy of the inspection results and making it difficult to accurately judge the surface treatment effect in these areas.

[0004] Therefore, there is an urgent need for a new machine vision-based quality inspection method for casting surface treatment effects to reduce the impact of artifacts on the quality inspection results, achieve high-precision, high-efficiency, and standardized inspection of casting surface treatment effects, and meet the stringent requirements of modern manufacturing industry for casting quality control. Summary of the Invention

[0005] In response, the present invention provides a quality inspection method, inspection device, electronic device, computer storage medium, and computer program product for inspecting the surface treatment effect of castings based on machine vision, so as to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a quality inspection method for the surface treatment effect of castings based on machine vision, comprising the following steps: extracting the real-time light intensity of a non-fully open area of ​​the casting from a first high-definition image; predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area; capturing a second high-definition image under the illumination parameter combination; identifying and removing artifact areas in the second high-definition image to obtain a third high-definition image.

[0007] Quality inspection parameters are extracted from the third high-definition image, and the surface treatment effect of the casting is evaluated based on the quality inspection parameters.

[0008] In a second aspect, the present invention provides a quality inspection device for the surface treatment effect of castings based on machine vision, the device comprising an image control module, an artifact recognition and processing module, and a quality inspection module.

[0009] The shooting control module extracts the real-time light intensity of the non-fully open area of ​​the casting from the first high-definition image, and predicts a combination of illumination parameters that is beneficial to reducing the probability of artifacts based on the real-time light intensity and the structural data of the non-fully open area.

[0010] The artifact recognition and processing module captures a second high-definition image under the illumination parameter combination, identifies and removes artifact regions in the second high-definition image, and obtains a third high-definition image.

[0011] The quality inspection module extracts quality inspection parameters from the third high-definition image and evaluates the surface treatment effect of the casting based on the quality inspection parameters.

[0012] In a third aspect, the present invention provides an electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the computer program, when executed by the processor, implements the method as described in any of the preceding claims.

[0013] In a fourth aspect, the present invention provides a computer storage medium storing a computer program that can be executed by a processor to implement the method as described in any of the preceding claims.

[0014] In a fifth aspect, the present invention provides a computer program product comprising a computer program executable by a processor to implement the method as described in any of the preceding claims.

[0015] This invention acquires real-time light intensity by collecting images of non-fully open areas, and combines this with regional structural data to predict the optimal illumination intensity, reducing light reflection artifacts. Further, image artifact recognition and removal, along with quality control parameter extraction and evaluation, effectively overcomes the artifact interference problem in traditional casting quality inspection methods. This achieves high-precision and high-efficiency detection of casting surface treatment effects, meeting the stringent quality control requirements of modern manufacturing and ensuring the performance and lifespan of casting products. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a quality inspection method for the surface treatment effect of castings based on machine vision, as disclosed in an embodiment of the present invention.

[0018] Figure 2 This is a schematic diagram of the structure of the gated neural network disclosed in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of a quality inspection device for the surface treatment effect of castings based on machine vision, as disclosed in an embodiment of the present invention. Detailed Implementation

[0020] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0022] When inspecting the surface treatment effect of non-fully open areas of castings, lighting is needed to assist in image capture. This process can easily produce artifacts, which may interfere with defect identification. For example: Reflection / Glare: When the casting surface is rough or has an angle, the lighting may produce localized highlights, obscuring minor defects (such as cracks or pores); Shadows: Uneven surfaces (such as sand holes or burrs) may create shadows due to the lighting angle, which may be misjudged as defects; Specular Reflection: Smooth areas (such as machined surfaces) may reflect the shape of the light source (such as lamp images), interfering with visual judgment; Color Deviation: The color temperature of the lighting (such as cool / warm light) may change the color of the casting, affecting the judgment of defects such as oxidation and inclusions.

[0023] In response to the above technical problems, such as Figure 1 As shown, this embodiment of the invention discloses a quality inspection method for the surface treatment effect of castings based on machine vision, including the following steps: S10, extracting the real-time light intensity of the non-fully open area of ​​the casting from a first high-definition image, and predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area.

[0024] The first high-definition image of the non-fully open area (such as holes and deep grooves) of the casting is acquired by machine vision equipment (such as high-definition industrial cameras). Image analysis technology is used to statistically analyze the brightness information of each pixel in the image, thereby calculating the real-time light intensity of the area, that is, obtaining the current lighting conditions of the area to be detected.

[0025] Based on real-time light intensity and structural data of the non-fully open region (such as the depth, diameter, and shape of holes, and the width and depth of grooves), the propagation and reflection of light within this region under different combinations of illumination parameters are simulated using the principles of light propagation and reflection. This predicts the optimal combination of illumination parameters to reduce light reflection and the probability of artifacts. For example, for deeper holes with complex internal structures, it is predicted that reducing the light intensity and using multi-angle scattered light illumination can effectively reduce the probability of reflection artifacts.

[0026] Understandably, when machine vision equipment captures images of non-fully open areas, it can first capture images region by region and then synthesize them into a panoramic image. Subsequent analysis is also based on the panoramic image. At the same time, lighting equipment (such as LED light sources, fiber optic light guides, etc.) can be integrated with the machine vision equipment, so that corresponding region-by-region illumination is implemented during region-by-region shooting.

[0027] S20: Capture a second high-definition image under the illumination parameter combination, identify and remove artifact regions in the second high-definition image, and obtain a third high-definition image.

[0028] Based on the optimal combination of illumination parameters predicted in step S10, the lighting equipment is adjusted, and the non-fully open area of ​​the casting is photographed again to obtain a second high-definition image (e.g., the panoramic image mentioned above). Under optimized lighting conditions, this second high-definition image reduces artifact interference caused by light reflection and more clearly presents the true features of the casting surface.

[0029] Simultaneously, artifact regions in the second high-resolution image are further identified and removed to obtain the third high-resolution image. For example, a deep learning-based artifact recognition algorithm is used to process the second high-resolution image. The algorithm accurately distinguishes artifact regions from real defects or normal surface areas through learning and analysis of image features. The identified artifact regions are marked and removed (e.g., by setting the pixel values ​​of artifact regions to specific values ​​or repairing them using image masking techniques), thus obtaining the third high-resolution image free of artifact interference.

[0030] This can further improve image quality, enabling subsequent detection to be based on more accurate image data and avoiding misjudgments caused by artifacts.

[0031] S30, extract quality inspection parameters from the third high-definition image, and evaluate the surface treatment effect of the casting based on the quality inspection parameters.

[0032] In this step, appropriate image analysis algorithms and techniques are used to extract various quality inspection parameters related to the surface treatment effect of the casting, including but not limited to surface roughness, coating thickness, defect size (such as crack length and pore diameter), and surface color uniformity. Then, the contours of surface defects are extracted and their dimensions are calculated using algorithms such as edge detection, and the uniformity and color consistency of the coating are evaluated using grayscale analysis or color feature extraction algorithms.

[0033] The extracted quality inspection parameters are compared and analyzed with pre-set quality standards (such as industry standards and enterprise internal control standards). By setting reasonable evaluation rules and algorithms (such as weighted scoring, assigning different weights to different parameters based on their importance to product quality, and calculating a comprehensive score), the surface treatment effect of the casting is quantitatively evaluated to determine whether it meets quality requirements. For example, if parameters such as surface roughness and coating thickness are all within the standard range, the surface treatment effect of the casting is deemed qualified; if some parameters exceed the standard, a corresponding quality level and improvement suggestions are given. Further details are omitted.

[0034] This invention acquires real-time light intensity by collecting images of non-fully open areas, and combines this with regional structural data to predict the optimal illumination intensity, reducing light reflection artifacts. Further, image artifact recognition and removal, along with quality control parameter extraction and evaluation, effectively overcomes the artifact interference problem in traditional casting quality inspection methods. This achieves high-precision and high-efficiency detection of casting surface treatment effects, meeting the stringent quality control requirements of modern manufacturing and ensuring the performance and lifespan of casting products.

[0035] As an example, the prediction of an illumination parameter combination that reduces the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area includes: constructing a three-dimensional geometric model based on the structural data of the non-fully open area; deriving a physical model of light propagation and reflection within the three-dimensional geometric model based on geometric optics principles, including equations for calculating the incident angle and reflection angle of light, the relationship function between the reflectivity of the material surface and the wavelength of light, and a model of light energy attenuation after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as optimization variables, and using an optimization algorithm to solve for the optimal illumination parameter combination.

[0036] First, a three-dimensional geometric model is constructed based on the structural data of the non-fully open area to digitize the actual shapes of areas such as casting holes and deep grooves. Dimensions of the structural data, such as hole diameter, depth, and curvature, and groove width and sidewall angles, are transformed into geometric shapes in three-dimensional space, providing a digital medium for ray simulation.

[0037] Based on the principles of geometric optics, a physical model of light propagation and reflection within a three-dimensional geometric model is derived. Through equations for calculating the incident angle and reflection angle of light (following the law of reflection), a function relating the reflectivity of a material surface to the wavelength of light (characterizing the absorption and reflection characteristics of different materials), and a model of light energy attenuation after multiple reflections (considering the energy loss of light during the reflection process), the behavior of light within complex structures is accurately simulated, making light propagation visible and calculable.

[0038] In a three-dimensional geometric model, the normal vector of each surface point Determined by the model's geometry. For example: planar region: normal vector. It is a fixed normal vector of a plane (such as the sidewall of a deep trench).

[0039] Surface region: normal vector The normal vector of the tangent plane of a surface at that point (such as the inner wall of a hole) can be calculated using parametric equations or triangular meshes.

[0040] Polyhedral intersection: normal vector Abrupt changes may occur at the junction (such as the stepped surface of a stepped hole).

[0041] (1) The equations for calculating the angle of incidence and the angle of reflection are: ;in, It is a unit vector, with its direction pointing from the object's surface to the reflection direction; It is a unit vector, with its direction pointing from the light source to the object surface (incident direction); It is the unit normal vector, perpendicular to the object's surface and pointing outwards; The dot product operation represents vectors and is used to calculate the cosine of the angle of incidence. (where the angle of incidence is 0).

[0042] When light shines on the surface of a 3D model, the intersection point between the light ray and the model must first be determined using a ray tracing algorithm (such as the BVH acceleration structure), and then the normal vector of that point must be obtained. Finally, substitute the values ​​into the reflection equation to calculate the reflection direction. For example: when reflected from the inner wall of a hole, the normal vector... The reflection direction dynamically adjusts as the curvature of the inner wall surface changes. When reflecting from the bottom of a deep trench, if the bottom is an inclined plane, the normal vector... When tilted, reflected light rays will bend in a specific direction.

[0043] (2) The relationship between the surface reflectivity of the material and the wavelength of light is: ;in, is the spectral reflectance function, which is the ratio of the energy of light reflected from the surface of the casting to the energy of the incident light. It is a dimensionless parameter, and the larger the ratio, the higher the proportion of light of that wavelength is reflected. Wavelength of light (unit: nm); Polynomial coefficients (unit: nm) -k ), characterizing the material's fundamental reflectance trend to the continuous spectrum, The highest degree of a polynomial term, for example When, the polynomial is The higher the highest number, the more detailed the characterization of the continuous spectrum reflectance law; Gaussian term (i.e. The amplitude coefficient (a dimensionless parameter) is used to adjust the Gaussian peak (i.e., The height of the curve shape depicted. The width factor of the Gaussian peak (unit: nm) -2 ), controlling the width of the Gaussian peak, The center wavelength of the Gaussian peak (unit: nm) corresponds to the characteristic wavelength at which the material's reflectivity abruptly changes. Used to fit the abrupt change in reflection of a material at a specific wavelength (such as the characteristic absorption peak of a metal).

[0044] Material surface reflectivity function Parameters (such as) In a 3D model, reflectivity is a spatial variable that depends on the material distribution at various points on the model's surface. For example, a casting surface may have a coating, and the reflectivity function parameters of coated and uncoated areas differ. Holes at different depths inside the casting may use different materials (such as a metal matrix and filling material), requiring independent reflectivity definitions for each region. .

[0045] First, determine the intersection point between the light ray and the 3D model; then, look up the material ID of that point; finally, call the corresponding reflectivity function based on the material ID. Calculate the reflectance.

[0046] (3) The light energy attenuation model is as follows: ;in, The energy of light remaining after m reflections inside the casting (unit: J). The initial incident light energy (unit: J) is the original energy emitted by the light source. Let be the spectral reflectance of the k-th reflecting surface (calculated using the above relationship function), representing the ability of the casting surface to reflect light. Indicates a complete launch. This indicates complete absorption; Let be the angle of incidence (in °) for the k-th reflection, i.e., the angle between the incident direction of the ray and the surface normal vector; the cosine of the angle of incidence. This is a correction factor for the energy projection area under oblique incidence. This represents the distance the light travels after the kth reflection (in mm), which is the distance the light travels during the kth reflection. The attenuation length (related to the scattering characteristics of the ambient medium, unit: mm) is used to describe the scattering characteristics of light by the ambient medium (such as air, coolant residue). The smaller the value, the stronger the scattering. The larger the value, the weaker the scattering. It characterizes the scattering loss during the propagation of light. It is a dimensionless scattering loss measure, which quantifies the severity of light scattering in a medium by the ratio of propagation distance to attenuation length.

[0047] Propagation distance in energy decay model and angle of incidence All are determined by the three-dimensional geometric model: propagation distance The straight-line distance between two reflections of light needs to be calculated using the coordinates of the starting and ending points in three-dimensional space (determined by ray tracing).

[0048] Angle of incidence From the direction of the incident ray Normal vector at the intersection point The dot product is determined .

[0049] In complex structures (such as deep holes), light may undergo multiple reflections, each reflection... They are all different: First reflection: The intersection point is located at the aperture opening, Determined by the material of the orifice. This is the distance from the orifice to the point of first reflection from the inner wall.

[0050] Second reflection: The intersection point is located on the inner wall. Determined by the material of the inner wall, This is the distance between the two reflection points on the inner wall.

[0051] Repeat the above process until the light energy is below the threshold or leaves the model area.

[0052] The processing procedure is illustrated below: Assume light enters a stepped aperture (a cylindrical hole with a sudden change in diameter): 1. Geometric model preparation: Discretize the stepped aperture into a triangular mesh, with each mesh storing a normal vector. And material ID. Assign different material parameters (e.g., ...) to hole segments of different diameters. ).

[0053] 2. Ray Tracing and Reflection Calculation: Rays originate from the light source and intersect the surface of the stepped aperture at a point. , obtain normal vector And material ID_1. Substitute into the reflection equation to calculate the reflection direction. .

[0054] 3. Energy decay calculation: through Calculate the wavelength as The light in reflectivity at Calculate the angle of incidence. and transmission distance (Light source to) (Distance). Update energy: .

[0055] 4. Multiple reflection iterations: reflected light rays Continue intersecting the model at points Repeat the above steps to calculate. .like Located at the change in diameter of the stepped hole, the normal vector The material ID_2 may mutate, leading to significant changes in reflection behavior and energy decay.

[0056] Then, using real-time light intensity as the initial condition, the current illumination conditions of the detection area are taken as the starting point for calculation. Minimizing the probability of artifact generation is the objective function, i.e., the optimization direction. Adjustable parameters of the lighting equipment (such as light intensity, illumination angle, and spectral type) are used as optimization variables; these adjustable parameters directly affect the reflection of light on the casting surface. Optimization algorithms (such as genetic algorithms and particle swarm optimization) are employed to quantitatively assess the probability of artifact generation by calculating relevant indicators of potential artifact regions under different combinations of illumination parameters (such as the area of ​​overexposed regions and the proportion of abnormally bright and dark regions). Through continuous iterative calculation and filtering, the optimal combination of illumination parameters is solved to minimize the probability of artifact generation when light propagates in non-fully open areas.

[0057] As an example, the process of identifying and removing artifact regions from the second high-definition image to obtain a third high-definition image includes: encoding the pixel features of the second high-definition image with real-time light intensity, structural data of the non-fully open region, and the illumination parameters to construct a multi-dimensional feature vector containing visual information, illumination parameters, and structural parameters; using a three-way attention mechanism to enhance the multi-dimensional feature vector to obtain an enhanced multi-dimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; using a trained semantic segmentation network to process the enhanced multi-dimensional feature vector to obtain several artifact regions; and using a multi-scale conditional random field to refine the boundaries of each artifact region.

[0058] In the aforementioned embodiments, although the probability of artifact generation was reduced by employing optimal parameter combinations, the complex structure and diverse materials of the non-fully open areas of the casting mean that even slight differences in lighting and structural details can still lead to artifacts. Therefore, artifact identification and removal are still required for the second high-definition image. Furthermore, due to the complex structure and variable lighting conditions of the non-fully open areas, artifact characteristics vary across different scenarios, and a single image pixel feature cannot fully reflect the causes and characteristics of artifacts.

[0059] To address the aforementioned issues, this embodiment encodes the pixel features of the second high-definition image with real-time light intensity, regional structure data, and illumination parameters to construct a multi-dimensional feature vector. This allows for comprehensive image analysis from multiple perspectives, effectively resolving the problem of misjudgment and missed detection of artifacts caused by incomplete information. Specifically: First, the pixel features of the second high-definition image are encoded with real-time light intensity, structural data of the non-fully open area, and illumination parameters. The pixel features of the second high-definition image include visual information such as texture, color, and grayscale of the casting surface, providing a direct basis for artifact identification. Real-time light intensity reflects the current illumination conditions of the non-fully open area, affecting the image's brightness and the generation of artifacts. The structural data of the non-fully open area (such as hole depth, shape, and groove width) determines the propagation path and reflection characteristics of light within the area, closely related to artifact generation. The illumination parameter combination (light intensity, angle, spectral type, etc.) represents the pre-optimized illumination conditions, directly impacting image quality. This information is fused using encoding techniques to construct a multi-dimensional feature vector, enabling comprehensive image analysis from multiple perspectives.

[0060] Next, through the collaborative work of the spatial-structure attention module, the spectral-illumination attention module, and the parameter-context attention module, feature enhancement of multidimensional feature vectors is achieved using a three-way attention mechanism.

[0061] Spatial-Structure Attention Module: Based on the spatial attention mechanism and combined with geometric feature extraction algorithms (such as curvature calculation and region topology analysis). The core of the spatial attention mechanism is to strengthen the feature representation of key regions (such as structurally complex locations or areas prone to artifacts) by calculating the weights of different spatial locations in the feature map.

[0062] The enhancement process involves: acquiring 3D structural data (such as geometric parameters of holes, deep grooves, and corners) and illumination parameters (such as light source angle and intensity) of non-fully open areas to construct a spatial feature map. Using geometric algorithms, the curvature of the structural surface (such as abrupt curvature changes) and topological relationships (such as closed / semi-closed regions) are calculated to locate areas prone to artifacts (such as corners of holes and the bottom of deep grooves—these areas have complex light reflections and are prone to artifacts due to multiple reflections or occlusions). Higher attention weights are assigned to these key areas, while lower weights are assigned to flat areas with fewer artifacts, generating a spatial attention map. The attention map is then weighted and fused with the original spatial feature map to enhance artifact features in areas of abrupt curvature changes (such as uneven brightness and blurred edges), preventing these areas from being missed due to weak features.

[0063] Spectrum-Illumination Attention Module: Based on the channel attention mechanism, combined with spectral feature mapping algorithms (such as reflectance-wavelength relationship modeling and color space conversion). The channel attention mechanism assigns weights to different channels of the feature map (such as RGB color channels and texture channels) to highlight task-relevant channel features.

[0064] The enhancement process involves: obtaining real-time light intensity and the material's spectral reflectance function. and the multi-channel features of the image (such as color channels, texture channels). Based on the wavelength of light ( The relationship between light intensity and material reflectivity is analyzed to calculate the reflection ratio of light at different wavelengths on the material surface. Combined with real-time illumination intensity, the color and texture characteristics of artifacts are determined (e.g., artifacts on metallic materials at specific wavelengths may exhibit a yellowish channel characteristic). A channel attention mechanism is used to assign high weights to color channels (e.g., RGB components corresponding to specific wavelengths) and texture channels (e.g., blurred / jagged textures of artifacts) related to artifacts, while weakening irrelevant channels (e.g., uniform color channels in the background area). The channel weights are then fused with the original image feature map to highlight the color and texture features of artifacts, improving the model's ability to distinguish artifacts from real areas.

[0065] Parameter-Context Attention Module: Based on the contextual attention mechanism, this module combines multi-parameter correlation modeling (such as the synergistic effect analysis of illumination parameters, structural parameters, and light intensity). The contextual attention mechanism models the dependencies between global features, allowing the model to understand the overall impact of parameter relationships on the target (artifacts).

[0066] The enhancement process involves: integrating illumination parameter combinations (such as light source angle and distance), real-time light intensity, and 3D structural data (such as region closure) to construct a multi-dimensional parameter feature matrix, capturing global correlations between parameters (e.g., "small-angle light source + deep trench structure" easily produces shadow artifacts). A self-attention mechanism (such as scaling dot product attention in Transformer) is used to calculate the correlation weights between different parameter combinations and artifact generation, constructing a global context model—for example, learning the strong correlation between "specific light source angle + highly reflective material + closed structure" and "spectral artifacts." Based on the global context model, the attention distribution of spatial, spectral, and other feature dimensions is dynamically adjusted (e.g., when "closed structure + strong light" is detected, the spatial weight and specular color channel weight of the corresponding region are automatically increased). Through correlation analysis between parameters, the model's semantic understanding of the artifact generation mechanism is strengthened (e.g., "why artifacts occur in this scene"), enabling the model to accurately distinguish artifacts from real areas even in complex scenes (such as multiple light sources and mixed materials).

[0067] After feature enhancement is completed, a trained semantic segmentation network (such as the U-Net model combined with Transformer) is used to process the enhanced multidimensional feature vector. During the training process, the semantic segmentation network learns the feature differences between artifacts and real regions through training on a large amount of labeled data. It can analyze the enhanced multidimensional feature vector, identify and mark the artifact regions in the image, and obtain several artifact regions.

[0068] However, due to the complexity of the image itself and noise interference, the boundaries of the initially identified artifact regions may not be accurate enough. To further refine the boundaries of each artifact region, a multi-scale conditional random field (MS-CRF) is used. The multi-scale conditional random field can consider the spatial neighborhood information of the image from different scales and combine the correlation between pixels to optimize the boundaries of the artifact regions, making the segmentation results more consistent with the real boundaries of the artifacts and improving the accuracy and completeness of artifact recognition.

[0069] In addition to removing identified artifact regions, image inpainting can also be performed as needed. For example, a multimodal inpainting model based on generative adversarial networks (GANs) can be used to generate appropriate inpainting content by combining structural data and illumination parameters. The inpainted region can then be fused with the non-artifact regions of the original image to generate a third high-resolution image free of artifact interference.

[0070] As an example, a three-way attention mechanism is used to enhance the multidimensional feature vector to obtain an enhanced multidimensional feature vector. This includes: analyzing the light reflection complexity, material spectral sensitivity, and parameter correlation based on the real-time light intensity, the structural data, and the combination of illumination parameters; inputting the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain scene classification; matching the scene classification to obtain a set of weight coefficients; inputting the multidimensional feature vector into the spatial-structural attention module, the spectral-illumination attention module, and the parameter-context attention module respectively to obtain corresponding enhanced feature vectors; and weighting and fusing the enhanced feature vectors based on the set of weight coefficients to obtain the enhanced multidimensional feature vector.

[0071] In the quality inspection of surface treatment effects in non-fully open areas of castings, real-time light intensity determines the initial energy and direction of incident light, structural data (hole depth, shape, groove width, etc.) affects the internal reflection path of light, and the combination of illumination parameters (light intensity, angle, spectral type, etc.) directly changes the interaction between light and the casting surface. By comprehensively analyzing these data, we can derive the light reflection complexity, that is, the complexity of multiple reflections of light on a complex structural surface; the higher the complexity, the greater the possibility of artifacts. Material spectral sensitivity reflects the absorption and reflection characteristics of different materials for specific spectral light; some materials are prone to reflection artifacts under specific spectra. Parameter correlation refers to the relationship between real-time light intensity, structural data, and the combination of illumination parameters; for example, a specific structure exhibits a specific pattern of light reflection under certain illumination intensities and angles.

[0072] Gated neural networks possess powerful pattern recognition and classification capabilities. Using the aforementioned light reflection complexity, material spectral sensitivity, and parameter correlation as inputs, the gated neural network learns the feature patterns of artifacts in different scenes from a large amount of historical data to classify the current detection scene.

[0073] For example, if the light reflection complexity is high and the material spectral sensitivity is prominent in a certain frequency band, it can be identified as a "high-reflectivity complex material scene"; if the parameter correlation shows that a specific combination of lighting parameters and structure is prone to artifacts, it is classified as a "parameter coupling sensitive scene".

[0074] Different scene classifications correspond to different artifact feature distribution patterns. Based on pre-defined matching rules, a corresponding set of weight coefficients is matched for each scene classification, that is, the weights of the spatial-structure attention module, the spectral-illumination attention module, and the parameter-context attention module are determined.

[0075] For example, in "highly reflective complex material scenes", the weight of the spectral-lighting attention module will be increased accordingly, making it more focused on capturing artifact features related to the material spectrum; in "parameter coupling sensitive scenes", the weight of the parameter-context attention module will be increased to better analyze artifacts under the combined influence of multiple parameters.

[0076] The multidimensional feature vector contains image pixel features as well as multi-source information such as real-time light intensity, structural data, and combinations of illumination parameters. Three attention modules enhance these features from different perspectives. Specifically: the spatial-structural attention module focuses on areas with complex light reflection based on structural data and illumination parameters, enhancing the features of these areas; the spectral-illumination attention module highlights spectral feature channels related to artifacts based on the material's spectral sensitivity; and the parameter-context attention module understands the semantic features of artifacts under the combined effect of multiple parameters from the perspective of parameter correlation.

[0077] After obtaining the three enhanced feature vectors, a weighted sum is performed based on the weight coefficients obtained from the aforementioned matching to achieve feature fusion. This ensures that each module contributes feature information that meets the needs of the current scene to the final enhanced multidimensional feature vector. For example, in a "high-reflectivity complex material scene," the enhanced feature vector output by the spectral-illumination attention module has a higher proportion during fusion, thereby strengthening the expression of artifact features in this scene and providing a more accurate and targeted feature vector for subsequent artifact recognition.

[0078] As an example, the step of inputting the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain scene classification includes: determining the number of preset scene categories in a preset scene classification set; determining the number of neurons in the output layer of the gated neural network based on the number; and adjusting the type of adaptive activation function in the hidden layer; inputting the normalized light reflection complexity, the material spectral sensitivity, and the parameter correlation into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer; calculating the probability value of each preset scene category through the Softmax function in the output layer; and selecting the scene category with the highest probability value as the final classification result, thus obtaining the scene classification.

[0079] Based on the structural type (such as straight hole, stepped hole, U-groove, V-groove), material type (such as aluminum alloy, cast iron, ceramic) and lighting conditions (such as low light intensity, high light intensity, multi-angle mixed light) of the non-fully open area of ​​the casting, N preset scene categories are divided in advance, and a preset scene category set is constructed.

[0080] The more preset scene categories (N) in the preset scene classification set, the more detailed the division of casting quality inspection scenes, such as distinguishing complex scenes with holes of different depths or different materials under specific lighting conditions. In this case, the neural network needs to have a stronger feature discrimination capability. To address this, this invention sets the number of output layer neurons in the gated neural network based on this number (N), and adjusts the type of adaptive activation function in the hidden layers. Specifically: the number of output layer neurons is directly set to N, so that each neuron corresponds to a preset scene category, ensuring that the network output can cover all classification results. Simultaneously, the hidden layer activation function is adjusted according to the size of N: for example, when N≤5, the ReLU function is used because it is simple and efficient, and can meet the feature extraction needs of scenarios with fewer categories; when N>5, the LeakyReLU function is switched to. This function solves the "neuron death" problem of the ReLU function on the negative half-axis by introducing a small non-zero slope, enhancing the network's feature expression capability for complex, multi-class scenarios, ensuring that the network structure adapts to different classification needs, and improving the accuracy of scene classification.

[0081] like Figure 2As shown, the gated neural network includes an input layer, a hidden layer, and an output layer. The light reflection complexity, the material spectral sensitivity, and the parameter correlation are normalized (e.g., the data is mapped to the [0,1] interval) to eliminate scale differences between data. The normalized feature data is then input into the input layer of the gated neural network.

[0082] Next, the hidden layer performs a nonlinear transformation on the input features, uncovering the complex relationships between light reflection complexity, material spectral sensitivity, and parameter correlations to extract more discriminative high-order features. The output layer then uses the Softmax function to transform the feature vector output by the hidden layer into a probability distribution, where each probability value represents the likelihood that the current input data belongs to a corresponding preset scene category. The scene category with the highest probability value is selected as the final result, achieving accurate judgment of the current casting quality inspection scene. For example, if the scene "deep-hole aluminum alloy material under high light intensity irradiation" has the highest probability value, the current scene is determined to belong to that category. This helps to improve the subsequent adjustment of artifact recognition weights based on scene classification and enhance the accuracy of quality inspection.

[0083] like Figure 3 As shown in the figure, this embodiment of the invention also provides a quality inspection device 10 for the surface treatment effect of castings based on machine vision. The device 10 includes an image capture control module 1001, an artifact recognition and processing module 1002, and a quality inspection module 1003. The image capture control module 1001 extracts the real-time light intensity of the non-fully open area of ​​the casting from a first high-definition image, and predicts a combination of illumination parameters that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area. The artifact recognition and processing module 1002 captures a second high-definition image under the illumination parameter combination, identifies and removes artifact areas in the second high-definition image, and obtains a third high-definition image. The quality inspection module 1003 extracts quality inspection parameters from the third high-definition image and evaluates the surface treatment effect of the casting based on the quality inspection parameters.

[0084] As an example, the shooting control module 1001 is used to: construct a three-dimensional geometric model based on the structural data of the non-fully open area; derive a physical model of the propagation and reflection of light within the three-dimensional geometric model based on the principles of geometric optics, including calculation equations for the incident angle and reflection angle of light, the relationship function between the reflectivity of the material surface and the wavelength of light, and a model of light energy attenuation after multiple reflections; and use the real-time light intensity as the initial condition, minimize the probability of artifact generation as the objective function, and use the adjustable parameters of the lighting equipment as optimization variables to solve for the optimal combination of illumination parameters using an optimization algorithm.

[0085] As an example, the artifact recognition processing module 1002 is used to: encode the pixel features of the second high-definition image with real-time light intensity, structural data of the non-fully open region, and the illumination parameters to construct a multi-dimensional feature vector containing visual information, illumination parameters, and structural parameters; enhance the multi-dimensional feature vector using a three-way attention mechanism to obtain an enhanced multi-dimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; process the enhanced multi-dimensional feature vector using a trained semantic segmentation network to obtain several artifact regions, and refine the boundaries of each artifact region using a multi-scale conditional random field.

[0086] As an example, the artifact recognition processing module 1002 is used to: analyze the light reflection complexity, material spectral sensitivity, and parameter correlation based on the real-time light intensity, the structural data, and the combination of illumination parameters; input the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain scene classification; and obtain a weight coefficient group based on the scene classification matching; input the multidimensional feature vector into the spatial-structural attention module, the spectral-illumination attention module, and the parameter-context attention module respectively to obtain the corresponding enhanced feature vector; and perform weighted fusion of each enhanced feature vector based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

[0087] As an example, the artifact recognition processing module 1002 is used to: determine the number of preset scene categories in the preset scene category set; determine the number of neurons in the output layer of the gated neural network based on the number; and adjust the type of the adaptive activation function of the hidden layer; input the normalized light reflection complexity, the material spectral sensitivity, and the parameter correlation into the input layer of the gated neural network; perform nonlinear mapping on the input features through the hidden layer; calculate the probability value of each preset scene category through the Softmax function in the output layer; and select the scene category with the highest probability value as the final classification result, thus obtaining the scene category.

[0088] This invention also provides an electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the computer program, when executed by the processor, implements the method as described in any of the foregoing embodiments.

[0089] This invention also provides a computer storage medium storing a computer program that can be executed by a processor to implement the methods described in any of the foregoing claims.

[0090] This invention also provides a computer program product comprising a computer program that can be executed by a processor to implement the method as described in any of the foregoing embodiments.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0092] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A quality inspection method for the surface treatment effect of castings based on machine vision, characterized in that: The method includes the following steps: extracting the real-time light intensity of the non-fully open area of ​​the casting from a first high-definition image; predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area; capturing a second high-definition image under the illumination parameter combination; identifying and removing artifact areas in the second high-definition image to obtain a third high-definition image; extracting quality inspection parameters from the third high-definition image; and evaluating the surface treatment effect of the casting based on the quality inspection parameters. Based on the real-time light intensity and the structural data of the non-fully open area, a combination of illumination parameters that is beneficial to reducing the probability of artifact generation is predicted. This includes: constructing a three-dimensional geometric model based on the structural data of the non-fully open area; deriving a physical model of light propagation and reflection within the three-dimensional geometric model based on the principles of geometric optics, including equations for calculating the incident angle and reflection angle of light, the relationship function between the reflectivity of the material surface and the wavelength of light, and a model of light energy attenuation after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as optimization variables, an optimization algorithm is used to solve for the optimal combination of illumination parameters. Identifying and removing artifact regions from the second high-definition image to obtain a third high-definition image includes: encoding the pixel features of the second high-definition image with real-time light intensity, structural data of the non-fully open region, and the illumination parameters to construct a multi-dimensional feature vector containing visual information, illumination parameters, and structural parameters; using a three-way attention mechanism to enhance the multi-dimensional feature vector to obtain an enhanced multi-dimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; using a trained semantic segmentation network to process the enhanced multi-dimensional feature vector to obtain several artifact regions; and using a multi-scale conditional random field to refine the boundaries of each artifact region.

2. The quality inspection method for the surface treatment effect of castings based on machine vision according to claim 1, characterized in that: A three-way attention mechanism is used to enhance the multidimensional feature vector to obtain an enhanced multidimensional feature vector. This includes: analyzing the light reflection complexity, material spectral sensitivity, and parameter correlation based on the real-time light intensity, the structural data, and the combination of illumination parameters; inputting the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain a scene classification; matching the scene classification to obtain a set of weight coefficients; inputting the multidimensional feature vector into the spatial-structural attention module, the spectral-illumination attention module, and the parameter-context attention module respectively to obtain corresponding enhanced feature vectors; and weighting and fusing the enhanced feature vectors based on the set of weight coefficients to obtain the enhanced multidimensional feature vector.

3. The quality inspection method for the surface treatment effect of castings based on machine vision according to claim 2, characterized in that: The process of inputting the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain scene classification includes: determining the number of preset scene categories in a preset scene classification set; determining the number of neurons in the output layer of the gated neural network based on the number; and adjusting the type of adaptive activation function in the hidden layer; inputting the normalized light reflection complexity, the material spectral sensitivity, and the parameter correlation into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer; calculating the probability value of each preset scene category through the Softmax function in the output layer; and selecting the scene category with the highest probability value as the final classification result, thus obtaining the scene classification.

4. A quality inspection device for the surface treatment effect of castings based on machine vision, characterized in that, The device includes a shooting control module, an artifact recognition and processing module, and a quality inspection module. The shooting control module extracts the real-time light intensity of the non-fully open area of ​​the casting from a first high-definition image, and predicts an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area. The artifact recognition and processing module captures a second high-definition image under the illumination parameter combination, identifies and removes artifact areas in the second high-definition image, and obtains a third high-definition image. The quality inspection module extracts quality inspection parameters from the third high-definition image, and evaluates the surface treatment effect of the casting based on the quality inspection parameters. The shooting control module is used to: construct a three-dimensional geometric model based on the structural data of the non-fully open area; derive a physical model of light propagation and reflection within the three-dimensional geometric model based on the principles of geometric optics, including calculation equations for the incident angle and reflection angle of light, the relationship function between the reflectivity of the material surface and the wavelength of light, and a model of light energy attenuation after multiple reflections; and use the real-time light intensity as the initial condition, minimize the probability of artifact generation as the objective function, and use the adjustable parameters of the lighting equipment as optimization variables to solve for the optimal combination of illumination parameters using an optimization algorithm. The artifact recognition processing module is used to: encode the pixel features of the second high-definition image with real-time light intensity, structural data of the non-fully open region, and the illumination parameters to construct a multi-dimensional feature vector containing visual information, illumination parameters, and structural parameters; enhance the multi-dimensional feature vector using a three-way attention mechanism to obtain an enhanced multi-dimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; process the enhanced multi-dimensional feature vector using a trained semantic segmentation network to obtain several artifact regions; and refine the boundaries of each artifact region using a multi-scale conditional random field.

5. The quality inspection device for casting surface treatment effect based on machine vision according to claim 4, characterized in that: The artifact recognition processing module is used to: analyze the light reflection complexity, material spectral sensitivity, and parameter correlation based on the real-time light intensity, the structural data, and the combination of illumination parameters; input the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain scene classification; and obtain a weight coefficient group based on the scene classification matching. The multidimensional feature vectors are respectively input into the spatial-structure attention module, the spectral-illumination attention module, and the parameter-context attention module to obtain the corresponding enhanced feature vectors. The enhanced feature vectors are then weighted and fused based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

6. The quality inspection device for casting surface treatment effect based on machine vision according to claim 5, characterized in that: The artifact recognition processing module is used to: determine the number of preset scene categories in the preset scene category set; determine the number of neurons in the output layer of the gated neural network based on the number; and adjust the type of the adaptive activation function of the hidden layer; input the normalized light reflection complexity, the material spectral sensitivity, and the parameter correlation into the input layer of the gated neural network; perform nonlinear mapping on the input features through the hidden layer; calculate the probability value of each preset scene category through the Softmax function in the output layer; and select the scene category with the highest probability value as the final classification result, thus obtaining the scene category.

Images