Casting surface treatment effect quality inspection method and device based on machine vision

Abstract

The invention belongs to the technical field of casting quality inspection, and provides a quality inspection method and a quality inspection device for a casting surface treatment effect based on machine vision. The method comprises the following steps: extracting real-time light intensity of a non-fully-open area of a casting from a first high-definition image, and predicting an irradiation parameter combination beneficial to reducing the generation probability of artifacts based on the real-time light intensity and structural data of the non-fully-open area; shooting a second high-definition image under the irradiation intensity, and identifying and removing an artifact area in the second high-definition image to obtain a third high-definition image; and quality inspection parameters are extracted from the third high-definition image, and the surface treatment effect of the casting is evaluated based on the quality inspection parameters. According to the method, the problem of artifact interference in a traditional casting quality inspection method can be effectively solved, and high-precision and high-efficiency detection of the casting surface treatment effect is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of casting quality inspection, and in particular to a method and device for inspecting the surface treatment effect of castings based on machine vision. Background Art

[0002] In modern manufacturing, castings are important basic components of mechanical products. The quality of their surface treatment directly affects the corrosion resistance, wear resistance and appearance quality of the products, which in turn is related to the overall performance and service life of the mechanical products.

[0003] While traditional sensor-based inspection methods (such as eddy current testing and ultrasonic testing) can detect internal and surface defects in castings to a certain extent, they are limited in their ability to detect surface treatment effects (such as coating uniformity, roughness variations, and surface color consistency), making it difficult to intuitively capture surface texture features and subtle defect information. While the recent rise of machine vision-based inspection technology has partially addressed these shortcomings of traditional inspection methods, existing technologies still present numerous challenges. These challenges primarily arise from the fact that casting surfaces are divided into fully open and partially open areas. When inspecting surface treatment effects in fully open areas, conventional feature extraction algorithms can effectively mitigate interference from ambient light, thereby ensuring high inspection accuracy. However, for partially open areas such as holes and deep grooves, existing machine vision-based inspection methods typically require the use of lighting (e.g., integrated with a camera) to capture images before analysis and inspection. Due to the complex internal structure of holes, light reflects multiple times off the inner walls, creating artifacts that can be misidentified as actual defects. This significantly impacts the accuracy of inspection results and makes it difficult to accurately determine the surface treatment effects in these areas.

[0004] Therefore, there is an urgent need for a new quality inspection method for the surface treatment effects of castings based on machine vision to reduce the impact of artifacts on quality inspection results, achieve high-precision, high-efficiency, and standardized detection of the surface treatment effects of castings, and meet the strict requirements of modern manufacturing for casting quality control. Summary of the Invention

[0005] To this end, the present invention provides a quality inspection method, quality inspection device, electronic equipment, computer storage medium and computer program product for the surface treatment effect of castings based on machine vision to solve at least one of the above technical problems.

[0006] In a first aspect, the present invention provides a quality inspection method for the surface treatment effect of a casting based on machine vision, comprising the following method steps: extracting the real-time light intensity of a non-fully open area of ​​the casting from a first high-definition image, and predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area; capturing a second high-definition image under the illumination parameter combination, identifying and eliminating the artifact area in the second high-definition image, and obtaining a third high-definition image.

[0007] Quality inspection parameters are extracted from the third high-definition image, and the surface treatment effect of the casting is evaluated based on the quality inspection parameters.

[0008] In a second aspect, the present invention provides a quality inspection device for the surface treatment effect of castings based on machine vision, the device comprising a shooting control module, an artifact recognition and processing module, and a quality inspection module.

[0009] The shooting control module extracts the real-time light intensity of the non-fully open area of ​​the casting from the first high-definition image, and predicts an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and structural data of the non-fully open area.

[0010] The artifact recognition and processing module captures a second high-definition image under the illumination parameter combination, identifies and removes artifact areas in the second high-definition image, and obtains a third high-definition image.

[0011] The quality inspection module extracts quality inspection parameters from the third high-definition image, and evaluates the surface treatment effect of the casting based on the quality inspection parameters.

[0012] In a third aspect of the present invention, an electronic device is provided, comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the computer program implements any of the methods described above when executed by the processor.

[0013] According to a fourth aspect of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program executable by a processor to implement any of the methods described above.

[0014] According to a fifth aspect of the present invention, a computer program product is provided, which comprises a computer program executable by a processor to implement any of the methods described above.

[0015] This method acquires real-time light intensity by capturing images of non-fully open areas, combines them with regional structural data to predict optimal illumination intensity, and reduces light reflection artifacts. This method then uses image artifact recognition and elimination, along with quality inspection parameter extraction and evaluation, to effectively overcome artifact interference issues in traditional casting quality inspection methods. This method enables high-precision and efficient testing of casting surface treatment effects, meeting the stringent quality control requirements of modern manufacturing and ensuring the performance and lifespan of castings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 The present invention is a flowchart of a method for inspecting the surface treatment effect of a casting based on machine vision.

[0018] Figure 2 It is a schematic diagram of the structure of the gated neural network disclosed in an embodiment of the present invention.

[0019] Figure 3 The present invention is a schematic structural diagram of a device for inspecting the surface treatment effect of a casting based on machine vision disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following specific embodiments illustrate the implementation of this application. Those familiar with the art can easily understand the other advantages and functions of this application from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of this application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0022] When inspecting the surface treatment of non-fully exposed areas of a casting, lighting is required to assist in capturing images. This can easily lead to artifacts, which can interfere with defect identification. For example, reflections / glare: When the casting surface is rough or tilted, the lighting can create local highlights that obscure subtle defects (such as cracks and pores); shadows: Uneven surfaces (such as pinholes and burrs) can cast shadows due to the lighting angle, leading to misinterpretations as defects; specular reflections: Smooth areas (such as machined surfaces) can reflect the shape of the light source (such as the image of a lamp tube), interfering with visual judgment; and color shift: The color temperature of the lighting (e.g., cold light vs. warm light) can alter the color of the casting, affecting the identification of defects such as oxidation and inclusions.

[0023] In response to the above technical issues, such as Figure 1 As shown, an embodiment of the present invention discloses a quality inspection method for the surface treatment effect of a casting based on machine vision, comprising the following method steps: S10, extracting the real-time light intensity of a non-fully open area of ​​the casting from a first high-definition image, and predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area.

[0024] The first high-definition image of the non-fully open area of ​​the casting (such as holes and deep grooves) is collected through machine vision equipment (such as high-definition industrial cameras). Using image analysis technology, the brightness information of each pixel in the image is statistically analyzed to calculate the real-time light intensity of the area, that is, to obtain the current lighting conditions of the area to be inspected.

[0025] Based on real-time light intensity and structural data of the non-fully open area (such as hole depth, diameter, and shape, and deep groove width and depth, among other geometric parameters), the system uses light propagation and reflection principles to simulate the propagation and reflection of light within the area under different illumination parameter combinations. The system predicts the optimal illumination parameter combination that minimizes light reflections and reduces the probability of artifacts. For example, for deep holes with complex internal structures, it is predicted that reducing light intensity and using multi-angle scattered light illumination can effectively reduce the probability of reflection artifacts.

[0026] It is understood that when capturing a non-fully visible area, machine vision equipment can first capture each area individually before synthesizing it into a panoramic image. Subsequent analysis also targets the panoramic image. Furthermore, lighting equipment (such as LED light sources and fiber optic light guides) can be integrated with the machine vision equipment to provide corresponding illumination for each area captured.

[0027] S20 , capturing a second high-definition image under the illumination parameter combination, identifying and removing artifact areas in the second high-definition image, and obtaining a third high-definition image.

[0028] Based on the optimal illumination parameter combination predicted in step S10, the lighting equipment is adjusted and the partially exposed area of ​​the casting is photographed again to obtain a second high-definition image (e.g., the panoramic image described above). Under the optimized lighting conditions, the second high-definition image now reduces artifacts caused by light reflections and more clearly presents the true features of the casting surface.

[0029] At the same time, artifact areas in the second HD image are further identified and removed to produce a third HD image. For example, a deep learning-based artifact recognition algorithm is used to process the second HD image. By learning and analyzing image features, the algorithm accurately distinguishes artifact areas from real defects or normal surface areas. The identified artifact areas are marked and removed (for example, by using image masking techniques to set pixel values ​​in the artifact areas to specific values ​​or perform repairs), resulting in a third HD image free of artifact interference.

[0030] In this way, the image quality can be further improved, so that subsequent detection can be performed based on more accurate image data, avoiding misjudgment of detection results caused by artifacts.

[0031] S30, extracting quality inspection parameters from the third high-definition image, and evaluating the surface treatment effect of the casting based on the quality inspection parameters.

[0032] In this step, appropriate image analysis algorithms and techniques are used to extract various quality inspection parameters related to the casting surface treatment effect, including but not limited to surface roughness, coating thickness, defect size (such as crack length and pore diameter), surface color uniformity, etc. Edge detection algorithms are then used to extract the outline of surface defects and calculate their size, and grayscale analysis or color feature extraction algorithms are used to evaluate the uniformity and color consistency of the coating.

[0033] The extracted quality inspection parameters are compared and analyzed against pre-established quality standards (such as industry standards or internal company control standards). By establishing appropriate evaluation rules and algorithms (such as a weighted scoring method, which assigns different weights to different parameters based on their importance to product quality and calculates a comprehensive score), the casting surface treatment effect is quantitatively evaluated to determine whether it meets quality requirements. For example, if parameters such as surface roughness and coating thickness are within the standard range, the casting surface treatment is considered acceptable; if some parameters exceed the standard, the corresponding quality grade and improvement suggestions are given. The details will not be elaborated here.

[0034] This method acquires real-time light intensity by capturing images of non-fully open areas, combines them with regional structural data to predict optimal illumination intensity, and reduces light reflection artifacts. This method then uses image artifact recognition and elimination, along with quality inspection parameter extraction and evaluation, to effectively overcome artifact interference issues in traditional casting quality inspection methods. This method enables high-precision and efficient testing of casting surface treatment effects, meeting the stringent quality control requirements of modern manufacturing and ensuring the performance and lifespan of castings.

[0035] As an example, the illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area is predicted, including: constructing a three-dimensional geometric model based on the structural data of the non-fully open area, and deriving a physical model of light propagation and reflection in the three-dimensional geometric model based on the principles of geometric optics, including calculation equations for the incident angle and reflection angle of light, a relationship function between the surface reflectivity of the material and the wavelength of light, and a light energy attenuation model after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as optimization variables, an optimization algorithm is used to solve and obtain the optimal illumination parameter combination.

[0036] First, a 3D geometric model is constructed based on the structural data of the non-fully open area to digitize the actual shape of the casting's holes, deep grooves, and other areas. Structural data such as hole diameter, depth, and curvature, and deep groove width and sidewall angles are converted into geometric shapes in 3D space, providing a digital carrier for light simulation.

[0037] Based on the principles of geometric optics, a physical model of light propagation and reflection within a three-dimensional geometric model is derived. Through the calculation equations for the incident and reflection angles of light (following the law of reflection), the relationship function between the surface reflectivity of the material and the wavelength of the light (characterizing the absorption and reflection characteristics of different materials for light), and the light energy attenuation model after multiple reflections (taking into account the energy loss of light during the reflection process), the behavior of light within complex structures is accurately simulated, making the propagation of light visual and calculable.

[0038] In a 3D geometric model, the normal vector of each surface point is Determined by the geometry of the model. For example: Planar area: Normal vector is a fixed normal vector to a plane (such as the sidewall of a deep groove).

[0039] Surface Area: Normal Vector is the normal vector of the tangent plane of the surface at that point (such as the inner wall of a hole), which can be calculated using parametric equations or triangular meshes.

[0040] Polyhedron Intersection: Normal Vector Abrupt changes may occur at the junction (such as the step surface of a stepped hole).

[0041] (1) The calculation equations for the incident angle and reflection angle of light are: ;in, is a unit vector pointing from the surface of the object to the reflection direction; is a unit vector pointing from the light source to the surface of the object (direction of incidence); is the unit normal vector, perpendicular to the surface of the object and pointing outward; Represents the dot product operation of the vector, which is used to calculate the cosine of the angle of incidence ( is the angle of incidence).

[0042] When light hits the surface of a 3D model, it is necessary to first determine the intersection of the light and the model using a ray tracing algorithm (such as the BVH acceleration structure), and then obtain the normal vector of that point. , and finally substitute it into the reflection equation to calculate the reflection direction For example, when reflecting off the inner wall of a hole, the normal vector As the curvature of the inner wall changes, the reflection direction is dynamically adjusted. When reflecting at the bottom of a deep groove, if the bottom is a slope, the normal vector When tilted, the reflected light will bend in a specific direction.

[0043] (2) The relationship function between the surface reflectivity of the material and the wavelength of the light is: ;in, is the spectral reflectance function, that is, the ratio of the energy of light reflected from the casting surface to the energy of incident light. It is a dimensionless parameter. The larger the ratio, the higher the proportion of light of this wavelength reflected. is the wavelength of light (unit: nm); is the polynomial coefficient (unit: nm -k ), characterizes the basic reflection trend of the material to the continuous spectrum, is the highest degree of the polynomial terms, e.g. When , the polynomial is , the higher the maximum number, the finer the description of the continuous spectrum reflection law; is the Gaussian term (i.e. ) is a dimensionless parameter used to adjust the Gaussian peak (i.e. The height of the curve shape depicted, is the width coefficient of the Gaussian peak (unit: nm -2 ), controls the width of the Gaussian peak, is the central wavelength of the Gaussian peak (unit: nm), corresponding to the characteristic wavelength where the material's reflection characteristics show a sudden change. Used to fit the reflection mutation of materials at specific wavelengths (such as the characteristic absorption peak of metals).

[0044] Material surface reflectivity function Parameters (such as ) is a spatial variable in a 3D model and depends on the material distribution at each point on the model surface. For example, a casting surface may have a coating, and the reflectivity function parameters of the coated area and the uncoated area are different. Holes of different depths inside a casting may use different materials (such as metal matrix and filler material), and independent definitions need to be defined for each area. .

[0045] First, determine the intersection of the light and the 3D model; query the material ID of the point through the intersection position; call the corresponding reflectivity function according to the material ID Calculate reflectivity.

[0046] (3) The light energy attenuation model is: ;in, The remaining detectable light energy after m reflections inside the casting (unit: J); is the initial incident light energy (unit: J), i.e. the original energy emitted by the light source; is the spectral reflectance of the kth reflection surface (calculated by the above relationship function), which represents the reflection ability of the casting surface to light. Indicates full launch, Indicates complete absorption; is the incident angle of the kth reflection (unit: °), that is, the angle between the incident direction of the light and the surface normal vector; the cosine value of the incident angle is the correction factor for the energy projection area at oblique incidence; is the propagation distance of the light after the kth reflection (unit: mm), that is, the distance traveled by the light during the k-th reflection process; is the attenuation length (related to the scattering characteristics of the ambient medium, unit: mm). This parameter is used to describe the scattering characteristics of the ambient medium (such as air, coolant residue) on light. The smaller the scattering, the stronger the The larger it is, the weaker the scattering; Characterizes the scattering loss during light propagation. It is a dimensionless scattering loss metric that quantifies the severity of light scattering in the medium by the ratio of the propagation distance to the attenuation length.

[0047] Propagation distance in the energy decay model and the angle of incidence All determined by the three-dimensional geometric model: propagation distance : The straight-line distance of a ray between two reflections needs to be calculated using the coordinates of the starting and ending points in three-dimensional space (determined by ray tracing).

[0048] Angle of incidence :From the direction of incident light Normal vector at the intersection The dot product of .

[0049] In complex structures (such as deep holes), light may experience multiple reflections, and each reflection All are different: 1st reflection: intersection point is at the hole mouth, Determined by the orifice material, is the distance from the orifice to the first inner wall reflection point.

[0050] Second reflection: the intersection point is on the inner wall, Determined by the inner wall material, is the distance between two inner wall reflection points.

[0051] The above process is repeated until the ray energy falls below the threshold or leaves the model area.

[0052] The processing process is illustrated as follows: Assume that light enters a stepped hole (a cylindrical hole with a sudden change in diameter): 1. Geometric model preparation: discretize the stepped hole into a triangular mesh, and each mesh stores the normal vector and material ID. Assign different material parameters to hole segments of different diameters (such as ).

[0053] 2. Ray tracing and reflection calculation: The light starts from the light source and intersects the stepped hole surface at point , get Normal vector And material ID_1. Substitute into the reflection equation to calculate the reflection direction .

[0054] 3. Energy decay calculation: through The calculated wavelength is The light in Reflectivity at Calculate the angle of incidence and propagation distance (Light source to Update energy: .

[0055] 4. Multiple reflection iterations: reflecting light Continue intersecting the model at point Repeat the above steps to calculate .like Located at the diameter change point of the stepped hole, the normal vector and material ID_2 may mutate, resulting in significant changes in reflection behavior and energy attenuation.

[0056] Then, using real-time light intensity as the initial condition and the current lighting conditions in the inspection area as the starting point for calculations, the objective function, or optimization direction, is to minimize the probability of artifact generation. Adjustable parameters of the lighting equipment (such as light intensity, illumination angle, and spectral type) are used as optimization variables. These adjustable parameters directly affect the reflection of light on the casting surface. Optimization algorithms (such as genetic algorithms and particle swarm algorithms) are used to quantitatively assess the likelihood of artifact generation by calculating relevant indicators of potential artifact areas (such as the area of ​​overexposed areas and the proportion of abnormally bright and dark areas) under different illumination parameter combinations. Through continuous iterative calculation and screening, the optimal illumination parameter combination is solved to minimize the probability of artifact generation when light propagates in non-fully open areas.

[0057] As an example, the identification and removal of artifact areas in the second high-definition image to obtain a third high-definition image includes: encoding the pixel features of the second high-definition image with the real-time light intensity, the structural data of the non-fully open area, and the illumination parameter combination to construct a multidimensional feature vector containing visual information, illumination parameters, and structural parameters; using a three-way attention mechanism to enhance the features of the multidimensional feature vector to obtain an enhanced multidimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; using a trained semantic segmentation network to process the enhanced multidimensional feature vector to obtain several artifact areas, and using a multi-scale conditional random field to refine the boundaries of each artifact area.

[0058] While the aforementioned embodiment reduces the probability of artifact generation by employing an optimal parameter combination, the complex structure and diverse materials of the non-fully open areas of the casting, along with subtle differences in lighting and structural details, can still lead to artifacts. Therefore, artifact identification and removal are still necessary in the second high-definition image. Furthermore, due to the complex structure and variable lighting conditions of the non-fully open areas, artifact characteristics vary in different scenarios, and a single image pixel feature cannot fully reflect the causes and characteristics of artifacts.

[0059] To address the above situation, this embodiment encodes the pixel features of the second HD image along with real-time light intensity, regional structural data, and illumination parameters to construct a multidimensional feature vector. This allows for comprehensive image analysis from multiple perspectives, effectively addressing artifact misidentification and omissions caused by incomplete information. Specifically, the pixel features of the second HD image are encoded along with real-time light intensity, structural data of the non-fully open region, and illumination parameters. The pixel features of the second HD image contain visual information such as the casting surface texture, color, and grayscale, providing intuitive insight into artifact detection. Real-time light intensity reflects the current lighting conditions in the non-fully open region, influencing the image's brightness and shadows and artifact generation. Structural data (such as hole depth, shape, and deep groove width) within the non-fully open region determines the internal light propagation path and reflection characteristics, and is closely related to artifact generation. The illumination parameter combination (light intensity, angle, spectral type, etc.) represents pre-optimized lighting conditions and directly impacts image quality. This information is fused through encoding technology to construct a multidimensional feature vector, enabling comprehensive image analysis from multiple perspectives.

[0060] Then, through the collaborative work of the spatial-structural attention module, the spectral-illumination attention module and the parameter-context attention module, the feature enhancement of the multi-dimensional feature vector is achieved using a three-way attention mechanism.

[0061] Spatial-Structural Attention Module: Based on the spatial attention mechanism and combined with geometric feature extraction algorithms (such as curvature calculation and regional topological relationship analysis), the core of the spatial attention mechanism is to strengthen the feature representation of key areas (such as those with complex structures and prone to artifacts) by calculating the weights of different spatial locations in the feature map.

[0062] The enhancement process is as follows: 3D structural data (such as geometric parameters of holes, deep grooves, and corners) and illumination parameters (such as light source angle and intensity) of non-fully open areas are acquired to construct a spatial feature map. Geometric algorithms are used to calculate the curvature of the structural surface (such as areas with sudden changes in curvature) and topological relationships (such as closed / semi-closed areas), and to locate areas prone to artifacts (such as hole corners and the bottoms of deep grooves—these areas have complex light reflections and are prone to artifacts due to multiple reflections or occlusion). Higher attention weights are assigned to these key areas, while lower weights are assigned to flat, artifact-free areas to generate a spatial attention map. The attention map is weightedly fused with the original spatial feature map to enhance artifact features in areas with sudden changes in curvature (such as uneven brightness and blurred edges), avoiding missed detections due to weak features in these areas.

[0063] Spectral-Illumination Attention Module: Based on the channel attention mechanism, it combines spectral feature mapping algorithms (such as reflectance-wavelength relationship modeling and color space conversion). The channel attention mechanism assigns weights to different channels of the feature map (such as RGB color channels and texture channels) to highlight channel features relevant to the task.

[0064] The enhancement process is: obtaining real-time light intensity and material spectral reflectance function and the multi-channel features of the image (such as color channel, texture channel). According to the wavelength of light ( ) and the material's reflectivity, calculating the proportion of light reflected at different wavelengths on the material's surface. Combined with real-time light intensity, this method determines the color and texture characteristics of artifacts (for example, reflection artifacts of metal materials at specific wavelengths may exhibit a yellowish channel characteristic). Through a channel attention mechanism, artifact-related color channels (such as the RGB components corresponding to specific wavelengths) and texture channels (such as the artifact's blurred or jagged texture) are given high weights, while irrelevant channels (such as the uniform color channel of the background area) are downplayed. The channel weights are then fused with the original image feature map to highlight the color and texture characteristics of the artifact, improving the model's ability to distinguish artifacts from real areas.

[0065] Parameter-Contextual Attention Module: Based on the contextual attention mechanism, it combines multi-parameter correlation modeling (such as analysis of the synergistic effects of illumination parameters, structural parameters, and light intensity). The contextual attention mechanism models the dependencies between global features, allowing the model to comprehensively understand the impact of correlations between parameters on the target (artifacts).

[0066] The enhancement process integrates illumination parameter combinations (such as light source angle and distance), real-time light intensity, and 3D structural data (such as area enclosure) to construct a multidimensional parameter feature matrix. This matrix captures global correlations between parameters (e.g., "small-angle light source + deep groove structure" is prone to shadow artifacts). A self-attention mechanism (such as the scaled dot-product attention in the Transformer) calculates the correlation weights between different parameter combinations and artifact generation, building a global context model. For example, it learns the strong correlation between "specific light source angle + highly reflective material + closed structure" and "highlight artifacts." Based on this global context model, it dynamically adjusts the attention distribution of spatial and spectral feature dimensions (e.g., when "closed structure + strong light" is detected, the spatial weight and highlight color channel weight of the corresponding area are automatically increased). This correlation analysis strengthens the model's semantic understanding of artifact generation mechanisms (e.g., "Why does artifact occur in this scene"), enabling the model to accurately distinguish artifacts from real areas in complex scenes (e.g., with multiple light sources and mixed materials).

[0067] After feature enhancement is completed, a trained semantic segmentation network (such as the U-Net model combined with Transformer) is used to process the enhanced multi-dimensional feature vector. During the training process, the semantic segmentation network learns the characteristic differences between artifacts and real areas through training with a large amount of labeled data. It can analyze the enhanced multi-dimensional feature vector, identify and mark the artifact areas in the image, and obtain several artifact areas.

[0068] However, due to the complexity of the image itself and noise interference, the boundaries of the artifact areas initially identified may not be accurate enough. Therefore, a multi-scale conditional random field (MS-CRF) is further used to refine the boundaries of each artifact area. The multi-scale conditional random field can consider the spatial neighborhood information of the image from different scales and combine the correlation between pixels to optimize the boundaries of the artifact area, so that the segmentation results are more consistent with the true boundaries of the artifacts, thereby improving the accuracy and completeness of artifact recognition.

[0069] In addition to removing artifacts, image restoration can also be performed on identified areas. For example, a multimodal restoration model based on a generative adversarial network (GAN) can be used to combine structural data and lighting parameters to generate appropriate restoration content. The restored area is then fused with the non-artifact areas of the original image to generate a third, high-definition image free of artifacts.

[0070] As an example, a three-way attention mechanism is used to perform feature enhancement on the multidimensional feature vector to obtain an enhanced multidimensional feature vector, including: obtaining light reflection complexity, material spectral sensitivity and parameter correlation based on the real-time light intensity, the structural data and the illumination parameter combination analysis; inputting the light reflection complexity, the material spectral sensitivity and the parameter correlation into a gated neural network to obtain scene classification, and obtaining a weight coefficient group based on the scene classification matching; inputting the multidimensional feature vector into the space-structure attention module, the spectrum-illumination attention module and the parameter-context attention module respectively to obtain corresponding enhanced feature vectors, and performing weighted fusion on each of the enhanced feature vectors based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

[0071] When inspecting the surface treatment effects of non-fully open areas of a casting, real-time light intensity determines the initial energy and direction of the incident light. Structural data (hole depth, shape, groove width, etc.) influences the internal reflection path of the light. The combination of illumination parameters (light intensity, angle, spectral type, etc.) directly changes how the light interacts with the casting surface. Comprehensive analysis of this data reveals light reflection complexity—the complexity of multiple reflections of light on complex structural surfaces. Higher complexity increases the likelihood of artifacts. Material spectral sensitivity reflects the absorption and reflection characteristics of different materials for light of specific spectra. Some materials are more susceptible to reflection artifacts under certain spectra. Parameter correlation refers to the interplay between real-time light intensity, structural data, and illumination parameter combinations. For example, under certain illumination intensities and angles, the light reflection pattern of a specific structure exhibits a specific pattern.

[0072] Gated neural networks have powerful pattern recognition and classification capabilities. Taking the previously determined light reflection complexity, material spectral sensitivity, and parameter correlation as input, the gated neural network classifies the current detection scene by learning the characteristic patterns of artifacts in different scenes from a large amount of historical data.

[0073] For example, if the light reflection complexity is high and the material spectral sensitivity is prominent in a certain frequency band, it can be determined as a "high-reflection complex material scene"; if the parameter correlation shows that a specific lighting parameter and structure combination is prone to artifacts, it is classified as a "parameter coupling sensitive scene."

[0074] Different scene classifications correspond to different artifact feature distribution patterns. Based on pre-set matching rules, a corresponding weight coefficient group is matched for each scene classification, that is, the weights of the spatial-structure attention module, the spectral-illumination attention module, and the parameter-context attention module are determined.

[0075] For example, in "highly reflective complex material scenes", the weight of the spectrum-illumination attention module will be increased accordingly, making it more focused on capturing artifact characteristics related to the material spectrum; in "parameter coupling sensitive scenes", the weight of the parameter-context attention module will be increased to better analyze the artifact situation under the combined influence of multiple parameters.

[0076] The multidimensional feature vector contains image pixel features as well as multi-source information such as real-time light intensity, structural data, and illumination parameter combinations. Three attention modules perform feature enhancement processing based on different aspects. Specifically, the spatial-structural attention module focuses on areas with complex light reflections based on structural data and illumination parameters, enhancing the characteristics of these areas. The spectral-illumination attention module highlights spectral feature channels related to artifacts based on the spectral sensitivity of the material. The parameter-contextual attention module understands the semantic characteristics of artifacts under the combined effects of multiple parameters from the perspective of parameter correlation.

[0077] After obtaining the three enhanced feature vectors, a weighted summation is performed based on the weight coefficients obtained from the aforementioned matching to achieve feature fusion, so that each module contributes feature information that meets the needs of the current scene to the final enhanced multi-dimensional feature vector. For example, in a "highly reflective complex material scene," the enhanced feature vector output by the spectral-illumination attention module accounts for a higher proportion during fusion, thereby strengthening the representation of artifact characteristics in this scene and providing a more accurate and targeted feature vector for subsequent artifact recognition.

[0078] As an example, the light reflection complexity, the material spectral sensitivity and the parameter correlation are input into a gated neural network to obtain a scene classification, including: determining the number of preset scene classifications in a preset scene classification set, determining the number of neurons in the output layer of the gated neural network based on the number, and adjusting the type of adaptive activation function of the hidden layer; normalizing the light reflection complexity, the material spectral sensitivity and the parameter correlation and inputting them into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer, and calculating the probability value of each preset scene classification through the Softmax function in the output layer; selecting the scene classification with the highest probability value as the final classification result, that is, obtaining the scene classification.

[0079] Based on the structural type (such as straight hole, stepped hole, U-shaped groove, V-shaped groove) of the non-fully open area of ​​the casting, the material type (such as aluminum alloy, cast iron, ceramic) and the lighting conditions (such as low light intensity, high light intensity, multi-angle mixed light), N preset scene categories are divided in advance to construct a preset scene classification set.

[0080] The more preset scene classifications (N) there are in the preset scene classification set, the more detailed the division of the casting quality inspection scene will be. For example, to distinguish complex scenes of holes of different depths and different materials under specific lighting, the neural network needs to have stronger feature differentiation capabilities. To this end, the present invention sets a method for determining the number of output layer neurons of the gated neural network based on this number (N), and adjusting the type of adaptive activation function of the hidden layer. Specifically, the number of output layer neurons is directly set to N, so that each neuron corresponds to a preset scene classification, ensuring that the network output can cover all classification results. At the same time, the hidden layer activation function is adjusted according to the size of N: for example, when N≤5, the ReLU function is used because it is simple and efficient and can meet the feature extraction requirements of fewer classification scenarios; when N>5, it switches to the LeakyReLU function, which solves the "neuron death" problem of the ReLU function on the negative semi-axis by introducing a small non-zero slope, thereby enhancing the network's feature expression ability for complex and multi-classification scenes, ensuring that the network structure adapts to different classification requirements, and improving the accuracy of scene classification.

[0081] like Figure 2As shown, the gated neural network includes an input layer, a hidden layer, and an output layer. The light reflection complexity, the material spectral sensitivity, and the parameter correlation are normalized (e.g., by mapping the data to the [0, 1] interval) to eliminate scale differences between the data. The resulting normalized feature data is input into the input layer of the gated neural network.

[0082] Next, the hidden layer performs a nonlinear transformation on the input features, mining the complex relationship between the complexity of light reflection, material spectral sensitivity, and parameter correlation, and extracting more discriminative high-order features. The output layer uses the Softmax function to convert the feature vector output by the hidden layer into a probability distribution. Each probability value represents the possibility that the current input data belongs to the corresponding preset scene classification. The scene classification with the highest probability value is selected as the final result to achieve accurate judgment of the current casting quality inspection scene. For example, if the probability value of the "deep hole aluminum alloy material under high light intensity" scene is the highest, it is determined that the current scene belongs to this classification. This is conducive to improving the subsequent adjustment of artifact recognition weights according to scene classification and improving quality inspection accuracy.

[0083] like Figure 3 As shown, an embodiment of the present invention further provides a quality inspection device 10 for the surface treatment effect of a casting based on machine vision, the device 10 comprising a shooting control module 1001, an artifact recognition and processing module 1002, and a quality inspection module 1003; the shooting control module 1001 extracts the real-time light intensity of a non-fully open area of ​​the casting from a first high-definition image, and predicts an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and the structural data of the non-fully open area; the artifact recognition and processing module 1002 shoots a second high-definition image under the illumination parameter combination, identifies and eliminates the artifact area in the second high-definition image, and obtains a third high-definition image; the quality inspection module 1003 extracts quality inspection parameters from the third high-definition image, and evaluates the surface treatment effect of the casting based on the quality inspection parameters.

[0084] As an example, the shooting control module 1001 is used to achieve: constructing a three-dimensional geometric model based on the structural data of the non-fully open area, and deriving a physical model of light propagation and reflection in the three-dimensional geometric model based on the principles of geometric optics, including calculation equations for the incident angle and reflection angle of light, the relationship function between the surface reflectivity of the material and the wavelength of light, and a light energy attenuation model after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as the optimization variables, an optimization algorithm is used to solve and obtain the optimal illumination parameter combination.

[0085] As an example, the artifact recognition and processing module 1002 is used to implement: encoding the pixel features of the second high-definition image with the real-time light intensity, the structural data of the non-fully open area, and the illumination parameter combination to construct a multidimensional feature vector containing visual information, illumination parameters, and structural parameters; using a three-way attention mechanism to enhance the features of the multidimensional feature vector to obtain an enhanced multidimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; using a trained semantic segmentation network to process the enhanced multidimensional feature vector to obtain several artifact areas, and using a multi-scale conditional random field to refine the boundaries of each artifact area.

[0086] As an example, the artifact recognition and processing module 1002 is used to achieve: based on the real-time light intensity, the structural data and the illumination parameter combination analysis, obtain the light reflection complexity, the material spectral sensitivity and the parameter correlation; input the light reflection complexity, the material spectral sensitivity and the parameter correlation into the gated neural network to obtain scene classification, and obtain a weight coefficient group based on the scene classification matching; input the multidimensional feature vector into the space-structure attention module, the spectrum-lighting attention module and the parameter-context attention module respectively to obtain the corresponding enhanced feature vector, and perform weighted fusion on each of the enhanced feature vectors based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

[0087] As an example, the artifact recognition processing module 1002 is used to implement: determining the number of preset scene classifications in the preset scene classification set, determining the number of output layer neurons of the gated neural network based on the number, and adjusting the type of adaptive activation function of the hidden layer; normalizing the light reflection complexity, the material spectral sensitivity and the parameter correlation and inputting them into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer, and calculating the probability value of each preset scene classification through the Softmax function in the output layer; selecting the scene classification with the highest probability value as the final classification result, that is, obtaining the scene classification.

[0088] An embodiment of the present invention further provides an electronic device comprising: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the computer program implements any of the aforementioned methods when executed by the processor.

[0089] An embodiment of the present invention further provides a computer storage medium storing a computer program that can be executed by a processor to implement any of the methods described above.

[0090] An embodiment of the present invention further provides a computer program product, which includes a computer program that can be executed by a processor to implement any of the methods described above.

[0091] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0092] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for quality inspection of casting surface treatment effects based on machine vision, characterized by: The method includes the following steps: extracting the real-time light intensity of a non-fully open area of ​​a casting from a first high-definition image, and predicting an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and structural data of the non-fully open area; capturing a second high-definition image under the illumination parameter combination, identifying and eliminating the artifact area in the second high-definition image, and obtaining a third high-definition image; extracting quality inspection parameters from the third high-definition image, and evaluating the surface treatment effect of the casting based on the quality inspection parameters.

2. The method for quality inspection of casting surface treatment effects based on machine vision according to claim 1, characterized in that: Based on the real-time light intensity and the structural data of the non-fully open area, an illumination parameter combination that is beneficial to reducing the probability of artifact generation is predicted, including: constructing a three-dimensional geometric model based on the structural data of the non-fully open area, and deriving a physical model of light propagation and reflection in the three-dimensional geometric model based on the principles of geometric optics, including a calculation equation for the incident angle and reflection angle of light, a relationship function between the surface reflectivity of the material and the wavelength of light, and a light energy attenuation model after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as the optimization variables, an optimization algorithm is used to solve and obtain the optimal illumination parameter combination.

3. The method for quality inspection of casting surface treatment effects based on machine vision according to claim 2, characterized in that: Identify and remove artifact areas in the second high-definition image to obtain a third high-definition image, including: encoding the pixel features of the second high-definition image with the real-time light intensity, the structural data of the non-fully open area, and the illumination parameter combination to construct a multidimensional feature vector containing visual information, illumination parameters, and structural parameters; use a three-way attention mechanism to enhance the features of the multidimensional feature vector to obtain an enhanced multidimensional feature vector; wherein the three-way attention mechanism is implemented by a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; use a trained semantic segmentation network to process the enhanced multidimensional feature vector to obtain several artifact areas, and use a multi-scale conditional random field to refine the boundaries of each artifact area.

4. The method for quality inspection of casting surface treatment effects based on machine vision according to claim 3, characterized in that: A three-way attention mechanism is used to perform feature enhancement on the multidimensional feature vector to obtain an enhanced multidimensional feature vector, including: obtaining light reflection complexity, material spectral sensitivity and parameter correlation based on the real-time light intensity, the structural data and the illumination parameter combination analysis; inputting the light reflection complexity, the material spectral sensitivity and the parameter correlation into a gated neural network to obtain scene classification, and obtaining a weight coefficient group based on the scene classification matching; inputting the multidimensional feature vector into the space-structure attention module, the spectrum-illumination attention module and the parameter-context attention module respectively to obtain corresponding enhanced feature vectors, and performing weighted fusion on each of the enhanced feature vectors based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

5. The method for quality inspection of casting surface treatment effects based on machine vision according to claim 4, characterized in that: The light reflection complexity, the material spectral sensitivity and the parameter correlation are input into a gated neural network to obtain a scene classification, including: determining the number of preset scene classifications in a preset scene classification set, determining the number of neurons in the output layer of the gated neural network based on the number, and adjusting the type of adaptive activation function of the hidden layer; normalizing the light reflection complexity, the material spectral sensitivity and the parameter correlation and inputting them into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer, and calculating the probability value of each preset scene classification through the Softmax function in the output layer; selecting the scene classification with the highest probability value as the final classification result, that is, obtaining the scene classification.

6. A quality inspection device for casting surface treatment effect based on machine vision, characterized in that: The device includes a shooting control module, an artifact recognition and processing module, and a quality inspection module; the shooting control module extracts the real-time light intensity of the non-fully open area of ​​the casting from a first high-definition image, and predicts an illumination parameter combination that is beneficial to reducing the probability of artifact generation based on the real-time light intensity and structural data of the non-fully open area; the artifact recognition and processing module shoots a second high-definition image under the illumination parameter combination, identifies and eliminates the artifact area in the second high-definition image, and obtains a third high-definition image; the quality inspection module extracts quality inspection parameters from the third high-definition image, and evaluates the surface treatment effect of the casting based on the quality inspection parameters.

7. The device for inspecting the surface treatment effect of castings based on machine vision according to claim 6, characterized in that: The shooting control module is used to achieve: constructing a three-dimensional geometric model based on the structural data of the non-fully open area, and deriving a physical model of light propagation and reflection within the three-dimensional geometric model based on the principles of geometric optics, including calculation equations for the incident angle and reflection angle of light, a relationship function between the surface reflectivity of the material and the wavelength of the light, and a light energy attenuation model after multiple reflections; using the real-time light intensity as the initial condition, minimizing the probability of artifact generation as the objective function, and the adjustable parameters of the lighting equipment as the optimization variables, and adopting an optimization algorithm to solve and obtain the optimal illumination parameter combination.

8. The device for inspecting the surface treatment effect of castings based on machine vision according to claim 7, characterized in that: The artifact recognition and processing module is used to achieve: encoding the pixel features of the second high-definition image with the real-time light intensity, the structural data of the non-fully open area, and the illumination parameter combination to construct a multidimensional feature vector containing visual information, illumination parameters, and structural parameters; using a three-way attention mechanism to enhance the features of the multidimensional feature vector to obtain an enhanced multidimensional feature vector; wherein the three-way attention mechanism is implemented through a spatial-structural attention module, a spectral-illumination attention module, and a parameter-context attention module; using a trained semantic segmentation network to process the enhanced multidimensional feature vector to obtain several artifact areas, and using a multi-scale conditional random field to refine the boundaries of each artifact area.

9. The device for inspecting the surface treatment effect of castings based on machine vision according to claim 8, characterized in that: The artifact recognition and processing module is configured to: determine light reflection complexity, material spectral sensitivity, and parameter correlation based on a combination analysis of the real-time light intensity, the structural data, and the illumination parameters; input the light reflection complexity, the material spectral sensitivity, and the parameter correlation into a gated neural network to obtain a scene classification; and determine a weight coefficient group based on the scene classification matching; The multidimensional feature vector is respectively input into the spatial-structural attention module, the spectral-illumination attention module and the parameter-context attention module to obtain the corresponding enhanced feature vector, and each enhanced feature vector is weightedly fused based on the weight coefficient group to obtain the enhanced multidimensional feature vector.

10. The device for inspecting the surface treatment effect of castings based on machine vision according to claim 9, characterized in that: The artifact recognition and processing module is used to implement the following: determining the number of preset scene classifications in the preset scene classification set, determining the number of neurons in the output layer of the gated neural network based on the number, and adjusting the type of the adaptive activation function of the hidden layer; normalizing the light reflection complexity, the material spectral sensitivity, and the parameter correlation and inputting them into the input layer of the gated neural network; performing nonlinear mapping on the input features through the hidden layer, and calculating the probability value of each preset scene classification through the Softmax function in the output layer; and selecting the scene classification with the highest probability value as the final classification result, thereby obtaining the scene classification.

Images