Metal nameplate surface detection system based on machine vision

Abstract

This invention relates to the field of machine vision and industrial defect detection, specifically a metal sign surface inspection system based on machine vision. It includes modules for image acquisition, feature analysis, strategy decision-making, dual detection, and result output. The system extracts surface image features to construct a visual feature space and calculates semantic drift entropy. Its core is to dynamically switch based on the comparison between the entropy value and a preset danger threshold: at low entropy values, a generative vision model is called to extract depth features; at high entropy values, a physical optical morphological filtering algorithm is called to extract basic morphological features. This invention breaks the limitations of a single detection architecture, utilizes adaptive scheduling to solve the problem of environmental light spot noise interference, and balances the ability to identify subtle surface defects under stable lighting with the robustness of structural detection under complex optical interference.

Description

Technical Field

[0001] This invention relates to the field of machine vision and industrial defect detection, specifically to a machine vision-based surface inspection system for metal signs. Background Technology

[0002] In the surface inspection environment of metal signs on industrial production lines, the signs often exhibit anisotropic reflectivity, and the inspection station is frequently subjected to external interference such as ambient light fluctuations, equipment reflections, and mechanical vibrations. This leads to high-frequency artifacts, local overexposure, or discrete noise in the acquired image sequences. To analyze surface images for defects, existing visual inspection solutions generally employ a single, fixed processing architecture, relying on a fixed grayscale difference threshold to extract defect features, or simply using a deep learning model for end-to-end judgment of surface images. Although this approach has certain processing capabilities under stable optical conditions, due to the complex reflections of metal surfaces and the dynamic drift of ambient light, single depth models are prone to unstable outputs and false alarms when faced with light spot contamination or strong reflective interference. Traditional morphological and fixed threshold methods are easily overwhelmed by background textures when dealing with minor defects such as scratches. Existing solutions lack adaptive perception of image visual semantic deviations and the ability to dynamically switch detection paths, resulting in poor system robustness in complex environments and difficulty in balancing the sensitivity of minor defect recognition with the reliability of structural defect detection.

[0003] Therefore, under the complex optical interference and anisotropic reflection characteristics of the target object, how to improve the stability and accuracy of surface defect detection of metal signs has become an urgent technical problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a machine vision-based metal sign surface inspection system. Specifically, the technical solution of this invention includes: The image acquisition module is used to acquire a sequence of surface images of the target object; The feature analysis module is used to extract the current image feature set of the surface image sequence to construct a visual feature space, and to calculate the semantic drift entropy based on the visual feature space; The strategy decision module is used to compare the semantic drift entropy with a preset danger threshold; if the semantic drift entropy is less than the preset danger threshold, the first detection module is triggered; if the semantic drift entropy is greater than or equal to the preset danger threshold, the second detection module is triggered. The first detection module is used to extract depth generation features from the surface image sequence using a pre-trained generative vision model, and output a first defect determination result based on the depth generation features; The second detection module is used to select structural elements of corresponding size and preset shape according to the physical scale of the target object and the type of structural defect to be detected, and to extract basic morphological features from the surface image sequence using a morphological filtering algorithm containing the structural elements, and to output a second defect judgment result based on the basic morphological features. The result output module is used to generate a final inspection report for the target object based on the first defect determination result or the second defect determination result.

[0005] Preferably, the process by which the feature analysis module calculates the semantic drift entropy includes: The current image feature set is compared with a preset benchmark feature distribution constructed based on standard defect-free samples to generate a feature distribution difference degree; Based on the feature distribution difference degree, the feature distribution difference degree is mapped to deviation probability through linear normalization or normalized exponential function to form a feature probability distribution, and the semantic drift entropy is calculated by calculating the information entropy of the feature probability distribution.

[0006] Preferably, the system further includes: The risk prediction module is used to predict the probability of missing structural defects in the current detection condition based on the semantic drift entropy. The threshold adjustment module is used to lower the preset danger threshold when the probability of missing a structural defect increases compared to the statistical result of the previous detection time window; and to raise the preset danger threshold when the probability of missing a structural defect decreases or becomes zero compared to the statistical result of the previous detection time window, thereby dynamically updating the preset danger threshold.

[0007] Preferably, the first detection module includes: A reconstruction unit is used to perform feature reconstruction on the surface image sequence using the pre-trained generative vision model, and generate a residual feature map as the depth generation feature. The first determination unit is used to determine the first defect determination result based on the residual feature map.

[0008] Preferably, the second detection module includes: A morphological processing unit is used to perform opening and closing operations on the surface image sequence based on the structural elements of a preset morphology to generate a morphological background image. A difference extraction unit is used to calculate the difference image between the surface image sequence and the morphological background image; The second determination unit is used to determine the second defect determination result based on the differential image.

[0009] Preferably, the first defect determination result includes the determination status of a first type of surface defect and a second type of structural defect; the second defect determination result only includes the determination status of a second type of structural defect. The first type of surface defects includes scratches, uneven coating, local stains, character burrs, and shallow indentations; the second type of structural defects includes character breakage, edge gaps, hole deformation, embossing collapse, and contour chipping.

[0010] Preferably, the image acquisition module is equipped with a ring-shaped side light source and a top diffuse light source, used to acquire a multi-angle surface image sequence for the metal signboard that is the target object.

[0011] Preferably, the feature analysis module, in response to the presence of high-frequency artifact features in the current image feature set, incorporates the high-frequency artifact features into the visual feature space for feature deviation calculation, thereby increasing the calculated semantic drift entropy.

[0012] The present invention has the following beneficial effects: 1. This system calculates semantic drift entropy through the feature analysis module, and compares it with a preset danger threshold by the strategy decision module to achieve dynamic switching between the first and second detection modules. When the entropy value increases due to high-frequency artifacts caused by ambient light spot noise, the physical optical morphological filtering algorithm is called; when the entropy value is low, the generative vision model is called. This mechanism breaks the limitations of a single detection architecture and takes into account both the ability to identify subtle surface defects under stable lighting and the robustness of structural feature detection under complex optical interference. 2. This system introduces a risk prediction module and a threshold adjustment module. Based on semantic drift entropy, it predicts the probability of missed detection of structural defects and dynamically updates the preset danger threshold, solving the problem that a fixed detection architecture is difficult to adapt to changing environments. At the same time, combined with the hierarchical output mechanism of the first defect judgment result and the second defect judgment result, only the judgment status of the second type of structural defect is output when there is high semantic drift, which effectively prevents false alarms of surface defects and achieves a dynamic balance and high stability between missed detection risk and detection efficiency. Attached Figure Description

[0013] The following drawings, which illustrate embodiments of this application, are incorporated herein by reference and are used to understand this application. The drawings show embodiments of this application and their descriptions, serving to explain the principles of this application. In the drawings: Figure 1 This is a structural diagram of the system of the present invention. Detailed Implementation

[0014] In the following description, numerous specific details are set forth to provide a more thorough understanding of this application. However, it will be apparent to those skilled in the art that embodiments of this application may be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described to avoid confusion with embodiments of this application.

[0015] Example 1 Please see Figure 1 A machine vision-based metal sign surface inspection system includes: The image acquisition module is used to acquire a sequence of surface images of the target object; The feature analysis module is used to extract the current image feature set of the surface image sequence to construct a visual feature space, and calculate the semantic drift entropy based on the visual feature space; The strategy decision module is used to compare the semantic drift entropy with a preset danger threshold; if the semantic drift entropy is less than the preset danger threshold, the first detection module is triggered; if the semantic drift entropy is greater than or equal to the preset danger threshold, the second detection module is triggered. The first detection module is used to extract depth generation features from the surface image sequence using a pre-trained generative vision model, and output the first defect judgment result based on the depth generation features. The second detection module is used to select structural elements of corresponding size and preset shape according to the physical scale of the target object and the type of structural defect to be detected, and to extract basic morphological features from the surface image sequence using a morphological filtering algorithm containing structural elements, and to output the second defect judgment result based on the basic morphological features. The results output module is used to generate the final inspection report of the target object based on the first defect determination result or the second defect determination result.

[0016] This embodiment provides a machine vision-based metal nameplate surface inspection mechanism; specifically, it takes the final inspection station of metal nameplates in the new energy vehicle drive assembly production line as a unified scenario, and the object to be inspected is a metal nameplate that has completed etching, spraying and imprinting processes; The front end of the workstation is equipped with a conveyor belt, positioning fixtures, a ring-shaped side light source, a top diffused light source, and an industrial camera array to continuously acquire surface image sequences as the sign passes through the inspection area. The image sequence is not a single static photograph, but a collection of multiple frames of images formed by the same sign under different exposure levels, different incident angles, and different camera positions, in order to cover the visual representation of different types of defects such as scratches, dents, broken characters, and edge gaps. The image acquisition module can read the positioning sensor signal after the sign enters the field of view and trigger continuous image acquisition accordingly; for example, a metal sign with the number P202 can obtain three frames of images I1, I2 and I3 in 0.3 seconds. Among them, I1 focuses on surface texture, I2 focuses on the boundary of the imprinted characters, and I3 focuses on the suppression of reflective areas. The feature analysis module extracts the current image feature set from each frame of the image; the feature set may include the mean brightness, texture orientation histogram, edge density, local frequency domain energy, and residual noise distribution, etc.; among them, the local frequency domain energy can be obtained by performing a fast Fourier transform on the local window of the image and extracting the amplitude of the high-frequency components; the residual noise distribution can be obtained by filtering out the low-frequency structure of the main body of the image with a high-pass filter and then statistically analyzing the gray-level variance of the background area.

[0017] For ease of explanation, let's assume that the feature vector is extracted from image I1. The feature vector is obtained from image I2. The feature vector is obtained from image I3. Then, the three sets of vectors can be mapped to the same visual feature space, and the statistical distribution of the current sample can be formed accordingly. After comparing the statistical distribution with the preset stable sample distribution, a quantitative value reflecting whether the current visual semantics deviates from the normal state is obtained, namely semantic drift entropy. The lower the value, the more likely it is that the current image is still within the effective coverage of the training samples and the existing visual model. The higher the value, the more likely it is that there is a significant deviation, such as light pollution, abnormal texture magnification, structural boundary distortion, etc. After receiving the semantic drift entropy, the strategy decision module performs path selection; the initial value of the preset danger threshold is determined by calculating the minimum error rate critical point when the two distributions intersect based on the statistical distribution of semantic drift entropy of normal images and abnormal interference images in historical production line data; the initial danger threshold can be preset to 0.5. When the semantic drift entropy calculated in a certain detection is 0.32, it indicates that the similarity between the current image sequence and the effective visual semantics in a normal production environment is within the preset range. At this time, it is appropriate to call the generative visual model to perform more fine-grained depth analysis, that is, to trigger the first detection module.

[0018] This module can use a generative visual model trained only on defect-free samples, such as a reconstruction network, to encode, reconstruct, and compare the current image sequence and output deep generated features. If the reconstruction error is concentrated in the character strokes, surface coatings, or fine scratches, the first defect judgment result is generated accordingly.

[0019] When the semantic drift entropy calculated in a certain detection is 0.71, it indicates that the image environment has experienced a semantic shift greater than the preset offset threshold. The depth model may be affected by reflection, noise, or high-frequency artifacts, resulting in unstable output. At this time, the second detection module is triggered instead.

[0020] The second detection module uses a physical optical morphological filtering algorithm to perform opening and closing operations, background suppression, differential enhancement, and region discrimination on the image, paying more attention to structural contours and significant morphological anomalies, thereby outputting a second defect judgment result. The result output module receives the corresponding judgment result and generates a final detection report according to the different paths taken. For example, if there are fine line scratches in the character area of ​​the sign, it is grade B, or if there are embossed gaps on the edge of the sign, it is grade A, and re-inspection is recommended.

[0021] In actual operation, there are still some boundary cases; if the image acquisition module causes a frame to lose focus due to transmission vibration, a sharpness score can be added to that frame; when the score is lower than the preset lower limit, the frame will not participate in the semantic drift entropy calculation of the current round, and the remaining valid frames will continue to form the feature set. If the entire set of images is below the lower limit of clarity, the result output module will directly generate a report of acquisition abnormality and need to be reloaded, instead of entering the defect judgment branch; for example, when the semantic drift entropy is exactly equal to the danger threshold, in order to avoid repeated jitter at the boundary, it can directly enter the second detection module, because this path has stronger robustness to complex interference environment. If both the first and second detection modules fail to produce reliable results due to abnormal input, the system can mark the sign as awaiting manual review and archive the original image sequence.

[0022] On the same new energy vehicle drivetrain production line, the ambient light is stable in the morning. The semantic drift entropy of most nameplates in batches P202 to P235 is concentrated between 0.21 and 0.38. The system prioritizes calling the first detection module to identify minute surface defects with high sensitivity. When the ambient light changes abruptly, some target object images show abnormal highlight stripes. At this time, the semantic drift entropy of multiple signs rises to above 0.62. The system automatically switches to the second detection module to prioritize ensuring that structural defects such as imprint fractures and edge gaps are not missed in a morphological manner. The same workstation can continuously output usable inspection reports in both stable and disturbed environmental conditions. The purpose of this step is to enable the system to recognize subtle defects under normal operating conditions and prioritize the reliability of structural defect detection under abnormal optical conditions by first evaluating the semantic stability of the image and then selecting an appropriate detection path, thereby achieving stable detection of the surface quality of metal signs.

[0023] The process of calculating semantic drift entropy by the feature analysis module includes: comparing the current image feature set with the preset benchmark feature distribution constructed based on standard defect-free samples to generate feature distribution difference degree; based on the feature distribution difference degree, mapping the feature distribution difference degree to deviation probability through linear normalization or normalized exponential function to form feature probability distribution, and solving the semantic drift entropy by calculating the information entropy of feature probability distribution.

[0024] This embodiment provides a specific solution mechanism for semantic drift entropy; specifically, in the aforementioned production line scenario, if the state of image visual features deviating from the preset baseline distribution is converted into path switching conditions based solely on human experience, it is easily affected by inconsistent operator standards, slight fluctuations in lighting, and differences in batch textures, resulting in unstable switching criteria. Therefore, this embodiment further establishes a quantitative comparison relationship between the current image feature set and standard defect-free samples, and calculates the semantic drift entropy through the information entropy of the feature probability distribution, so that the path switching has a verifiable data basis.

[0025] During the modeling phase, the system first collects a batch of standard, defect-free sign samples. For example, it selects 300 qualified signs from a stable production line and extracts the baseline feature distribution. This distribution does not need to be publicly disclosed as a complex high-dimensional formula; in implementation, it can be understood as a set of normal intervals and corresponding probabilities. For example, for the three features of brightness uniformity, texture direction consistency, and edge continuity, the statistical center of normal samples can be approximately represented as a vector. To facilitate the explanation of the subsequent difference calculation logic, another set of example data is provided here. It is assumed that the current feature center vector is obtained by fusing the three frames of the current sign to be tested. Then we can first calculate the characteristic distribution difference, for example, obtain the following: , , ; The degree of difference can be mapped to a probability weight. The degree of difference in feature distribution is positively correlated with the deviation probability from the normal semantics. Specifically, a linear normalization function or a normalization exponential function is used to convert the degree of difference in feature distribution of each dimension into a value that sums to 1, which is then used as the deviation probability corresponding to that dimension. Furthermore, assuming the mapping yields three deviation probabilities of 0.2, 0.5, and 0.3, the semantic drift entropy can be calculated based on the information entropy of these probabilities, resulting in a comprehensive uncertainty value. In specific quantization operations, the semantic drift entropy... The calculation formula can be expressed as: ; The calculation is based on the total number of dimensions used to construct the visual feature space. and the normalized first Deviation probability corresponding to dimensional feature The results are summed and then normalized to a preset danger threshold range. ,For example Possible values This is to unify the semantic drift entropy under different feature dimensions to the value range of [0,1], thereby improving the stability and reliability of the system detection algorithm, and to prevent overflow caused by the true number being zero in logarithmic calculations by a minimal safety constant. Substituting this formula into the discrete distributions of 0.2, 0.5, and 0.3 above, the absolute value of the entropy obtained will be significantly greater than that of a concentrated distribution dominated by a single feature, such as 0.8, 0.1, and 0.1. Based on this logic, if the three probabilities are relatively evenly distributed, it means that multiple dimensions have shown moderate deviations, and the entropy value will be high; if only one dimension deviates slightly, and the other dimensions remain stable, then the deviation probability after mapping is concentrated in that single dimension, and the entropy value is relatively low.

[0026] The above calculations can be dynamically updated with each batch using a sliding window approach. For example, after the system has inspected 20 labels, it adds the 20 most recently confirmed defect-free samples to the buffer and performs a dynamic update on the baseline distribution based on a preset smoothing factor to adapt to the characteristic changes in process such as sand texture and coating thickness within the preset tolerance range. However, this correction is not unlimited; instead, an upper limit is set for the update weight to prevent the baseline distribution from being contaminated by short-term abnormal samples. When there are missing items in the image feature set, such as when edge continuity cannot be stably extracted due to reflection occlusion, a two-level processing can be adopted. First, if the number of missing dimensions does not exceed one-third of the total dimensions, the average value of the same type of features in adjacent frames is used for compensation, and a confidence label is added to the entropy value result. Second, if there are too many missing dimensions, the entropy value is not directly output, but the group of images is marked as having incomplete features, and the image is entered into a more conservative second detection module by default.

[0027] For example, when the reference variance of a certain feature in the baseline distribution is close to zero, to prevent abnormal amplification in the information entropy calculation process, a minimum fluctuation lower limit can be set for that feature to keep the calculation bounded. When the afternoon lighting fluctuates slightly, the brightness uniformity of a sign drops from 0.80 at the normal center to 0.76, the texture direction consistency increases from 0.20 to 0.22, and the edge continuity decreases from 0.60 to 0.58. The difference corresponding to this set of changes is very small, and the mapped deviation probability is concentrated in the low-value area. The final semantic drift entropy is, for example, 0.19, and the system continues to detect along the deep model path. Another sign had large areas of high-gloss stripes on its surface, causing its brightness uniformity to drop to 0.52, texture direction consistency to rise to 0.48, and edge continuity to drop to 0.36. Multiple features deviated simultaneously, resulting in a more dispersed deviation probability after mapping. The semantic drift entropy rose to 0.68, and the system immediately switched to the morphological detection path. The purpose of this step is to transform the degree of deviation of the image sequence relative to normal samples into a quantifiable and comparable entropy value index, thereby realizing a unified criterion for path selection and improving the stability and reliability of the system's detection algorithm.

[0028] The system also includes: a risk prediction module, which predicts the probability of missing structural defects under the current detection condition based on semantic drift entropy; and a threshold adjustment module, which lowers the preset danger threshold when the probability of missing structural defects increases compared to the statistical result of the previous detection time window, and raises the preset danger threshold when the probability of missing structural defects decreases or becomes zero compared to the statistical result of the previous detection time window, thereby dynamically updating the preset danger threshold. This embodiment provides a dynamic update mechanism for hazard thresholds. Specifically, after the aforementioned production line has been running continuously for a period of time, if the hazard threshold remains fixed, new problems will emerge when the operating conditions drift over a long period. For example, the environmental lighting, lens temperature rise, and conveyor vibration conditions are different between winter night shifts and summer day shifts. The same fixed threshold may be too conservative in a certain shift, causing a large number of samples to enter the morphological pathway, reducing the identification efficiency of subtle surface defects. In actual engineering, it may also be too aggressive in another shift, causing some structural defects to still be sent to the depth model pathway, resulting in the risk of missed detection. Therefore, this embodiment further introduces a risk prediction module and a threshold adjustment module to dynamically correct the hazard threshold.

[0029] The risk prediction module receives the semantic drift entropy sequence within a recent time window and, in conjunction with the posterior verification results, establishes a correspondence between the entropy level and the probability of missing structural defects. Specifically, the process of establishing the correspondence includes dividing the preset semantic drift entropy range into several continuous discrete intervals. The total number of samples in each discrete interval within the statistical time window, and the number of missed samples that were confirmed to have structural defects after posterior verification but were not detected by the first detection module; the ratio of the number of missed samples to the total number of samples in the corresponding interval is calculated as the probability of missed structural defects in that interval; for ease of explanation, it is assumed that among the most recent 100 signs, samples with semantic drift entropy below 0.40 have a statistically significant probability of missed structural defects of 1%; samples with a semantic drift entropy between 0.40 and 0.55 have a statistically significant probability of missed structural defects of 4%; and samples with a semantic drift entropy above 0.55 have a statistically significant probability of missed structural defects of 11%. When the system detects a significant increase in the proportion of samples with entropy values ​​greater than 0.50 in the current shift, the risk prediction module can determine that if the original threshold of 0.55 is maintained, a higher proportion of samples will remain in the deep model pathway, increasing the risk of missing structural defects. Therefore, the threshold adjustment module lowers the danger threshold from 0.55 to 0.48, allowing more high-risk samples to enter the second detection module earlier. Conversely, when the system runs smoothly for multiple consecutive batches, and the verification results show that no structural defects are missed in samples within the 0.45 to 0.50 range, the threshold adjustment module can lower the threshold back from 0.48 to 0.52 to reduce unnecessary path switching. To demonstrate the micro-process of dynamic adjustment, we can assume that five representative samples appear within a certain time window, with semantic drift entropies of 0.43, 0.47, 0.49, 0.58, and 0.61, respectively. Subsequent manual sampling reveals that samples with entropies of 0.49 and 0.58 exhibit edge chipping, and the depth model has not yet reached a stable conclusion. At this point, the risk prediction module recalculates and increases the probability of missing structural defects in the 0.47 to 0.50 range based on the newly observed missed detection events. In the next round of threshold adjustment, the threshold adjustment module adjusts the preset danger threshold from 0.50 to 0.47 or 0.48 based on the updated probability of missing structural defects, thus enabling samples within this entropy range to trigger the second detection module earlier.

[0030] In case of boundary anomalies, the trigger frequency of dynamic adjustments is controlled to avoid frequent oscillations in the detection strategy within the same time period on the production line. To this end, the threshold adjustment module can be set with a minimum adjustment cycle, such as allowing updates only once every 50 labels or every 10 minutes. At the same time, a maximum adjustment step size can be set, such as not exceeding 0.05. If the risk prediction module finds that the current sample quantity is insufficient or the post-verification results are not yet sufficient, the threshold of the previous cycle will remain unchanged. For example, when a sudden environmental change causes the increase rate of the entropy value of all samples to exceed the preset jump threshold, and the manual verification data has not yet been updated synchronously, the system can temporarily implement a conservative strategy, that is, first lower the threshold to the preset safety lower limit with a first preset step size, and then, after the environment recovers, pull back to the initial threshold with a second preset step size smaller than the first preset step size. During the night shift on the same drive assembly production line, due to a slight change in lens focal length caused by equipment heat dissipation, coupled with enhanced local lighting reflection, the system statistics showed that the proportion of signs in the 0.48 to 0.55 range increased from 8% during the day shift to 23%. The risk prediction module, combined with the sampling results, determined that the probability of missing edge defects in samples within this range was significantly increased. Therefore, the threshold adjustment module lowered the danger threshold from 0.52 to 0.47. In the following two hours, more suspected high-risk samples were sent to the second detection module, and the structural defect miss rate decreased. Once the lighting stabilized in the latter half of the night shift, the miss rate dropped, and the threshold was gradually adjusted back up. The purpose of this mechanism is to adaptively modify the diversion conditions based on the actual production line status, thereby achieving a dynamic balance between the risk of missed detection of structural defects and detection efficiency.

[0031] The first detection module includes: a generation and reconstruction unit, used to perform feature reconstruction on the surface image sequence using a pre-trained generative vision model to generate a residual feature map as a depth generation feature; and a first determination unit, used to determine a first defect determination result based on the residual feature map. This embodiment provides a first detection mechanism based on generative reconstruction. Specifically, after the aforementioned path selection is completed, when the semantic drift entropy is within a safe range, it indicates that the current image sequence is close to normal working conditions. At this time, if only simple thresholds or edge differences are relied upon, it is impossible to effectively extract weak defects such as minor scratches, uneven spraying, and shallow indentations, because the change value of these defects in the overall grayscale is lower than the preset grayscale difference threshold, causing their features to be lost in the background texture. Therefore, this embodiment introduces a generation and reconstruction unit and a first determination unit in the first detection module, and outputs fine-grained determination results by first reconstructing the normal appearance and then observing the residual regions that cannot be explained by the normal model.

[0032] The generative reconstruction unit can employ a generative visual model trained using samples of defect-free metal signs. Specifically, the generative visual model can use an autoencoder network architecture, including an encoder and a decoder. When the current image sequence is input, the encoder maps the surface image sequence to a low-dimensional latent space to filter out local high-frequency anomalous details that deviate from the normal distribution, thereby extracting its deep representation. The decoder, based on the deep representation, performs inverse mapping according to the feature manifold of normal samples to reconstruct a set of approximate images that should appear if the sign is defect-free. Subtracting the original image from the reconstructed image yields a residual feature map; regions with larger residuals are more likely to be anomalous regions. The residual feature map and the residual amplitude, distribution location, and other information it contains constitute the specific representation of the aforementioned deep generative features. For ease of explanation, let the grayscale value of a certain 3×3 local area in the original image be: ; The grayscale values ​​of the corresponding defect-free regions reconstructed by the model are: ; The residual plot can then be represented as: ; The presence of significant residuals in the central pixel and its vicinity indicates the presence of anomalies in that area that are difficult for the model to reconstruct, potentially corresponding to shallow scratches, coating bubbles, or embossing noise. The first judgment unit then classifies the residuals based on their magnitude, area, connected component shape, and location within the functional area of ​​the residual feature map. For example, if high residual points are distributed within character strokes and extend linearly, they can be identified as a risk of broken strokes; if they are located in the blank base area and are elongated, they can be identified as surface scratches; and if they are located at the edge of the fixing hole and are clustered, they can be identified as embossing deformation.

[0033] The first judgment unit can set different thresholds for different regions; for example, the character area allows a lower residual trigger threshold to improve the sensitivity to missing strokes; the background area uses a slightly higher threshold to avoid false alarms caused by natural fluctuations in the texture; for multi-frame image sequences, residual consistency comparison can also be performed: if a certain residual region is located in a similar spatial position in I1, I2, and I3, its confidence is improved; if it only appears in a single frame and is highly consistent with the reflection direction, it is preferentially marked as an anomaly to be confirmed rather than directly judged as missing. As a boundary case handling mechanism, generative vision models cannot maintain stable output under all extreme conditions. When the input image is oversaturated, the reconstructed image may be distorted as a whole, resulting in a large increase in residuals. In this case, pre-screening can be performed based on the saturated pixel ratio. If the saturated area exceeds the preset ratio, such as exceeding 20% ​​of the entire image, the first judgment result will not be output, and the process will fall back to the second detection module or manual review. When multiple discrete small spots appear in the residual map, but these spots only exist in a single frame and are randomly distributed, they can be determined as sensor noise rather than real defects. If the first determination unit has insufficient confidence in the defect category, such as the difference between scratches and brushed texture deviation is not obvious, it can output suspected surface anomalies and record the coordinates of the corresponding area for verification.

[0034] Under the aforementioned stable day shift conditions, the semantic drift entropy of a nameplate numbered P218 is 0.27, and it enters the first detection module. After the generation and reconstruction unit reconstructs the character area, it is found that the middle horizontal stroke area of ​​the letter E has a thin line gap in the original image, while the reconstructed image remains intact. After subtracting the two, a continuous high residual strip is formed at this position. The first judgment unit, taking into account the conditions that the residual strip is located in the character area and its length exceeds the preset stroke missing threshold, outputs the first type of surface defect: character broken stroke, medium severity; another sign in the same batch produces an isolated single-point residual in the background area, but it only appears in one frame of the image, so the system identifies it as noise and does not judge it as a defect. The purpose of this step is to amplify small-scale anomalies by utilizing the generation and reconstruction capabilities of normal samples, thereby improving the sensitivity of identifying subtle surface defects and local structural anomalies.

[0035] The second detection module includes: a morphological processing unit, used to perform opening and closing operations on the surface image sequence based on the structuring elements of a preset shape to generate a morphological background image; a difference extraction unit, used to calculate the difference image between the surface image sequence and the morphological background image; and a second determination unit, used to determine the second defect determination result based on the difference image. This embodiment provides a second detection mechanism based on physical optical morphological filtering. Specifically, in the aforementioned main scenario, when the semantic drift entropy increases, it indicates that the image semantics have significantly deviated from the stable sample distribution. If the generative reconstruction method is still relied upon at this time, the generative visual model is prone to misjudging features such as strong reflection, high-frequency interference, or abnormal lighting as detail defects, thereby causing unstable output. Therefore, this embodiment adopts a processing path that emphasizes the stability of structural morphology. It constructs a background through opening and closing operations and then uses difference to highlight structurally abnormal areas to ensure the reliability of identifying defects such as edge gaps, imprint fractures, and hole deformation.

[0036] Physical optical morphological filtering algorithm refers to a filtering algorithm that dynamically determines the size, shape and orientation of morphological structural elements by combining the physical optical reflection characteristics of metal signs; the morphological processing unit can first perform grayscale normalization and noise smoothing on the input image, and then select structural elements with preset shapes according to the physical scale of the sign; The specific logic of opening and closing operations is as follows: morphological opening operations eliminate high-frequency noise points in bright areas of the image that are smaller than the corresponding structuring element; morphological closing operations fill in tiny holes and breaks in dark areas of the image that are smaller than the corresponding structuring element, thereby generating a morphological background image. For example, for edge gap detection, a circular structuring element with a radius of 3 pixels can be selected; for character break detection, a linear structuring element with a length of 5 pixels and an orientation that adapts to the main texture can be selected. For ease of explanation, let a certain local original image be: ; Here, the center value 1 can be understood as a local depression or hole; after the closing operation, the background image may be restored as: ; After the difference extraction unit subtracts the two values, a significant difference of 8 is obtained at the center position, which can be used as a candidate anomaly area. The structural information of the candidate anomaly area, such as its area, aspect ratio, and relative position to the edge, is extracted, forming the aforementioned basic morphological features. The second judgment unit further judges based on the area, aspect ratio, and relative position to the edge or character template of the candidate area. For example, when the difference region is close to the outer boundary of the sign and forms a concave contour, it can be judged as an edge gap; when the difference region penetrates the main stroke of the embossed character, it can be judged as a structural break; when the difference region falls on a blank background and has no structural relationship, it can be downweighted. This pathway can cross-validate multiple frames of images; if a candidate defect area maintains a stable position and approximate outline under different lighting angles, it is more likely to be a physical structural anomaly; if the candidate area moves with the direction of illumination, it is more likely to be a reflection artifact, which can be eliminated by the second judgment unit; in this way, even in a high-light interference environment, the system can still prioritize capturing real structural defects with fixed spatial positions.

[0037] As a boundary case handling mechanism, when the global grayscale difference between the background image after opening and closing operations and the original image exceeds the preset tolerance threshold, it indicates that the current preset structuring element size does not match the image feature scale. For example, if the structuring element is too large, it will smooth out the real small holes as well; if the structuring element is too small, it will not be able to sufficiently suppress high-frequency noise. At this time, the system can use two or three structuring elements to perform parallel calculations and then select the one with a higher matching degree with the template to output the result. If there are too many candidate regions in the difference image, such as exceeding the preset upper limit, it indicates that the current frame may be subject to large-area optical pollution. The system can reduce the weight of this frame and let the other frames complete the main judgment. If all frames are polluted, the output image environment is abnormal, and it is recommended to retake the image or manually review it. Under the detection conditions with light spot interference, the semantic drift entropy of a certain target object reaches 0.73, and the system sends it to the second detection module. After the morphological processing unit constructs the background in the edge region, it finds a stable concave difference region between the outer contour of the lower left corner and the background. This region is in the same position in the three frames of images, and the area is higher than the lower limit threshold of the gap. The second determination unit outputs the second type of structural defect: edge chipping; a bright stripe appears above the characters on another sign in the same batch, but the position of the stripe drifts significantly in different lighting frames, and the system identifies it as a reflective artifact rather than a structural defect. The purpose of this step is to highlight real structural anomalies under complex optical interference by using morphological background modeling and differential enhancement, thereby achieving robust detection under conditions of high semantic drift.

[0038] The first defect determination result includes the determination status of the first type of surface defects and the second type of structural defects; the second defect determination result only includes the determination status of the second type of structural defects. The first category of surface defects includes scratches, uneven coating, localized stains, burrs on characters, and shallow indentations; the second category of structural defects includes broken characters, edge gaps, deformed holes, embossed collapse, and chipped corners. This embodiment provides a dual-path output aperture separation mechanism. Specifically, during the operation of the aforementioned system, if the two detection modules output the same set of defect categories, two problems are likely to occur: First, under high semantic drift conditions, the detection confidence of the morphological path for subtle surface defects is lower than the preset requirement. If the surface defect conclusion is still forced to be output, the false alarm probability will increase. Second, although the depth generation path can cover subtle surface anomalies, it also has a certain ability to identify structural anomalies when the operating conditions are stable. To address the aforementioned issues, this embodiment defines the detection results in layers based on defect attributes, making the output range of the first detection module more complete, while the second detection module retains only its most reliable structural judgments.

[0039] The system predefines two types of defects: the first type of surface defects mainly corresponds to surface quality problems, such as minor scratches, uneven spraying, local stains, character burrs, and shallow indentations; the second type of structural defects mainly corresponds to geometric or contour anomalies, such as character breakage, edge gaps, hole deformation, embossing collapse, and contour chipping. In the first detection module, since the generation and reconstruction mechanism can simultaneously generate residual responses for detailed textures and local structures, the first judgment result can simultaneously include the judgment status of the above two types of defects; for example, a sign can be judged as having surface scratches: yes; edge gaps: no; broken characters: yes. In the second detection module, since the focus is on the difference in structural contour after the opening and closing operation, the result only outputs the judgment status of the second type of structural defect; for example, the result can be expressed as edge notch: yes; hole deformation: no; character breakage: no, while no direct conclusion is made for surface stains, shallow scratches, etc. For ease of understanding, we can assume that a sign, under stable operating conditions, receives two candidate images from the first detection module: one residual image shows a thin, elongated high residual line segment in the background area, corresponding to surface scratches; the other residual image shows a residual breakpoint in the upper half of the character 8, corresponding to structural fracture; at this time, the output simultaneously covers both types of defects. Another sign, after being processed by the second detection module under strong reflective conditions, only obtained a stable notch difference on the left side of the outer contour, while a large number of unstable small spots existed in the background area; the system only output that the edge notch was true, without judging whether those small spots constituted scratches; When the first detection module finds both surface defects and structural defects, it can sort them by severity level in the report and give priority to structural anomalies that affect assembly and traceability. If the second detection module does not directly determine the surface defects, but finds that the differential region is close to the sensitive surface region and is different from the historical false detection pattern, it can output a prompt that the surface condition is abnormal and needs to be reviewed, instead of directly classifying it as a first-class defect, so as to avoid exceeding the stability capability range of this path. If two different paths lead to different conclusions after re-photographing the same sign, the output module can adopt a merging principle that prioritizes structural types and confirms surface types from the latest stable frame.

[0040] In the final inspection of the same batch of signs, sign number P218 went through the first inspection module due to stable operating conditions, and the system simultaneously identified two anomalies: broken characters and scratches on the background. Sign number P276 went through the second inspection module due to light pollution, and the system only confirmed the chipped corner of the lower left edge, noting in the report that surface-related conclusions were not enabled and suggesting that a detailed inspection be carried out after re-shooting. In this way, the different output conclusions of the same workstation still maintained a consistent interpretation. The purpose of this mechanism is to limit the output range according to the capability boundaries of the two detection modules, thereby achieving interpretability and stability of the detection results.

[0041] Example 2 The image acquisition module is equipped with a ring-shaped side light source and a top diffused light source to acquire multi-angle surface image sequences of metal signs as the target object; The feature analysis module, in response to the presence of high-frequency artifact features in the current image feature set, incorporates the high-frequency artifact features into the visual feature space for feature deviation calculation, thereby increasing the calculated semantic drift entropy.

[0042] This embodiment provides an adaptation mechanism for anisotropic reflection and ambient light spot interference in metal nameplates. Specifically, in the aforementioned new energy vehicle drive assembly production line, the nameplates to be inspected are usually made of brushed aluminum plates, stainless steel etched plates, or materials that have been sandblasted and then coated with a protective layer. These materials have obvious anisotropic reflection characteristics, that is, when the incident light direction, viewing angle, or surface texture direction changes, the distribution of bright and dark stripes in the image will change accordingly. If they are simply regarded as ordinary diffuse reflection objects, it is easy to mistake the reflective stripes for scratches, broken pen, or edge abnormalities in certain work environments. Therefore, this embodiment explicitly incorporates anisotropic reflection and high-frequency artifacts introduced by ambient light spots into the image semantic analysis process.

[0043] During the image acquisition phase, industrial cameras can be used with ring side lighting and top diffused lighting to form multi-angle imaging. For metal signs with the same wire drawing direction as the conveying direction, bright stripes extending along the wire drawing direction will appear on the surface under the incident light source. When the sign is slightly rotated, these bright stripes may shift in position. The system does not immediately treat such changes as real defects, but encodes them into the visual feature space through the feature analysis module. For example, texture direction consistency, local frequency domain high-frequency energy, and bright spot density will jointly reflect whether the current frame is affected by anisotropic reflection. If it is just the regular reflection of the material itself, these features usually still fluctuate around the baseline distribution; if ambient light spot noise is superimposed, such as moving light spots reflected from the metal casing of the side equipment, irregular high-frequency artifacts will be added to the image, which are manifested as a sudden increase in local frequency domain energy, discrete bright spot size, and obvious positional drift across frames. At this time, the semantic drift entropy increases accordingly. For ease of explanation, we can assume that the three features of a standard sign under stable operating conditions are texture direction consistency 0.78, high frequency energy 0.24, and bright spot density 0.12; when it is just normal anisotropic reflection, these three features may fluctuate to 0.75, 0.28, and 0.15, but the overall value is still close to the baseline distribution, and the semantic drift entropy remains at around 0.30. When a moving light spot appears in the environment, the same sign may change to 0.61, 0.49, and 0.33, with a significant increase in high-frequency energy and bright spot density, indicating the introduction of additional artifacts into the image. At this point, the semantic drift entropy can rise above 0.65, triggering the system to switch to the second detection module. Thus, anisotropic reflection is not considered abnormal in all cases, but is incorporated into the modeling as part of normal material properties; what truly causes the current switching is the increase in high-frequency artifacts caused by the added environmental noise.

[0044] To distinguish between inherent material reflections and environmental noise artifacts, the system can use a multi-frame position consistency criterion; if the bright fringes are always distributed along the main direction of the wire drawing and only undergo a small overall translation between different frames, it tends to be identified as a normal reflection mode. If the bright spots exhibit discrete shapes, abrupt position changes, and a sharp increase in edge frequency in adjacent frames, they are more likely to be injected with external spot noise. This difference will be directly reflected in the increase of semantic drift entropy and drive the switching of detection strategies.

[0045] When the reflection pattern of a metal sign is abnormal due to oil film, water stains, or residual protective film, its behavior may be similar to that of ambient light spot interference. In this case, the system can further judge by combining the continuity of time: if the abnormal bright spot is fixed in the same physical position as the sign moves, it is likely to be surface attachments; if the abnormal bright spot is synchronous with the change of external lighting and appears repeatedly between different signs, it is likely to be ambient noise. If the two still cannot be reliably distinguished, the system can directly proceed to the second detection module and add a prompt for optical environment abnormalities or surface attachment interference in the report to avoid erroneously relying on the depth model to give conclusions about fine-grained surfaces. When the inspection station is subjected to external optical interference such as intermittent specular reflection from surrounding equipment, random bright spots appear in some signs with brushed texture. For unaffected signs, although the image has bright lines that change with angle, its texture direction is consistent with the brushed direction, and the semantic drift entropy remains in the low value range. For signs affected by light spots, the system detects local high-frequency artifacts jumping across frames, and the semantic drift entropy rises rapidly. These signs are then imported into the second inspection module to prioritize the structural inspection of edges, holes, and imprint contours. The purpose of this mechanism is to establish an image interpretation framework that is more in line with real working conditions based on the inherent anisotropic reflective properties of metal signs, and to take the high-frequency artifacts caused by ambient light spot noise as an important triggering factor for the increase of entropy value, so as to achieve more accurate path switching and defect detection.

[0046] This application has been described through the above embodiments; however, it should be understood that the above embodiments are for illustrative purposes only and are not intended to limit this application to the described embodiments. Those skilled in the art will understand that many more variations and modifications can be made based on the teachings of this application, and all such variations and modifications fall within the scope of protection claimed in this application.

Claims

1. A machine vision-based metal sign surface inspection system, characterized in that, include: The image acquisition module is used to acquire a sequence of surface images of the target object; The feature analysis module is used to extract the current image feature set of the surface image sequence to construct a visual feature space, and to calculate the semantic drift entropy based on the visual feature space; The strategy decision module is used to compare the semantic drift entropy with a preset danger threshold; if the semantic drift entropy is less than the preset danger threshold, the first detection module is triggered. If the semantic drift entropy is greater than or equal to the preset danger threshold, the second detection module is triggered; The first detection module is used to extract depth generation features from the surface image sequence using a pre-trained generative vision model, and output a first defect determination result based on the depth generation features; The second detection module is used to select structural elements of corresponding size and preset shape according to the physical scale of the target object and the type of structural defect to be detected, and to extract basic morphological features from the surface image sequence using a morphological filtering algorithm containing the structural elements, and to output a second defect judgment result based on the basic morphological features. The result output module is used to generate a final inspection report for the target object based on the first defect determination result or the second defect determination result.

2. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The process by which the feature analysis module calculates the semantic drift entropy includes: The current image feature set is compared with a preset benchmark feature distribution constructed based on standard defect-free samples to generate a feature distribution difference degree; Based on the feature distribution difference degree, the feature distribution difference degree is mapped to deviation probability through linear normalization or normalized exponential function to form a feature probability distribution, and the semantic drift entropy is calculated by calculating the information entropy of the feature probability distribution.

3. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The system also includes: The risk prediction module is used to predict the probability of missing structural defects in the current detection condition based on the semantic drift entropy. The threshold adjustment module is used to lower the preset danger threshold when the probability of missing a structural defect increases compared to the statistical result of the previous detection time window; and to raise the preset danger threshold when the probability of missing a structural defect decreases or becomes zero compared to the statistical result of the previous detection time window, thereby dynamically updating the preset danger threshold.

4. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The first detection module includes: A reconstruction unit is used to perform feature reconstruction on the surface image sequence using the pre-trained generative vision model, and generate a residual feature map as the depth generation feature. The first determination unit is used to determine the first defect determination result based on the residual feature map.

5. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The second detection module includes: A morphological processing unit is used to perform opening and closing operations on the surface image sequence based on the structural elements of a preset morphology to generate a morphological background image. A difference extraction unit is used to calculate the difference image between the surface image sequence and the morphological background image; The second determination unit is used to determine the second defect determination result based on the differential image.

6. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The first defect determination result includes the determination status of the first type of surface defects and the second type of structural defects; the second defect determination result only includes the determination status of the second type of structural defects. The first type of surface defects includes scratches, uneven coating, local stains, character burrs, and shallow indentations; the second type of structural defects includes character breakage, edge gaps, hole deformation, embossing collapse, and contour chipping.

7. The machine vision-based metal sign surface inspection system according to claim 1, characterized in that, The image acquisition module is equipped with a ring-shaped side light source and a top diffused light source, which are used to acquire multi-angle surface image sequences of the metal signboard, which is the target object.

8. The machine vision-based metal sign surface inspection system according to claim 7, characterized in that, The feature analysis module, in response to the presence of high-frequency artifact features in the current image feature set, incorporates the high-frequency artifact features into the visual feature space for feature deviation calculation, thereby increasing the calculated semantic drift entropy.

Images