A visual inspection method and system compatible with multiple types of workpieces to be inspected

Abstract

This invention discloses a visual inspection method and system compatible with multiple workpiece models, relating to the field of machine vision inspection technology. The method includes acquiring the original image of the workpiece at the inspection station; preprocessing to extract its shape contour information; matching the contour with a feature template library to identify the workpiece model; loading preset standard features and detection area definitions associated with that model; locating the corresponding image sub-region in the original image; extracting multi-scale features to obtain actual feature data; comparing each feature with preset standard features to generate a feature difference map; analyzing the feature difference map to determine if the workpiece has defects; and generating a visual inspection result containing the workpiece model, defect location, and defect type. This method can automatically adapt to the inspection of multiple workpiece models, accurately locate the detection area and extract multi-scale features, complete defect determination through the feature difference map, and output complete inspection result information.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection technology, specifically a visual inspection method and system compatible with multiple types of workpieces to be inspected. Background Technology

[0002] Existing workpiece visual inspection technologies mostly adopt inspection modes adapted to a single workpiece model. The inspection process, feature extraction method, and defect judgment criteria are all set around a fixed workpiece model. During the inspection process, it is necessary to manually confirm the workpiece model to be inspected, then manually switch to the matching inspection program, and enter the corresponding standard features and inspection area parameters. Feature extraction adopts a fixed-scale processing method, and defect judgment is achieved through routine comparison of the overall image.

[0003] Existing inspection technologies cannot automatically identify the model of multiple workpieces under test, nor can they adaptively load corresponding inspection parameters based on the identification results. When switching workpiece models, the inspection program and configuration parameters need to be re-adjusted. When multiple workpiece models are inspected together, the process continuity is poor, and the automation adaptability is insufficient. Fixed-scale feature extraction cannot cover feature details in different dimensions, global inspection cannot accurately focus on the effective inspection area, feature comparison is limited in form, and cannot form an intuitive difference representation. Defect judgment results are limited in information and cannot simultaneously integrate relevant content such as workpiece model, defect location, and defect type.

[0004] This invention achieves automatic model recognition by analyzing the outline of the workpiece to be tested. Based on the recognition results, it loads corresponding preset standard features and detection area definitions, accurately locates the detection sub-region, and performs multi-scale feature extraction. By comparing features item by item, it generates a feature difference map to complete defect analysis. Finally, it outputs detection results that include workpiece model, defect location, and defect type, realizing adaptive processing and accurate defect judgment for visual inspection of multiple workpiece models. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art;

[0006] Therefore, this invention proposes a visual inspection method compatible with multiple types of workpieces to be tested, comprising:

[0007] Acquire the original image of the workpiece to be tested placed on the inspection station;

[0008] The original image is preprocessed to extract the shape contour information of the workpiece to be tested;

[0009] Based on the extracted shape contour information, contour matching is performed with a pre-generated feature template library to identify the workpiece model corresponding to the workpiece to be tested from the feature template library;

[0010] Based on the identified workpiece model, load the preset standard features and detection area definitions associated with the workpiece model from the feature template library;

[0011] Locate the image sub-region in the original image that corresponds to the defined detection region;

[0012] Multi-scale feature extraction is performed on the image sub-region to obtain actual feature data;

[0013] The extracted actual feature data is compared item by item with the preset standard features to generate a feature difference map;

[0014] The feature difference map is analyzed to determine whether the workpiece under test has defects;

[0015] Generate visual inspection results that include the workpiece model, defect location, and defect type.

[0016] Further, the original image is preprocessed to extract the external contour information of the workpiece to be tested, including:

[0017] Perform grayscale conversion on the original image to obtain a grayscale image;

[0018] An adaptive threshold segmentation algorithm is applied to the grayscale image to separate the workpiece area to be tested from the background area, thereby obtaining a binarized image;

[0019] Perform morphological closing operation on the binarized image to fill the small holes in the area of ​​the workpiece to be tested caused by surface texture or reflection;

[0020] Detect the boundaries of all connected components in the binarized image, and select the connected component with the largest area as the target region of the workpiece to be tested;

[0021] Extract the outermost pixel sequence of the target region, which constitutes the outline information of the workpiece to be tested.

[0022] Further, based on the extracted shape contour information, contour matching is performed with a pre-generated feature template library to identify the workpiece model corresponding to the workpiece under test from the feature template library, including:

[0023] Calculate the shape context descriptor of the extracted outline information, the shape context descriptor being used to characterize the distance and angle distribution of each point on the outline relative to the centroid of the outline;

[0024] Traverse the reference contour information of each standard workpiece template stored in the feature template library, and calculate the reference shape context descriptor corresponding to the reference contour information;

[0025] The similarity distance between the shape context descriptor of the current workpiece under test and each of the reference shape context descriptors is calculated sequentially.

[0026] Select the reference shape context descriptor that has the minimum similarity distance to the shape context descriptor of the current workpiece to be tested;

[0027] The model number marked on the standard workpiece template to which the reference shape context descriptor belongs is identified as the workpiece model corresponding to the workpiece to be tested.

[0028] Further, locating the image sub-region corresponding to the detection region definition in the original image includes:

[0029] Read the detection area definition, which includes the relative coordinate range and area shape description with the centroid of the contour of the workpiece to be tested as the origin;

[0030] Based on the actual position of the workpiece to be tested in the original image, calculate the actual coordinates of the centroid of the contour in the image coordinate system;

[0031] Based on the actual coordinates and the relative coordinate range defined in the detection area, the pixel coordinate boundaries of the image sub-region in the original image are calculated through coordinate transformation;

[0032] Based on the calculated pixel coordinate boundaries, corresponding image blocks are cropped from the original image, and these image blocks are the image sub-regions.

[0033] Furthermore, multi-scale feature extraction is performed on the image sub-region to obtain actual feature data, including:

[0034] A Gaussian pyramid containing multiple Gaussian kernels of different sizes is constructed, and the Gaussian pyramid is used to perform multi-scale decomposition on the image sub-region to obtain multiple copies of the image sub-region at different scales.

[0035] For each scale image sub-region copy, its local binary pattern features are calculated to obtain a texture feature vector reflecting the micro-texture;

[0036] On each scale of the image sub-region copy, its directional gradient histogram is calculated to obtain the gradient feature vector that reflects edge and shape information;

[0037] The texture feature vectors and gradient feature vectors obtained at each scale are concatenated and fused in scale order to form the actual feature data.

[0038] Further, the extracted actual feature data is compared item by item with the preset standard features to generate a feature difference map, including:

[0039] Read the preset standard features corresponding to the current workpiece model and detection area from the feature template library. The preset standard features are standard feature vectors with the same structure as the actual feature data.

[0040] Calculate the absolute difference between the feature vector in the actual feature data and the standard feature vector in the preset standard features in each feature dimension;

[0041] The calculated absolute difference of each feature dimension is mapped onto a two-dimensional grid corresponding to the spatial resolution of the image sub-region, where each pixel position of the grid represents a feature dimension, and the gray value of the pixel represents the magnitude of the absolute difference of the feature dimension.

[0042] The two-dimensional grid diagram is the feature difference diagram.

[0043] Further, the feature difference map is analyzed to determine whether the workpiece under test has defects, including:

[0044] Threshold segmentation is applied to the feature difference map to mark pixel regions with gray values ​​exceeding a preset difference threshold as difference regions;

[0045] Calculate the pixel area, region compactness, and average difference intensity of each of the said difference regions;

[0046] If any difference region exists whose pixel area is greater than a preset area threshold and whose average difference intensity is greater than a preset intensity threshold, then the workpiece under test is determined to have a defect in the current detection area.

[0047] The location of the difference region that meets the judgment condition in the original image is recorded as the defect location, and the defect type is determined by matching the texture feature pattern of the difference region with a predefined defect type library.

[0048] Furthermore, before performing contour matching with the pre-generated feature template library based on the extracted shape contour information, the method further includes a step of constructing the feature template library, which includes:

[0049] Collect sample images of standard workpieces of various models to ensure that there are multiple sample images of each model from different angles;

[0050] Perform contour extraction and alignment operations on the sample image set for each model to generate the standard shape contour of the corresponding model;

[0051] For each model, the key areas to be detected are manually marked on its sample image, and the coordinates of the key areas relative to the centroid of its standard shape contour are recorded to form the detection area definition for the corresponding model.

[0052] For each key region image of each model, perform multi-scale feature extraction and calculate the standard feature vector of the corresponding key region as the preset standard feature;

[0053] The model identifier, standard shape outline, corresponding detection area definition, and preset standard features of each area are associated and stored to form the feature template library.

[0054] Furthermore, after loading preset standard features and detection area definitions associated with the identified workpiece model from the feature template library, the method further includes an adaptive template fine-tuning step, which includes:

[0055] Based on the actual feature data of multiple workpieces to be tested that have been inspected in the current batch, calculate the average vector of the features of each dimension of the current batch within the defined detection area;

[0056] The calculated average vector is weighted and fused with the preset standard features loaded from the feature template library to generate dynamic standard features suitable for the current batch.

[0057] In subsequent inspections of the same batch of workpieces, the dynamic standard features are used to replace the original preset standard features for comparison.

[0058] Furthermore, the present invention also includes a visual inspection system compatible with multiple models of workpieces to be tested. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the visual inspection method compatible with multiple models of workpieces to be tested described above.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] By extracting the outline information of the workpiece to be tested and matching it with a pre-generated feature template library, the corresponding workpiece model can be directly determined. Based on the identified workpiece model, associated preset standard features and detection area definitions are loaded from the feature template library, eliminating the need for manual confirmation of workpiece model, manual switching of detection programs, and configuration of detection parameters. The detection parameters and workpiece model form an automatic correspondence, adapting to the detection needs of different workpiece models without manual adjustment. When switching between multiple workpiece models, the process does not need to be interrupted to reset parameters, maintaining a stable connection between the detection process and applications requiring simultaneous detection of multiple workpiece models. The matching process of detection parameters is unaffected by manual operation, and the accuracy of parameter loading and workpiece model correspondence remains consistent, resulting in a higher degree of autonomy in model recognition and parameter configuration.

[0061] The process involves locating and defining corresponding image sub-regions within the original image to detect the defect. Multi-scale feature extraction is then performed on these sub-regions to obtain actual feature data. This data is then compared item by item with preset standard features to generate a feature difference map. Analysis of this map directly determines whether the workpiece under test has defects. The detection processing range is limited to the specified image sub-regions, reducing interference from invalid image information. Multi-scale feature extraction captures feature details across different dimensions, and item-by-item feature comparison fully reflects the deviation between actual and standard features. The feature difference map visually illustrates the distribution of feature anomalies, making defect detection more targeted. The final detection result integrates workpiece model, defect location, and defect type information, providing more comprehensive information dimensions. The accuracy and completeness of defect detection and result output are guaranteed, and the detailed processing and result representation of the detection process better meet actual detection needs. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating the steps of a visual inspection method compatible with multiple types of workpieces described in this invention.

[0063] Figure 2 The flowchart shows the preprocessing and extraction of shape contour information;

[0064] Figure 3 A flowchart for contour matching and workpiece model recognition;

[0065] Figure 4 For the time consumption analysis of the entire visual inspection process;

[0066] Figure 5 A comparison chart of the feature distribution of the weld end area of ​​each batch of workpieces with the adaptive benchmark. Detailed Implementation

[0067] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] See Figure 1 This invention provides a visual inspection method compatible with multiple types of workpieces to be tested, and its overall implementation scheme is as follows:

[0069] Acquire the original image of the workpiece to be tested placed at the inspection station. Preprocess the original image to extract the outline information of the workpiece. Based on the extracted outline information, perform contour matching with a pre-generated feature template library to identify the workpiece model corresponding to the workpiece. According to the identified workpiece model, load the preset standard features and detection area definitions associated with the workpiece model from the feature template library. Locate the image sub-region corresponding to the detection area definition in the original image. Perform multi-scale feature extraction on the image sub-region to obtain actual feature data. Compare the extracted actual feature data with the preset standard features item by item to generate a feature difference map. Analyze the feature difference map to determine whether the workpiece has defects. Generate a visual inspection result containing the workpiece model, defect location, and defect type.

[0070] In one embodiment of the present invention, see [reference] Figure 2 The original image is converted to grayscale to obtain a grayscale image. An adaptive threshold segmentation algorithm is applied to the grayscale image to separate the workpiece region from the background region to obtain a binarized image. Morphological closing operations are performed on the binarized image to fill small holes in the workpiece region caused by surface texture or reflection. The boundaries of all connected components in the binarized image are detected, and the connected component with the largest area is selected as the target region of the workpiece. The outermost pixel sequence of the target region is extracted, and the outermost pixel sequence constitutes the shape contour information of the workpiece.

[0071] In practical implementation, a visual inspection method compatible with multiple types of workpieces can be described by combining the image preprocessing and shape contour information extraction process with the inspection scenario of metal stamping parts. An industrial camera captures a color raw image containing the metal stamping part to be inspected, with the workpiece placed on a dark backlight panel in the raw image. The raw image is then converted to grayscale, converting the red, green, and blue channel color values ​​of each pixel into a single grayscale value using a weighted formula. One formula for grayscale conversion is expressed as follows:

[0072]

[0073] in: Represents the grayscale value of a pixel after conversion. Represents the intensity value of the red channel of a pixel. Represents the intensity value of the green channel of a pixel. The intensity value of the blue channel of a pixel is used to convert the image into a grayscale image that reflects the overall brightness distribution.

[0074] In some embodiments, an adaptive thresholding segmentation algorithm is applied to the obtained grayscale image. This algorithm dynamically calculates the segmentation threshold based on the grayscale distribution of pixels in the neighborhood of each pixel in the grayscale image. In areas where the metal stamping part contrasts significantly with the dark background, the adaptive thresholding algorithm calculates a higher threshold. In areas with coatings or reflections on the workpiece surface, the algorithm calculates a relatively lower threshold. By traversing the entire grayscale image and applying the dynamic threshold corresponding to each location, the workpiece area to be tested is separated from the background area, generating a clear black-and-white binary image. In the binary image, the workpiece area is represented by white pixels, and the background area is represented by black pixels.

[0075] Optionally, since the surface of metal stamping parts may have textures, scratches, or localized reflections, these appear as tiny black holes or depressions within the white area of ​​the workpiece in the binarized image. Morphological closing operations are performed on these tiny holes. The morphological closing operation uses a circular structuring element of a specific size to perform dilation and erosion operations on the binarized image sequentially. The dilation operation expands the boundary of the white area outwards, filling in black holes of a size comparable to the structuring element. The subsequent erosion operation shrinks the expanded boundary inwards, restoring the approximate original outer edge of the workpiece, thus filling in the tiny holes caused by surface textures or reflections and obtaining an internally connected white area.

[0076] In some embodiments, in the binarized image after morphological operations, all connected white pixel regions are scanned and marked. The total number of pixels in each connected region is calculated as the area of ​​the connected region. From all detected connected regions, the connected region with the largest area is compared and selected as the target region of the workpiece to be tested. The connected region with the largest area corresponds to the main body of the workpiece to be tested. Connected regions with smaller areas may be image noise, small attachments on the workpiece, or residual background interference. Selecting the connected region with the largest area can effectively eliminate these interferences.

[0077] It can be understood that when extracting the outermost pixel sequence of the target region, starting from any boundary pixel of the target region in the binarized image, the coordinates of the continuous boundary pixels surrounding the entire target region are tracked and recorded in a fixed clockwise or counterclockwise direction. This continuous coordinate sequence describes the external boundary shape of the workpiece, and the outermost pixel sequence constitutes the outline information of the workpiece under test, providing input data for subsequent workpiece model outline matching.

[0078] In practical implementation, taking a ring-shaped gasket as an example, after the above processing, the original color image is converted into a grayscale image. The grayscale values ​​of the gasket differ from the background grayscale values ​​in the grayscale image. After adaptive thresholding, the gasket area is completely segmented as the foreground, but the central hole may appear black due to light transmission, becoming a void in the binarized image. Morphological closing operations, through dilation followed by erosion, fill the central hole, temporarily transforming the gasket area into a solid disk to facilitate contour extraction. Subsequently, connected component analysis confirms that this solid disk area is the largest region in the image. Finally, the outer boundary of this solid disk area is traced, and the coordinates of all edge pixels are recorded. This sequence of outer boundary coordinates represents the shape contour information of the gasket workpiece, accurately reflecting the maximum outer diameter shape of the gasket.

[0079] In one embodiment of the present invention, see [reference] Figure 3 Before performing contour matching based on the extracted shape contour information and the pre-generated feature template library, the feature template library is constructed by collecting a set of sample images of standard workpieces of each model, ensuring that each model has multiple sample images from different angles. Contour extraction and alignment operations are performed on the sample image set of each model to generate the standard shape contour of the corresponding model. For each model, the key areas to be detected are manually marked on its sample images, and the coordinates of the key areas relative to the centroid of its standard shape contour are recorded to form the detection area definition for the corresponding model. Multi-scale feature extraction is performed on each key area image of each model, and the standard feature vector of the corresponding key area is calculated as a preset standard feature. The model identifier, its standard shape contour, the corresponding detection area definition, and the preset standard features of each area are associated and stored together to constitute the feature template library.

[0080] During contour matching, a shape context descriptor is calculated for the extracted contour information. This shape context descriptor characterizes the distance and angle distribution of each point on the contour relative to the contour centroid. The reference contour information of each standard workpiece template stored in the feature template library is traversed, and a reference shape context descriptor corresponding to the reference contour information is calculated. The similarity distance between the shape context descriptor of the current workpiece to be tested and each reference shape context descriptor is calculated sequentially. The reference shape context descriptor with the smallest similarity distance to the shape context descriptor of the current workpiece to be tested is selected, and the model number marked on the standard workpiece template to which the reference shape context descriptor belongs is identified as the workpiece model corresponding to the workpiece to be tested.

[0081] In practical implementation, a visual inspection method compatible with multiple models of workpieces can be described by combining the feature template library construction and workpiece model recognition process with the inspection scenario of mobile phone casing workpieces. In the step of constructing the feature template library, it is necessary to pre-collect sample image sets of standard workpieces for each model. For example, for two standard mobile phone casing workpieces, Model A and Model B, multiple high-definition images of each model are collected under different angles and lighting conditions to ensure that each model has multiple sample images from different angles to cover possible posture changes. A contour extraction operation is performed on the sample image set of Model A to extract the outer contour of the mobile phone casing in each image. All extracted contours are then aligned to ensure that the centroids of the contours coincide and the principal directions are consistent, thereby generating a stable and uniform standard outer contour for Model A. The same operation is performed on Model B to generate its standard outer contour.

[0082] In some embodiments, for each model, key areas to be detected are manually calibrated on its standard sample image. For mobile phone casing components, key areas include the camera hole area, side button area, and charging port area. During calibration, the operator uses an interactive tool to precisely select these areas on the image. The system records the relative coordinates of the circumscribed rectangle vertices of each key area in its corresponding standard outline coordinate system, with the centroid of the standard outline as the origin. For example, the coordinates of the camera hole area can be recorded as the offset relative to the centroid of the outline and the length and width of the area. The set of these relative coordinates forms the detection area definition for the corresponding model. On each key area image of each model, a multi-scale feature extraction operation is performed. The extraction operation includes constructing a Gaussian pyramid and calculating local binary pattern features and directional gradient histogram features. The multi-scale features are spliced ​​and fused to calculate a high-dimensional standard feature vector. This standard feature vector serves as the preset standard feature for that model and that area.

[0083] Optionally, all processed standard workpiece information is systematically stored. The model identifier of model A, the standard outline of model A, the detection area definitions for the camera hole area, side button area, and charging interface area of ​​model A, and the preset standard features of each of these three areas are stored as a complete record. The same associated storage operation is performed on model B. All records for model A and model B together constitute a structured, queryable feature template library, which serves as the benchmark for subsequent automatic workpiece model identification and feature comparison.

[0084] In practical implementation, when it is necessary to identify a mobile phone casing workpiece of an unknown model, its outline information is extracted, and the shape context descriptor of the extracted outline information is calculated. The shape context descriptor describes the outline shape by dividing the outline point set into polar coordinate logarithmic radius and equal angles. The shape context descriptor is used to characterize the distance and angular distribution of each point on the outline relative to the outline centroid. Its calculation constructs a histogram reflecting the relative positional distribution of the remaining points for each outline point. The reference outline information of each standard workpiece template stored in the feature template library is traversed. For example, the standard outlines of models A and B in the feature template library are read sequentially, and the reference shape context descriptor corresponding to each standard outline is calculated.

[0085] In some embodiments, the similarity distance between the shape context descriptor of the workpiece under test and each reference shape context descriptor in the template library is calculated. A formula for measuring the difference between two shape context descriptors is expressed as:

[0086]

[0087] in: This represents the chi-square similarity distance between two shape context descriptors. This represents the total number of dimensions of the descriptive sub-histogram. The shape context descriptor of the workpiece under test is in the first... Values ​​on a histogram interval The context descriptor of a certain reference shape in the template library is in the first position. Values ​​on each histogram interval. Calculate the distance between the shape context descriptor of the workpiece under test and the reference shape context descriptor of model A sequentially. and the distance to the model B reference shape context descriptor .

[0088] It is understandable that by comparing all the calculated similarity distance values, the benchmark shape context descriptor with the smallest similarity distance value to the shape context descriptor of the current workpiece to be tested is selected. Assume... , ,because Less than If the model is selected, the reference shape context descriptor corresponding to model A is chosen. The model number marked on the standard workpiece template to which the selected reference shape context descriptor belongs, i.e., model A, is identified as the workpiece model corresponding to the workpiece to be tested. This identification result determines that the preset standard features associated with model A and the detection area definition will be loaded from the feature template library for defect detection.

[0089] In one embodiment of the present invention, an image sub-region corresponding to the detection region definition is located in the original image. The detection region definition is read, and the detection region definition includes a relative coordinate range and a region shape description with the centroid of the workpiece under test as the origin. Based on the actual position of the workpiece under test in the original image, the actual coordinates of the centroid of the contour in the image coordinate system are calculated. Based on the actual coordinates and the relative coordinate range in the detection region definition, the pixel coordinate boundary of the image sub-region in the original image is calculated through coordinate transformation. Based on the calculated pixel coordinate boundary, a corresponding image block is cropped from the original image, and the image block is the image sub-region.

[0090] Multi-scale feature extraction is performed on the image sub-regions to obtain actual feature data. A Gaussian pyramid containing multiple Gaussian kernels of different sizes is constructed, and the image sub-regions are decomposed at multiple scales using the Gaussian pyramids to obtain multiple image sub-region copies at different scales. For each scale image sub-region copy, its local binary pattern features are calculated to obtain a texture feature vector reflecting micro-texture, and its directional gradient histogram is calculated to obtain a gradient feature vector reflecting edge and shape information. The texture feature vectors and gradient feature vectors obtained at each scale are concatenated and fused in scale order to form the actual feature data.

[0091] In practical implementation, a visual inspection method compatible with multiple workpiece models can describe the image sub-region localization and multi-scale feature extraction process in conjunction with the inspection scenario of circuit board assemblies. After completing the workpiece model identification, the system loads the preset standard features and detection area definitions associated with the workpiece model "PCB-TypeA" from the feature template library based on the identified workpiece model "PCB-TypeA". The detection area definition includes the relative coordinate range and area shape description of multiple key areas with the centroid of the "PCB-TypeA" standard outline as the origin. For example, "pad area 1" is defined as a rectangular area with a relative centroid coordinate of (10.5, -3.2), a width of 20, and a height of 15.

[0092] In some embodiments, an image sub-region corresponding to the detection region definition is located in the original image. First, the detection region definition of "pad region 1" is read, which specifies that the relative coordinates of the upper left corner of its rectangular region are (10.5, -3.2), as well as its width and height. Based on the actual position of the workpiece "PCB-Type A" in the original image, the actual coordinates of the centroid of the contour in the image coordinate system are calculated using its extracted outline. Assuming the calculated actual coordinates of the centroid are (255, 128), the pixel coordinate boundaries of "pad region 1" in the original image are calculated through coordinate transformation based on the actual coordinates of the centroid and the relative coordinate range in the detection region definition. A formula for converting relative coordinates to absolute image coordinates is expressed as follows:

[0093]

[0094]

[0095] in: Represents the absolute pixel coordinates of the vertices of the key region in the image. This represents the actual coordinates of the calculated centroid of the profile. This represents the relative coordinates relative to the centroid of the contour, recorded in the definition of the detection region. Adding the relative coordinates (10.5, -3.2) to the centroid coordinates (255, 128), and considering the region width and height, the pixel coordinate boundaries of "Pad Region 1" are finally calculated as the top left corner (265, 124) and the bottom right corner (285, 139). Based on the calculated pixel coordinate boundaries, the corresponding rectangular image block is cropped from the original image; this rectangular image block is the image sub-region to be subjected to subsequent feature analysis.

[0096] In the specific implementation, multi-scale feature extraction is performed on the cropped "pad region 1" image sub-region to obtain actual feature data. A Gaussian pyramid containing multiple Gaussian kernels of different sizes is constructed, and the Gaussian pyramid is used to decompose the image sub-region at multiple scales. The standard deviation of the Gaussian kernel and the image size are scaled layer by layer according to the scaling factor to obtain multiple copies of the image sub-region at different scales. The image resolution and Gaussian kernel parameters at different scales are shown in Table 1.

[0097] Table 1: Multiscale Decomposition Parameter Table

[0098]

[0099] Optionally, local binary pattern features are calculated for each scale-scale image sub-region copy. Taking a scale-1 (14×10 pixels) copy as an example, the algorithm traverses the 3x3 neighborhood of each pixel in the image, compares the gray value of the center pixel with the gray values ​​of the surrounding 8 pixels, generates an 8-bit binary pattern string, and counts the frequency of these binary patterns throughout the entire image range to obtain a texture feature vector reflecting the micro-texture distribution. Simultaneously, for the same scale-1 image sub-region copy, its histogram of oriented gradients is calculated. The histogram of oriented gradients calculates the gradient magnitude and direction of each pixel in the image. The image is divided into small spatial units, and the histogram of gradient direction is calculated within each unit. Finally, the histograms of all units are concatenated to obtain a gradient feature vector reflecting edge and shape information.

[0100] In one embodiment of the present invention, the extracted actual feature data is compared item by item with the preset standard features to generate a feature difference map. Preset standard features corresponding to the current workpiece model and detection area are read from the feature template library. The preset standard features are standard feature vectors with the same structure as the actual feature data. The absolute difference between the feature vectors in the actual feature data and the standard feature vectors in the preset standard features is calculated for each feature dimension. The calculated absolute difference for each feature dimension is mapped onto a two-dimensional grid map corresponding to the spatial resolution of the image sub-region. Each pixel position in the grid represents a feature dimension, and the gray value of the pixel represents the magnitude of the absolute difference of the feature dimension. The two-dimensional grid map is the feature difference map.

[0101] The feature difference map is analyzed to determine whether the workpiece under test has a defect. Threshold segmentation is applied to the feature difference map, and pixel regions with grayscale values ​​exceeding a preset difference threshold are marked as difference regions. The pixel area, region compactness, and average difference intensity of each difference region are calculated. If any difference region has a pixel area greater than a preset area threshold and its average difference intensity greater than a preset intensity threshold, then the workpiece under test is determined to have a defect in the current detection area. The positions of the difference regions that meet the determination conditions in the original image are recorded as defect locations, and the defect type is determined by matching the texture feature patterns of the difference regions against a predefined defect type library.

[0102] In practical implementation, a visual inspection method compatible with multiple workpiece models can be described by combining the feature difference map generation and defect judgment process with the inspection scenario of the precision gear tooth surface region. After completing the multi-scale feature extraction of the image sub-region, the system obtains the actual feature data reflecting the current state of the gear tooth surface region under test. Preset standard features corresponding to the current workpiece model "Gear-TypeB" and the specific inspection region "tooth surface region-3" are read from the feature template library. These preset standard features are standard feature vectors extracted and stored from the corresponding regions of standard defect-free gear samples during the feature template library construction phase. The preset standard features have the same structure as the actual feature data, both being one-dimensional vectors composed of multi-scale local binary pattern features and directional gradient histogram features.

[0103] In some embodiments, the extracted actual feature data is compared item by item with preset standard features to generate a feature difference map, and the absolute difference between the feature vector in the actual feature data and the standard feature vector in the preset standard features is calculated for each feature dimension. Assuming the total dimension of the feature vector is N, for the k-th feature, its absolute difference is:

[0104]

[0105] in: Represents the absolute difference of the k-th dimension feature. This represents the value of the actual feature data extracted from a sub-region of the image of the workpiece under test in the k-th dimension. This represents the value of a predefined standard feature read from the feature template library in the k-th dimension. Calculate the values ​​of all features from k to N sequentially. The value is used to obtain a difference vector containing N absolute differences.

[0106] In practice, the calculated absolute difference of each feature dimension is mapped onto a two-dimensional grid map corresponding to the spatial resolution of the image sub-region. Assuming that after multi-scale decomposition and feature extraction, the spatial layout of the image sub-region is organized into a 5x5 grid, each grid cell corresponds to a spatial block within the image sub-region and contains multiple feature dimensions, the system unfolds this 5x5 grid into a one-dimensional sequence, where the multiple feature dimensions within each grid cell are arranged sequentially. When generating the feature difference map, a grayscale value is assigned to each pixel position in this 5x5 grid. The pixel position (i,j) corresponds to the m-th feature dimension in the unfolded sequence, and its grayscale value... That is, the absolute difference of the feature in that dimension. A simplified example of feature difference graph grid grayscale representation data is shown in Table 1.

[0107] Table 2: Local Mesh Gray Values ​​of Feature Difference Map

[0108]

[0109] Optionally, the aforementioned two-dimensional grid diagram is the feature difference map. The feature difference map visually presents the point-by-point differences between the features of the region under test and the standard features in grayscale image form. High grayscale value regions (such as regions with values ​​close to 255 in the table) indicate significant differences in the feature dimension corresponding to that location. Threshold segmentation is applied to the feature difference map, setting a preset difference threshold T=200. All pixels in the feature difference map with grayscale values ​​exceeding the preset difference threshold of 200 are marked. These marked connected pixels constitute one or more difference regions. The pixel area of ​​each difference region is calculated, i.e., the total number of pixels in the region. The region compactness is calculated, which can be estimated by the ratio of the square of the region's perimeter to its area. Simultaneously, the average grayscale value of all pixels in the region is calculated as the average difference intensity.

[0110] Understandably, defect judgment is performed according to preset judgment rules. If any difference region exists whose pixel area is greater than a preset area threshold (e.g., the area threshold is set to 5 pixels) and whose average difference intensity is greater than a preset intensity threshold (e.g., the intensity threshold is set to 220), then the workpiece under test is judged to have a defect in the current detection area. For example, in the table data, the pixel group located near rows 2 to 4 and columns 2 to 4 constitutes a difference region. The pixel area of ​​this region is greater than 5, and its average gray value is much higher than 220, thus meeting the defect judgment condition. The position of the difference region that meets the judgment condition in the original image is recorded as the defect location. The defect location is obtained by mapping the grid coordinates of the difference region back to the original image sub-region and then combining it with the global coordinates of the image sub-region in the original image. The defect type is determined by matching the texture feature pattern of the difference region with a predefined defect type library. For example, if the difference region is distributed in a continuous strip shape, it is matched as the "scratch" type; if the difference region is clustered in discrete points, it may be matched as the "pitting" type.

[0111] See Figure 4In visual inspection methods compatible with multiple workpiece models, the time consumption distribution of the entire visual inspection process exhibits clear modular performance characteristics, providing a quantitative basis for system performance optimization and bottleneck identification. Specifically, the entire visual inspection process is broken down into seven core execution stages: image acquisition, preprocessing, contour matching, region localization, feature extraction, difference analysis, and defect determination. The horizontal axis represents the inspection stage, and the vertical axis represents the time consumption of a single inspection at each stage (unit: ms). From the time consumption distribution, the feature extraction stage is the performance bottleneck of the entire process, with a single inspection taking approximately 48ms, accounting for the highest proportion of the total time consumption. This stage requires performing multi-scale Gaussian pyramid decomposition on sub-regions of the image and calculating Local Binary Pattern (LBP) texture features and Histogram of Oriented Gradients (HOG) gradient features at each scale, ultimately completing the splicing and fusion of multi-scale feature vectors. The complex multi-scale operations and feature calculations are the core reason for its high time consumption. The next stage is contour matching, which takes approximately 32ms. This stage requires calculating the shape context descriptor of the workpiece's outline and traversing the feature template library to complete multi-template similarity matching. The computational complexity is positively correlated with the size of the template library. The preprocessing stage takes approximately 25ms and requires a series of image preprocessing operations, including grayscale conversion, adaptive threshold segmentation, morphological closing operations, and connected component filtering, to provide a foundation for subsequent contour extraction and matching. The difference analysis stage takes approximately 22ms and requires calculating the absolute difference between the actual feature data and the preset standard features, generating feature difference maps, and threshold segmentation. This is a core preliminary step for defect determination. The image acquisition stage takes approximately 18ms, corresponding to the image exposure and data transmission process of the industrial camera. The region localization stage takes the least time, only about 16ms, and only requires calculating the centroid of the contour, transforming relative coordinates to the image coordinate system, and cropping image sub-regions, resulting in low computational complexity. The defect determination stage takes approximately 10ms and only requires threshold determination of the area, compactness, and average difference intensity of the difference regions and defect type matching, resulting in minimal computation. The time consumption analysis results intuitively verify the performance characteristics of each module of the visual inspection method, providing a clear direction for subsequent algorithm optimization of high-time-consuming modules such as feature extraction and contour matching (such as multi-threaded parallelism, template library index optimization, feature dimensionality reduction, etc.), and providing quantitative data support for the real-time evaluation of the system and the matching of industrial production line cycle time.

[0112] In one embodiment of the present invention, after loading preset standard features associated with the workpiece model and a detection area definition from the feature template library based on the identified workpiece model, an adaptive template fine-tuning step is performed. This involves calculating the average vector of features in each dimension within the detection area definition for the current batch based on the actual feature data of multiple workpieces already detected in the current batch. The calculated average vector is then weighted and fused with the preset standard features loaded from the feature template library to generate dynamic standard features suitable for the current batch. These dynamic standard features are then used to replace the original preset standard features for comparison in subsequent detections of workpieces in the same batch.

[0113] In practical implementation, a visual inspection method compatible with multiple workpiece models can be described using an adaptive template fine-tuning step in the context of visual inspection of mass-produced ceramic capacitors. After loading the preset standard features associated with the workpiece model "Capacitor-C" from the feature template library, and defining the detection area, the system does not always use fixed preset standard features. Instead, it introduces an adaptive update mechanism to adapt to minor process fluctuations that may exist within the same production batch. When inspecting the first few workpieces in the current batch, the system still uses the original preset standard features loaded from the feature template library for comparison and judgment, but simultaneously records the actual feature data extracted from these inspected workpieces within the defined detection area.

[0114] In some embodiments, the average vector of features in each dimension of the current batch within the defined detection area is calculated based on the actual feature data of multiple workpieces already inspected in the current batch. Assuming that M "Capacitor-C type" workpieces have been continuously inspected in the current batch, for the defined "weld end region," the actual feature data extracted from each workpiece in this region is an N-dimensional feature vector. The system performs an algebraic average of these M N-dimensional feature vectors in each dimension to calculate the average vector of features in each dimension of the current batch within the "weld end region":

[0115]

[0116] in: This represents the calculated average value vector for the current batch; it is an N-dimensional vector. This represents the number of workpieces that have been inspected in the current batch. This represents the N-dimensional actual feature data vector extracted from the "weld end region" of the i-th inspected workpiece. (Average value vector) This reflects the statistically central position of the "weld end area" feature in the current production batch.

[0117] In practice, the calculated average vector is weighted and fused with preset standard features loaded from the feature template library. This refers to the N-dimensional feature vectors determined based on standard samples during the feature template library construction phase. Weighted fusion uses a preset weight coefficient β to balance historical standard information with current batch statistics, generating dynamic standard features suitable for the current batch. A weighted fusion formula is expressed as follows:

[0118]

[0119] in: This represents the final generated dynamic standard feature vector used for the current batch alignment. This represents the original preset standard feature vector loaded from the feature template library. This represents the average value vector calculated based on the inspected workpieces in the current batch. This represents the weighting coefficient assigned to the statistics of the current batch, and its value is between 0 and 1. For example, setting the weighting coefficient... Then the dynamic standard eigenvector It contains 70% of the original standard feature information and 30% of the current batch average feature information.

[0120] Optionally, after completing the dynamic standard features After calculation, the system will use dynamic standard features in subsequent inspections of the same batch of workpieces. Replace the original preset standard features A comparison is then performed. This means that for any subsequent "Capacitor-C type" workpiece within the same batch, when judging defects in its "weld end area," the reference benchmark used for comparison is no longer the fixed feature template library. Instead, it is an updated version that incorporates features of the inspected workpieces in this batch. This replacement remains in effect until the production batch inspection task is completed. When a new production batch inspection begins, the system resets this process, reloading the original preset standard features from the feature template library. Used for the inspection of the first few workpieces in a new batch, and the average vector is recalculated and new dynamic standard features are generated based on the data of the new batch.

[0121] It is understandable that the adaptive template fine-tuning step, by incorporating statistical information from batch data, enables the comparison standard to adapt to minor, batch-specific manufacturing deviations. For example, due to variations in raw material batches, a slight but consistent overall increase in surface gloss across all workpieces in a particular batch might cause the extracted actual feature data to systematically deviate from the original preset standard features in the dimension reflecting gloss. By calculating the average vector of the inspected workpieces in this batch. and Fusion, generating dynamic standard features Adaptive adjustments will be made in the corresponding dimensions. Subsequent comparisons will more accurately reflect the anomalies of individual workpieces relative to the average level of the batch, rather than deviations relative to a fixed absolute standard. This will reduce the misjudgment of consistent batch-wide process deviations as individual defects and improve the robustness of the detection system to normal fluctuations between batches.

[0122] See Figure 5 In a visual inspection method compatible with multiple workpiece models, the effectiveness of the adaptive template fine-tuning mechanism is verified by comparing the feature distribution of the weld end region of each batch of workpieces with the baseline. Specifically, the box plots for the five batches in the figure represent the statistical distribution of the actual feature data extracted from the weld end region of the workpiece under the corresponding production batch. The upper and lower boundaries of the box represent the upper and lower quartiles of the feature values, the horizontal line inside the box is the median of the batch features, the upper and lower whisker lines represent the normal fluctuation range of the batch features, and outliers are the feature outliers of individual workpieces within the batch. The red dashed line represents the original standard features (fixed baseline value 0.70) stored in the feature template library, calibrated based on the standard workpiece. The core logic of adaptive template fine-tuning is as follows: at the beginning of batch inspection, the system uses the original standard features as a baseline, synchronously collects the actual feature data of the inspected workpieces in the current batch, calculates the batch feature average vector, and generates dynamic standard features suitable for the current batch through a weighted fusion formula. As shown in the figure, the median values ​​of the weld end region features in batches 1 to 5 (0.71, 0.72, 0.722, 0.739, 0.738) are all systematically higher than the original standard feature value of 0.70, reflecting consistent process fluctuations across different production batches. If the original standard feature is always used as the comparison benchmark, normal workpieces within the batch that meet the process level will be misjudged as defects. Through adaptive template fine-tuning, the dynamic standard feature will shift towards the center of the batch feature distribution (batch median), making the comparison benchmark for subsequent inspections adapt to the process level of the current batch. Only outliers that deviate from the batch distribution (such as 0.78 in batch 1 and 0.679 in batch 3) will be judged as defects, significantly reducing the false detection rate caused by batch-based process fluctuations and improving the robustness of the vision inspection system for multiple batches of production. At the parameter configuration level, the weighted fusion coefficient β is set to 0.3, which means that the dynamic standard features retain 70% of the original standard feature information and fuse 30% of the current batch statistical information. This adapts to batch fluctuations while avoiding excessive deviation of abnormal batch data from the benchmark. The batch feature statistical sample size is set to the first 20 inspected workpieces in the batch to ensure the representativeness and stability of the statistical results.

[0123] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A visual inspection method compatible with multiple types of workpieces to be tested, characterized in that, The method includes: Acquire the original image of the workpiece to be tested placed on the inspection station; The original image is preprocessed to extract the shape contour information of the workpiece to be tested; Based on the extracted shape contour information, contour matching is performed with a pre-generated feature template library to identify the workpiece model corresponding to the workpiece to be tested from the feature template library; Based on the identified workpiece model, load the preset standard features and detection area definitions associated with the workpiece model from the feature template library; Locate the image sub-region in the original image that corresponds to the defined detection region; Multi-scale feature extraction is performed on the image sub-region to obtain actual feature data; The extracted actual feature data is compared item by item with the preset standard features to generate a feature difference map; The feature difference map is analyzed to determine whether the workpiece under test has defects; Generate visual inspection results that include the workpiece model, defect location, and defect type.

2. The visual inspection method compatible with multiple models of workpieces according to claim 1, characterized in that, The original image is preprocessed to extract the external contour information of the workpiece to be tested, including: Perform grayscale conversion on the original image to obtain a grayscale image; An adaptive threshold segmentation algorithm is applied to the grayscale image to separate the workpiece area to be tested from the background area, thereby obtaining a binarized image; Perform morphological closing operation on the binarized image to fill the small holes in the area of ​​the workpiece to be tested caused by surface texture or reflection; Detect the boundaries of all connected components in the binarized image, and select the connected component with the largest area as the target region of the workpiece to be tested; Extract the outermost pixel sequence of the target region, which constitutes the outline information of the workpiece to be tested.

3. The visual inspection method compatible with multiple models of workpieces according to claim 2, characterized in that, Based on the extracted shape contour information, contour matching is performed with a pre-generated feature template library to identify the workpiece model corresponding to the workpiece under test from the feature template library, including: Calculate the shape context descriptor of the extracted outline information, the shape context descriptor being used to characterize the distance and angle distribution of each point on the outline relative to the centroid of the outline; Traverse the reference contour information of each standard workpiece template stored in the feature template library, and calculate the reference shape context descriptor corresponding to the reference contour information; The similarity distance between the shape context descriptor of the current workpiece under test and each of the reference shape context descriptors is calculated sequentially. Select the reference shape context descriptor that has the minimum similarity distance to the shape context descriptor of the current workpiece to be tested; The model number marked on the standard workpiece template to which the reference shape context descriptor belongs is identified as the workpiece model corresponding to the workpiece to be tested.

4. The visual inspection method compatible with multiple models of workpieces according to claim 3, characterized in that, Locating the image sub-region in the original image that corresponds to the defined detection region includes: Read the detection area definition, which includes the relative coordinate range and area shape description with the centroid of the contour of the workpiece to be tested as the origin; Based on the actual position of the workpiece to be tested in the original image, calculate the actual coordinates of the centroid of the contour in the image coordinate system; Based on the actual coordinates and the relative coordinate range defined in the detection area, the pixel coordinate boundaries of the image sub-region in the original image are calculated through coordinate transformation; Based on the calculated pixel coordinate boundaries, corresponding image blocks are cropped from the original image, and these image blocks are the image sub-regions.

5. The visual inspection method compatible with multiple models of workpieces according to claim 4, characterized in that, Multi-scale feature extraction is performed on the image sub-region to obtain actual feature data, including: A Gaussian pyramid containing multiple Gaussian kernels of different sizes is constructed, and the Gaussian pyramid is used to perform multi-scale decomposition on the image sub-region to obtain multiple copies of the image sub-region at different scales. For each scale image sub-region copy, its local binary pattern features are calculated to obtain a texture feature vector reflecting the micro-texture; On each scale of the image sub-region copy, its directional gradient histogram is calculated to obtain the gradient feature vector that reflects edge and shape information; The texture feature vectors and gradient feature vectors obtained at each scale are concatenated and fused in scale order to form the actual feature data.

6. The visual inspection method compatible with multiple models of workpieces according to claim 5, characterized in that, The extracted actual feature data is compared item by item with the preset standard features to generate a feature difference map, including: Read the preset standard features corresponding to the current workpiece model and detection area from the feature template library. The preset standard features are standard feature vectors with the same structure as the actual feature data. Calculate the absolute difference between the feature vector in the actual feature data and the standard feature vector in the preset standard features in each feature dimension; The calculated absolute difference of each feature dimension is mapped onto a two-dimensional grid corresponding to the spatial resolution of the image sub-region, where each pixel position of the grid represents a feature dimension, and the gray value of the pixel represents the magnitude of the absolute difference of the feature dimension. The two-dimensional grid diagram is the feature difference diagram.

7. The visual inspection method compatible with multiple models of workpieces according to claim 6, characterized in that, Analyzing the feature difference map to determine whether the workpiece under test has defects includes: Threshold segmentation is applied to the feature difference map to mark pixel regions with gray values ​​exceeding a preset difference threshold as difference regions; Calculate the pixel area, region compactness, and average difference intensity of each of the said difference regions; If any difference region exists whose pixel area is greater than a preset area threshold and whose average difference intensity is greater than a preset intensity threshold, then the workpiece under test is determined to have a defect in the current detection area. The location of the difference region that meets the judgment condition in the original image is recorded as the defect location, and the defect type is determined by matching the texture feature pattern of the difference region with a predefined defect type library.

8. The visual inspection method compatible with multiple models of workpieces according to claim 7, characterized in that, Before performing contour matching with a pre-generated feature template library based on the extracted shape contour information, the method further includes a step of constructing the feature template library. The step of constructing the feature template library includes: Collect sample images of standard workpieces of various models to ensure that there are multiple sample images of each model from different angles; Perform contour extraction and alignment operations on the sample image set for each model to generate the standard shape contour of the corresponding model; For each model, the key areas to be detected are manually marked on its sample image, and the coordinates of the key areas relative to the centroid of its standard shape contour are recorded to form the detection area definition for the corresponding model. For each key region image of each model, perform multi-scale feature extraction and calculate the standard feature vector of the corresponding key region as the preset standard feature; The model identifier, standard shape outline, corresponding detection area definition, and preset standard features of each area are associated and stored to form the feature template library.

9. The visual inspection method compatible with multiple models of workpieces according to claim 8, characterized in that, After loading preset standard features and detection area definitions associated with the identified workpiece model from the feature template library, the method further includes an adaptive template fine-tuning step, which includes: Based on the actual feature data of multiple workpieces to be tested that have been inspected in the current batch, calculate the average vector of the features of each dimension of the current batch within the defined detection area; The calculated average vector is weighted and fused with the preset standard features loaded from the feature template library to generate dynamic standard features suitable for the current batch. In subsequent inspections of the same batch of workpieces, the dynamic standard features are used to replace the original preset standard features for comparison.

10. A vision inspection system compatible with multiple types of workpieces to be inspected, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the visual inspection method for compatible multi-model workpieces as described in any one of claims 1 to 9.

Images