Pen holder surface defect detection system and method based on machine vision

Abstract

The invention relates to the technical field of defect detection, and particularly discloses a pen holder surface defect detection system and method based on machine vision, and the system comprises an image acquisition module, an image processing module, a feature map construction module and a result generation module. The image processing module obtains a to-be-detected surface feature map and a normal surface feature map by extracting features of the images, then constructs a penholder surface difference feature map between the to-be-detected surface feature map and the normal surface feature map, and finally obtains a classification result through a classifier based on the penholder surface difference feature map. The pen holder surface detection device is widely applied to pen manufacturing on an industrial assembly line, and the production efficiency and the quality level are improved.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and more specifically, to a machine vision-based pen barrel surface defect detection system and method. Background Technology

[0002] Surface inspection of pen barrels is a common task in pen manufacturing, but it cannot be taken lightly. Only by ensuring the most basic quality can pen manufacturing reach a higher level. Considering the rapid advancement of technology, improving inspection efficiency is also urgent. While inspecting for cracks and other defects on the pen barrel surface, it is also necessary to consider the burrs that may remain on the surface after demolding. The final inspection should be able to decisively eliminate both major pen barrel surface problems simultaneously, making it both convenient and efficient.

[0003] For a long time, the quality inspection of pen barrels on industrial production lines has relied on manual inspection, which suffers from low accuracy and efficiency, becoming a pressing problem. With the continuous development of machine vision technology, surface defect detection methods based on machine vision are gradually replacing manual inspection due to their advantages of high efficiency, high precision, and low false positive rate.

[0004] Therefore, a machine vision-based pen barrel surface defect detection system and method are desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a machine vision-based pen barrel surface defect detection system and method, comprising an image acquisition module, an image processing module, a feature map construction module, and a result generation module. First, surface images of the pen barrel to be inspected and normal pen barrels are acquired. The image processing module extracts features from these images to obtain feature maps of the surface to be inspected and normal surfaces. Next, a pen barrel surface difference feature map is constructed between the feature maps of the surface to be inspected and normal surfaces. Finally, based on the pen barrel surface difference feature map, a classifier obtains a classification result indicating whether the surface of the pen barrel to be inspected meets the requirements of a normal pen barrel. This system can be widely applied in pen manufacturing on industrial production lines to improve production efficiency and quality levels.

[0006] According to one aspect of this application, a machine vision-based pen barrel surface defect detection system is provided, comprising:

[0007] The image acquisition module is used to acquire surface images of the pen barrel to be detected and surface images of a normal pen barrel;

[0008] The image processing module is used to extract features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain the surface feature map to be detected and the normal surface feature map;

[0009] The feature map construction module is used to construct a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimize it to obtain an optimized pen barrel surface difference feature map.

[0010] The result generation module is used to determine whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel based on the optimized pen barrel surface differential feature map.

[0011] According to another aspect of this application, a machine vision-based method for detecting surface defects in pen barrels is also provided, comprising:

[0012] Acquire surface images of the pen barrel to be inspected and a normal pen barrel;

[0013] Extract features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain the surface feature map to be detected and the normal surface feature map;

[0014] Construct a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimize it to obtain an optimized pen barrel surface difference feature map;

[0015] Based on the optimized pen barrel surface differential feature map, it is determined whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel.

[0016] Compared with existing technologies, this application provides a machine vision-based pen barrel surface defect detection system and method, which includes an image acquisition module, an image processing module, a feature map construction module, and a result generation module. First, surface images of the pen barrel to be detected and normal pen barrels are acquired. The image processing module extracts features from these images to obtain feature maps of the surface to be detected and normal surfaces. Next, a pen barrel surface difference feature map is constructed between the feature maps of the surface to be detected and the normal surfaces. Finally, a classification result is obtained based on the pen barrel surface difference feature map by a classifier, indicating whether the surface of the pen barrel to be detected meets the requirements of a normal pen barrel. This method can be widely applied in pen manufacturing on industrial production lines to improve production efficiency and quality levels. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a block diagram of a machine vision-based pen barrel surface defect detection system according to an embodiment of this application.

[0019] Figure 2 This is a block diagram of the image processing module in a machine vision-based pen barrel surface defect detection system according to an embodiment of this application.

[0020] Figure 3 This is a block diagram of an image block encoding unit in a machine vision-based pen barrel surface defect detection system according to an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of the architecture of a machine vision-based pen barrel surface defect detection system according to an embodiment of this application.

[0022] Figure 5 This is a flowchart of a machine vision-based pen barrel surface defect detection method according to an embodiment of this application. Detailed Implementation

[0023] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0024] Exemplary System

[0025] Figure 1 The illustration shows a block diagram of a machine vision-based pen barrel surface defect detection system according to an embodiment of this application. Figure 1 As shown, a machine vision-based pen barrel surface defect detection system 100 according to an embodiment of this application includes: an image acquisition module 110, used to acquire a surface image of a pen barrel to be detected and a surface image of a normal pen barrel; an image processing module 120, used to extract features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain a surface feature map to be detected and a normal surface feature map; a feature map construction module 130, used to construct a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimize it to obtain an optimized pen barrel surface difference feature map; and a result generation module 140, used to determine whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel based on the optimized pen barrel surface difference feature map.

[0026] In this embodiment, the image acquisition module 110 is used to acquire surface images of the pen barrel to be inspected and surface images of a normal pen barrel. It should be understood that acquiring the surface image of the pen barrel to be inspected allows for the capture of the actual condition of the pen barrel surface, including potential defects such as cracks and burrs. Acquiring the surface image of a normal pen barrel provides a reference for comparing the surface condition of the pen barrel to be inspected and determining whether defects exist. By comparing and analyzing the surface images of the pen barrel to be inspected and the normal pen barrel, the system can extract their features and construct a differential feature map. The differential feature map reflects the differences between the pen barrel to be inspected and the normal pen barrel, i.e., the defects on the pen barrel surface. By analyzing the differential feature map, the system can determine whether the surface of the pen barrel to be inspected meets the requirements of a normal pen barrel, thereby performing defect detection and quality control. Therefore, acquiring the surface images of the pen barrel to be inspected and the normal pen barrel is based on the need for comparison and analysis to enable effective defect detection and quality judgment. Such comparison and analysis can help manufacturers improve the quality level of pen manufacturing and ensure that products meet predetermined requirements. Specifically, appropriate equipment, such as a camera or other image acquisition device, is selected to acquire surface images of the pen barrel. The camera or image acquisition device is installed in a suitable location and subjected to necessary debugging and calibration to ensure image quality and accuracy. The acquired surface images of the pen barrel to be inspected and normal pen barrels are preprocessed, such as denoising and contrast enhancement, to extract image features and reduce the impact of noise. Image processing algorithms are used to extract features from the surface images of the pen barrel to be inspected and normal pen barrels, such as texture features, edge features, and color features. A difference operation is performed between the feature map of the pen barrel to be inspected and the feature map of the normal pen barrel surface to obtain a difference feature map. The difference feature map highlights the differences between the surface of the pen barrel to be inspected and the surface of the normal pen barrel, i.e., potential defects. Defect detection and analysis are performed based on the difference feature map. By analyzing the location, shape, size, and other features of the differences, it is determined whether defects exist on the surface of the pen barrel to be inspected.

[0027] In this embodiment, the image processing module 120 is used to extract features from the surface image of the pen barrel to be detected and the surface image of a normal pen barrel to obtain a surface feature map to be detected and a normal surface feature map. It should be understood that by extracting features from the image, the system can capture key properties and texture information of the surface of the pen barrel to be detected. These features can be edges, textures, colors, shapes, etc. By extracting features, complex information in the image can be transformed into more descriptive and comparable feature vectors. By comparing the surface feature map to be detected and the normal surface feature map, the system can detect differences and defects on the surface of the pen barrel to be detected. Defects are usually manifested as abnormal areas in the feature map, such as texture changes, color anomalies, edge breaks, etc. By analyzing the feature map, the system can locate and identify these abnormal areas and determine whether they meet the requirements of a normal pen barrel. By extracting feature maps and performing comparative analysis, the complexity of pixel-level comparison of the entire image can be reduced. Feature maps provide a higher level of abstraction and can capture important features of the pen barrel surface. This can improve the efficiency and accuracy of the detection algorithm. Therefore, extracting features from the surface images of the pen barrel to be inspected and normal pen barrels to obtain surface feature maps of the surface to be inspected and normal surfaces is to transform the complex information in the images into more descriptive and comparable feature vectors. Comparative analysis is then used to detect and evaluate defects on the surface of the pen barrel to be inspected. This improves the accuracy and efficiency of defect detection.

[0028] Specifically, in one embodiment of this application, Figure 2 The illustration shows a block diagram of the image processing module in a machine vision-based pen barrel surface defect detection system according to an embodiment of this application. Figure 2 As shown, in the above-mentioned machine vision-based pen barrel surface defect detection system 100, the image processing module 120 includes: a block segmentation unit 121, used to perform image block processing on the surface image of the pen barrel to be detected and the surface image of the normal pen barrel respectively to obtain the image block sequence to be detected and the normal image block sequence; and an image block encoding unit 122, used to extract from the image block sequence to be detected and the normal image block sequence to obtain the surface feature map to be detected and the normal surface feature map.

[0029] Specifically, the segmentation unit 121 is used to perform image segmentation processing on the surface image of the pen barrel to be detected and the surface image of the normal pen barrel, respectively, to obtain a sequence of image blocks to be detected and a sequence of normal image blocks. It should be understood that segmenting the image allows the system to focus more on the local features of the pen barrel surface. Different defects may appear in different areas of the pen barrel, such as the nib and body. By segmenting the image, each image block can be independently feature extracted and analyzed to more accurately detect and locate defects. Segmenting the image allows for more precise location and description of the defect's position. Each image block can be considered an independent region; by comparing the sequence of image blocks to be detected and the sequence of normal image blocks, it is possible to determine which regions are abnormal. This helps to locate defects and provides more specific information for subsequent processing and repair. Segmenting the image provides more data samples, enabling more accurate statistical analysis. By comparing the sequence of image blocks to be detected and the sequence of normal image blocks, difference measures between each image block, such as mean, variance, and gradient, can be calculated. This statistical information can be used to establish a defect detection model or to determine appropriate thresholds to judge whether an image block contains a defect. Performing pixel-level comparison and analysis of the entire image can lead to excessively high computational complexity. Dividing the image into blocks reduces the amount of data that needs to be processed, thereby improving processing efficiency. Analyzing only the image blocks of interest allows for faster detection and identification of defects. Therefore, image block processing of the surface image of the pen to be inspected and the surface image of a normal pen allows for more detailed analysis and comparison of local features on the pen surface, locating and describing the position of defects, providing more accurate statistical information, and reducing computational complexity. This enhances the accuracy and efficiency of defect detection. Specifically, image block processing is performed on the surface image of the pen to be inspected and the surface image of a normal pen to obtain a sequence of image blocks to be inspected and a sequence of normal image blocks, including: determining the size and shape of the image blocks (using square or rectangular blocks of fixed size, or selecting blocks of different sizes and shapes depending on the specific situation); dividing the surface image of the pen to be inspected and the surface image of a normal pen according to the selected block size; and for each image block, extracting features, calculating statistical information, or performing other analytical operations. The resulting image block sequence is obtained after dividing the surface image of the pen to be inspected and the surface image of a normal pen into blocks. Each image patch contains local features of the pen barrel surface to be detected and local features of the normal pen barrel surface.

[0030] further, Figure 3 The illustration shows a block diagram of an image block encoding unit in a machine vision-based pen barrel surface defect detection system according to an embodiment of this application. Figure 3As shown, in the image processing module 120 of the quality control system 100 for manufacturing electronic components, the image block encoding unit 122 includes: a detection image encoding subunit 1221, used to perform convolutional encoding on the detection image block sequence to obtain a detection surface feature map; and a normal image encoding subunit 1222, used to perform convolutional encoding on the normal image block sequence to obtain a normal surface feature map.

[0031] Accordingly, in a specific example of this application, the image encoding subunit 1221 is used to perform convolutional encoding on the sequence of image blocks to be detected to obtain a surface feature map to be detected. It should be understood that convolutional encoding can extract features from image blocks layer by layer through operations such as convolution and pooling. Compared to simple feature extraction methods, convolutional encoding can capture higher-level features, including image texture, shape, and structure. The sequence of image blocks to be detected may contain a large amount of data, and convolutional encoding can reduce storage and computational requirements by reducing the dimensionality of the data. The encoded surface feature map to be detected can more compactly represent the feature information of the image blocks. Convolutional encoding considers the spatial relationships between image blocks when extracting features. Through convolution and pooling operations, the encoded feature map preserves the spatial structure of the image blocks and can better capture the correlation between local features. The surface feature map to be detected obtained based on convolutional encoding can be used for subsequent classification and detection tasks. This allows for the use of methods such as deep learning to train and infer the feature map, further improving the accuracy and robustness of defect detection. Therefore, convolutional encoding of the sequence of image patches to be detected can yield a surface feature map, thereby extracting higher-level feature representations, compressing data dimensionality, strengthening the spatial relationships between features, and improving the accuracy of classification and detection. This can provide more informative and effective feature representations for subsequent defect detection tasks.

[0032] It is worth mentioning that, in other specific examples of this application, the sequence of image patches to be detected can also be convolutionally encoded in other ways to obtain the feature map of the surface to be detected. For example, a convolutional autoencoder is designed, including an encoder and a decoder. The encoder consists of multiple convolutional layers and pooling layers, used to compress the input image patch sequence into a lower-dimensional feature representation. The decoder consists of multiple deconvolutional layers and upsampling layers, used to reconstruct the feature representation to the same size as the input image patch sequence. The convolutional autoencoder is trained using a large number of normal sample image patch sequences. During training, the encoder compresses the input image patch sequence into a feature representation, and the decoder reconstructs the feature representation back into the original image patch sequence. The parameters of the autoencoder are optimized by minimizing the reconstruction error (e.g., mean squared error loss). Using the trained convolutional autoencoder, the sequence of image patches to be detected is input into the encoder to obtain the corresponding feature representation. These feature representations can be regarded as feature vectors of the surface to be detected. The feature representation is reconstructed to the same size as the input image patch sequence to obtain the feature map of the surface to be detected. The obtained feature map of the surface to be detected is used for defect detection. Various machine learning or deep learning algorithms, such as Support Vector Machines (SVM), Random Forest, or Convolutional Neural Networks (CNN), can be used to classify feature maps or detect defects. By using a convolutional autoencoder for convolutional encoding, a compact representation of the input image patch sequence can be learned, and important surface features can be extracted. This approach can effectively capture the spatial relationships and texture information between image patches, providing informative feature representations for subsequent defect detection tasks.

[0033] Specifically, the image encoding subunit includes: a second-level convolutional encoding subunit for the image to be detected, used to pass the image block sequence to be detected through a first convolutional neural network model as a filter to obtain multiple image block feature vectors; and a second-level Siamese network encoding subunit for arranging the multiple image block feature vectors to be detected into a global feature matrix of the image to be detected and then passing it through a Siamese network model containing a first image encoder to obtain a surface feature map to be detected.

[0034] Accordingly, in a specific example of this application, the second-level sub-unit of the convolutional encoding of the image to be detected is used to pass the sequence of image patches to be detected through a first convolutional neural network model acting as a filter to obtain multiple feature vectors of the image patches to be detected. It should be understood that the convolutional neural network model can gradually extract features at different levels of image patches through multiple layers of convolution and pooling operations. By using multiple convolutional layers, each layer can capture features at different scales and levels of abstraction. Therefore, the feature vectors obtained through multiple convolutional layers can provide richer and more diverse feature representations. The convolutional neural network model can learn more complex and abstract feature representations by automatically adjusting the weights of the convolutional kernels during training. This allows it to learn more suitable feature representations according to a specific detection task, thereby improving the accuracy and robustness of defect detection. By stacking multiple convolutional layers, the convolutional neural network model can capture the correlation between image patch features. The output of each convolutional layer can be seen as a higher-level representation of the features of the previous layer, containing more contextual information and semantic relationships. The multiple feature vectors obtained in this way can better describe the spatial structure of the image patches and the relationships between local features. By using multiple convolutional layers, the size of the feature map can be gradually reduced, thereby reducing the dimensionality of the feature vectors. This reduces storage and computational requirements, improving algorithm efficiency. By passing the sequence of image patches to be detected through a first convolutional neural network model acting as a filter, multiple feature vectors for each patch can be obtained. This allows for the extraction of multi-level feature representations, learning more complex feature representations, strengthening the correlation between features, reducing feature dimensionality, and improving computational efficiency. This provides richer and more diverse feature representations for subsequent defect detection tasks, improving detection accuracy and efficiency.

[0035] Specifically, the sequence of image blocks to be detected is passed through a first convolutional neural network model, which acts as a filter, to obtain multiple feature vectors of the image blocks to be detected. This includes: performing convolution processing on the input data to obtain a convolutional feature map; pooling each feature matrix along the channel dimension of the convolutional feature map to obtain a pooled feature map; and performing nonlinear activation on the pooled feature map to obtain an activation feature map. The output of the last layer of the first convolutional neural network model is the multiple feature vectors of the image blocks to be detected, and the input of the first layer of the first convolutional neural network model is each image block to be detected in the sequence of image blocks to be detected.

[0036] Accordingly, in another specific example of this application, the first Siamese network encoding secondary subunit is used to arrange the feature vectors of the multiple image blocks to be detected into a global feature matrix of the image to be detected, and then pass it through a Siamese network model containing a first image encoder to obtain a feature map of the surface to be detected. It should be understood that the image to be detected is usually composed of multiple image blocks, and there are spatial relationships between these image blocks. By arranging the feature vectors of the image blocks into a global feature matrix, their relative positions and layout information can be preserved. This is crucial for characterizing features such as texture, shape, and structure of the surface to be detected. The feature vectors of image blocks usually only provide local information and cannot capture the contextual information of the entire image. By combining the feature vectors into a global feature matrix, the content and context of the entire image can be comprehensively considered, thereby more comprehensively describing the features of the surface to be detected. This helps improve the accuracy and robustness of defect detection. The Siamese network model containing the first image encoder can further extract high-level feature representations from the global feature matrix. This model structure can maintain symmetry by sharing parameters, making the processing of the image blocks to be detected and the global feature matrix consistent. Through the encoder's processing, we can capture more abstract and semantic features, which helps to better distinguish between defective and non-defective regions. Therefore, by arranging the feature vectors of multiple image blocks to be detected into a global feature matrix and processing it through a Siamese network model containing the first image encoder, we can comprehensively consider spatial relationships and global context information to extract richer feature representations, thereby improving the representation capability of the surface feature map to be detected and the accuracy of defect detection.

[0037] Specifically, after arranging the feature vectors of the multiple image blocks to be detected into a global feature matrix of the image to be detected, a surface feature map to be detected is obtained by using a Siamese network model containing a first image encoder. This includes: using each hybrid convolutional layer of the first image encoder to perform multi-scale convolution processing, pooling processing, and nonlinear activation processing on the input data during the forward pass of the layer, so that the surface feature map to be detected is output by the last hybrid convolutional layer of the first image encoder.

[0038] Specifically, in another specific example of this application, the normal image encoding subunit 1222 is used to perform convolutional encoding on the normal image patch sequence to obtain a normal surface feature map. It should be understood that convolutional encoding is an effective feature extraction method that can extract useful local and global features from an input image. By inputting the normal image patch sequence into the convolutional encoder, convolutional operations can be used to capture features such as texture, edges, and shape in the image. These features are crucial for representing normal surfaces and can help us distinguish between normal and abnormal samples. Normal image patch sequences typically have high dimensionality and contain a large amount of redundant information. Through convolutional encoding, high-dimensional input data can be mapped to a low-dimensional feature space. The normal surface feature map obtained through convolutional encoding can be used to model the distribution of normal samples. These feature maps contain the commonalities and general features of normal samples and can be used to construct models of normal samples or generate the distribution of normal samples. Thus, in subsequent defect detection tasks, we can compare the sample to be detected with the model of normal samples to determine whether it has defects.

[0039] It is worth mentioning that, in other specific examples of this application, the normal image patch sequence can also be convolutionally encoded in other ways to obtain normal surface feature maps. For example, a normal image patch sequence is prepared as input data. Each image patch can be resized to the same size and preprocessed (e.g., normalized) to make it suitable for a convolutional neural network. A convolutional neural network is constructed as an encoder to convert the normal image patch sequence into normal surface feature maps. A convolutional neural network typically consists of convolutional layers, pooling layers, and fully connected layers. A series of convolutional kernels are used to perform convolution operations on the input image patches to extract local features. Each convolutional kernel generates a feature map representing different features. Pooling operations (e.g., max pooling or average pooling) reduce the spatial size of the feature map while retaining important feature information. The pooled feature map is flattened into a one-dimensional vector and further feature extraction and mapping are performed through fully connected layers. The normal image patch sequence is used as training data, and the convolutional encoder is trained using a backpropagation algorithm. Loss functions such as mean squared error or cross-entropy can be used to measure the difference between the output feature map and the true normal surface feature map, and network weights can be updated using optimization algorithms such as stochastic gradient descent. After training, the trained convolutional encoder is used to perform forward propagation on the sequence of normal image patches to obtain a normal surface feature map as the output. This feature map can capture important features and patterns of normal surfaces.

[0040] Specifically, the normal image coding subunit includes: a normal image convolutional coding secondary subunit, used to pass the normal image block sequence through a second convolutional neural network model as a filter to obtain multiple normal image block feature vectors; and a second Siamese network coding secondary subunit, used to arrange the multiple normal image block feature vectors into a normal image global feature matrix and then pass them through a Siamese network model containing a second image encoder to obtain a normal surface feature map.

[0041] Accordingly, in another specific example of this application, the normal image convolutional encoding secondary subunit is used to pass the normal image patch sequence through a second convolutional neural network model acting as a filter to obtain multiple normal image patch feature vectors. It should be understood that different convolutional kernels capture features at different scales when performing convolution operations on input image patches. For example, smaller convolutional kernels can extract local detail features, while larger convolutional kernels can capture broader contextual information. By using multiple convolutional kernels, features at different scales can be obtained simultaneously, thus providing a more comprehensive description of the features of normal image patches. Each convolutional kernel learns a limited number of features during the convolution process; they typically focus on certain specific shapes, textures, or edges. By using multiple convolutional kernels, each kernel can learn different semantic features, such as edges, textures, and shapes. This results in richer and more diverse normal image patch features, improving the representation ability of normal surfaces. By using multiple convolutional kernels, each normal image patch can obtain multiple feature vectors, each feature vector corresponding to one convolutional kernel. These feature vectors have different semantic and scale features, providing descriptions of normal image patches at different levels and angles. This diversity helps to represent the features of normal surfaces more comprehensively, improving the accuracy and robustness of subsequent tasks (such as anomaly detection). Therefore, by passing the sequence of normal image patches separately through a second convolutional neural network model acting as a filter, multiple feature vectors of normal image patches can be obtained, thus achieving multi-scale and multi-semantic feature representations. This improves the ability to represent normal surfaces and provides richer and more diverse features for subsequent tasks.

[0042] Specifically, the normal image patch sequence is passed through a second convolutional neural network model, which acts as a filter, to obtain multiple normal image patch feature vectors, including: defining a normal image patch sequence X = [X1, X2, ..., X...]. N ], where X i Let represent the i-th normal image patch; the second convolutional neural network model is represented as a function f(·), which takes a normal image patch as input and outputs the corresponding feature vector. That is, f(X) = f(·). iThe expression represents the feature vector obtained after processing the i-th normal image patch through model f. The feature vector is obtained by iterating through each normal image patch sequence X and inputting it into the second convolutional neural network model f. This can be represented as: Feature vector sequence V = [V1, V2, ..., V...]. N ], where V1=f(X i Let Vi represent the feature vector of the i-th normal image patch. Through the above steps, we obtain the feature vectors of N normal image patches. These feature vectors represent the representation of normal image patches in the second convolutional neural network model. We can represent these feature vectors as a feature matrix, where each row corresponds to the feature vector of a normal image patch. This can be expressed as: Feature matrix F = [V1, V2, ..., Vi]N. N ], where V i It is the feature vector of the i-th normal image patch.

[0043] Accordingly, in another specific example of this application, the second Siamese network encoding secondary subunit is used to arrange the feature vectors of the multiple normal image blocks into a global feature matrix of the normal image, and then pass it through a Siamese network model containing a second image encoder to obtain a normal surface feature map. It should be understood that by arranging the feature vectors of multiple normal image blocks into a global feature matrix, the relative positional relationships and global contextual information between the normal image blocks can be preserved. This global contextual information is crucial for understanding the overall pattern and structure of the normal surface. By introducing a second image encoder into the Siamese network model, its learning ability and parameter sharing can be utilized to extract global features. After arranging the feature vectors of multiple normal image blocks into a global feature matrix, the global features can be further integrated and mapped through the second image encoder Siamese network model. The Siamese network model is a structure with shared weights, which can jointly learn the feature representation of the normal surface during training. By introducing a second image encoder, the network weights can be optimized through backpropagation, enabling it to better capture the features and patterns of the normal surface. By arranging the feature vectors of multiple normal image blocks into a global feature matrix and extracting the normal surface feature map through the second image encoder Siamese network model, the robustness and generalization ability of the model can be improved. The global feature matrix contains more comprehensive and richer feature information of normal image patches, while the Siamese network model, by sharing weights and parameters, can better adapt to different normal surface samples and extract universal feature representations. Therefore, by arranging the feature vectors of multiple normal image patches into a global feature matrix for normal images, and by using a Siamese network model that includes a second image encoder, local features can be integrated into global features, capturing the overall pattern and structure of normal surfaces. This method can improve the model's robustness, generalization ability, and representation ability of normal surfaces.

[0044] Specifically, the first image encoder and the second image encoder have the same network structure.

[0045] In this embodiment, the feature map construction module 130 is used to construct a pen barrel surface difference feature map between the feature map of the surface to be detected and the feature map of the normal surface, and to optimize it to obtain an optimized pen barrel surface difference feature map. It should be understood that by calculating the difference between the feature map of the surface to be detected and the feature map of the normal surface, we can obtain a feature map representing the difference between the two. This difference may be caused by damage, defects, or other anomalies. By analyzing the difference feature map, we can detect and locate potentially abnormal areas, thereby enabling subsequent processing and judgment. The difference operation can compare the features of the normal surface with the features of the surface to be detected, highlighting the changes and differences between the two. By calculating the difference feature map, we can focus our attention on these changing areas, making it easier to discover and analyze potential problems. Normal surfaces typically contain some inherent textures and structures that are unrelated to the pen barrel features of the surface to be detected. By calculating the difference feature map, we can reduce or eliminate these background interferences, making anomalies or changes more obvious and visible. Therefore, by calculating the positional difference between the feature map of the surface to be detected and the feature map of the normal surface, we can obtain the surface difference feature map of the pen barrel. This feature map highlights the differences and changes between the surface to be detected and the normal surface, which helps in anomaly detection, localization and analysis.

[0046] Specifically, in this embodiment, the feature map construction module includes: a difference calculation unit, used to calculate the positional difference between the surface feature map to be detected and the normal surface feature map to obtain a pen barrel surface difference feature map; and an optimization unit, used to extract the latent feature representation of the motion distribution model of the pen barrel surface difference feature map relative to the target classification function to obtain an optimized pen barrel surface difference feature map.

[0047] Accordingly, the positional difference between the surface feature map to be detected and the normal surface feature map is calculated to obtain a pen barrel surface difference feature map, used to: calculate the pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map using the following formula, wherein the formula is:

[0048]

[0049] Among them, F a This represents the feature map of the surface to be detected. F represents the difference based on position. b This represents the normal surface feature map, and F n This represents the differential feature map of the pen barrel surface.

[0050] It is worth mentioning that, in other specific examples of this application, the penholder surface difference feature map can also be obtained in other ways. For example, for the surface to be detected and the normal surface, their feature vectors need to be extracted first. The feature vectors can contain various numerical or statistical quantities describing surface features, such as color, texture, shape, etc. These feature vectors can be extracted through image processing and computer vision techniques. Then, the feature vectors of the surface to be detected and the feature vectors of the normal surface are differentially analyzed. This can be done by subtracting corresponding positions one by one, or by using other difference methods, such as calculating Euclidean distance or cosine similarity. This yields a set of feature vector differences representing the differences between the surface to be detected and the normal surface. Next, the feature vector differences are aggregated to obtain the penholder surface difference feature map. The numerical values ​​of the feature vector differences can be weighted averaged, summed, or otherwise aggregated to obtain a global difference feature map. This feature map can reflect the differences and changes between the entire penholder surface and the normal surface. Finally, the obtained penholder surface difference feature map is input into a classifier for classification. The classifier can be various machine learning models, such as support vector machines (SVM), decision trees, neural networks, etc. By training a classifier using known normal and abnormal samples, classification rules and decision boundaries can be established to classify the surface of the pen barrel to be tested.

[0051] Specifically, in the technical solution of this application, after image block processing, the resulting sequence of image blocks to be detected and the sequence of normal image blocks are respectively extracted using a convolutional neural network model to extract feature vectors. These feature vectors are then arranged into a global feature matrix, and surface feature maps are further extracted using an image encoder Siamese network model. Finally, by calculating the positional difference between the surface feature map to be detected and the normal surface feature map, a pen barrel surface difference feature map is obtained. Improving the compatibility of local features along the correlation dimension with the desired distribution within the overall feature map is to ensure consistency and correlation between local features and overall features in the feature map. If there are inconsistencies or irrelevance between local features and overall features in the feature map, it may lead to inaccurate classification results. By improving the compatibility between local and overall features, relevant information in the feature map can be enhanced, and irrelevant noise can be reduced. This allows for better capture of subtle changes and features on the pen barrel surface, improving the classifier's ability to judge whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel. In other words, by extracting the latent feature representation of the motion distribution model of the pen barrel surface difference feature map relative to the target classification function, the compatibility of the feature local of the pen barrel surface difference feature map with the expected distribution along the correlation dimension within the feature whole is improved, thereby improving the accuracy and reliability of the classification results, and thus better determining whether the pen barrel surface to be detected meets the predetermined requirements of a normal pen barrel.

[0052] Specifically, to improve the classification ability of the penholder surface difference feature map, the technical solution of this application uses the latent feature representation of the motion distribution model of the penholder surface difference feature map relative to the target classification function to replace or supplement the original features. Specifically, the motion distribution model is a probabilistic model that assumes each point in the feature space is generated by a random variable whose distribution is determined by the gradient of the target classification function. The latent feature representation of the motion distribution model refers to the potential motion state of each point in the feature space, which can reflect the motion information in the feature map, i.e., the relative changes between features.

[0053] The steps for extracting the latent feature representation of the motion distribution model are as follows: First, calculate the gradient value of the target classification function corresponding to the feature value at each position in the differential feature map of the pen surface, which serves as the motion direction. Then, based on the magnitude and direction of the gradient value, divide the feature space into several motion regions, representing different motion modes. For each motion region, fit the distribution of gradient values ​​to obtain the parameters of the motion distribution model. Next, for each motion region, use Cauchy normalization or other methods to eliminate the influence of outliers or noise, making the motion distribution model more stable. Then, for each motion region, use the parameters of the motion distribution model or other features to represent the motion information, serving as the latent feature representation of the motion distribution model. Finally, concatenate the latent feature representations of each motion region to obtain the latent feature representation of the motion distribution model for the entire feature map.

[0054] In one embodiment of this application, the optimization unit is configured to: extract the latent feature representation of the motion distribution model of the penholder surface difference feature map relative to the target classification function using the following formula to obtain an optimized penholder surface difference feature map; wherein, the formula is:

[0055]

[0056] Among them, f i,j,k The eigenvalue at position (i,j,k) of the differential feature map on the pen barrel surface is represented by _log_, where _log_ represents the base-2 logarithmic function value, _softmax_ represents the normalized exponential function, and _f_ represents the eigenvalue. i,j,k ′ represents the feature value at position (i,j,k) of the optimized pen barrel surface difference feature map.

[0057] In this way, by extracting the latent feature representation of the motion distribution model of the penholder surface difference feature map relative to the target classification function, higher-level features in the penholder surface difference feature map can be extracted to improve the consistency and robustness of its feature distribution, adapt to changes in different scales and angles, maintain the invariance and distinguishability of features, and thus improve classification ability.

[0058] In this embodiment, the result generation module 140, based on the optimized pen barrel surface difference feature map, determines whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel. It should be understood that by calculating the optimized pen barrel surface difference feature map, we can capture the differences and variations between the surface to be tested and the normal surface. These differences often correspond to potential defects, anomalies, or problems. Inputting the difference feature map into a classifier allows for further extraction and expression of the features of these differences, enabling more accurate classification and judgment. A classifier is a machine learning model that, after training, can learn the differences and patterns between different categories. By inputting the optimized pen barrel surface difference feature map into the classifier, the classifier can learn and understand the relationship between these features and the predetermined requirements of a normal pen barrel. It can establish classification rules and decision boundaries using normal and abnormal samples in the training data, thereby classifying the surface of the pen barrel to be tested. The output of the classifier indicates whether the surface of the pen barrel to be tested is classified as normal or abnormal. This classification result can be used to indicate whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel. If the classification result is normal, it means that the surface of the pen barrel to be tested is similar to the normal surface and meets the predetermined requirements. If the classification result is abnormal, it means that the surface of the pen barrel to be tested differs from the normal surface and may not meet the predetermined requirements, requiring further processing or inspection.

[0059] Specifically, in one embodiment of this application, the result generation module is used to: pass the optimized pen barrel surface difference feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel. Specifically, the classifier is used to process the optimized pen barrel surface difference feature map using the following formula to obtain the classification result; wherein, the formula is: O=softmax{(W c B c )|Project(F)}, where Project(F) represents projecting the optimized pen barrel surface difference feature map into a vector, W c Let B be the weight matrix. c represents the bias vector, softmax represents the normalization exponential function, and O represents the classification result.

[0060] In summary, this application also provides a system architecture diagram, specifically as follows: Figure 4 As shown. Figure 4This is a schematic diagram of the architecture of a machine vision-based pen barrel surface defect detection system according to an embodiment of this application. In this system architecture, firstly, surface images of the pen barrel to be detected and normal pen barrels are acquired. Secondly, the surface images of the pen barrel to be detected and the normal pen barrels are processed into image blocks to obtain a sequence of image blocks to be detected and a sequence of normal image blocks. Then, the sequence of image blocks to be detected is passed through a first convolutional neural network model as a filter to obtain multiple feature vectors of image blocks to be detected. Next, the sequence of normal image blocks is passed through a second convolutional neural network model as a filter to obtain multiple feature vectors of normal image blocks. Then, the multiple feature vectors of image blocks to be detected are arranged into a global feature matrix of the image to be detected and passed through a Siamese network model containing a first image encoder to obtain a surface feature map to be detected. Next, the multiple feature vectors of normal image blocks are arranged into a global feature matrix of the normal image and passed through a Siamese network model containing a second image encoder to obtain a normal surface feature map. Finally, the positional difference between the surface feature map to be detected and the normal surface feature map is calculated to obtain a pen barrel surface difference feature map. Next, the latent feature representation of the motion distribution model of the pen barrel surface difference feature map relative to the target classification function is extracted to obtain an optimized pen barrel surface difference feature map. Finally, the optimized pen barrel surface difference feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel.

[0061] As described above, the machine vision-based pen barrel surface defect detection system 100 according to the embodiments of this application can be implemented in various terminal devices, such as servers for the machine vision-based pen barrel surface defect detection system. In one example, the machine vision-based pen barrel surface defect detection system 100 can be integrated into the terminal device as a software module and / or hardware module. For example, the machine vision-based pen barrel surface defect detection system 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the machine vision-based pen barrel surface defect detection system 100 can also be one of many hardware modules of the terminal device.

[0062] Alternatively, in another example, the machine vision-based pen barrel surface defect detection system 100 and the terminal device can also be separate devices, and the machine vision-based pen barrel surface defect detection system 100 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0063] Exemplary methods

[0064] Figure 5This is a flowchart of a machine vision-based pen barrel surface defect detection method according to an embodiment of this application. Figure 5 As shown, the machine vision-based pen barrel surface defect detection method according to an embodiment of this application includes: S110, acquiring a surface image of a pen barrel to be detected and a surface image of a normal pen barrel; S120, extracting features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain a surface feature map to be detected and a normal surface feature map; S130, constructing a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimizing it to obtain an optimized pen barrel surface difference feature map; S140, based on the optimized pen barrel surface difference feature map, determining whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel.

[0065] Here, those skilled in the art will understand that the specific operations of each step in the above-described machine vision-based pen barrel surface defect detection method have been referenced above. Figures 1 to 4 The description of the machine vision-based pen barrel surface defect detection system is detailed here, and therefore, its repeated description will be omitted.

[0066] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the module division is only a logical functional division, and other division methods may be used in actual implementation.

[0067] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0070] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0071] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim can also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A machine vision-based pen barrel surface defect detection system, characterized in that, include: The image acquisition module is used to acquire surface images of the pen barrel to be detected and surface images of a normal pen barrel; The image processing module is used to extract features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain the surface feature map to be detected and the normal surface feature map; The feature map construction module is used to construct a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimize it to obtain an optimized pen barrel surface difference feature map. The result generation module is used to determine whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel based on the optimized pen barrel surface differential feature map.

2. The pen barrel surface defect detection system based on machine vision according to claim 1, characterized in that, The image processing module includes: The block segmentation unit is used to perform image block processing on the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain the image block sequence to be detected and the normal image block sequence. An image block encoding unit is used to extract surface feature maps to be detected and normal surface feature maps from the image block sequence to be detected and the normal image block sequence.

3. The pen barrel surface defect detection system based on machine vision according to claim 2, characterized in that, The image block coding unit includes: The image encoding subunit is used to perform convolutional encoding on the image block sequence to obtain the surface feature map to be detected. A normal image coding subunit is used to perform convolutional coding on the normal image block sequence to obtain a normal surface feature map.

4. The pen barrel surface defect detection system based on machine vision according to claim 3, characterized in that, The image encoding subunit to be detected includes: The image to be detected is a second-level convolutional coding subunit, which is used to pass the sequence of image blocks to be detected through a first convolutional neural network model as a filter to obtain multiple feature vectors of image blocks to be detected; The first twin network encoding secondary subunit is used to arrange the feature vectors of the multiple image blocks to be detected into a global feature matrix of the image to be detected, and then obtain the surface feature map to be detected by passing it through a twin network model containing the first image encoder.

5. The pen barrel surface defect detection system based on machine vision according to claim 4, characterized in that, The normal image coding subunit includes: A normal image convolutional coding second-level subunit is used to pass the normal image block sequence through a second convolutional neural network model as a filter to obtain multiple normal image block feature vectors; The second twin network encoding secondary subunit is used to arrange the feature vectors of the multiple normal image blocks into a normal image global feature matrix, and then obtain a normal surface feature map by passing it through a twin network model containing a second image encoder.

6. The pen barrel surface defect detection system based on machine vision according to claim 5, characterized in that, The first image encoder and the second image encoder have the same network structure.

7. The pen barrel surface defect detection system based on machine vision according to claim 6, characterized in that, The feature map construction module includes: A differential calculation unit is used to calculate the positional difference between the surface feature map to be detected and the normal surface feature map to obtain the pen barrel surface differential feature map; An optimization unit is used to extract the latent feature representation of the motion distribution model of the pen barrel surface difference feature map relative to the target classification function to obtain an optimized pen barrel surface difference feature map.

8. The pen barrel surface defect detection system based on machine vision according to claim 7, characterized in that, The optimization unit is configured to: extract the latent feature representation of the motion distribution model of the penholder surface difference feature map relative to the target classification function using the following formula to obtain an optimized penholder surface difference feature map; wherein, the formula is: Among them, f i,j,k The eigenvalue at position (i,j,k) of the differential feature map on the pen barrel surface is represented by _log_, where _log_ represents the base-2 logarithmic function value, _softmax_ represents the normalized exponential function, and _f_ represents the eigenvalue. i,j,k ′ represents the feature value at position (i,j,k) of the optimized pen barrel surface difference feature map.

9. The pen barrel surface defect detection system based on machine vision according to claim 8, characterized in that, The result generation module is used to: pass the optimized pen barrel surface differential feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the surface of the pen barrel to be detected meets the predetermined requirements of a normal pen barrel.

10. A method for detecting surface defects in pen barrels based on machine vision, characterized in that, include: Acquire surface images of the pen barrel to be inspected and a normal pen barrel; Extract features from the surface image of the pen barrel to be detected and the surface image of the normal pen barrel to obtain the surface feature map to be detected and the normal surface feature map; Construct a pen barrel surface difference feature map between the surface feature map to be detected and the normal surface feature map, and optimize it to obtain an optimized pen barrel surface difference feature map; Based on the optimized pen barrel surface differential feature map, it is determined whether the surface of the pen barrel to be tested meets the predetermined requirements of a normal pen barrel.

Images