Intelligent detection system and method based on machine vision

Abstract

The present application relates to the field of intelligent detection, and specifically discloses an intelligent detection system and method based on machine vision, which first obtains a multi-angle image of a polished Hu brush barrel collected by a camera and a spectral image of the polished Hu brush barrel collected by a camera, then passes the multi-angle image of the Hu brush barrel after polishing through a multi-angle pen feature extractor and calculates the difference, and then passes through an angle difference feature extractor based on a deep convolutional neural network to obtain a pen barrel associated feature map, then, pre-processes the spectral image of the Hu brush barrel after polishing and passes through a channel attention mechanism to obtain a pen barrel spectral feature map, finally, the pen barrel associated feature map and the pen barrel spectral feature map are fused and passed through a classifier to determine whether the quality of the polished pen barrel is qualified, thereby improving the accuracy of quality detection, thereby improving production efficiency and the controllability of product quality.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to an intelligent detection system and method based on machine vision. Background Art

[0002] Hu Bi, also known as Huzhou Brush, is one of the four traditional writing tools. Hu Bi is famous for its unique craftsmanship and excellent writing performance. It uses high-quality mixed hair as the nib and is made through multiple complicated manual processes. The nib of Hu Bi is soft and elastic, which can flexibly control the flow of ink, making writing more fluent and free. Its handwriting is full and round, which can express the unique charm and personality of calligraphy. As an important part of traditional culture, Hu Bi carries the writing tradition and wisdom of thousands of years and is widely used in calligraphy, painting and cultural and artistic creation.

[0003] The production process of Hu brushes is delicate, including material selection, sharpening, adjustment and assembly. Among them, polishing the pen holder is one of the key steps. Craftsmen use meticulous frosting to make the surface of the pen holder smooth and delicate, increasing the comfort and aesthetics of holding. In the traditional process, the inspection of the polishing defects of the Hu brush pen holder requires workers to spend a long time to carefully observe and evaluate the polishing quality of each product, which limits the production speed and capacity. At the same time, manual inspection requires a lot of human resources and may not fully cover every detail and corner of each product. Some hidden or difficult-to-observe areas may be ignored, resulting in quality problems not being discovered.

[0004] Therefore, an intelligent detection system and method based on machine vision is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent detection system and method based on machine vision, which first obtains a multi-angle image of the polished pen shaft of the Hu pen collected by a camera and a spectral image of the polished pen shaft of the Hu pen collected by a camera, and then passes the multi-angle image of the polished pen shaft of the Hu pen through a multi-angle pen shaft feature extractor and calculates the difference and then passes through an angle difference feature extractor based on a deep convolutional neural network to obtain a pen shaft associated feature map, then, the spectral image of the polished pen shaft of the Hu pen is preprocessed and passed through a channel attention mechanism to obtain a pen shaft spectral feature map, and finally, the pen shaft associated feature map and the pen shaft spectral feature map are fused and passed through a classifier to determine whether the quality of the polished pen shaft is qualified, thereby improving the accuracy of quality detection, thereby improving production efficiency and controllability of product quality.

[0006] According to one aspect of the present application, there is provided an intelligent detection system based on machine vision, comprising:

[0007] A module for acquiring images of the Hu brush after polishing, used for acquiring multi-angle images of the Hu brush shaft after polishing acquired by the camera and a spectral image of the Hu brush shaft after polishing acquired by the camera;

[0008] A Hu brush image feature extraction module is used to extract a brush shaft correlation feature map and a brush shaft spectral feature map from the multi-angle image of the Hu brush shaft after polishing collected by the camera and the spectral image of the Hu brush shaft after polishing collected by the camera;

[0009] The polishing quality result generation module is used to judge whether the quality of the pen holder after polishing is qualified based on the pen holder association feature map and the pen holder spectrum feature map.

[0010] In the above-mentioned intelligent detection system based on machine vision, the polishing quality result generation module includes: a defect construction unit, which is used to fuse the pen shaft association feature map and the pen shaft spectral feature map based on the smooth parameterized expression of the output relative objective function to obtain a pen shaft polishing defect feature map; a defect result generation unit, which is used to pass the pen shaft polishing defect feature map through a classifier to obtain a classification result, and the classification result is used to judge whether the quality of the pen shaft after polishing is qualified.

[0011] In the above-mentioned intelligent detection system based on machine vision, the defect construction unit includes: calculating the smooth parameterized expression factor of the pen shaft associated feature map relative to the pen shaft spectral feature map; using the smooth parameterized expression factor as a weight to calculate the position-weighted sum of the pen shaft associated feature map and the pen shaft spectral feature map to obtain the pen shaft polishing defect feature map.

[0012] In the above-mentioned intelligent detection system based on machine vision, the defect construction unit includes: calculating the smooth parameterized expression factor of the penholder associated characteristic graph relative to the penholder spectral characteristic graph using the following smooth parameterized expression factor formula; wherein the smooth parameterized expression factor formula is:

[0013]

[0014] Among them, F1 represents the pen shaft association feature map, F2 represents the pen shaft spectral feature map, d(F1, F2) represents the Euclidean distance between the pen shaft association feature map and the pen shaft spectral feature map, cos(F1, F2) represents the cosine similarity between the pen shaft association feature map and the pen shaft spectral feature map, α and β represent predetermined hyperparameters, and w represents the smoothing parameterized expression factor.

[0015] In the above-mentioned intelligent detection system based on machine vision, the defect construction unit includes: calculating the Euclidean distance between the penholder association feature map and the penholder spectrum feature map by the following Euclidean distance formula;

[0016] Wherein, the Euclidean distance formula is:

[0017]

[0018] Among them, F 1k,l,m represents the feature value of the (k, l, m)th position of the pen-related feature map, F 2k,l,m represents the eigenvalue of the (k, l, m)th position of the penholder spectrum feature map, W, H, C represent the width, height and number of channels of the feature map respectively, and d(F1, F2) represents the Euclidean distance between the penholder association feature map and the penholder spectrum feature map.

[0019] In the above-mentioned intelligent detection system based on machine vision, the defect construction unit includes: calculating the cosine similarity between the penholder association feature map and the penholder spectrum feature map using the following cosine similarity formula;

[0020] Wherein, the cosine similarity formula is:

[0021]

[0022] Among them, F 1k,l,m represents the feature value of the (k, l, m)th position of the pen-related feature map, F 2k,l,m represents the eigenvalue of the (k, l, m)th position of the penholder spectrum feature map, W, H, C represent the width, height and number of channels of the feature map respectively, and cos(F1, F2) represents the cosine similarity between the penholder association feature map and the penholder spectrum feature map.

[0023] According to another aspect of the present application, an intelligent detection method based on machine vision is provided, which includes: acquiring multi-angle images of the Hu brush pen shaft after polishing collected by a camera and a spectral image of the Hu brush pen shaft after polishing collected by the camera; extracting a pen shaft association feature map and a pen shaft spectral feature map from the multi-angle images of the Hu brush pen shaft after polishing collected by the camera and the spectral image of the Hu brush pen shaft after polishing collected by the camera; based on the pen shaft association feature map and the pen shaft spectral feature map, judging whether the quality of the polished pen shaft is qualified.

[0024] Compared with the prior art, the present application provides an intelligent detection system and method based on machine vision, which first obtains multi-angle images of the polished pen shaft of the Hu brush collected by a camera and a spectral image of the polished pen shaft of the Hu brush collected by the camera, then passes the multi-angle images of the polished pen shaft of the Hu brush through a multi-angle pen shaft feature extractor and calculates the difference, and then passes through an angle difference feature extractor based on a deep convolutional neural network to obtain a pen shaft associated feature map, then, pre-processes the spectral image of the polished pen shaft of the Hu brush and passes through a channel attention mechanism to obtain a pen shaft spectral feature map, finally, the pen shaft associated feature map and the pen shaft spectral feature map are fused and passed through a classifier to determine whether the quality of the polished pen shaft is qualified, thereby improving the accuracy of quality detection, thereby improving production efficiency and the controllability of product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 4 is a block diagram of an intelligent detection system based on machine vision according to an embodiment of the present application.

[0027] Figure 2 It is a block diagram of the lake brush image feature extraction module in the machine vision-based intelligent detection system according to an embodiment of the present application.

[0028] Figure 3 It is a block diagram of a feature extraction unit for a polished pen shaft in a machine vision-based intelligent detection system according to an embodiment of the present application.

[0029] Figure 4 It is a block diagram of a polishing quality result generation module in a machine vision-based intelligent detection system according to an embodiment of the present application.

[0030] Figure 5 Schematic diagram of the architecture of an intelligent detection system based on machine vision according to an embodiment of the present application.

[0031] Figure 6 The figure is a flow chart of an intelligent detection method based on machine vision according to an embodiment of the present application.

[0032] Figure 7 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0034] Exemplary Systems

[0035] Figure 1 FIG. 1 is a block diagram of an intelligent detection system based on machine vision according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the intelligent detection system 100 based on machine vision includes: a module 110 for acquiring images of the Hu brush after polishing collected by a camera, which is used to acquire multi-angle images of the Hu brush shaft after polishing and a spectral image of the Hu brush shaft after polishing collected by the camera; a Hu brush image feature extraction module 120, which is used to extract a pen shaft association feature map and a pen shaft spectral feature map from the multi-angle images of the Hu brush shaft after polishing collected by the camera and the spectral image of the Hu brush shaft after polishing collected by the camera; a polishing quality result generation module 130, which is used to judge whether the quality of the polished pen shaft is qualified based on the pen shaft association feature map and the pen shaft spectral feature map.

[0036] In the above-mentioned intelligent detection system 100 based on machine vision, the polished Hu pen image acquisition module 110 is used to acquire the multi-angle image of the Hu pen after polishing collected by the camera and the spectral image of the Hu pen after polishing collected by the camera. It should be understood that the multi-angle image of the Hu pen after polishing collected by the camera can provide the appearance information of the Hu pen after polishing under different viewing angles. By collecting images at multiple angles, more comprehensive visual features can be obtained, including surface smoothness, concave-convex defects and other information, which can be used to judge the overall quality of the polished pen. The spectral image of the Hu pen after polishing collected by the camera can provide information about the optical properties of the Hu pen after polishing. Since the pen surfaces of different planes have different reflection and absorption characteristics under different wavelengths of light, by collecting spectral images, the spectral distribution of the pen at different wavelengths can be obtained, thereby extracting spectral features for judging the quality of the pen. By integrating the information of multi-angle images and spectral images, the quality of the polished pen can be more accurately evaluated and whether it is qualified.

[0037] In the above-mentioned intelligent detection system 100 based on machine vision, the lake pen image feature extraction module 120 is used to extract the pen shaft associated feature map and the pen shaft spectral feature map from the multi-angle image of the lake pen shaft after polishing collected by the camera and the spectral image of the lake pen shaft after polishing collected by the camera. It should be understood that through the feature extraction of multi-angle images, information about the smoothness and texture of the pen shaft surface can be obtained. The smooth pen shaft surface should have a consistent gloss, while the defective pen shaft surface may show uneven gloss, uneven or rough texture. The spectrum is a graphical representation that visualizes the relationship between the intensity of the light signal and its corresponding wavelength or frequency. It shows the absorption, emission or scattering of light of different wavelengths or frequencies by the material, and provides information about the composition, structure and properties of the material. The pen shaft surface with concave-convex defects will have different reflection and absorption characteristics under different wavelengths of light. The concave-convex defects may cause color changes or abnormal spectral characteristics of the pen shaft surface. By comprehensively utilizing the feature extraction results of multi-angle images and spectral images, a comprehensive evaluation of the smoothness and defects of the pen shaft can be obtained.

[0038] Figure 2 FIG. 1 is a block diagram of a lake brush image feature extraction module in an intelligent detection system based on machine vision according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, the lake brush image feature extraction module 120 includes: a polished pen feature extraction unit 121, which is used to extract features from the multi-angle images of the lake brush after polishing collected by the camera to obtain the pen shaft associated feature map; a spectral image preprocessing unit 122, which is used to perform data preprocessing on the spectral image of the lake brush after polishing collected by the camera to obtain a denoised pen shaft spectrum map; a denoised spectrum map feature extraction unit 123, which is used to obtain the pen shaft spectrum feature map through the channel attention mechanism. It should be understood that feature extraction can help detect defects or flaws on the pen shaft. By analyzing the features in the image, defects such as cracks, scratches, and bubbles on the surface of the pen shaft can be found, which helps to achieve quality control and timely detection of problems. Furthermore, when collecting spectral images, it may be interfered by factors such as ambient light and noise, resulting in noise points or unnecessary fluctuations in the image. These noises will interfere with subsequent spectral analysis and feature extraction, affecting the accuracy of the results. Therefore, by preprocessing the spectral image, the influence of noise can be reduced and cleaner and more reliable pen spectral information can be extracted. Common data preprocessing methods include noise filtering, spectral correction, background removal, etc. Applying filtering algorithms, such as median filtering and Gaussian filtering, can remove noise points in the image, thereby smoothing the spectral curve. Correcting the spectral image eliminates the spectral offset caused by light source fluctuations or uneven sensor response, so that the spectral curve more accurately reflects the spectral characteristics of the pen. If there is background interference in the spectral image, the background removal algorithm can be used to subtract the background signal from the spectrum to highlight the spectral characteristics of the pen. Through the preprocessing steps, the denoised pen spectrum can be obtained, which improves the accuracy and reliability of subsequent spectral analysis and feature extraction.

[0039] Furthermore, in the spectral image, the information of different channels (wavelengths) has different importance for describing the characteristics of the pen shaft. Some channels may contain more useful information, while other channels may contain noise or irrelevant information. Therefore, through the channel attention mechanism, the weights of different channels can be adjusted so that important channels get higher weights and unimportant channels get lower weights. The channel attention mechanism can be implemented by learning a set of channel weights, which can be done by using the attention mechanism in the deep learning model. By inputting the denoised pen shaft spectrum image, the model can learn the weight of each channel in order to highlight the important spectral features. Specifically, based on the channel attention mechanism, the input data is convolved to generate a channel convolution feature map; the convolution feature map is pooled to generate a channel pooling feature map; the pooling feature map is activated to generate a channel activation feature map; the quotient of the eigenvalue mean of the feature matrix corresponding to each channel in the channel activation feature map and the sum of the eigenvalue mean of the feature matrix corresponding to all channels is calculated as the weighting coefficient of the feature matrix corresponding to each channel; and the feature matrix of each channel is weighted by the weighting coefficient of each channel in the channel activation feature map to generate a channel attention feature map; wherein the output of the last layer of the channel attention mechanism is the pen shaft spectrum feature map.

[0040] Figure 3 FIG. 1 is a block diagram of a polished pen shaft feature extraction unit in a machine vision-based intelligent detection system according to an embodiment of the present application. Figure 3As shown, in a specific embodiment of the present application, the polished pen shaft feature extraction unit 121 includes: a spatial attention subunit 1211, which is used to pass the multi-angle images of the polished pen shaft of the lake brush collected by the camera through a multi-angle pen shaft feature extractor based on spatial attention to obtain a multi-angle pen shaft feature map; an associated feature encoding subunit 1212, which is used to perform feature encoding on the multi-angle pen shaft feature map to obtain the pen shaft associated feature map. It should be understood that the multi-angle pen shaft feature extractor based on spatial attention can improve the expressiveness of features by paying attention to different areas and details. It can apply the spatial attention mechanism to the image at each angle, automatically learn and emphasize the important areas related to the pen shaft, and through the multi-angle pen shaft feature extractor, feature maps of multiple angles can be obtained, and each feature map represents the key features in the corresponding angle image. Specifically, the multi-angle image of the polished pen shaft of the lake brush collected by the camera is subjected to deep convolution encoding using the convolution encoding part of the multi-angle pen shaft feature extractor based on spatial attention to obtain an initial convolution feature map; the initial convolution feature map is input into the spatial attention part of the multi-angle pen shaft feature extractor based on spatial attention to obtain a spatial attention map; the spatial attention map is activated by the Softmax function to obtain a spatial attention feature map; the spatial attention feature map and the initial convolution feature map are multiplied by the position points to obtain the multi-angle pen shaft feature map. More specifically, the initial convolution feature map is subjected to average pooling and maximum pooling along the channel dimension to obtain an average feature matrix and a maximum feature matrix; the average feature matrix and the maximum feature matrix are cascaded and channel-adjusted to obtain a channel feature matrix; the channel feature matrix is ​​convolution-encoded using the convolution layer of the spatial attention feature map to obtain a spatial attention map.

[0041] Furthermore, through feature encoding, key features related to the pen shaft can be extracted and encoded into a more compact, high-dimensional vector representation, namely, the pen shaft-related feature map. Considering that the multi-angle pen shaft feature map may have a high dimension, containing a lot of redundant and irrelevant information, through feature encoding, the dimension of the original feature map can be compressed to a lower dimension, reducing storage and computing costs.

[0042] In a specific embodiment of the present application, the associated feature encoding subunit 1212 includes: calculating the difference between the feature maps of every two pen shafts in the multi-angle pen shaft feature map to obtain multiple pen shaft differential feature maps; passing the multiple pen shaft differential feature maps through an angle difference feature extractor based on a deep convolutional neural network to obtain the pen shaft associated feature map. It should be understood that by calculating the difference between the feature maps, subtle changes and differences between different angles can be extracted, and these differential feature maps can be used to highlight the differences in pen shaft shape, smoothness or convexity at different angles. For example, if the pen shaft feature map of a certain angle shows obvious defect details in a certain area, while the feature maps of other angles show fewer defect details in the same area, then by calculating the differential feature map, we can more easily capture this difference, thereby better distinguishing pen shafts at different angles. Furthermore, each differential feature map represents a specific difference between different angles, thereby providing more details and change patterns to better capture defect features. Specifically, the difference between the feature maps of every two pen shafts in the multi-angle pen shaft feature map is calculated using the following differential formula, including:

[0043] Wherein, the difference formula is:

[0044]

[0045] Among them, F a and F b represents a characteristic diagram of each two angle pen holders in the multi-angle pen holder characteristic diagram, Indicates the difference by position, F n Represents the pen holder differential feature map.

[0046] Furthermore, the purpose of obtaining the pen-related feature map by passing multiple pen-barrel differential feature maps through the angle difference feature extractor based on the deep convolutional neural network is to capture the feature differences between different angles and extract the feature representation related to the pen-barrel quality. In the actual application scenario of the quality inspection of the lake brush polishing process, images at different angles may present different viewing angles, lighting conditions and shape changes, and these differences are very important for evaluating the quality and performance of the pen-barrel. By performing differential operations on images at multiple angles, the differences between different angles can be highlighted, making the relevant features more obvious. It should be understood that the angle difference feature extractor based on the deep convolutional neural network is a model that uses deep learning methods to extract feature differences between different angles. It can capture the differences and changes between images at different angles by learning effective feature representations. This feature extractor is usually composed of multiple convolutional layers, pooling layers, and fully connected layers. It accepts images at multiple angles as input and extracts local features in the image through the convolutional layer. The convolutional layer uses a series of filters (also called convolution kernels) to perform convolution operations on the input image to extract features of different scales and directions. In the angle difference feature extractor, multiple parallel convolution branches are usually used to process images at each angle. Each branch has the same network structure, but the parameters are learned independently, which ensures that each branch can learn the feature representation related to the image at its corresponding angle. After the convolution layer, a pooling layer is usually used to downsample and reduce the dimension of the features to reduce the number of parameters of the model and improve computational efficiency. Finally, the extracted features are mapped to specific output categories or feature representations for subsequent tasks through a fully connected layer. When training the angle difference feature extractor, supervised learning is usually performed using labeled data. By minimizing the loss function, the network can learn to map the differences between images at different angles to a specific feature representation space. The use of an angle difference feature extractor based on a deep convolutional neural network can effectively capture the feature differences between different angles. This method can automatically learn feature representations related to angle changes without manually designing feature extraction algorithms. Specifically, each layer of the angle difference feature extractor based on the deep convolutional neural network is used to perform convolution processing, pooling processing and non-linear activation processing on the input data in the forward pass of the layer so that the last layer of the angle difference feature extractor based on the deep convolutional neural network outputs the pen shaft associated feature map.

[0047] In the above-mentioned intelligent detection system 100 based on machine vision, the polishing quality result generation module 130 is used to judge whether the quality of the pen barrel after polishing is qualified based on the pen barrel associated feature map and the pen barrel spectral feature map. It should be understood that the combined use of the pen barrel associated feature map and the pen barrel spectral feature map can comprehensively consider the appearance and morphological characteristics and defect characteristics of the pen barrel. Such a comprehensive evaluation can more comprehensively judge whether the quality of the pen barrel after polishing is qualified. For example, if the pen barrel associated feature map shows that the pen barrel has an irregular shape or an uneven surface, then it can be judged that the quality of the pen barrel is unqualified.

[0048] Figure 4 FIG. 1 is a block diagram of a polishing quality result generation module in a machine vision-based intelligent detection system according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the polishing quality result generation module 130 includes: a defect construction unit 131, which is used to fuse the pen barrel associated feature map and the pen barrel spectral feature map based on the smooth parameterized expression of the output relative objective function to obtain a pen barrel polishing defect feature map; a defect result generation unit 132, which is used to pass the pen barrel polishing defect feature map through a classifier to obtain a classification result, and the classification result is used to judge whether the quality of the pen barrel after polishing is qualified. It should be understood that in the technical solution of the present application, considering that the pen barrel associated feature map is extracted from the multi-angle image based on the multi-angle pen barrel feature extractor and the angle difference feature extractor, it mainly includes the shape, texture and angle difference of the pen barrel. The pen barrel associated feature map focuses on the shape and texture information of the pen barrel, as well as the difference between different angles, so the data manifold in the feature space may be simpler and smoother, and the changes in shape and position are relatively small. The pen barrel spectral feature map is obtained by performing noise reduction and channel attention mechanism processing on the spectral image of the pen barrel, which mainly reflects the spectral distribution characteristics of the pen barrel. The penholder spectral feature map focuses on the spectral information of the penholder, so the data manifold in the feature space may be more complex, and the changes in shape and position are more sensitive. Therefore, simply fusing the penholder-associated feature map and the penholder spectral feature map may not fully utilize their differences in the feature space. This simple fusion may cause information between different feature maps to interfere with or be lost, thereby reducing the classification accuracy. In order to better utilize the information of the two feature maps, the penholder-associated feature map and the penholder spectral feature map are fused based on a smooth parameterized expression of the output relative objective function to retain and fully utilize the information of the two feature maps, thereby improving the accuracy of classification. In this way, the shape, texture and angle difference characteristics of the penholder, as well as the spectral distribution characteristics, can be better captured, thereby improving the classification accuracy and judgment accuracy of the quality of the polished penholder.

[0049] Specifically, the pen shaft associated feature map and the pen shaft spectral feature map are fused based on the smooth parameterized expression of the output relative objective function to obtain the pen shaft polishing defect feature map, including: calculating the smooth parameterized expression factor of the pen shaft associated feature map relative to the pen shaft spectral feature map; using the smooth parameterized expression factor as a weight to calculate the position-weighted sum of the pen shaft associated feature map and the pen shaft spectral feature map to obtain the pen shaft polishing defect feature map.

[0050] The smooth parameterized expression factor of the penholder-related characteristic graph relative to the penholder spectrum characteristic graph is calculated using the following smooth parameterized expression factor formula; wherein the smooth parameterized expression factor formula is:

[0051]

[0052] Among them, F1 represents the pen shaft association feature map, F2 represents the pen shaft spectral feature map, d(F1, F2) represents the Euclidean distance between the pen shaft association feature map and the pen shaft spectral feature map, cos(F1, F2) represents the cosine similarity between the pen shaft association feature map and the pen shaft spectral feature map, α and β represent predetermined hyperparameters, and w represents the smoothing parameterized expression factor.

[0053] The cosine similarity between the penholder association feature graph and the penholder spectrum feature graph is expressed as:

[0054]

[0055] Among them, F 1k,l,m represents the feature value of the (k, l, m)th position of the pen-related feature map, F 2k,l,m represents the eigenvalue of the (k, l, m)th position of the penholder spectrum feature map, W, H, C represent the width, height and number of channels of the feature map respectively, and cos(F1, F2) represents the cosine similarity between the penholder association feature map and the penholder spectrum feature map;

[0056] Among them, the Euclidean distance is expressed as:

[0057]

[0058] Among them, F 1k,l,m represents the feature value of the (k, l, m)th position of the pen-related feature map, F 2k,l,m represents the eigenvalue of the (k, l, m)th position of the penholder spectrum feature map, W, H, C represent the width, height and number of channels of the feature map respectively, and d(F1, F2) represents the Euclidean distance between the penholder association feature map and the penholder spectrum feature map.

[0059] In the technical solution of the present application, in order to improve the fusion performance of the pen shaft associated feature map and the pen shaft spectral feature map in a high-dimensional feature space, the smooth parameterized expression factor of the pen shaft associated feature map relative to the pen shaft spectral feature map is calculated, which can use the distribution similarity of the feature manifold to achieve the transformation and alignment of the feature manifold. Specifically, the smooth parameterized expression factor constructs a parameterized surface function according to the eigenvalues ​​of the pen shaft associated feature map and the pen shaft spectral feature map, which is used to smoothly transform and adjust the pen shaft associated feature map to make it more similar to the feature manifold of the pen shaft spectral feature map. Furthermore, by measuring the distribution similarity of the feature manifolds of the pen shaft associated feature map and the pen shaft spectral feature map observed from different dimensional perspectives, the weighted fusion method of the two is determined to enhance and supplement the information of the pen shaft associated feature map, which can not only maintain the original information and semantics of the feature, but also improve the information volume, quality, distribution, consistency, expression ability, generalization ability, alignment and matching degree of the feature.

[0060] More specifically, the classifier is used to process the pen holder polishing defect feature map using the following classification formula to obtain the classification result; wherein the classification formula is:

[0061] O=softmax{(W c ,B c )│Project(F)}, where Project(F) represents the projection of the pen holder polishing defect feature map into a vector, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.

[0062] Figure 5 FIG. 1 is a schematic diagram of the architecture of an intelligent detection system based on machine vision according to an embodiment of the present application. Figure 5As shown, according to the embodiment of the present application, the intelligent detection system based on machine vision includes: first, obtaining a multi-angle image of the polished pen shaft of the Hu brush collected by a camera and a spectral image of the polished pen shaft of the Hu brush collected by the camera, then, the multi-angle images of the polished pen shaft of the Hu brush collected by the camera are respectively passed through a multi-angle pen shaft feature extractor based on spatial attention to obtain a multi-angle pen shaft feature map, then, the difference between the feature maps of every two angles of the pen shaft in the multi-angle pen shaft feature map is calculated to obtain a plurality of pen shaft differential feature maps, and then, the plurality of pen shaft differential feature maps are passed through a base An angle difference feature extractor of a deep convolutional neural network is used to obtain a pen shaft associated feature map. Then, the spectral image of the polished pen shaft collected by the camera is denoised to obtain a denoised pen shaft spectrum map. Further, the denoised pen shaft spectrum map is passed through a channel attention mechanism to obtain a pen shaft spectral feature map. Furthermore, the pen shaft associated feature map and the pen shaft spectral feature map are fused to obtain a pen shaft polishing defect feature map. Finally, the pen shaft polishing defect feature map is passed through a classifier to obtain a classification result, and the classification result is used to determine whether the quality of the polished pen shaft is qualified.

[0063] In summary, the embodiment of the present application obtains multi-angle images of the Hu brush shaft after polishing collected by the camera and spectral images of the Hu brush shaft after polishing collected by the camera, uses deep learning technology to perform feature extraction and correlation analysis on the two, and obtains classification results through a classifier to determine whether the quality of the polished pen shaft is qualified.

[0064] As described above, the intelligent detection system 100 based on machine vision according to the embodiment of the present application can be implemented in various terminal devices, such as a server deployed with an intelligent detection control algorithm based on machine vision. In one example, the intelligent detection system 100 based on machine vision can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent detection system 100 based on machine vision can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the intelligent detection system 100 based on machine vision can also be one of the many hardware modules of the terminal device.

[0065] Alternatively, in another example, the machine vision-based intelligent detection system 100 and the terminal device may also be separate devices, and the machine vision-based intelligent detection system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0066] Exemplary Methods

[0067] Figure 6FIG. 1 is a flow chart of an intelligent detection method based on machine vision according to an embodiment of the present application. Figure 6 As shown, according to the intelligent detection method based on machine vision according to the embodiment of the present application, it includes: S110, acquiring multi-angle images of the Hu brush pen shaft after polishing collected by a camera and a spectral image of the Hu brush pen shaft after polishing collected by the camera; S120, extracting a pen shaft association feature map and a pen shaft spectral feature map from the multi-angle images of the Hu brush pen shaft after polishing collected by the camera and the spectral image of the Hu brush pen shaft after polishing collected by the camera; S130, judging whether the quality of the polished pen shaft is qualified based on the pen shaft association feature map and the pen shaft spectral feature map.

[0068] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned intelligent detection method based on machine vision have been referred to above. Figures 1 to 5 The invention has been introduced in detail in the description of the machine vision-based intelligent detection system, and therefore, its repeated description will be omitted.

[0069] Exemplary Electronic Devices

[0070] Below, reference Figure 7 To describe an electronic device according to an embodiment of the present application.

[0071] Figure 7 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0072] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned intelligent voice control based on deep learning when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0073] See also Figure 7 , Figure 7 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0074] The processor 11 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solution provided in the embodiment of the present application; the memory 12 can be implemented in the form of a read-only memory (Read Only Memory, ROM), a static storage device, a dynamic storage device or a random access memory (Random Access Memory, RAM), etc.

[0075] The memory 12 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 12, and the processor 11 calls and executes the intelligent detection method based on machine vision in the embodiment of this application;

[0076] Input / output interface 13, used to implement information input and output;

[0077] The communication interface 14 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0078] A bus 15 that transmits information between the various components of the device (e.g., the processor 11, the memory 12, the input / output interface 13, and the communication interface 14);

[0079] The processor 11 , the memory 12 , the input / output interface 13 and the communication interface 14 are connected to each other in communication within the device via a bus 15 .

[0080] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned intelligent detection method based on machine vision is implemented.

[0081] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0082] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0083] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0084] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0086] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0087] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0088] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0089] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs. The preferred embodiment of the embodiment of the present application is described above with reference to the accompanying drawings, and the scope of rights of the embodiment of the present application is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of rights of the embodiments of the present application.

Claims

1. An intelligent detection system based on machine vision, characterized in that: include: A module for acquiring images of the Hu brush after polishing, used for acquiring multi-angle images of the Hu brush shaft after polishing acquired by the camera and a spectral image of the Hu brush shaft after polishing acquired by the camera; A Hu brush image feature extraction module is used to extract a brush shaft correlation feature map and a brush shaft spectral feature map from the multi-angle image of the Hu brush shaft after polishing collected by the camera and the spectral image of the Hu brush shaft after polishing collected by the camera; A polishing quality result generation module, used for judging whether the quality of the pen holder after polishing is qualified based on the pen holder association characteristic diagram and the pen holder spectrum characteristic diagram; Wherein, the lake brush image feature extraction module includes: A unit for extracting features of the polished pen shaft, used for extracting features from the multi-angle images of the polished pen shaft of the Hu brush collected by the camera to obtain a pen shaft-related feature map; A spectral image preprocessing unit, used for performing data preprocessing on the spectral image of the polished pen shaft of the Hu brush collected by the camera to obtain a spectral image of the pen shaft after noise reduction; A denoised spectrum feature extraction unit, used for obtaining the penholder spectrum feature map by subjecting the denoised penholder spectrum map to a channel attention mechanism; Wherein, the post-polishing penholder feature extraction unit comprises: A spatial attention subunit, used for respectively passing the multi-angle images of the polished shaft of the Hu brush collected by the camera through a multi-angle shaft feature extractor based on spatial attention to obtain a multi-angle shaft feature map; A correlation feature encoding subunit, used for performing feature encoding on the multi-angle penholder feature graph to obtain the penholder correlation feature graph; Wherein, the associated feature encoding subunit includes: Calculating the difference between the feature maps of every two pen holder angles in the multi-angle pen holder feature map to obtain a plurality of pen holder difference feature maps; The multiple pen shaft differential feature maps are passed through an angle difference feature extractor based on a deep convolutional neural network to obtain the pen shaft associated feature map.

2. The intelligent detection system based on machine vision according to claim 1, characterized in that: The polishing quality result generating module comprises: A defect construction unit, used for fusing the penholder correlation feature map and the penholder spectrum feature map based on a smooth parameterized expression of an output relative objective function to obtain a penholder polishing defect feature map; The defect result generating unit is used to pass the pen holder polishing defect feature map through a classifier to obtain a classification result, and the classification result is used to judge whether the quality of the pen holder after polishing is qualified.

3. The intelligent detection system based on machine vision according to claim 2, characterized in that: The defective construction unit comprises: Calculating a smooth parameterized expression factor of the penholder-related characteristic graph relative to the penholder spectral characteristic graph; The smooth parameterized expression factor is used as a weight to calculate the position-weighted sum of the pen holder associated feature map and the pen holder spectrum feature map to obtain the pen holder polishing defect feature map.

4. The intelligent detection system based on machine vision according to claim 3, characterized in that: The defective construction unit comprises: The smooth parameterized expression factor of the penholder-related characteristic graph relative to the penholder spectrum characteristic graph is calculated using the following smooth parameterized expression factor formula; The smoothing parameterization expression factor formula is: ， in, represents the penholder-related feature graph, represents the penholder spectral characteristic diagram, represents the Euclidean distance between the penholder association feature graph and the penholder spectral feature graph, represents the cosine similarity between the pen-related feature map and the pen-spectral feature map, and represents the predetermined hyperparameters, represents the smooth parameterized expression factor.

5. The intelligent detection system based on machine vision according to claim 4, characterized in that: The defective construction unit comprises: The Euclidean distance between the penholder association feature map and the penholder spectrum feature map is calculated using the following Euclidean distance formula; Wherein, the Euclidean distance formula is: ， in, The first The eigenvalues ​​of the positions, The first graph representing the penholder spectral characteristic graph The eigenvalues ​​of the positions, , , Respectively represent the width, height and number of channels of the feature map, Represents the Euclidean distance between the penholder association feature graph and the penholder spectral feature graph.

6. The intelligent detection system based on machine vision according to claim 5, characterized in that: The defective construction unit comprises: The cosine similarity between the penholder association feature graph and the penholder spectrum feature graph is calculated using the following cosine similarity formula; Wherein, the cosine similarity formula is: ， in, The first The eigenvalues ​​of the positions, The first graph representing the penholder spectral characteristic graph The eigenvalues ​​of the positions, , , Respectively represent the width, height and number of channels of the feature map, Represents the cosine similarity between the pen-holder association feature map and the pen-holder spectral feature map.

7. An intelligent detection method based on machine vision of the intelligent detection system based on machine vision according to claim 1, characterized in that: include: Acquire a multi-angle image of the polished shaft of the Hu brush collected by a camera and a spectral image of the polished shaft of the Hu brush collected by the camera; Extracting a penholder correlation feature map and a penholder spectrum feature map from the multi-angle image of the penholder after being polished collected by the camera and the spectrum image of the penholder after being polished collected by the camera; Based on the pen holder associated characteristic diagram and the pen holder spectral characteristic diagram, it is judged whether the quality of the pen holder after polishing is qualified.

Images