Product quality detection auxiliary method and system based on machine vision

Abstract

The invention discloses a product quality detection auxiliary method and system based on machine vision, and belongs to the technical field of intelligent production of products, and the method comprises the steps of data acquisition, image processing, product defect identification, product quality detection and detection result feedback. According to the method, the deep convolution multi-scale attention network model is adopted to perform product defect identification and efficiently extract multi-scale features, so that the model has relatively good perception ability from details to the global situation, the identification ability for small defects is improved, background interference can be effectively removed, and then efficient and accurate product defect identification is realized; according to the method, the residual error network extreme learning machine with the optimized parameters is adopted for product quality detection, the model parameters are optimized by improving the sparrow search algorithm, the model can adapt to detection of various product types, good universality and adaptability are achieved, the overall practicability of the method is improved, and the product quality can be better guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of product intelligent production, and specifically refers to a product quality detection auxiliary method and system based on machine vision. Background Art

[0002] Product quality inspection assistance based on machine vision uses machine vision technology and artificial intelligence technology to perform automated quality inspection on products in the production process, aiming to improve the accuracy of product quality inspection, effectively identify subtle defects that are difficult to detect with the naked eye, ensure the quality of each product, and significantly increase the inspection speed, thereby improving product production efficiency, reducing labor costs, and promoting the development of intelligent product production.

[0003] However, in the existing product quality inspection auxiliary process, there are technical problems such as complex textures, uneven lighting and other interferences on the product surface in actual production scenarios, and the traditional defect recognition method has weak ability to recognize small defects, resulting in poor sensitivity and accuracy of defect recognition, which in turn affects the subsequent product quality inspection performance; there are technical problems such as the actual production of products with various types and complex shapes, and large differences in the shapes and defect types between different products. It is difficult for traditional product quality inspection models to effectively adapt to different product types, resulting in poor actual application effects of the models. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a product quality inspection auxiliary method and system based on machine vision. In view of the technical problems that in the existing product quality inspection auxiliary process, there are interferences such as complex textures and uneven lighting on the product surface in actual production scenarios, and the traditional defect recognition method has weak ability to recognize small defects, resulting in poor sensitivity and accuracy of defect recognition, which in turn affects the subsequent product quality inspection performance, this solution creatively adopts a deep convolutional multi-scale attention network model for product defect recognition, efficiently extracts multi-scale features, and enables the model to have better perception capabilities from details to the overall situation, thereby improving the recognition ability of small defects and being able to Effectively remove background interference, thereby achieving efficient and accurate product defect identification; in view of the technical problem that in the existing product quality inspection auxiliary process, there are many types of products in actual production and complex shapes, and the shapes and defect types of different products vary greatly. The traditional product quality inspection model is difficult to effectively adapt to different product types, resulting in poor actual application effect of the model. This solution creatively uses parameter-optimized residual network extreme learning machine for product quality inspection, and optimizes model parameters by improving the sparrow search algorithm, so that the model can adapt to the inspection of various product types, has good versatility and adaptability, thereby improving the overall practicality of the method and helping to better ensure product quality.

[0005] The technical solution adopted by the present invention is as follows: The product quality detection auxiliary method based on machine vision provided by the present invention comprises the following steps:

[0006] Step S1: data collection;

[0007] Step S2: image processing;

[0008] Step S3: product defect identification;

[0009] Step S4: product quality inspection;

[0010] Step S5: Feedback of detection results.

[0011] Furthermore, in step S1, the data acquisition is used to acquire original image data related to the product, specifically, to acquire product image data from the product production line through an industrial camera.

[0012] Further, in step S2, the image processing is used to pre-process the image data, including the following steps:

[0013] Step S21: image denoising, specifically, performing a denoising operation on each image in the product image data through a median filter to obtain product denoised image data;

[0014] Step S22: image enhancement, specifically, image enhancement is performed by adjusting the contrast and brightness of each image in the product denoised image data to obtain product enhanced image data.

[0015] Further, in step S3, the product defect recognition is used to identify product surface defects, specifically, based on the product enhanced image data, a deep convolutional multi-scale attention network model is used to perform product defect recognition to obtain product defect information;

[0016] The deep convolutional multi-scale attention network model includes a backbone improvement network, a feature fusion network and a target detection head;

[0017] The backbone improved network is used to gradually extract high-level features, including a basic convolution block, a spatial pyramid hole convolution block, a cross-stage feature fusion block, and a spatial pyramid pooling block;

[0018] The feature fusion network is used to fuse multi-scale features to enhance the context perception capability of the model;

[0019] The target detection head is used to generate target category and position prediction;

[0020] The product defect identification comprises the following steps:

[0021] Step S31: constructing a backbone improvement network, specifically introducing a spatial pyramid hole convolution block and a cross-stage feature fusion block into the backbone network to improve the backbone network, extracting features from the image in the product enhanced image data through the backbone improvement network, and obtaining a multi-scale pooling feature map, including the following steps:

[0022] Step S311: construct a basic convolution block for extracting initial features, wherein the basic convolution block includes a convolution layer, a batch normalization layer, and a SiLU activation function, and processes the image in the product enhanced image data through the basic convolution block to obtain a basic convolution feature map;

[0023] Step S312: constructing a spatial pyramid atrous convolution block to extract multi-scale information and thus improve the model's ability to recognize small targets. Specifically, multi-scale information is extracted from the basic convolution feature map through atrous convolutions with three different dilation rates to obtain a multi-scale feature map.

[0024] Step S313: construct a cross-stage feature fusion block to reduce the computational cost, enhance the feature expression capability through feature segmentation, learning and feature fusion, and obtain a multi-scale fusion feature map. The calculation formula is:

[0025] ;

[0026] In the formula, F 1 is the first segmentation feature subgraph, F 2 is the second segmentation feature subgraph, Split(·) is the feature segmentation function, which is used to perform feature segmentation according to the channel dimension, and F SPD is a multi-scale feature map, ResBlock(·) is a residual block, and F res is the residual learning feature map, Cat(·) is the feature fusion function, F CSP It is a multi-scale fusion feature map;

[0027] Step S315: constructing a spatial pyramid pooling block for extracting multi-scale global information, specifically processing the multi-scale fusion feature map through the spatial pyramid pooling block to obtain a multi-scale pooling feature map;

[0028] Step S32: constructing a feature fusion network, specifically setting an improved multi-scale attention block, a dynamic upsampling block, and a spatial pyramid dilated convolution block in the feature fusion network, processing the multi-scale pooling feature map and the multi-scale fusion feature map through the feature fusion network, and obtaining a multi-scale global feature map, including the following steps:

[0029] Step S321: feature enhancement is performed on the multi-scale fusion feature map by improving the multi-scale attention block, and upsampling is performed by the dynamic upsampling block to obtain a multi-scale up-sampled feature map;

[0030] The improved multi-scale attention block includes initial convolution processing, channel enhancement, spatial enhancement and feature enhancement splicing;

[0031] The calculation formula for the initial convolution process is:

[0032] ;

[0033] In the formula, F one is the initial convolution processing feature map, Conv 1×1 (·) is a 1×1 convolution, used for the initial convolution process;

[0034] The calculation formula of the channel enhancement is:

[0035] ;

[0036] In the formula, F x is the pooled feature vector in the horizontal direction, AvgPool x (·) is the global average pooling operation along the horizontal axis, F y is the pooled feature vector in the vertical direction, AvgPool y (·) is the global average pooling operation along the vertical axis, F cat is the fused feature vector, Conv 1×1 (·) is a 1×1 convolution operation, W c is the channel attention weight, Sig(·) is the S-type activation function, and F cha is the channel enhanced feature map;

[0037] The calculation formula of the spatial enhancement is:

[0038] ;

[0039] In the formula, F s is the convolutional feature map, Conv 3×3 (·) is a 3×3 convolution operation, W s1 is the first attention weight, Softmax(·) is the Softmax activation function, W s2 is the second attention weight, W s3 is the spatial attention weight, F spa is the spatial enhancement feature map;

[0040] The calculation formula of the feature enhancement splicing is:

[0041] ;

[0042] In the formula, F out It is to improve the output feature map of the multi-scale attention block;

[0043] Step S322: performing feature fusion through the multi-scale fusion feature map and the multi-scale up-sampling feature map to obtain a multi-scale up-sampling fusion feature map;

[0044] Step S323: performing an upsampling operation on the multi-scale pooling feature map through a dynamic upsampling block to obtain a multi-scale pooling upsampling feature map, and performing feature fusion on the multi-scale pooling upsampling feature map and the multi-scale upsampling fusion feature map to obtain a multi-scale multi-level feature map;

[0045] Step S324: Process the multi-scale and multi-level feature map through the spatial pyramid dilated convolution block to further extract semantic information and obtain a multi-scale global feature map;

[0046] Step S33: constructing an object detection head, specifically setting classification and bounding box regression tasks in the object detection head, processing the multi-scale global feature map through the object detection head, and obtaining a model detection result;

[0047] Step S34: model training, specifically, constructing a deep convolutional multi-scale attention network model by constructing the backbone improvement network, constructing the feature fusion network and constructing the target detection head, and using the product enhanced image data for model training to obtain a product defect recognition model;

[0048] Step S35: Calculate the product defect recognition result, specifically, perform product defect recognition through a product defect recognition model to obtain product defect recognition information, wherein the product defect recognition information includes defect type, defect location and defect severity.

[0049] Further, in step S4, the product quality detection is used to detect the product form and classify the product quality level, specifically, based on the product defect recognition information and the product enhanced image data, a residual network extreme learning machine with optimized parameters is used to perform product quality detection to obtain product quality detection information, including the following steps:

[0050] Step S41: constructing a residual network extreme learning machine, wherein the residual network extreme learning machine includes a residual network block, an information vectorization layer, a feature fusion layer and an extreme learning machine, and includes the following steps:

[0051] Step S411: constructing a residual network block, specifically, performing deep feature extraction on the image in the product enhanced image data through the residual network block to obtain product deep image features;

[0052] Step S412: constructing an information vectorization layer, specifically, converting the product defect identification information into a vector of fixed dimension through the information vectorization layer, and performing normalization processing to obtain product defect identification features;

[0053] Step S413: constructing a feature fusion layer, specifically, fusing the product high-level image features and the product defect recognition features through the feature fusion layer to obtain the product defect comprehensive image features;

[0054] Step S414: constructing an extreme learning machine, specifically, processing the comprehensive image features of product defects through the extreme learning machine to obtain a model classification result;

[0055] Step S42: model parameter tuning, specifically using an improved sparrow search algorithm to tune the model parameters of the extreme learning machine to obtain a residual network extreme learning machine with optimized parameters;

[0056] The improved sparrow search algorithm improves the sparrow search algorithm by introducing tent chaos mapping, adaptive weight and adaptive random perturbation into the traditional sparrow search algorithm, and comprises the following steps:

[0057] Step S421: Initializing the position of the sparrow, specifically generating a chaotic map value through a tent chaotic map, and initializing the position of the sparrow according to the chaotic map value;

[0058] The calculation formula for generating the chaotic map value by using the tent chaotic map is:

[0059] ;

[0060] In the formula, s k+1 is the chaotic mapping value generated by the k+1th mapping, k is the mapping number index, is the tent chaos map parameter, s k is the chaotic map value generated by the k-th mapping;

[0061] The calculation formula for initializing the sparrow position based on the chaotic map value is:

[0062] ;

[0063] In the formula, s ij is the position component of the i-th sparrow in the j-th dimension, i is the sparrow index, j is the search space dimension index, lb j is the lower bound of the j-th dimension in the search space, ub j is the upper bound of the jth dimension in the search space, S i is the i-th sparrow position, which is used to represent the model parameter combination, s i1 is the position component of the ith sparrow in the first dimension, s i2 is the position component of the ith sparrow in the second dimension, s iD is the position component of the i-th sparrow in the D-th dimension, where D is the dimension of the search space;

[0064] Step S422: Evaluate the fitness value, which is used to evaluate the quality of the model parameters represented by each sparrow position. Specifically, design a fitness function based on the mean square error, mean absolute error and root mean square error of the model, and calculate the fitness value of each sparrow position through the fitness function. The sparrow position with the smallest fitness value is taken as the global optimal position, and the sparrow position with the largest fitness value is taken as the global worst position.

[0065] The calculation formula of the fitness function is:

[0066] ;

[0067] Where f(·) is the fitness function, a 1 is the mean square error weight, MSE(·) is the mean square error function, a 2 is the mean absolute error weight, MAE(·) is the mean absolute error function, a 3 is the root mean square error weight, RMSE(·) is the root mean square error function;

[0068] Step S423: updating the position of the sparrow, specifically updating the position of the sparrow by improving the position of the discoverer, updating the position of the participant and updating the position of the scout;

[0069] The discoverer specifically refers to the sparrow responsible for exploring the search space and leading the group to find the optimal solution; the participant specifically refers to the sparrow that follows the discoverer to explore the potential search space; the scout specifically refers to the sparrow responsible for exploring the unknown area of ​​the search space;

[0070] The discovery improved location update improves the discoverer location update method by introducing adaptive weights. The calculation formula is:

[0071] ;

[0072] ;

[0073] In the formula, S i (t+1) is the position of the i-th sparrow at the t+1-th iteration, t is the iteration index, S i (t) is the position of the i-th sparrow at the t-th iteration, w is the adaptive weight, is a tuning parameter with a value range of [0,1], t max is the maximum number of iterations, R is the warning value in the range of [0,1], ST is the safety value in the range of [0.5,1], R 1 is a random number that follows a normal distribution, cos(·) is the cosine function, is pi, sin(·) is the sine function;

[0074] Step S424: sparrow position adjustment, specifically, adjusting the sparrow position through adaptive random disturbance, the calculation formula is:

[0075] ;

[0076] ;

[0077] In the formula, is the adjusted position of the i-th sparrow at the t+1th iteration, u is the perturbation value, ub is the upper bound of the search space, lb is the lower bound of the search space, R 4 is a random number in the range [0,1];

[0078] Step S425: iterative updating of the population, specifically, iteratively updating the sparrow positions by repeatedly executing steps S422 to S424 until the maximum number of iterations is reached, and taking the sparrow position with the smallest fitness value as the optimal parameter combination of the model;

[0079] Step S43: constructing a product quality detection model, specifically, obtaining a product quality detection model by training a residual network extreme learning machine with optimized parameters;

[0080] Step S44: Calculate the product quality test results, specifically, perform product quality test through a product quality test model to obtain product quality test information, wherein the product quality test information includes product quality grade, product qualification information and product morphology test results.

[0081] Furthermore, in step S5, the test result feedback is used to timely feedback the product quality test results, specifically to generate a product quality test report based on the product quality test information and product defect identification information. When serious defects are detected in the product, an alarm is issued and unqualified products are removed through sorting equipment.

[0082] The product quality inspection auxiliary system based on machine vision provided by the present invention comprises: a data acquisition module, an image processing module, a product defect recognition module, a product quality inspection module and a inspection result feedback module;

[0083] The data acquisition module is used for data acquisition, obtains product image data through data acquisition, and sends the product image data to the image processing module;

[0084] The image processing module is used for image processing, obtains product enhanced image data through image processing, and sends the product enhanced image data to the product defect recognition module and the product quality detection module;

[0085] The product defect recognition module is used for product defect recognition, obtains product defect information through product defect recognition, and sends the product defect information to the product quality detection module and the detection result feedback module;

[0086] The product quality detection module is used for product quality detection, obtains product quality detection information through product quality detection, and sends the product quality detection information to the detection result feedback module;

[0087] The test result feedback module is used for test result feedback, and generates a product quality test report through the test result feedback.

[0088] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0089] (1) In view of the technical problems in the existing product quality inspection auxiliary process, in actual production scenarios, the product surface contains complex textures, uneven lighting and other interferences, and the traditional defect recognition method has weak ability to recognize small defects, resulting in poor sensitivity and accuracy of defect recognition, which in turn affects the performance of subsequent product quality inspection. This solution creatively uses a deep convolutional multi-scale attention network model for product defect recognition, efficiently extracts multi-scale features, and enables the model to have better perception capabilities from details to the overall situation, thereby improving the recognition ability of small defects and effectively removing background interference, thereby achieving efficient and accurate product defect recognition;

[0090] (2) In view of the technical problem that the existing product quality inspection auxiliary process has the following problems: the products are of various types and complex shapes during actual production, and the shapes and defect types of different products vary greatly. The traditional product quality inspection model is difficult to effectively adapt to different product types, resulting in poor actual application effect of the model. This scheme creatively uses parameter-optimized residual network extreme learning machine for product quality inspection. By improving the sparrow search algorithm to optimize the model parameters, the model can adapt to the inspection of various product types, has good versatility and adaptability, and thus improves the overall practicality of the method, which helps to better ensure product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 A flow chart of a product quality inspection auxiliary method based on machine vision provided by the present invention;

[0092] Figure 2 A schematic diagram of a product quality inspection auxiliary system based on machine vision provided by the present invention;

[0093] Figure 3 is a schematic flow chart of step S3;

[0094] Figure 4 It is a schematic diagram of the process of step S4.

[0095] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0096] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0097] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0098] Example 1, see Figure 1 The present invention provides a product quality detection auxiliary method based on machine vision, which comprises the following steps:

[0099] Step S1: data collection;

[0100] Step S2: image processing;

[0101] Step S3: product defect identification;

[0102] Step S4: product quality inspection;

[0103] Step S5: Feedback of detection results.

[0104] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, the data acquisition is used to acquire original image data related to the product, specifically, to acquire product image data from the product production line through an industrial camera.

[0105] Example 3, see Figure 1 This embodiment is based on the above embodiment. In step S2, the image processing is used to pre-process the image data, including the following steps:

[0106] Step S21: image denoising, specifically, performing a denoising operation on each image in the product image data through a median filter to obtain product denoised image data;

[0107] Step S22: image enhancement, specifically, image enhancement is performed by adjusting the contrast and brightness of each image in the product denoised image data to obtain product enhanced image data.

[0108] Example 4, see Figure 1 and Figure 3 , this embodiment is based on the above embodiment. In step S3, the product defect recognition is used to identify product surface defects. Specifically, based on the product enhanced image data, a deep convolutional multi-scale attention network model is used to perform product defect recognition to obtain product defect information;

[0109] The deep convolutional multi-scale attention network model includes a backbone improvement network, a feature fusion network and a target detection head;

[0110] The backbone improved network is used to gradually extract high-level features, including a basic convolution block, a spatial pyramid hole convolution block, a cross-stage feature fusion block, and a spatial pyramid pooling block;

[0111] The feature fusion network is used to fuse multi-scale features to enhance the context perception capability of the model;

[0112] The target detection head is used to generate target category and position prediction;

[0113] The product defect identification comprises the following steps:

[0114] Step S31: constructing a backbone improvement network, specifically introducing a spatial pyramid hole convolution block and a cross-stage feature fusion block into the backbone network to improve the backbone network, extracting features from the image in the product enhanced image data through the backbone improvement network, and obtaining a multi-scale pooling feature map, including the following steps:

[0115] Step S311: construct a basic convolution block for extracting initial features, wherein the basic convolution block includes a convolution layer, a batch normalization layer, and a SiLU activation function, and processes the image in the product enhanced image data through the basic convolution block to obtain a basic convolution feature map;

[0116] Step S312: construct a spatial pyramid dilated convolution block to extract multi-scale information and improve the model's ability to recognize small targets. Specifically, multi-scale information is extracted from the basic convolution feature map through dilated convolutions with three different expansion rates to obtain a multi-scale feature map. The calculation formula is:

[0117] ;

[0118] In the formula, F SPDis a multi-scale feature map, specifically the output feature map of the spatial pyramid dilated convolution block, u is the dilated convolution index, SiLU(·) is the SiLU activation function, BatchNorm2d(·) is the batch normalization layer, and W u is the convolution kernel weight of the u-th atrous convolution, d u is the dilation rate of the u-th atrous convolution, F base is the basic convolution feature map, specifically the output feature map of the basic convolution block, b u is the bias term of the u-th atrous convolution;

[0119] Step S313: construct a cross-stage feature fusion block to reduce the computational cost, enhance the feature expression capability through feature segmentation, learning and feature fusion, and obtain a multi-scale fusion feature map. The calculation formula is:

[0120] ;

[0121] In the formula, F 1 is the first segmentation feature subgraph, F 2 is the second segmentation feature subgraph, Split(·) is the feature segmentation function, which is used to perform feature segmentation according to the channel dimension, and F SPD is a multi-scale feature map, ResBlock(·) is a residual block, and F res is the residual learning feature map, Cat(·) is the feature fusion function, F CSP It is a multi-scale fusion feature map;

[0122] Step S315: construct a spatial pyramid pooling block for extracting multi-scale global information. Specifically, the multi-scale fusion feature map is processed by the spatial pyramid pooling block to obtain a multi-scale pooling feature map. The calculation formula is:

[0123] ;

[0124] In the formula, F SPPF It is a multi-scale pooling feature map, MaxPool 1×1 (·) is a 1×1 maximum pooling operation, MaxPool 3×3 (·) is the 3×3 maximum pooling operation, MaxPool 5×5 (·) is a 5×5 max pooling operation;

[0125] Step S32: constructing a feature fusion network, specifically setting an improved multi-scale attention block, a dynamic upsampling block, and a spatial pyramid dilated convolution block in the feature fusion network, processing the multi-scale pooling feature map and the multi-scale fusion feature map through the feature fusion network, and obtaining a multi-scale global feature map, including the following steps:

[0126] Step S321: feature enhancement is performed on the multi-scale fusion feature map by improving the multi-scale attention block, and upsampling is performed by the dynamic upsampling block to obtain a multi-scale up-sampled feature map;

[0127] The improved multi-scale attention block includes initial convolution processing, channel enhancement, spatial enhancement and feature enhancement splicing;

[0128] The calculation formula for the initial convolution process is:

[0129] ;

[0130] In the formula, F one is the initial convolution processing feature map, Conv 1×1 (·) is a 1×1 convolution, used for the initial convolution process;

[0131] The calculation formula of the channel enhancement is:

[0132] ;

[0133] In the formula, F x is the pooled feature vector in the horizontal direction, AvgPool x (·) is the global average pooling operation along the horizontal axis, F y is the pooled feature vector in the vertical direction, AvgPool y (·) is the global average pooling operation along the vertical axis, F cat is the fused feature vector, Conv 1×1 (·) is a 1×1 convolution operation, W c is the channel attention weight, Sig(·) is the S-type activation function, and F cha is the channel enhanced feature map;

[0134] The calculation formula of the spatial enhancement is:

[0135] ;

[0136] In the formula, F s is the convolutional feature map, Conv 3×3 (·) is a 3×3 convolution operation, W s1 is the first attention weight, Softmax(·) is the Softmax activation function, W s2 is the second attention weight, W s3 is the spatial attention weight, F spa is the spatial enhancement feature map;

[0137] The calculation formula of the feature enhancement splicing is:

[0138] ;

[0139] In the formula, F out It is to improve the output feature map of the multi-scale attention block;

[0140] Step S322: performing feature fusion through the multi-scale fusion feature map and the multi-scale up-sampling feature map to obtain a multi-scale up-sampling fusion feature map;

[0141] Step S323: performing an upsampling operation on the multi-scale pooling feature map through a dynamic upsampling block to obtain a multi-scale pooling upsampling feature map, and performing feature fusion on the multi-scale pooling upsampling feature map and the multi-scale upsampling fusion feature map to obtain a multi-scale multi-level feature map;

[0142] Step S324: Process the multi-scale and multi-level feature map through the spatial pyramid dilated convolution block to further extract semantic information and obtain a multi-scale global feature map;

[0143] Step S33: constructing an object detection head, specifically setting classification and bounding box regression tasks in the object detection head, processing the multi-scale global feature map through the object detection head, and obtaining a model detection result;

[0144] Step S34: model training, specifically, constructing a deep convolutional multi-scale attention network model by constructing the backbone improvement network, constructing the feature fusion network and constructing the target detection head, and using the product enhanced image data for model training to obtain a product defect recognition model;

[0145] Step S35: calculating a product defect recognition result, specifically, performing product defect recognition through a product defect recognition model to obtain product defect recognition information, wherein the product defect recognition information includes defect type, defect location, and defect severity;

[0146] By performing the above operations, in view of the technical problem that in the existing product quality inspection auxiliary process, in actual production scenarios, the product surface contains complex textures, uneven lighting and other interferences, and the traditional defect recognition method has weak recognition ability for small defects, resulting in poor sensitivity and accuracy of defect recognition, which in turn affects the subsequent product quality inspection performance, this solution creatively adopts a deep convolutional multi-scale attention network model for product defect recognition, efficiently extracts multi-scale features, and enables the model to have good perception capabilities from details to the overall situation, thereby improving the recognition ability of small defects and effectively removing background interference, thereby achieving efficient and accurate product defect recognition.

[0147] Example 5, see Figure 1 and Figure 4This embodiment is based on the above embodiment. In step S4, the product quality detection is used to detect the product form and classify the product quality level. Specifically, according to the product defect recognition information and the product enhanced image data, a residual network extreme learning machine with optimized parameters is used to perform product quality detection to obtain product quality detection information, including the following steps:

[0148] Step S41: constructing a residual network extreme learning machine, wherein the residual network extreme learning machine includes a residual network block, an information vectorization layer, a feature fusion layer and an extreme learning machine, and includes the following steps:

[0149] Step S411: constructing a residual network block, specifically, performing deep feature extraction on the image in the product enhanced image data through the residual network block to obtain product deep image features;

[0150] Step S412: constructing an information vectorization layer, specifically, converting the product defect identification information into a vector of fixed dimension through the information vectorization layer, and performing normalization processing to obtain product defect identification features;

[0151] Step S413: constructing a feature fusion layer, specifically, fusing the product high-level image features and the product defect recognition features through the feature fusion layer to obtain the product defect comprehensive image features;

[0152] Step S414: constructing an extreme learning machine, specifically, processing the comprehensive image features of product defects through the extreme learning machine to obtain a model classification result;

[0153] Step S42: model parameter tuning, specifically using an improved sparrow search algorithm to tune the model parameters of the extreme learning machine to obtain a residual network extreme learning machine with optimized parameters;

[0154] The improved sparrow search algorithm improves the sparrow search algorithm by introducing tent chaos mapping, adaptive weight and adaptive random perturbation into the traditional sparrow search algorithm, and comprises the following steps:

[0155] Step S421: Initializing the position of the sparrow, specifically generating a chaotic map value through a tent chaotic map, and initializing the position of the sparrow according to the chaotic map value;

[0156] The calculation formula for generating the chaotic map value by using the tent chaotic map is:

[0157] ;

[0158] In the formula, s k+1 is the chaotic mapping value generated by the k+1th mapping, k is the mapping number index, is the tent chaos mapping parameter, the value of the tent chaos mapping parameter is 0.499, sk is the chaotic map value generated by the k-th mapping;

[0159] The calculation formula for initializing the sparrow position based on the chaotic map value is:

[0160] ;

[0161] In the formula, s ij is the position component of the i-th sparrow in the j-th dimension, i is the sparrow index, j is the search space dimension index, lb j is the lower bound of the j-th dimension in the search space, ub j is the upper bound of the jth dimension in the search space, S i is the i-th sparrow position, which is used to represent the model parameter combination, s i1 is the position component of the i-th sparrow in the first dimension, s i2 is the position component of the ith sparrow in the second dimension, s iD is the position component of the i-th sparrow in the D-th dimension, where D is the dimension of the search space;

[0162] Step S422: Evaluate the fitness value, which is used to evaluate the quality of the model parameters represented by each sparrow position. Specifically, design a fitness function based on the mean square error, mean absolute error and root mean square error of the model, and calculate the fitness value of each sparrow position through the fitness function. The sparrow position with the smallest fitness value is taken as the global optimal position, and the sparrow position with the largest fitness value is taken as the global worst position.

[0163] The smaller the fitness value of the sparrow position is, the smaller the detection error of the model is and the better the model performance is;

[0164] The calculation formula of the fitness function is:

[0165] ;

[0166] Where f(·) is the fitness function, a 1 is the mean square error weight, MSE(·) is the mean square error function, a 2 is the mean absolute error weight, MAE(·) is the mean absolute error function, a 3 is the root mean square error weight, RMSE(·) is the root mean square error function;

[0167] Step S423: updating the position of the sparrow, specifically updating the position of the sparrow by improving the position of the discoverer, updating the position of the participant and updating the position of the scout;

[0168] The discoverer specifically refers to the sparrow responsible for exploring the search space and leading the group to find the optimal solution;

[0169] The discovery improved location update improves the discoverer location update method by introducing adaptive weights. The calculation formula is:

[0170] ;

[0171] ;

[0172] In the formula, S i (t+1) is the position of the i-th sparrow at the t+1-th iteration, t is the iteration index, S i (t) is the position of the i-th sparrow at the t-th iteration, w is the adaptive weight, is a tuning parameter with a value range of [0,1], t max is the maximum number of iterations, R is the warning value in the range of [0,1], ST is the safety value in the range of [0.5,1], R 1 is a random number that follows a normal distribution, cos(·) is the cosine function, is pi, sin(·) is the sine function;

[0173] The participant specifically refers to the sparrow that follows the discoverer to explore the potential search space. The calculation formula for the participant position update is:

[0174] ;

[0175] Where exp(·) is the natural exponential function, S worst (t) is the global worst position at the tth iteration, N is the number of sparrows, S p (t+1) is the best position occupied by the finder at the t+1th iteration, A is a 1×D matrix with values ​​of {-1, 1}, T is the transpose operation, and L is a 1×D matrix with all elements being 1;

[0176] The scout specifically refers to the sparrow responsible for exploring the unknown area of ​​the search space. The calculation formula for updating the position of the scout is:

[0177] ;

[0178] In the formula, S best (t) is the global optimal position at the tth iteration, R 2 is a random number that follows a standard normal distribution, f i is the fitness value of the i-th sparrow position, f g is the fitness value of the global optimal position, f w is the fitness value of the global worst position, R 3 is a random number uniformly distributed in the interval [-1,1]. It is a very small positive number, used to avoid the situation where the denominator is 0;

[0179] Step S424: sparrow position adjustment, specifically, adjusting the sparrow position through adaptive random disturbance, the calculation formula is:

[0180] ;

[0181] ;

[0182] In the formula, is the adjusted position of the i-th sparrow at the t+1th iteration, u is the perturbation value, ub is the upper bound of the search space, lb is the lower bound of the search space, R 4 is a random number in the range [0,1];

[0183] Step S425: iterative updating of the population, specifically, iteratively updating the sparrow positions by repeatedly executing steps S422 to S424 until the maximum number of iterations is reached, and taking the sparrow position with the smallest fitness value as the optimal parameter combination of the model;

[0184] Step S43: constructing a product quality detection model, specifically, obtaining a product quality detection model by training a residual network extreme learning machine with optimized parameters;

[0185] Step S44: calculating the product quality test result, specifically, performing product quality test through a product quality test model to obtain product quality test information, wherein the product quality test information includes product quality grade, product qualification information and product form test result;

[0186] By performing the above operations, in view of the technical problem that in the existing product quality inspection auxiliary process, there are many types of products with complex shapes in actual production, and the shapes and defect types between different products are quite different, and the traditional product quality inspection model is difficult to effectively adapt to different product types, resulting in poor actual application effect of the model, this solution creatively uses parameter-optimized residual network extreme learning machine for product quality inspection, and optimizes model parameters by improving the sparrow search algorithm, so that the model can adapt to the inspection of various product types, has good versatility and adaptability, thereby improving the overall practicality of the method, which helps to better ensure product quality.

[0187] Example 6, see Figure 1 This embodiment is based on the above embodiment. In step S5, the test result feedback is used to timely feedback the product quality test results. Specifically, a product quality test report is generated based on the product quality test information and the product defect identification information. When a serious defect is detected in the product, an alarm is issued and unqualified products are removed through sorting equipment.

[0188] Embodiment 7, see Figure 2 , this embodiment is based on the above embodiment, the product quality inspection auxiliary system based on machine vision provided by the present invention includes: a data acquisition module, an image processing module, a product defect recognition module, a product quality inspection module and a detection result feedback module;

[0189] The data acquisition module is used for data acquisition, obtains product image data through data acquisition, and sends the product image data to the image processing module;

[0190] The image processing module is used for image processing, obtains product enhanced image data through image processing, and sends the product enhanced image data to the product defect recognition module and the product quality detection module;

[0191] The product defect recognition module is used for product defect recognition, obtains product defect information through product defect recognition, and sends the product defect information to the product quality detection module and the detection result feedback module;

[0192] The product quality detection module is used for product quality detection, obtains product quality detection information through product quality detection, and sends the product quality detection information to the detection result feedback module;

[0193] The test result feedback module is used for test result feedback, and generates a product quality test report through the test result feedback.

[0194] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0195] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

[0196] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A product quality inspection auxiliary method based on machine vision, characterized in that: The method comprises the following steps: Step S1: data collection; Step S2: image processing; Step S3: product defect recognition, which is used to identify product surface defects. Specifically, based on product enhanced image data, a deep convolutional multi-scale attention network model is used to perform product defect recognition to obtain product defect information; The deep convolutional multi-scale attention network model includes a backbone improvement network, a feature fusion network and a target detection head; The backbone improved network is used to gradually extract high-level features, including a basic convolution block, a spatial pyramid hole convolution block, a cross-stage feature fusion block, and a spatial pyramid pooling block; The feature fusion network is used to fuse multi-scale features to enhance the context perception capability of the model; The target detection head is used to generate target category and position prediction; Step S4: product quality inspection, which is used to detect the product form and classify the product quality level. Specifically, based on the product defect recognition information and the product enhanced image data, a residual network extreme learning machine with optimized parameters is used to perform product quality inspection to obtain product quality inspection information; Step S5: Feedback of detection results.

2. The product quality inspection auxiliary method based on machine vision according to claim 1 is characterized in that: In step S3, the product defect identification includes the following steps: Step S31: constructing a backbone improvement network, specifically introducing a spatial pyramid hole convolution block and a cross-stage feature fusion block into the backbone network to improve the backbone network, extracting features from the image in the product enhanced image data through the backbone improvement network, and obtaining a multi-scale pooling feature map, including the following steps: Step S311: construct a basic convolution block for extracting initial features, wherein the basic convolution block includes a convolution layer, a batch normalization layer, and a SiLU activation function, and processes the image in the product enhanced image data through the basic convolution block to obtain a basic convolution feature map; Step S312: constructing a spatial pyramid atrous convolution block to extract multi-scale information and thus improve the model's ability to recognize small targets. Specifically, multi-scale information is extracted from the basic convolution feature map through atrous convolutions with three different dilation rates to obtain a multi-scale feature map. Step S313: construct a cross-stage feature fusion block to reduce the computational cost, enhance the feature expression capability through feature segmentation, learning and feature fusion, and obtain a multi-scale fusion feature map. The calculation formula is: ； Where F1 is the first segmentation feature subgraph, F2 is the second segmentation feature subgraph, Split(·) is the feature segmentation function, which is used to perform feature segmentation according to the channel dimension, and F SPD is a multi-scale feature map, ResBlock(·) is a residual block, and F res is the residual learning feature map, Cat(·) is the feature fusion function, F CSP It is a multi-scale fusion feature map; Step S315: constructing a spatial pyramid pooling block for extracting multi-scale global information, specifically processing the multi-scale fusion feature map through the spatial pyramid pooling block to obtain a multi-scale pooling feature map; Step S32: constructing a feature fusion network, specifically setting an improved multi-scale attention block, a dynamic upsampling block and a spatial pyramid dilated convolution block in the feature fusion network, processing the multi-scale pooling feature map and the multi-scale fusion feature map through the feature fusion network to obtain a multi-scale global feature map; Step S33: constructing an object detection head, specifically setting classification and bounding box regression tasks in the object detection head, processing the multi-scale global feature map through the object detection head, and obtaining a model detection result; Step S34: model training, specifically, constructing a deep convolutional multi-scale attention network model by constructing the backbone improvement network, constructing the feature fusion network and constructing the target detection head, and using the product enhanced image data for model training to obtain a product defect recognition model; Step S35: Calculate the product defect recognition result, specifically, perform product defect recognition through a product defect recognition model to obtain product defect recognition information.

3. The product quality inspection auxiliary method based on machine vision according to claim 2 is characterized in that: In step S32, the construction of the feature fusion network includes the following steps: Step S321: feature enhancement is performed on the multi-scale fusion feature map by improving the multi-scale attention block, and upsampling is performed by the dynamic upsampling block to obtain a multi-scale up-sampled feature map; The improved multi-scale attention block includes initial convolution processing, channel enhancement, spatial enhancement and feature enhancement splicing; The calculation formula for the initial convolution process is: ； In the formula, F one is the initial convolution processing feature map, Conv 1×1 (·) is a 1×1 convolution, used for the initial convolution process; The calculation formula of the channel enhancement is: ； In the formula, F x is the pooled feature vector in the horizontal direction, AvgPool x (·) is the global average pooling operation along the horizontal axis, F y is the pooled feature vector in the vertical direction, AvgPool y (·) is the global average pooling operation along the vertical axis, F cat is the fused feature vector, Conv 1×1 (·) is a 1×1 convolution operation, W c is the channel attention weight, Sig(·) is the S-type activation function, and F cha is the channel enhanced feature map; The calculation formula of the spatial enhancement is: ； In the formula, F s is the convolutional feature map, Conv 3×3 (·) is a 3×3 convolution operation, W s1 is the first attention weight, Softmax(·) is the Softmax activation function, W s2 is the second attention weight, W s3 is the spatial attention weight, F spa is the spatial enhancement feature map; The calculation formula of the feature enhancement splicing is: ； In the formula, F out It is to improve the output feature map of the multi-scale attention block; Step S322: performing feature fusion through the multi-scale fusion feature map and the multi-scale up-sampling feature map to obtain a multi-scale up-sampling fusion feature map; Step S323: performing an upsampling operation on the multi-scale pooling feature map through a dynamic upsampling block to obtain a multi-scale pooling upsampling feature map, and performing feature fusion on the multi-scale pooling upsampling feature map and the multi-scale upsampling fusion feature map to obtain a multi-scale multi-level feature map; Step S324: Process the multi-scale and multi-level feature map through the spatial pyramid dilated convolution block to further extract semantic information and obtain a multi-scale global feature map.

4. The product quality inspection auxiliary method based on machine vision according to claim 3 is characterized in that: In step S4, the product quality detection includes the following steps: Step S41: constructing a residual network extreme learning machine, wherein the residual network extreme learning machine includes a residual network block, an information vectorization layer, a feature fusion layer and an extreme learning machine; Step S42: model parameter tuning, specifically using an improved sparrow search algorithm to tune the model parameters of the extreme learning machine to obtain a residual network extreme learning machine with optimized parameters; Step S43: constructing a product quality detection model, specifically, obtaining a product quality detection model by training a residual network extreme learning machine with optimized parameters; Step S44: Calculate the product quality test result, specifically, perform product quality test through the product quality test model to obtain product quality test information.

5. The product quality inspection auxiliary method based on machine vision according to claim 4 is characterized in that: In step S41, the construction of the residual network extreme learning machine includes the following steps: Step S411: constructing a residual network block, specifically, performing deep feature extraction on the image in the product enhanced image data through the residual network block to obtain product deep image features; Step S412: constructing an information vectorization layer, specifically, converting the product defect identification information into a vector of fixed dimension through the information vectorization layer, and performing normalization processing to obtain product defect identification features; Step S413: constructing a feature fusion layer, specifically, fusing the product high-level image features and the product defect recognition features through the feature fusion layer to obtain the product defect comprehensive image features; Step S414: constructing an extreme learning machine, specifically, processing the comprehensive image features of product defects through the extreme learning machine to obtain a model classification result.

6. The product quality inspection auxiliary method based on machine vision according to claim 5 is characterized in that: In step S42, the improved sparrow search algorithm is improved by introducing tent chaos mapping, adaptive weight and adaptive random perturbation into the traditional sparrow search algorithm, and includes the following steps: Step S421: Initializing the position of the sparrow, specifically generating a chaotic map value through a tent chaotic map, and initializing the position of the sparrow according to the chaotic map value; The calculation formula for generating the chaotic map value by using the tent chaotic map is: ； In the formula, s k+1 is the chaotic mapping value generated by the k+1th mapping, k is the mapping number index, is the tent chaos map parameter, s k is the chaotic map value generated by the k-th mapping; The calculation formula for initializing the sparrow position based on the chaotic map value is: ； In the formula, s ij is the position component of the i-th sparrow in the j-th dimension, i is the sparrow index, j is the search space dimension index, lb j is the lower bound of the j-th dimension in the search space, ub j is the upper bound of the jth dimension in the search space, S i is the i-th sparrow position, which is used to represent the model parameter combination, s i1 is the position component of the ith sparrow in the first dimension, s i2 is the position component of the ith sparrow in the second dimension, s iD is the position component of the i-th sparrow in the D-th dimension, where D is the dimension of the search space; Step S422: Evaluate the fitness value, which is used to evaluate the quality of the model parameters represented by each sparrow position. Specifically, design a fitness function based on the mean square error, mean absolute error and root mean square error of the model, and calculate the fitness value of each sparrow position through the fitness function. The sparrow position with the smallest fitness value is taken as the global optimal position, and the sparrow position with the largest fitness value is taken as the global worst position. The calculation formula of the fitness function is: ； Where, f(·) is the fitness function, a1 is the mean square error weight, MSE(·) is the mean square error function, a2 is the mean absolute error weight, MAE(·) is the mean absolute error function, a3 is the root mean square error weight, RMSE(·) is the root mean square error function; Step S423: updating the position of the sparrow, specifically updating the position of the sparrow by improving the position of the discoverer, updating the position of the participant and updating the position of the scout; The discoverer specifically refers to the sparrow responsible for exploring the search space and leading the group to find the optimal solution; the participant specifically refers to the sparrow that follows the discoverer to explore the potential search space; the scout specifically refers to the sparrow responsible for exploring the unknown area of ​​the search space; The discovery improved location update improves the discoverer location update method by introducing adaptive weights. The calculation formula is: ； ； In the formula, S i (t+1) is the position of the i-th sparrow at the t+1-th iteration, t is the iteration index, S i (t) is the position of the i-th sparrow at the t-th iteration, w is the adaptive weight, is a tuning parameter with a value range of [0,1], t max is the maximum number of iterations, R is the warning value in the range [0,1], ST is the safety value in the range [0.5,1], R1 is a random number that follows a normal distribution, cos(·) is the cosine function, is pi, sin(·) is the sine function; Step S424: sparrow position adjustment, specifically, adjusting the sparrow position through adaptive random disturbance, the calculation formula is: ； ； In the formula, is the adjusted position of the i-th sparrow at the t+1-th iteration, u is the disturbance value, ub is the upper bound of the search space, lb is the lower bound of the search space, and R4 is a random number in the range [0,1]; Step S425: iterative updating of the population, specifically, iteratively updating the sparrow positions by repeatedly executing steps S422 to S424 until the maximum number of iterations is reached, and taking the sparrow position with the smallest fitness value as the optimal parameter combination of the model.

7. The product quality inspection auxiliary method based on machine vision according to claim 6 is characterized in that: In step S5, the test result feedback is used to timely feedback the product quality test results, specifically generating a product quality test report based on the product quality test information and product defect identification information. When a serious defect is detected in a product, an alarm is issued and unqualified products are removed through sorting equipment; In step S1, the data acquisition is used to acquire original image data related to the product, specifically, to acquire product image data from the product production line through an industrial camera; In step S2, the image processing is used to pre-process the image data, including the following steps: Step S21: image denoising, specifically, performing a denoising operation on each image in the product image data through a median filter to obtain product denoised image data; Step S22: image enhancement, specifically, image enhancement is performed by adjusting the contrast and brightness of each image in the product denoised image data to obtain product enhanced image data.

8. A product quality inspection auxiliary system based on machine vision, used to implement the product quality inspection auxiliary method based on machine vision as described in any one of claims 1 to 7, characterized in that: It includes a data acquisition module, an image processing module, a product defect recognition module, a product quality inspection module and a test result feedback module.

9. The product quality inspection auxiliary system based on machine vision according to claim 8 is characterized in that: The data acquisition module is used for data acquisition, obtains product image data through data acquisition, and sends the product image data to the image processing module; The image processing module is used for image processing, obtains product enhanced image data through image processing, and sends the product enhanced image data to the product defect recognition module and the product quality detection module; The product defect recognition module is used for product defect recognition, obtains product defect information through product defect recognition, and sends the product defect information to the product quality detection module and the detection result feedback module; The product quality detection module is used for product quality detection, obtains product quality detection information through product quality detection, and sends the product quality detection information to the detection result feedback module; The test result feedback module is used for test result feedback, and generates a product quality test report through the test result feedback.

Images