A method for defect detection and conformity quantitative evaluation of industrial flexible products

Abstract

The present invention provides a method for defect detection and conformity quantitative evaluation of industrial flexible products, belonging to the technical field of detection and evaluation; the steps include: step 1: obtaining multiple groups of target blank flexible products as sample sets, obtaining initial images to obtain initial image sets, aligning and superimposing the initial image sets, and generating template images; step 2: obtaining the flexible product to be detected as a test sample, and obtaining a test image of the test sample, mapping the test image and the template image to the same coordinate system, and obtaining a comparison image; step 3: constructing a neural network model, inputting the comparison image to extract features; step 4: outputting the segmentation result map and the deformation field map of the features through a fully connected layer, and training the neural network model; step 5: deploying the parameters of the optimal network model on a field server, outputting the segmentation map and the deformation field of the image to be detected, and performing defect detection and conformity quantitative evaluation by analyzing the deformation field. The error is reduced and the reasoning speed is fast.

Description

Technical Field

[0001] The present invention relates to the field of detection and evaluation technology, and in particular to a method for detecting defects of industrial flexible products and quantitatively evaluating conformity. Background Art

[0002] At present, in the field of traditional industrial testing, there is no good means to test the conformity of flexible products.

[0003] The method used in single-modality medical image registration cannot be directly applied in the field of traditional industrial inspection. First, single-modality medical images are single-channel images with clear edges and no background texture influence. Second, floating images and fixed images are images of the patient's own organs or lesions, and there is a certain temporal relationship between the two images. Compared with industrial actual production images and design draft images, the image similarity faced by medical image registration is higher, and it is only used as a registration method without precise measurement and analysis, and there is no constraint on deformed images.

[0004] Therefore, the present invention proposes a method for defect detection and conformity quantitative evaluation of industrial flexible products. Summary of the invention

[0005] The present invention provides a method for defect detection and quantitative conformity evaluation of industrial flexible products, which realizes a rigid and non-rigid image transformation from image to template, so that end-to-end quantifiable conformity evaluation can be performed in the printing quality inspection of industrial flexible products. The method can greatly reduce the false positives of template matching differences caused by the production process and physical characteristics of flexible products. The end-to-end method is easy to use, the model architecture is simple and effective, and the reasoning speed is fast.

[0006] In one aspect, the present invention provides a method for defect detection and conformity quantitative evaluation of industrial flexible products, comprising:

[0007] Step 1: obtaining multiple groups of target blank flexible products as sample sets, and obtaining an initial image of each target blank flexible product in the sample set to obtain an initial image set, aligning and superimposing the initial image sets to generate a template image;

[0008] Step 2: obtaining a flexible product to be tested as a test sample, obtaining a test image of the test sample, and mapping the test image and the template image into the same coordinate system to obtain a comparison image;

[0009] Step 3: construct a neural network model, input the contrast image, and extract features of the contrast image based on a convolutional layer, wherein each convolutional neuron is connected to a local area of ​​the contrast image;

[0010] Step 4: Output the segmentation result map and deformation field map through the fully connected layer, and train the neural network model;

[0011] Step 5: Deploy the parameters of the trained optimal network model on the on-site server, output the segmentation map and deformation field of the image to be detected, and perform defect detection and quantitative evaluation of conformity by analyzing the deformation field.

[0012] On the other hand, multiple groups of target blank flexible products are obtained as sample sets, and an initial image of each target blank flexible product in the sample set is obtained to obtain an initial image set, including:

[0013] Acquire multiple target blank flexible products with completely similar shapes and properties to form a sample set;

[0014] The initial image of each target blank flexible product in the sample set is collected based on the video acquisition device, and finally the initial image set corresponding to the sample set is obtained.

[0015] On the other hand, the initial image sets are aligned and superimposed to generate a template image, including:

[0016] Convert each initial image in the initial image set into a grayscale image, and normalize the pixels of any grayscale image to obtain a standard image of the grayscale image, obtain a standard image set corresponding to the initial image set, and obtain key points of each standard image based on manual screening;

[0017] Select any image in the standard image set as a reference image, map the reference image to an alignment coordinate system, and identify key point coordinates of the reference image;

[0018] Obtain any other standard image in the standard image set as the image to be aligned, and align the image to be aligned with the reference image through perspective transformation according to the matching relationship between the key points of the image to be aligned and the reference image;

[0019] All aligned standard images are superimposed, and the pixel mean at the corresponding position is obtained by averaging the pixel values ​​at the same position of all standard images;

[0020] Generate a template image based on the image composed of the mean values ​​of pixels at all positions.

[0021] On the other hand, obtaining a flexible product to be inspected as a test sample, obtaining a test image of the test sample, and mapping the test image and the template image to the same coordinate system to obtain a comparison image, including:

[0022] A flexible product to be inspected is obtained as a test sample, a first image of the test sample is obtained based on recognition by an image detection system, and the first image is processed based on standard image information of a template image to obtain a test image;

[0023] Construct a standard coordinate system, locate the upper left corner of the test image and the template image to coincide with each other, and determine the matching score of the corresponding point located between the upper left corner of the test image and the template image:

[0024] ; Among them, MT represents the pixel length of the fixed-size image in the upper left corner, and NT represents the pixel width of the fixed-size image in the upper left corner. represents the matching score between the test image point A and the template image point B, Represents the test image point The pixel value of Represents the template image point The pixel value of represents the dot product operation, Represents the mean pixel value of the test image in a fixed-size image. Represents the mean pixel value of the template image in a fixed-size image;

[0025] Construct a sliding window and search for the point with the maximum matching score between the test image and the template image in a fixed-size image based on macro-region positioning for:

[0026] ;in, Indicates the location coordinates of the maximum matching score point, represents the set of positions of the sliding window, represents the macro area positioning function based on sliding window;

[0027] Based on the point with the maximum matching score, the test image and the template image are mapped into the same coordinate system to obtain a comparison image.

[0028] On the other hand, a neural network model is constructed, a contrast image is input, and features of the contrast image are extracted based on a convolutional layer, wherein each convolutional neuron is connected to a local area of ​​the contrast image, including:

[0029] Build a model based on convolutional neural network, and pass the input contrast image to the first layer of convolutional neural network;

[0030] The contrast image is segmented into multiple local regions based on a pixel-level segmentation network, a convolution operation is performed on any convolutional neuron of the first layer and any region to obtain a local receptive field of the corresponding region, the local receptive field is processed based on a nonlinear activation function, and all pixel coordinates corresponding to the first feature map are generated;

[0031] Performing dimension reduction and size control operations on the first feature map based on the pooling layer to obtain a second feature map;

[0032] The second feature map is output to other layers in the convolutional neural network based on the fully connected layer.

[0033] On the other hand, the features are output through the fully connected layer to generate the segmentation result map and the deformation field map, and the neural network model is trained, including:

[0034] Obtain a second feature map based on the fully connected layer, and perform a first segmentation on the second feature map using an encoder of a pixel-level segmentation network;

[0035] Based on the initial parameters of the pixel-level segmentation network, a spatial transformation network is configured. The spatial transformation network performs an affine transformation on the first segmentation result based on the preset affine transformation parameters to obtain a transformation matrix. At the second feature map input end, a position encoding method using trigonometric functions to alternately represent the coordinate position is used to input the position information of the foreground coordinate point of the template image into the network pixel-level segmentation network.

[0036] Based on the positioning network, the transformed image network is output using the transformation matrix, and interpolated and reconstructed to output a pixel-level segmentation deformation map;

[0037] In the training phase, the mean square error between the pixel-level segmentation deformation map and the template image is calculated to obtain the total variation of the pixel-level segmentation deformation map. The mean square error and total variation are used as parameters of the loss function, and the total loss coefficient is output:

[0038] ;in, represents the total loss coefficient, represents the mean square error between the pixel-level segmentation deformation map and the template image, represents the total variation of the pixel-level segmentation deformation map, represents the weight coefficient of the mean square error, represents the weight coefficient of total variation, represents the pixel value of the kth pixel in the pixel-level segmentation deformation map, represents the pixel value of the kth pixel of the template image, m represents the minimum value of the two pixel numbers of the template image and the pixel-level segmentation deformation image, Represents the X-direction gradient of the point at position pos in the pixel-level segmentation deformation map, Represents the Y-direction gradient of the point at position pos in the pixel-level segmentation deformation map, represents the pixel-level segmentation deformation map range, d represents the dimension, Indicates tuning of hyperparameters;

[0039] The total loss coefficient is combined with the stochastic gradient descent algorithm for back propagation to optimize the initial parameters of the pixel-level segmentation network and the preset affine transformation parameters of the spatial transformation network to generate the optimal network model.

[0040] On the other hand, the parameters of the trained optimal network model are deployed on the on-site server to output the segmentation map and deformation field of the image to be detected, including:

[0041] Configure and initialize the field server according to preset parameters, and import the optimal network model into the field server;

[0042] By inputting the image to be detected transmitted on the production line and the template image in the template library, after forward inference of the optimal network model, the segmentation map and deformation field are obtained, where the segmentation map is displayed as the segmentation result, and the deformation field is used as the key analysis image to analyze the connected domain and energy field;

[0043] The pixel points at each coordinate of the deformation field represent the horizontal and vertical coordinate displacements from the segmentation result of the image to be detected to each pixel point of the segmentation deformation map.

[0044] On the other hand, by analyzing the deformation field, defect detection and quantitative evaluation of conformity are performed, including:

[0045] Obtain the dual-channel grayscale values ​​of the deformation field and construct a grayscale value matrix;

[0046] The deformation field is subjected to binary processing using a preset area threshold to obtain a binary image, and a connected domain labeling algorithm is applied to the binary image to obtain a label and an area of ​​each connected domain;

[0047] For any connected domain, the energy of the connected domain is generated by using the area integration of the dual-channel grayscale values;

[0048] If the absolute value difference between the energy of the connected domain and the energy standard threshold, as well as the absolute value difference between the area and the area standard threshold are both greater than the standard qualified threshold, it is determined that the image to be detected has defects within the pixel range of the connected domain.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention provides a method for defect detection and quantitative conformity evaluation of industrial flexible products, which realizes a rigid and non-rigid image transformation from image to template, so that end-to-end quantifiable conformity evaluation can be performed in the printing quality inspection of industrial flexible products. The method can greatly reduce the false positives of template matching differences caused by the production process and physical characteristics of flexible products. The end-to-end method is easy to use, the model architecture is simple and effective, and the reasoning speed is fast. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flow chart of a method for defect detection and conformity quantitative evaluation of industrial flexible products provided by an embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of a method for quantitative evaluation of defects detection and conformity of industrial flexible products;

[0054] Figure 3 This is a simplified diagram of the convolutional neural network structure. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Embodiment 1:

[0057] like Figure 1 As shown, an industrial flexible product defect detection and conformity quantitative evaluation method provided by an embodiment of the present invention includes:

[0058] Step 1: obtaining multiple groups of target blank flexible products as sample sets, and obtaining an initial image of each target blank flexible product in the sample set to obtain an initial image set, aligning and superimposing the initial image sets to generate a template image;

[0059] Step 2: obtaining a flexible product to be tested as a test sample, obtaining a test image of the test sample, and mapping the test image and the template image into the same coordinate system to obtain a comparison image;

[0060] Step 3: construct a neural network model, input the contrast image, and extract features of the contrast image based on a convolutional layer, wherein each convolutional neuron is connected to a local area of ​​the contrast image;

[0061] Step 4: Output the segmentation result map and deformation field map through the fully connected layer, and train the neural network model;

[0062] Step 5: Deploy the parameters of the trained optimal network model on the on-site server, output the segmentation map and deformation field of the image to be detected, and perform defect detection and quantitative evaluation of conformity by analyzing the deformation field.

[0063] In this embodiment, the target blank flexible product refers to an original flexible product that has not been processed during the production process or does not have any logo, pattern, or structure.

[0064] In this embodiment, the sample set refers to a set of image data of a target blank flexible product used to build and train a model.

[0065] In this embodiment, the initial image refers to an original image obtained from each target blank flexible product.

[0066] In this embodiment, superposition refers to merging multiple initial images into one image, and aligning these images in space so that their pixel positions correspond.

[0067] In this embodiment, the template image is a "standard" image obtained by superimposing and aligning multiple groups of initial images of target blank flexible products.

[0068] In this embodiment, the flexible products to be inspected refer to those flexible products that need to undergo quality inspection, defect analysis, and deformation evaluation.

[0069] In this embodiment, the test sample refers to a target flexible product used for comparison and analysis with the template image during the detection process.

[0070] In this embodiment, the test image refers to an image obtained using a test sample for comparison and analysis with a template image.

[0071] In this embodiment, the comparison image refers to a comparison result image of the test image of the flexible product to be inspected and the template image after mapping and alignment.

[0072] In this embodiment, the neural network model is a computational model that simulates the way neurons in the human brain are connected, and can automatically learn features from input data and make predictions or classifications. Figure 2 shown.

[0073] In this embodiment, the convolution layer is a core component in a convolutional neural network (CNN), which is applied to fields such as image processing and speech recognition. The specific structure is as follows: Figure 3 shown.

[0074] In this embodiment, features refer to information representing local and global patterns, textures, and shapes in an image.

[0075] In this embodiment, the convolution neuron is a basic computing unit that automatically extracts local features in the image, such as edges, corners, textures, etc., by performing a convolution operation on the input image.

[0076] In this embodiment, the local area is in the convolution layer, and a small matrix is ​​selected as the "convolution kernel", such as a size of 3x3, 5x5, etc. The convolution kernel will gradually slide on the input image, covering a small area of ​​the image each time, and the area is the local area.

[0077] In this embodiment, the fully connected layer is used in the later stage of the neural network to comprehensively process the features extracted by the convolutional layer and finally output the prediction results of the model.

[0078] In this embodiment, the segmentation result map represents the result of classifying or marking different regions in the image.

[0079] In this embodiment, the deformation field map is an image containing displacement information, wherein each pixel contains one or more displacement components, indicating the offset of the position after deformation.

[0080] In this embodiment, the "optimal network model" refers to a convolutional neural network (CNN) model with the best performance obtained through the training process.

[0081] In this embodiment, the field server refers to a server deployed and used in production, testing, monitoring or other actual working environments.

[0082] In this embodiment, conformity refers to evaluating whether the sample meets the expected standard design specifications by analyzing the difference between the flexible product and the template image.

[0083] The working principle and beneficial effects of the above technical solution are: by constructing a neural network model, utilizing the comparison between the template image and the image to be detected, extracting features and outputting the segmentation map and deformation field map, defect detection and quantitative evaluation of conformity of flexible products can be achieved, thereby improving detection accuracy and efficiency.

[0084] Embodiment 2:

[0085] On the basis of the above-mentioned embodiment 1, multiple groups of target blank flexible products are obtained as sample sets, and an initial image of each target blank flexible product in the sample set is obtained to obtain an initial image set, including:

[0086] Acquire multiple target blank flexible products with completely similar shapes and properties to form a sample set;

[0087] The initial image of each target blank flexible product in the sample set is collected based on the video acquisition device, and finally the initial image set corresponding to the sample set is obtained.

[0088] In this embodiment, shape properties refer to various features or attributes that describe the shape of an object, such as size, geometry, structure, etc.

[0089] In this embodiment, the video acquisition device refers to a hardware device used to obtain and capture video images.

[0090] The working principle and beneficial effects of the above technical solution are: by acquiring multiple target blank flexible products of similar shapes and using video equipment to capture initial images, an initial image set is formed to provide a high-quality, standardized data basis for subsequent image analysis and detection, thereby improving detection accuracy and efficiency.

[0091] Embodiment 3:

[0092] Based on the above embodiment 2, the initial image sets are aligned and superimposed to generate a template image, including:

[0093] Convert each initial image in the initial image set into a grayscale image, and normalize the pixels of any grayscale image to obtain a standard image of the grayscale image, obtain a standard image set corresponding to the initial image set, and obtain key points of each standard image based on manual screening;

[0094] Select any image in the standard image set as a reference image, map the reference image to an alignment coordinate system, and identify key point coordinates of the reference image;

[0095] Obtain any other standard image in the standard image set as the image to be aligned, and align the image to be aligned with the reference image through perspective transformation according to the matching relationship between the key points of the image to be aligned and the reference image;

[0096] All aligned standard images are superimposed, and the pixel mean at the corresponding position is obtained by averaging the pixel values ​​at the same position of all standard images;

[0097] Generate a template image based on the image composed of the mean values ​​of pixels at all positions.

[0098] In this embodiment, the grayscale image is an image that contains only grayscale level information, and is created by converting each pixel in the color image into a single brightness value (ie, grayscale value).

[0099] In this embodiment, a pixel is the most basic unit of an image, and each pixel represents a small area in the image.

[0100] In this embodiment, the normalization process refers to scaling the pixel values ​​in the image to a uniform range, so that the pixel values ​​of different images can be compared and processed at the same scale.

[0101] In this embodiment, the standard image is an image that has been preprocessed and normalized and is used for image alignment, feature extraction, and template generation.

[0102] In this embodiment, key points refer to those feature points in the image that are significant, stable and easy to identify.

[0103] In this embodiment, the reference image is an image used as a reference in the alignment process.

[0104] In this embodiment, the alignment coordinate system refers to a unified reference coordinate system used in the image alignment process.

[0105] In this embodiment, perspective transformation is used to map points in one image to corresponding points in another image through a transformation matrix.

[0106] The working principle and beneficial effects of the above technical solution are: by converting the initial image into a standard image and performing key point matching and perspective transformation on it, the images are aligned and superimposed, and finally a template image is generated through the pixel mean, thereby improving image consistency and accuracy, and having a better image alignment and fusion effect.

[0107] Embodiment 4:

[0108] On the basis of the above-mentioned embodiment 1, a flexible product to be tested is obtained as a test sample, and a test image of the test sample is obtained, and the test image and the template image are mapped to the same coordinate system to obtain a comparison image, including:

[0109] A flexible product to be inspected is obtained as a test sample, a first image of the test sample is obtained based on recognition by an image detection system, and the first image is processed based on standard image information of a template image to obtain a test image;

[0110] Construct a standard coordinate system, locate the upper left corner of the test image and the template image to coincide with each other, and determine the matching score of the corresponding point located between the upper left corner of the test image and the template image:

[0111] ; Among them, MT represents the pixel length of the fixed-size image in the upper left corner, and NT represents the pixel width of the fixed-size image in the upper left corner. represents the matching score between the test image point A and the template image point B, Represents the test image point The pixel value of Represents the template image point The pixel value of represents the dot product operation, Represents the mean pixel value of the test image in a fixed-size image. Represents the mean pixel value of the template image in a fixed-size image;

[0112] Construct a sliding window and search for the point with the maximum matching score between the test image and the template image in a fixed-size image based on macro-region positioning for:

[0113] ;in, Indicates the location coordinates of the maximum matching score point, represents the set of positions of the sliding window, represents the macro area positioning function based on sliding window;

[0114] Based on the point with the maximum matching score, the test image and the template image are mapped into the same coordinate system to obtain a comparison image.

[0115] In this embodiment, the image detection system is a system that uses computer vision and image processing technology to automatically analyze, identify and process target information in an image.

[0116] In this embodiment, the first image refers to an original image of the flexible product to be inspected, which is collected in the image inspection system.

[0117] In this embodiment, the standard image information refers to the image data contained in the template image, including: size, resolution, pixel distribution, etc.

[0118] In this embodiment, the matching score is a measure of the consistency of similarity between the test image and the template image.

[0119] In this embodiment, the sliding window uses a window of a fixed size, "slides" on the image at a certain step length, and processes or analyzes the image area at each sliding position.

[0120] In this embodiment, the macro region is located in a larger area (ie, the macro region), and the position of the feature target in the image is determined based on a sliding window.

[0121] In this embodiment, the maximum matching score refers to the best matching score point calculated during the image matching process.

[0122] The working principle and beneficial effects of the above technical solution are: through image processing and template matching, the maximum matching score point is located using a sliding window, the test image is aligned with the template image, and a comparison image is generated. This method improves the accuracy of detection and image alignment and is suitable for flexible product detection.

[0123] Embodiment 5:

[0124] On the basis of the above-mentioned embodiment 1, a neural network model is constructed, a contrast image is input, and features of the contrast image are extracted based on a convolutional layer, wherein each convolutional neuron is connected to a local area of ​​the contrast image, including:

[0125] Build a model based on convolutional neural network, and pass the input contrast image to the first layer of convolutional neural network;

[0126] The contrast image is segmented into multiple local regions based on a pixel-level segmentation network, a convolution operation is performed on any convolutional neuron of the first layer and any region to obtain a local receptive field of the corresponding region, the local receptive field is processed based on a nonlinear activation function, and all pixel coordinates corresponding to the first feature map are generated;

[0127] Performing dimension reduction and size control operations on the first feature map based on the pooling layer to obtain a second feature map;

[0128] The second feature map is output to other layers in the convolutional neural network based on the fully connected layer.

[0129] In this embodiment, the pixel-level segmentation network is a neural network structure used for image segmentation, which classifies each pixel of the image into a specific label category, and is used to extract areas with specific meanings from the image, such as objects, boundaries or other structures.

[0130] In this embodiment, the convolution operation performs weighted sum operations on the input data by sliding a matrix called a convolution kernel to generate an output feature map. This process is usually called convolution.

[0131] In this embodiment, the local receptive field refers to the input data area of ​​a specific neuron or convolution kernel in a certain layer of the neural network.

[0132] In this embodiment, the nonlinear activation function is a key component in the neural network. The introduction of nonlinear features enables the network to learn and approximate more complex functions.

[0133] In this embodiment, the first feature map refers to a feature map generated by the first layer of convolutional neurons after the convolution operation.

[0134] In this embodiment, the pooling layer is used to reduce the dimension and compress the feature graph in the convolutional neural network, reduce the computational complexity, and improve the robustness of the model.

[0135] In this embodiment, the second feature map is the result of processing the first feature map through a pooling layer.

[0136] The working principle and beneficial effects of the above technical solution are: by combining convolutional neural network (CNN) with pixel-level segmentation, local receptive field analysis of contrast images is realized, and features are extracted and dimension reduced using convolution, activation function and pooling operations, and finally output through a fully connected layer, thereby enhancing the ability of image detail processing and feature expression, and improving the accuracy and robustness of the model.

[0137] Embodiment 6:

[0138] On the basis of the above-mentioned embodiment 5, the features are output through the fully connected layer to obtain the segmentation result map and the deformation field map, and the neural network model is trained, including:

[0139] Obtain a second feature map based on the fully connected layer, and perform a first segmentation on the second feature map using an encoder of a pixel-level segmentation network;

[0140] Based on the initial parameters of the pixel-level segmentation network, a spatial transformation network is configured. The spatial transformation network performs an affine transformation on the first segmentation result based on the preset affine transformation parameters to obtain a transformation matrix. At the second feature map input end, a position encoding method using trigonometric functions to alternately represent the coordinate position is used to input the position information of the foreground coordinate point of the template image into the network pixel-level segmentation network.

[0141] Based on the positioning network, the transformed image network is output using the transformation matrix, and interpolated and reconstructed to output a pixel-level segmentation deformation map;

[0142] In the training phase, the mean square error between the pixel-level segmentation deformation map and the template image is calculated to obtain the total variation of the pixel-level segmentation deformation map. The mean square error and total variation are used as parameters of the loss function, and the total loss coefficient is output:

[0143] ;in, represents the total loss coefficient, represents the mean square error between the pixel-level segmentation deformation map and the template image, represents the total variation of the pixel-level segmentation deformation map, represents the weight coefficient of the mean square error, represents the weight coefficient of total variation, represents the pixel value of the kth pixel in the pixel-level segmentation deformation map, represents the pixel value of the kth pixel of the template image, m represents the minimum value of the two pixel numbers of the template image and the pixel-level segmentation deformation image, Represents the X-direction gradient of the point at position pos in the pixel-level segmentation deformation map, Represents the Y-direction gradient of the point at position pos in the pixel-level segmentation deformation map, represents the pixel-level segmentation deformation map range, d represents the dimension, Indicates tuning of hyperparameters;

[0144] The total loss coefficient is combined with the stochastic gradient descent algorithm for back propagation to optimize the initial parameters of the pixel-level segmentation network and the preset affine transformation parameters of the spatial transformation network to generate the optimal network model.

[0145] In this embodiment, the encoder refers to a part of the pixel-level segmentation network, which is mainly responsible for extracting high-level feature representations from the input image.

[0146] In this embodiment, the initial parameters refer to the weights and biases of the initial state in the network, which are randomly initialized at the beginning of training.

[0147] In this embodiment, the spatial transformer network is a neural network module that is used to automatically learn the spatial transformation of input data, thereby improving the performance of downstream tasks.

[0148] In this embodiment, the preset affine transformation parameters refer to initial parameters set in advance in the spatial transformation network, which are used to control the spatial transformation of the image.

[0149] In this embodiment, the affine transformation performs operations such as translation, rotation, scaling, and shearing on the image without changing the linearity and relative proportions of the image.

[0150] In this embodiment, the transformation matrix refers to a matrix learned through a spatial transformation network, which is used to perform spatial transformation on an input image.

[0151] In this embodiment, the foreground coordinate point refers to the pixel coordinate representing the target area in the image.

[0152] In this embodiment, the image network refers to a deep learning model for image processing, feature extraction and segmentation, and a label is assigned to each region.

[0153] In this embodiment, interpolation reconstruction is used to calculate new pixel values ​​based on existing pixel data.

[0154] In this embodiment, the pixel-level segmentation deformation map refers to an image generated by the pixel-level segmentation network and the spatial transformation network.

[0155] In this embodiment, the mean square error is a coefficient used to measure the difference between two images or signals.

[0156] In this embodiment, total variation is a regularization method in image processing, which is used for denoising, smoothing images, or controlling the smoothness of images.

[0157] In this embodiment, the loss function is a function used to measure the difference between the predicted value and the actual value and guide the model optimization.

[0158] In this embodiment, the total loss coefficient refers to a measure that comprehensively measures the difference between the network output result and the target image during the training process.

[0159] In this embodiment, the stochastic gradient descent algorithm is an algorithm that updates the parameters of the model by randomly extracting one or a small number of samples from a data set and calculating the gradient of the sample.

[0160] In this embodiment, back propagation calculates the gradient of the loss function with respect to the weights of each layer of the network, propagates from the output layer to the input layer layer by layer, and updates the weights in the network in reverse to minimize the loss.

[0161] The working principle and beneficial effects of the above technical solution are: by combining the pixel-level segmentation network with the spatial transformation network, using affine transformation and position encoding to optimize the image segmentation process, calculating the mean square error and total variation as the loss function, combining the gradient descent algorithm to optimize the model parameters, thereby improving the image segmentation accuracy and deformation robustness.

[0162] Embodiment 7:

[0163] On the basis of the above-mentioned embodiment 1, the parameters of the trained optimal network model are deployed on the on-site server, and the segmentation map and deformation field of the image to be detected are output, including:

[0164] Configure and initialize the field server according to preset parameters, and import the optimal network model into the field server;

[0165] By inputting the image to be detected transmitted on the production line and the template image in the template library, after forward inference of the optimal network model, the segmentation map and deformation field are obtained, where the segmentation map is displayed as the segmentation result, and the deformation field is used as the key analysis image to analyze the connected domain and energy field;

[0166] The pixel points at each coordinate of the deformation field represent the horizontal and vertical coordinate displacements from the segmentation result of the image to be detected to each pixel point of the segmentation deformation map.

[0167] In this embodiment, the preset parameters include: system IP, model parameters, initial weights, etc.

[0168] In this embodiment, a production line refers to a continuous process or network of workstations used for production or manufacturing.

[0169] In this embodiment, the template library refers to a database storing a plurality of template images.

[0170] In this embodiment, forward reasoning refers to the process of processing input data through a trained deep learning model to generate output results.

[0171] In this embodiment, the segmentation map refers to dividing the input image into different regions through image segmentation technology.

[0172] In this embodiment, the deformation field is a mathematical representation that describes the deformation of the image shape.

[0173] In this embodiment, the connected domain refers to a region with the same characteristics (usually the same pixel value) in the binary image, and the pixels in these regions are connected to each other in a certain connection manner.

[0174] In this embodiment, the energy field is used to describe the distribution of a certain "force" or "influence" in different regions of the image, and can be regarded as a global or local metric describing the image content or deformation.

[0175] The working principle and beneficial effects of the above technical solution are: by deploying the optimal network model to the on-site server, real-time image segmentation and shape field analysis are realized, the segmentation map is used for result display, the deformation field is used for key analysis, accurate image displacement and connected domain analysis are provided, and the detection efficiency and accuracy of the production line are improved.

[0176] Embodiment 8:

[0177] On the basis of the above-mentioned embodiment 7, defect detection and quantitative evaluation of conformity are performed by analyzing the deformation field, including:

[0178] Obtain the dual-channel grayscale values ​​of the deformation field and construct a grayscale value matrix;

[0179] The deformation field is subjected to binary processing using a preset area threshold to obtain a binary image, and a connected domain labeling algorithm is applied to the binary image to obtain a label and an area of ​​each connected domain;

[0180] For any connected domain, the energy of the connected domain is generated by using the area integration of the dual-channel grayscale values;

[0181] If the absolute value difference between the energy of the connected domain and the energy standard threshold, as well as the absolute value difference between the area and the area standard threshold are both greater than the standard qualified threshold, it is determined that the image to be detected has defects within the pixel range of the connected domain.

[0182] In this embodiment, the dual-channel grayscale value refers to a data representation method with two grayscale channels, which means that each pixel point of the image contains two different grayscale value channels.

[0183] In this embodiment, the gray value matrix refers to a matrix formed by arranging the gray value of each pixel in an image according to its spatial position in image processing.

[0184] In this embodiment, the preset area threshold is a parameter used to screen out connected domains of a specific size during the binarization process.

[0185] In this embodiment, binary processing is a processing method in image processing for converting an image into an image containing only two gray levels.

[0186] In this embodiment, each pixel in the binary image contains only two possible values: usually black (0) and white (1).

[0187] In this embodiment, the connected region labeling algorithm is an algorithm for identifying different connected regions in a binary image and assigning a unique label to each connected region.

[0188] In this embodiment, the standard qualified threshold is a reference value used to determine whether there is a defect in the image.

[0189] The working principle and beneficial effects of the above technical solution are: by binarizing the deformation field and marking the connected domain, the energy is calculated by integrating the grayscale values ​​of the connected domain and comparing it with the standard threshold to determine the defective area in the image, thereby improving the accuracy and automation level of image defect detection.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for defect detection and conformity quantitative evaluation of industrial flexible products, characterized in that: include: Step 1: obtaining multiple groups of target blank flexible products as sample sets, and obtaining an initial image of each target blank flexible product in the sample set to obtain an initial image set, aligning and superimposing the initial image sets to generate a template image; Step 2: obtaining a flexible product to be detected as a test sample, obtaining a test image of the test sample, mapping the test image and the template image to the same coordinate system, and obtaining a comparison image, including: obtaining a flexible product to be detected as a test sample, obtaining a first image of the test sample based on recognition by an image detection system, and performing image processing on the first image based on standard image information of the template image to obtain a test image; Construct a standard coordinate system, locate the upper left corner of the test image and the template image to coincide with each other, and determine the matching score of the corresponding point located between the upper left corner of the test image and the template image: ; Among them, MT represents the pixel length of the fixed-size image in the upper left corner, and NT represents the pixel width of the fixed-size image in the upper left corner. represents the matching score between the test image point A and the template image point B, Represents the test image point The pixel value of Represents the template image point The pixel value of represents the dot product operation, Represents the mean pixel value of the test image in a fixed-size image. Represents the mean pixel value of the template image in a fixed-size image; Construct a sliding window and search for the point with the maximum matching score between the test image and the template image in a fixed-size image based on macro-region positioning for: ;in, Indicates the location coordinates of the maximum matching score point, represents the set of positions of the sliding window, represents the macro area positioning function based on sliding window; Based on the point with the maximum matching score, the test image and the template image are mapped to the same coordinate system to obtain a comparison image; Step 3: construct a neural network model, input the contrast image, and extract features of the contrast image based on a convolutional layer, wherein each convolutional neuron is connected to a local area of ​​the contrast image; Step 4: Output the segmentation result map and the deformation field map through the fully connected layer, and train the neural network model, including: obtaining the second feature map based on the fully connected layer, and performing the first segmentation on the second feature map using the encoder of the pixel-level segmentation network; Based on the initial parameters of the pixel-level segmentation network, a spatial transformation network is configured. The spatial transformation network performs an affine transformation on the first segmentation result based on the preset affine transformation parameters to obtain a transformation matrix. At the second feature map input end, a position encoding method using trigonometric functions to alternately represent the coordinate position is used to input the position information of the foreground coordinate point of the template image into the pixel-level segmentation network. Based on the positioning network, the transformed image network is output using the transformation matrix, and interpolated and reconstructed to output a pixel-level segmentation deformation map; In the training phase, the mean square error between the pixel-level segmentation deformation map and the template image is calculated to obtain the total variation of the pixel-level segmentation deformation map. The mean square error and total variation are used as parameters of the loss function, and the total loss coefficient is output: The total loss coefficient is combined with the stochastic gradient descent algorithm for back propagation to optimize the initial parameters of the pixel-level segmentation network and the preset affine transformation parameters of the spatial transformation network to generate the optimal network model; Step 5: Deploy the parameters of the trained optimal network model on the on-site server, output the segmentation map and deformation field of the image to be detected, and perform defect detection and quantitative evaluation of conformity by analyzing the deformation field.

2. The method for defect detection and conformity quantitative evaluation of industrial flexible products according to claim 1, characterized in that: A plurality of groups of target blank flexible products are obtained as sample sets, and an initial image of each target blank flexible product in the sample set is obtained to obtain an initial image set, including: Acquire a plurality of target blank flexible products with similar shapes to form a sample set; The initial image of each target blank flexible product in the sample set is collected based on the video acquisition device, and finally the initial image set corresponding to the sample set is obtained.

3. The method for defect detection and conformity quantitative evaluation of industrial flexible products according to claim 2, characterized in that: Align and overlay the initial image sets to generate a template image, including: Convert each initial image in the initial image set into a grayscale image, and normalize the pixels of any grayscale image to obtain a standard image of the grayscale image, obtain a standard image set corresponding to the initial image set, and obtain key points of each standard image based on manual screening; Select any image in the standard image set as a reference image, map the reference image to an alignment coordinate system, and identify key point coordinates of the reference image; Obtain any other standard image in the standard image set as the image to be aligned, and align the image to be aligned with the reference image through perspective transformation according to the matching relationship between the key points of the image to be aligned and the reference image; All aligned standard images are superimposed, and the pixel mean at the corresponding position is obtained by averaging the pixel values ​​at the same position of all standard images; Generate a template image based on the image composed of the mean values ​​of pixels at all positions.

4. The method for defect detection and conformity quantitative evaluation of industrial flexible products according to claim 1, characterized in that: Construct a neural network model, input a contrast image, and extract features of the contrast image based on a convolution layer, wherein each convolution neuron is connected to a local area of ​​the contrast image, including: Build a model based on convolutional neural network, and pass the input contrast image to the first layer of convolutional neural network; The contrast image is segmented into multiple local regions based on a pixel-level segmentation network, a convolution operation is performed on any convolutional neuron of the first layer and any region to obtain a local receptive field of the corresponding region, the local receptive field is processed based on a nonlinear activation function, and all pixel coordinates corresponding to the first feature map are generated; Based on the pooling layer, dimension reduction and size control operations are performed on the first feature map to obtain a second feature map.

5. The method for defect detection and conformity quantitative evaluation of industrial flexible products according to claim 1, characterized in that: The parameters of the trained optimal network model are deployed on the on-site server, and the segmentation map and deformation field of the image to be detected are output, including: Configure and initialize the field server according to preset parameters, and import the optimal network model into the field server; By inputting the image to be detected transmitted on the production line and the template image in the template library, after forward reasoning of the optimal network model, the segmentation map and deformation field are obtained, where the segmentation map is displayed as the segmentation result, and the deformation field is used as the key analysis image to analyze the connected domain and energy field; The pixel points at each coordinate of the deformation field represent the horizontal and vertical coordinate displacements from the segmentation result of the image to be detected to each pixel point of the segmentation deformation map.

6. The method for defect detection and conformity quantitative evaluation of industrial flexible products according to claim 5, characterized in that: By analyzing the deformation field, defect detection and quantitative evaluation of conformity are performed, including: Obtain the dual-channel grayscale values ​​of the deformation field and construct a grayscale value matrix; The deformation field is subjected to binary processing using a preset area threshold to obtain a binary image, and a connected domain labeling algorithm is applied to the binary image to obtain a label and an area of ​​each connected domain; For any connected domain, the energy of the connected domain is generated by using the area integration of the dual-channel grayscale values; If the absolute value difference between the energy of the connected domain and the energy standard threshold, as well as the absolute value difference between the area and the area standard threshold are both greater than the standard qualified threshold, it is determined that the image to be detected has defects within the pixel range of the connected domain.

Images