Intelligent visual inspection system and method for non-standard automatic production line

Abstract

The invention discloses an intelligent visual detection system and method for a non-standard automatic production line, and relates to the technical field of intelligent visual detection, and the method comprises the steps: carrying out the enhancement of a to-be-detected sample image, carrying out the normalization of the image, carrying out the edge detection of the image through a Canny algorithm, the image enhancement effect is indirectly evaluated according to the peak signal-to-noise ratio and the edge number of the image, and the enhancement effect of the image is analyzed and adjusted according to the indirect evaluation effect and the threshold value for limiting the contrast ratio; obtaining the number of known defect samples, calculating the omission ratio, the accuracy rate and the false detection rate in the detection process in combination with the number of the defect samples, evaluating the visual detection performance based on a machine learning algorithm, and judging whether to optimize the detection system or not according to an evaluation result; the problems of inaccurate manual detection and poor detection effect in the prior art are solved, and the working efficiency and the detection accuracy of a non-standard automatic production line are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and more specifically, to an intelligent visual detection system and method for a non-standard automated production line. Background Art

[0002] Non-standard automated production lines refer to automated production systems customized according to specific products or production requirements, which usually do not conform to the design of traditional standardized equipment. This type of production line is highly flexible and can meet diverse and personalized production requirements, and is widely used in different fields. With the rapid development of computer technology and artificial intelligence, machine vision has been widely used in the field of industrial product inspection by obtaining surface images of identified objects to determine defects.

[0003] Deficiencies of existing technologies:

[0004] In the current tire production process of the automotive industry, manual inspection and handling are generally performed by operators, which is inefficient and labor-intensive, and has gradually failed to meet production needs. Therefore, it is necessary to replace the original manual operations with automated equipment and improve the efficiency of the production line through intelligent visual inspection of the production line.

[0005] In view of the above problems, the present invention proposes a solution. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an intelligent visual inspection system and method for a non-standard automated production line, which solves the problems raised in the above-mentioned background technology through intelligent visual inspection of the production line.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An intelligent visual inspection system for a non-standard automated production line comprises: an image acquisition module, which is used to simulate an automated production line inspection process, perform image acquisition on known samples to be inspected, and obtain images of the samples to be inspected; an image processing module, which is used to enhance the images of the samples to be inspected according to an OpenCV-Python visual algorithm library, normalize the images, perform edge detection on the images using a Canny algorithm, and indirectly evaluate the effect of image enhancement according to an image peak signal-to-noise ratio and the number of edges, and analyze and adjust the image enhancement effect of a threshold value for limiting contrast in the OpenCV-Python visual algorithm according to the indirect evaluation effect; an image recognition module, which is used to perform end-to-end defect recognition on images according to a CNN model, and obtain the number of defective samples; a performance evaluation module, which is used to obtain the number of known defective samples, calculate the missed detection rate, the precision rate and the false detection rate in the inspection process in combination with the number of defective samples, evaluate the visual inspection performance based on a machine learning algorithm, and determine whether to optimize the inspection system according to the evaluation result.

[0009] In a preferred embodiment, the enhancing of the sample image to be detected according to the OpenCV-Python visual algorithm library includes enhancing the image according to adaptive histogram equalization, gamma transformation, median filtering and multi-scale Retinex algorithm.

[0010] In a preferred embodiment, the specific process of image enhancement by the multi-scale Retinex algorithm is as follows: the input image is divided into an incident component and a reflected component; the input image is processed according to the single-scale Retinex algorithm, the incident component in the input image is filtered out by Gaussian low-pass filtering, and the reflected component is extracted; the input image is processed using the single-scale Retinex algorithm at multiple scales to obtain reflected components of different scales, and the reflected components of different scales are weightedly fused to obtain the output image enhanced by the multi-scale Retinex algorithm.

[0011] In a preferred embodiment, the steps of edge detection of an image using the Canny algorithm are as follows: performing Gaussian filtering on the image to remove noise; performing edge detection on the image using the Sobel operator to calculate the gradients of the image in the horizontal and vertical directions to obtain the gradient strength and direction of each pixel; checking the two neighboring pixel values ​​of each pixel along the gradient direction based on the gradient direction, if the gradient amplitude of the current pixel is greater than the values ​​of the two neighboring pixels, the pixel is retained, otherwise it is suppressed; using a high threshold and a low threshold to determine the strength of the edge, pixels stronger than the high threshold are considered to be "definite edges", pixels lower than the low threshold are considered to be "non-edge pixels", and pixels between the two need to be further judged through a connection step; confirming whether the pixel marked as "possibly an edge" belongs to an actual edge through edge connection, if it is connected to a "definite edge" pixel, it is retained as an edge pixel, otherwise it is suppressed.

[0012] In a preferred embodiment, the specific steps of adjusting the threshold size of limiting contrast are as follows: defining the number of edges and the image peak signal-to-noise ratio as input variables, and dividing them into different fuzzy sets respectively; defining whether to adjust the threshold size of limiting contrast as an output variable, and dividing them into fuzzy sets; formulating fuzzy rules to describe the influence of the number of edges and the image peak signal-to-noise ratio on the adjustment of the threshold size of limiting contrast; and performing fuzzy reasoning according to the fuzzy rules to determine whether to adjust the threshold size of limiting contrast.

[0013] In a preferred embodiment, the specific process of performing end-to-end defect recognition on images according to the CNN model to obtain the number of defect samples is as follows: performing data enhancement on the image data to avoid overfitting; extracting local features of the image layer by layer according to the convolution layer, and gradually reducing the size of the feature map through the pooling layer to extract more abstract features; using a fully connected layer to map the features extracted by the convolution layer to the final layer, and outputting whether the defect exists in the final layer; and calculating the number of defect samples based on the output result of the final layer.

[0014] In a preferred embodiment, the process of determining whether to optimize the detection system based on the evaluation results is as follows: comparing and analyzing the detection evaluation coefficient with a preset evaluation threshold; if the detection evaluation coefficient is greater than the preset evaluation threshold, the system visual detection performance is good and the detection system does not need to be optimized; if the detection evaluation coefficient is less than the preset evaluation threshold, the system visual detection performance is poor and the detection system needs to be optimized.

[0015] An intelligent visual inspection method for a non-standard automated production line comprises: simulating an automated production line inspection process, collecting images of known samples to be inspected, and obtaining images of samples to be inspected; enhancing the images of samples to be inspected according to an OpenCV-Python visual algorithm library, normalizing the images, using a Canny algorithm to detect edges of the images, indirectly evaluating the effect of image enhancement according to an image peak signal-to-noise ratio and the number of edges, and analyzing and adjusting the image enhancement effect of a threshold value for limiting contrast in an OpenCV-Python visual algorithm according to the indirect evaluation effect; performing end-to-end defect recognition on images according to a CNN model, and obtaining the number of defective samples; obtaining the number of known defective samples, calculating the missed detection rate, the precision rate, and the false detection rate in the inspection process in combination with the number of defective samples, evaluating the visual inspection performance based on a machine learning algorithm, and judging whether to optimize the inspection system according to the evaluation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure is a schematic diagram of the structure of an intelligent visual inspection system for a non-standard automated production line of the present invention.

[0017] Figure 2 The present invention is a schematic structural diagram of an intelligent visual inspection method for a non-standard automated production line. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1, Figure 1 The invention provides an intelligent visual inspection system for a non-standard automated production line.

[0020] The image acquisition module is used to simulate the detection process of the automated production line, collect images of known samples to be detected, and obtain images of the samples to be detected;

[0021] The automated production line inspection system consists of multiple components working together, including conveyor belts, camera systems, light sources, processing units, etc.; its main goal is to use machine vision technology to perform online inspection, defect detection, dimensional measurement, etc. of products on the production line.

[0022] Production line: Conveyors move products from one location to another, usually at a steady speed;

[0023] Camera system: fixedly installed at an appropriate location on the production line to capture product images in real time;

[0024] Light source: Provide sufficient and uniform lighting conditions to ensure the quality of image acquisition;

[0025] Processing unit: analyzes the image through image processing algorithms, such as edge detection, object recognition, defect detection, etc.

[0026] The first step in image acquisition is to select a suitable camera device. Usually, you can choose an industrial camera or a high-resolution camera. The main considerations are:

[0027] Resolution: You need to choose the appropriate resolution based on the size and detail requirements of the product. High-resolution cameras help capture the tiny defects of the product more finely.

[0028] Frame rate: On high-speed production lines, it is critical to select a camera with a sufficient frame rate to ensure that images can be captured synchronously and do not affect the line speed;

[0029] Lens: According to the inspection requirements, select the appropriate focal length and field of view (FOV) lens to ensure that the image of the entire product can be clearly captured;

[0030] Interface and compatibility: Consider the compatibility of the camera with other systems (such as computers and image acquisition cards) to ensure stable data transmission;

[0031] The camera should be installed on the production line and maintain an appropriate relative position with the conveyor belt to ensure that the image of the sample to be tested can be clearly captured. The position and angle of the camera need to be adjusted to ensure that the key parts of the product can be captured in all directions to avoid image quality being affected by incorrect angles or obstructions;

[0032] Fixed installation position: The camera is usually installed above, on the side or below the production line. The specific location needs to be determined according to the size and appearance characteristics of the product.

[0033] Multi-camera collaboration: If the product has a complex shape, multiple cameras may be required to shoot from different angles simultaneously to obtain images from multiple perspectives;

[0034] Synchronous acquisition: Image acquisition of multiple cameras needs to be synchronized to ensure that image data is collected at the same time point to avoid image distortion due to time differences;

[0035] Appropriate lighting is the key to image acquisition. The choice and position of the light source directly affects the image quality. Common light source configurations include:

[0036] Uniform lighting: Ensure that the entire surface of the product is evenly illuminated to avoid shadows or highlights;

[0037] Directional lighting: Depending on the inspection requirements, it may be necessary to use directional light sources such as side lighting or backlighting to highlight specific features of the product, such as surface defects, contours, etc.

[0038] Avoid reflections: On products made of certain materials (such as metal and glass), special attention should be paid to the angle of the light source to avoid image distortion caused by reflections.

[0039] The images captured by the image acquisition device are transmitted to the computer or processing unit through a digital interface (such as USB, Ethernet, Camera Link, etc.). The acquired images are usually processed by the following steps:

[0040] Image preprocessing: including denoising, white balance adjustment, image enhancement, etc., to ensure that the image quality meets the analysis requirements;

[0041] Image storage and transmission: Images can be directly transferred to a computer for real-time processing, or stored on a local disk or database for subsequent analysis.

[0042] The image processing module is used to enhance the sample image to be detected according to the OpenCV-Python visual algorithm library, normalize the image, use the Canny algorithm to detect the edge of the image, and indirectly evaluate the effect of image enhancement based on the image peak signal-to-noise ratio and the number of edges. Based on the indirect evaluation effect, the threshold size of the contrast limit in the OpenCV-Python visual algorithm is analyzed and adjusted for the image enhancement effect.

[0043] The image enhancement according to the OpenCV-Python visual algorithm library includes adaptive histogram equalization, gamma transformation, median filtering and multi-scale Retinex algorithm.

[0044] The adaptive histogram equalization is to divide the image into multiple rectangular blocks and process them separately. For each rectangular block image, the corresponding histogram distribution function is statistically calculated, and then a threshold value for limiting contrast is added to optimize the transition between the rectangular blocks using linear interpolation and bilinear interpolation methods. In this way, the effect of reducing the noise amplification problem can be achieved, making the image look more coordinated and reasonable.

[0045] In adaptive histogram equalization, limiting the contrast threshold, that is, contrast-limited adaptive histogram equalization, can help control the image enhancement effect and avoid excessive enhancement leading to noise amplification or excessive pseudo-colorization of the image. In order to achieve better image enhancement effect, it is very important to adjust the threshold of limiting the contrast.

[0046] The gamma transformation is a nonlinear operation on the grayscale value of the input image, so that the grayscale value of the output image is exponentially related to the grayscale value of the input image to achieve image enhancement. The basic form of the function is:

[0047] S=c*r a

[0048] In the formula, S is the gray value after the change, c is the gray scaling coefficient, r is the gray value of the original pixel, and a is the transformation coefficient;

[0049] When a is less than 1, the brightness of the image can be enhanced, and the pixels with smaller grayscale values ​​can be quickly enlarged, which can brighten the dark areas of the image. When a is greater than 1, the pixels with larger grayscale values ​​can be rapidly enlarged, which can enhance the contrast of the image and suppress the highlight parts of the image.

[0050] The median filter is a statistical sorting filter, and its main method is to select a neighborhood template, such as 3×3, 5×5, etc., with each pixel of the input image as the center, sort the pixels in the template according to the size of the pixel value, and use the pixel value in the middle position as the pixel value of the current pixel.

[0051] The specific process of multi-scale Retinex algorithm image enhancement is as follows:

[0052] First, the input image is divided into two parts: the incident component and the reflected component, which can be expressed as: S(x,y)=R(x,y)*L(x,y); where S(x,y) represents the input image, R(x,y) represents the reflected component, and L(x,y) represents the incident component.

[0053] The input image is processed according to the single-scale Retinex algorithm, including filtering out the incident component L(x, y) in the input image with a Gaussian low-pass filter, thereby extracting the reflected component R(x, y). The specific formula is as follows: logR(x, y) = logS(x, y) - log[G(x, y)*S(x, y)]; where R(x, y) represents the reflected component, S(x, y) represents the input image, * is the convolution symbol, and G(x, y) is the center surround function;

[0054] Among them, the center surround function G(x,y) needs to meet the following conditions, the specific formula is: Where c is the Gaussian surround scale, λ is the central surround constant;

[0055] After processing the output image using the single-scale Retinex algorithm at multiple scales, the reflection components of different scales are obtained, and the reflection components of different scales are weighted fused to obtain the output image enhanced by the multi-scale Retinex algorithm. The specific formula is as follows: Where r(x,y) is the output function;

[0056] The normalization scales the image pixel values ​​to a specific range to facilitate subsequent processing, usually scaling the pixel values ​​to the range of [0, 1] or [0, 255];

[0057] The Canny algorithm is used to detect the edge of the image. The following steps are mainly used to detect the edge of the image:

[0058] First, perform Gaussian filtering on the image to remove noise; noise can cause false edge detection, so denoising is necessary;

[0059] The Sobel operator is used to perform edge detection on the image, and the gradients of the image in the horizontal and vertical directions are calculated to obtain the gradient strength and direction of the pixel points; the gradient strength calculation formula is as follows: Where q is the gradient strength, q x is the gradient in the horizontal direction, q y is the gradient in the vertical direction;

[0060] Based on the gradient direction, for each pixel, check the values ​​of the two neighboring pixels along the gradient direction. If the gradient magnitude of the current pixel is greater than the values ​​of the two neighboring pixels, retain the pixel, otherwise suppress it;

[0061] Two thresholds (high threshold and low threshold) are used to determine the strength of the edge. Pixels stronger than the high threshold are considered "definite edges", pixels lower than the low threshold are considered "non-edge pixels", and pixels between the two may be part of the edge and need to be further judged through the connection step;

[0062] Through edge connection, it is confirmed whether the pixels marked as "possible edges" belong to the actual edges. If these pixels are connected to the "determined edge" pixels, they are retained as edge pixels, otherwise they are suppressed; and the number of edges formed after edge connection is calculated;

[0063] If the number of edges increases significantly after enhancement, it means that the enhancement operation over-enhanced the details, resulting in increased edge noise;

[0064] The image peak signal-to-noise ratio (PSNR) is a commonly used indicator in image quality assessment, which is used to measure the difference between an enhanced image and an original image, and is particularly used to evaluate the effects of compression, denoising, image enhancement and other processing. The higher the PSNR value, the better the image enhancement effect and the smaller the distortion.

[0065] The PSNR is the ratio between the maximum value of the signal (image) and the noise (image distortion), and the specific calculation formula is as follows: PSNR is the peak signal-to-noise ratio of the image, S(j,k) is the pixel value of the jth row and kth column in the input image, r(j,k) is the pixel value of the jth row and kth column in the output function, MAX is the maximum pixel value, M and N are the height and width of the image respectively;

[0066] PSNR is an objective evaluation index, which is often used to evaluate the image enhancement effect. By comparing the PSNR values ​​of the original image and the enhanced image, the effectiveness of the enhancement method can be obtained:

[0067] A higher PSNR value usually means that the enhancement method has less damage to the image quality and retains more original information. A lower PSNR value means that the enhancement method introduces greater distortion or noise, resulting in a decrease in image quality.

[0068] The effect of image enhancement is indirectly evaluated based on the image peak signal-to-noise ratio and the number of edges, and whether the threshold value for limiting contrast is adjusted is determined based on the evaluation results. The specific steps are as follows:

[0069] In step C1, the number of edges and the peak signal-to-noise ratio of the image are defined as input variables, and they are divided into different fuzzy sets respectively.

[0070] For example, "Few", "Right", "Many" for the number of edges, "Low", "Medium", "High" for the image peak signal-to-noise ratio.

[0071] Step C2, whether to adjust the threshold value of limiting contrast is defined as an output variable, and it is divided into fuzzy sets, for example, "No", "Yes", for whether to adjust the threshold value of limiting contrast.

[0072] Step C3, formulate a set of fuzzy rules to describe the impact of different input variables on output variables. The definition of rules can be based on professional knowledge or obtained through data analysis and experiments. For example:

[0073] The number of edges is marked as B, the peak signal-to-noise ratio of the image is marked as D, and whether the threshold value of limiting contrast is adjusted is marked as P, then it can be defined

[0074] Rule 1:IF(B is Few)AND(D is High)THEN(P is No)

[0075] Rule 2:IF(B is Many)AND(D is Low)THEN(P is Yes) ...

[0077] Step C4, performing fuzzy reasoning according to fuzzy rules to determine whether to adjust the threshold value for limiting contrast.

[0078] It should be noted that the division of fuzzy sets can be adjusted according to actual conditions. For example, although the present embodiment takes three fuzzy sets as examples, in fact, the image peak signal-to-noise ratio and the number of edges and whether to adjust the threshold size for limiting the contrast can be divided into more than three sets to facilitate better and more precise adjustment.

[0079] Furthermore, for the judgment of whether to adjust the threshold value of limiting contrast, the threshold can be set according to the actual situation. For example, when the peak signal-to-noise ratio of the image exceeds 40 decibels, it is marked as "High", and when the number of edges is higher than 10, it is marked as "Many", etc., which will not be elaborated here.

[0080] Lower clip limit (smaller value): When a lower clip limit is set, the contrast enhancement effect will be more restricted, the details of the image may be more suppressed, and the enhancement effect will be more conservative. At this time, the noise gain of the image is less, and the overall effect may be smoother, which is suitable for scenes with more noise or low requirements for details.

[0081] Higher clip limit (larger value): A larger clip limit allows greater contrast enhancement in local areas, and the details and structure of the image will be more prominent, but it may also cause noise to be over-amplified, especially in low-contrast or noisy areas. A wide range of clip limits will result in significant image contrast enhancement, but it needs to be used with caution to avoid introducing unnecessary artifacts or noise.

[0082] Moderate clip limit: By gradually adjusting, a moderate clip limit value can be found to make the contrast enhancement of the image obvious without excessively amplifying the noise. Generally speaking, the clip limit can be adjusted between 0.01 and 0.05, depending on the characteristics of the image and the noise level.

[0083] The image recognition module is used to perform end-to-end defect recognition on the image based on the CNN model to obtain the number of defect samples;

[0084] End-to-end image defect recognition based on convolutional neural network (CNN) is a deep learning technology that identifies defects in images by automatically extracting image features and applying these features to classification tasks. The end-to-end process means that no manual feature engineering or additional intermediate steps are required from the input image to the output results (such as the classification or location of defects). The whole process includes data preprocessing, training of CNN models, and output of defect recognition results. The following are the specific steps:

[0085] First, image data containing defects and normal samples needs to be collected;

[0086] In order to enhance the robustness of the model and avoid overfitting, data augmentation is performed on the training image data;

[0087] Use convolutional layers to automatically extract local features of the image, such as edges, textures, shapes, etc., extract local features in the image layer by layer, and gradually reduce the size of the feature map through pooling layers;

[0088] Use the pooling layer to reduce the size of the feature map, reduce the amount of calculation, and extract more abstract features;

[0089] Use a fully connected layer to map the features extracted by the convolutional layer to the output space, usually in the last layer of the network for classification tasks, and output whether the defect exists in the final layer;

[0090] Calculate the number of defect samples based on the final layer output results.

[0091] The performance evaluation module is used to obtain the number of known defect samples, and calculate the missed detection rate, precision rate and false detection rate during the detection process based on the number of defect samples; evaluate the visual inspection performance based on the machine learning algorithm, and determine whether to optimize the inspection system based on the evaluation results.

[0092] Calculate the missed detection rate, precision rate and false detection rate in the detection based on the number of known defect samples and the number of defect samples;

[0093] The missed detection rate indicates the percentage of the number of undetected defect samples to the actual number of defect samples. It is an extremely important indicator in industrial surface defect detection. Missed detection will directly lead to defective products flowing to end customers, which will have a serious impact on the company's image and customer satisfaction. Therefore, reducing the missed detection rate is of great significance to the company.

[0094] The precision rate indicates the percentage of the number of detected defect samples to the actual number of defect samples;

[0095] The false detection rate refers to the percentage of qualified samples detected as defective samples to the actual number of qualified samples. False detection will have a direct negative impact on the production efficiency of industrial enterprises.

[0096] According to the missed detection rate, precision rate and false detection rate, an evaluation model is built based on the machine learning algorithm to generate the detection evaluation coefficient. The specific calculation formula is as follows: Where α is the missed detection rate, μ1 is the missed detection rate weight coefficient, β is the precision rate, μ2 is the precision rate weight coefficient, γ is the false detection rate, and μ3 is the false detection rate weight coefficient.

[0097] The detection evaluation coefficient is compared and analyzed with the preset evaluation threshold. If the detection evaluation coefficient is greater than the preset evaluation threshold, the system visual detection performance is good and the detection system does not need to be optimized; if the detection evaluation coefficient is less than the preset evaluation threshold, the system visual detection performance is poor and the detection system needs to be optimized.

[0098] Embodiment 2, Figure 2 The present invention is a schematic structural diagram of an intelligent visual inspection method for a non-standard automated production line.

[0099] Simulate the automated production line inspection process, collect images of known samples to be inspected, and obtain images of samples to be inspected;

[0100] According to the OpenCV-Python visual algorithm library, the sample image to be detected is enhanced and normalized. The Canny algorithm is used to detect the edge of the image, and the effect of image enhancement is indirectly evaluated based on the image peak signal-to-noise ratio and the number of edges. Based on the indirect evaluation effect, the contrast-limiting threshold in the OpenCV-Python visual algorithm is analyzed and adjusted for the image enhancement effect.

[0101] Perform end-to-end defect recognition on the image according to the CNN model to obtain the number of defect samples;

[0102] Obtain the number of known defect samples, calculate the missed detection rate, precision rate and false detection rate during the detection process based on the number of defect samples, evaluate the visual inspection performance based on the machine learning algorithm, and determine whether to optimize the inspection system based on the evaluation results.

[0103] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0104] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0105] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0106] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0108] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent visual inspection system for non-standard automated production lines, characterized in that: include: The image acquisition module is used to simulate the detection process of the automated production line, collect images of known samples to be detected, and obtain images of the samples to be detected; The image processing module is used to enhance the sample image to be detected according to the OpenCV-Python visual algorithm library, normalize the image, use the Canny algorithm to detect the edge of the image, and indirectly evaluate the effect of image enhancement according to the image peak signal-to-noise ratio and the number of edges, and analyze and adjust the image enhancement effect of the threshold value of limiting contrast in the OpenCV-Python visual algorithm according to the indirect evaluation effect; The image recognition module is used to perform end-to-end defect recognition on the image based on the CNN model to obtain the number of defect samples; The performance evaluation module is used to obtain the number of known defect samples, calculate the missed detection rate, precision rate and false detection rate during the detection process based on the number of defect samples, evaluate the visual inspection performance based on the machine learning algorithm, and determine whether to optimize the inspection system based on the evaluation results.

2. According to claim 1, the intelligent visual inspection system for non-standard automated production lines is characterized in that: The enhancing of the sample image to be detected according to the OpenCV-Python visual algorithm library includes enhancing the image according to adaptive histogram equalization, gamma transformation, median filtering and multi-scale Retinex algorithm.

3. The intelligent visual inspection system for non-standard automated production lines according to claim 2 is characterized in that: The specific process of the multi-scale Retinex algorithm to enhance the image is as follows: The input image is divided into an incident component and a reflected component; The input image is processed according to the single-scale Retinex algorithm, the incident component in the input image is filtered out using a Gaussian low-pass filter, and the reflected component is extracted; The input image is processed using a single-scale Retinex algorithm at multiple scales to obtain reflection components of different scales, and the reflection components of different scales are weighted fused to obtain an output image enhanced by a multi-scale Retinex algorithm.

4. The intelligent visual inspection system for non-standard automated production lines according to claim 3 is characterized in that: The steps of using the Canny algorithm to perform edge detection on an image are as follows: Perform Gaussian filtering on the image to remove noise; Use the Sobel operator to detect the edge of the image, calculate the horizontal and vertical gradients of the image, and obtain the gradient intensity and direction of each pixel; Based on the gradient direction, check the two neighboring pixel values ​​of each pixel along the gradient direction. If the gradient magnitude of the current pixel is greater than the values ​​of the two neighboring pixels, retain the pixel, otherwise suppress it; Use high and low thresholds to determine the strength of the edge. Pixels stronger than the high threshold are considered "definite edges", pixels lower than the low threshold are considered "non-edge pixels", and pixels between the two need to be further judged through the connection step; Through edge connection, it is confirmed whether the pixel marked as "possible edge" belongs to the actual edge. If it is connected to the "confirmed edge" pixel, it is retained as an edge pixel, otherwise it is suppressed.

5. The intelligent visual inspection system for non-standard automated production lines according to claim 4, characterized in that: The specific steps of adjusting the threshold value for limiting contrast are as follows: The number of edges and image peak signal-to-noise ratio are defined as input variables and divided into different fuzzy sets respectively; Whether to adjust the threshold value of limiting contrast is defined as the output variable, and it is divided into fuzzy sets; Fuzzy rules are formulated to describe the influence of edge number and image peak signal-to-noise ratio on the threshold size for limiting contrast. Fuzzy reasoning is performed according to fuzzy rules to determine whether the threshold value for limiting contrast should be adjusted.

6. The intelligent visual inspection system for non-standard automated production lines according to claim 5, characterized in that: The specific process of performing end-to-end defect recognition on an image according to the CNN model and obtaining the number of defect samples is as follows: Perform data enhancement on image data to avoid overfitting; The local features of the image are extracted layer by layer according to the convolution layer, and the size of the feature map is gradually reduced through the pooling layer to extract more abstract features; Use a fully connected layer to map the features extracted by the convolutional layer to the final layer, and output whether the defect exists in the final layer; Calculate the number of defect samples based on the final layer output results.

7. The intelligent visual inspection system for non-standard automated production lines according to claim 6, characterized in that: The process of determining whether to optimize the detection system according to the evaluation results is as follows: Compare and analyze the detection evaluation coefficient with the preset evaluation threshold. If the detection evaluation coefficient is greater than the preset evaluation threshold, the system visual detection performance is good and there is no need to optimize the detection system. If the detection evaluation coefficient is less than the preset evaluation threshold, the system's visual detection performance is poor and the detection system needs to be optimized.

8. The intelligent visual inspection system for non-standard automated production lines according to claim 7, characterized in that: The specific formula of the output image after the multi-scale Retinex algorithm enhancement is as follows: Where r(x,y) represents the output image, S(x,y) represents the input image, * is the convolution symbol, and G(x,y) is the center-surround function.

9. The intelligent visual inspection system for non-standard automated production lines according to claim 8, characterized in that: The specific calculation formula of the image peak signal-to-noise ratio is as follows: PSNR is the peak signal-to-noise ratio of the image, S ( j,k) is the pixel value of the jth row and kth column in the input image, r(j,k) is the pixel value of the jth row and kth column in the output function, MAX is the maximum pixel value, and M and N are the height and width of the image respectively.

10. An intelligent visual inspection method for non-standard automated production lines, characterized in that: include: Simulate the automated production line inspection process, collect images of known samples to be inspected, and obtain images of samples to be inspected; According to the OpenCV-Python visual algorithm library, the sample image to be detected is enhanced and normalized. The Canny algorithm is used to detect the edge of the image, and the effect of image enhancement is indirectly evaluated based on the image peak signal-to-noise ratio and the number of edges. Based on the indirect evaluation effect, the contrast-limiting threshold in the OpenCV-Python visual algorithm is analyzed and adjusted for the image enhancement effect. Perform end-to-end defect recognition on the image according to the CNN model to obtain the number of defect samples; Obtain the number of known defect samples, calculate the missed detection rate, precision rate and false detection rate during the detection process based on the number of defect samples, evaluate the visual inspection performance based on the machine learning algorithm, and determine whether to optimize the inspection system based on the evaluation results.

Images