Visual detection workpiece production line based on CCD and detection method

Abstract

The invention discloses a CCD-based visual detection workpiece production line and a detection method, the method is based on a plurality of CCD cameras for photographing detected workpieces from different angles and then carrying out defect detection and product screening, and the method comprises the following steps: carrying out median filtering on an obtained image, and carrying out graying processing to obtain a grayscale image; extracting texture feature information of the image; applying an Ostu threshold value method to the images of the corresponding parts, and comparing the extracted texture feature information of each image with the threshold value of the image; and judging whether the workpiece has defects or not or marking defect risks according to the number of features of which the texture feature information of the image of each part exceeds a threshold value, and further repeating the steps for the defects of the corresponding parts according to the image of the subsequent visual inspection component to detect and screen the product. According to the invention, intelligent detection and identification of workpiece surface defects are realized, the detection efficiency can be effectively improved, the cost is reduced, and the application prospect is good.

Description

Technical Field

[0001] The present invention relates to the technical field of production lines for visual inspection products, and in particular to a CCD-based visual inspection workpiece production line and an inspection method. Background Art

[0002] With the rapid development of China's manufacturing industry, people's requirements for product quality are constantly increasing. In the production process of industrial strips, affected by factors such as production technology and working environment, cracks, holes, spots and other defects will appear on the surface of the product, which not only affects the appearance of the product, but also affects the use of the product. Therefore, in order to improve the product inspection pass rate and quality, and thus ensure greater economic benefits, it is necessary to conduct defect detection on the surface of industrial strip products. Traditional detection methods such as manual detection and single mechanism detection have gradually been eliminated by the market due to their shortcomings such as low detection efficiency, cumbersome processes and many limitations. The surface defect detection method based on machine vision is based on the latest artificial intelligence technology. With the characteristics of high degree of automation, high precision and high speed, it is widely used in industrial strip defect detection. The input data is processed, analyzed and judged by the neural network model, thereby greatly reducing human intervention, achieving high-precision defect identification and classification, and improving the accuracy of defect detection. Compared with traditional detection methods, the surface defect detection method based on machine vision can shorten the detection time and improve the detection efficiency.

[0003] At present, traditional area array cameras are often used in industrial production processes for visual inspection of strips. However, due to the limitation of shutter speed, traditional area array cameras have the disadvantages of low inspection efficiency and great influence of light source. Therefore, there is an urgent need for a CCD-based intelligent workpiece defect inspection production line. Summary of the invention

[0004] The purpose of the present invention is to provide a CCD-based visual inspection workpiece production line and inspection method to solve the shortcomings of the background technology. The present invention realizes intelligent detection and identification of workpiece surface defects, can be applied to the inspection of production workpieces, can effectively improve the inspection efficiency, reduce costs, and has good application prospects.

[0005] In order to achieve the above object, the present invention provides the following technical solutions: a CCD-based visual inspection workpiece production line and inspection method, the method is based on multiple CCD cameras to take pictures of the inspected workpiece from different angles and then perform defect detection and product screening, the method comprises the following steps:

[0006] S1. Multiple workpieces produced are vibrated and transmitted to the workpiece transmission disc in sequence by the vibration material assembly;

[0007] S2, control the workpiece transmission disc to rotate in situ, and make multiple workpieces fall into the top of the starting end of the feeding conveyor belt after being photographed and inspected by the first visual inspection component in turn; when the inspection result obtained by the first visual inspection component is that there is a problem with the bottom of the workpiece, control the first blowing nozzle to open, and blow the defective workpiece at the first position into the corresponding first product recovery bucket;

[0008] S3, controlling the feeding conveyor belt to inspect the workpieces dropped thereon through the second visual inspection component and the third visual inspection component in sequence; when the inspection result obtained by the second visual inspection component indicates that there is a problem with the top and / or the front side of the workpiece, controlling the second blowing nozzle to open, and blowing the defective workpiece at the second position into the corresponding second product recovery bucket;

[0009] S4. After the workpiece is inspected by the third visual inspection component, the central control microcontroller performs point cloud fusion on the three-dimensional images of the workpiece from the first visual inspection component to the third visual inspection component to detect defects in other parts of the workpiece;

[0010] S5. When the inspection result obtained by the third visual inspection component shows that there is a problem on the rear side of the workpiece, or when the inspection result obtained by the step S4 shows that other parts of the workpiece have defects, the third blowing nozzle is controlled to open to blow the defective workpiece at the third part into the corresponding third product recovery bucket.

[0011] As a preferred embodiment of the present invention, the first defect is a bottom defect; the second defect is a top defect and / or a front defect; the third defect is a back defect and / or other defects; other defects refer to any defects on the three-dimensional solid surface of the standard workpiece CAD drawing.

[0012] As a preferred embodiment of the present invention, the method in which the central control single chip controls the first visual detection component to detect the first part defect, controls the second visual detection component to detect the second part defect, and controls the third visual detection component to detect the third part defect comprises the following steps:

[0013] Obtain a photographic image of the workpiece where the defect is located;

[0014] Perform median filtering on the acquired image and grayscale it to obtain a grayscale image I(x,y);

[0015] Calculate the gray level co-occurrence matrix of the image, and further extract the texture feature information of the image based on the gray level co-occurrence matrix;

[0016] Apply the Ostu threshold method to the images of the corresponding parts, and compare the texture feature information of each image extracted with its threshold;

[0017] If none of the texture feature information of the image exceeds the threshold, the corresponding part does not have defects; if any two of the feature information exceed the threshold, it is concluded that the corresponding part has defects; if only one feature information exceeds the threshold, the workpiece is marked as having a defect risk, and the subsequent visual inspection components are continued to wait for repeating the above steps and making judgments based on the same rules; when only one feature of the image texture feature of the subsequent visual inspection component exceeds its threshold, if the inspected component is marked as having a defect risk, the workpiece is identified as having defects.

[0018] As a preferred embodiment of the present invention, in the process of controlling the second visual inspection component to detect the front side defects and in the process of controlling the third visual inspection component to detect the rear side defects, after performing image fusion on the corresponding side multiple cameras, texture feature extraction of the image is performed to perform defect detection, and the image fusion on the corresponding side multiple cameras includes the following steps:

[0019] S301, obtaining photographic images I of multiple cameras on the corresponding side m,n (x, y); where m is a or b, a represents the front side, and b represents the back side; n = 3, 4, 5, 6, 7; x is the x-axis coordinate of the image point in the image two-dimensional coordinate system, and y is the y-axis coordinate of the image point in the image two-dimensional coordinate system;

[0020] S302, calculate the internal parameter matrix J of the nth CCD camera n :

[0021] Among them, f x,n is the x-axis focal length of the intersection of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, f y,n The y-axis focal length of the intersection of the nth CCD camera and the line connecting the image point in the two-dimensional image coordinate system; c x,n is the x-axis coordinate position of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, c y,n is the y-axis coordinate position of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system;

[0022] S303, calculate the rotation matrix R of the nth CCD camera n :

[0023] R n =R n,z (γ n )R n,y (β n )R n,x (α n ), where α n , β n and γ nThey are the x-axis rotation angle, y-axis rotation angle and z-axis rotation angle of the nth CCD camera around the three-dimensional coordinate system of the production line body;

[0024] α3=60°, α4=120°, α5=-135°, α6=-90°, α7=-45°; β n =0°,γ n =0°;

[0025]

[0026] The three-dimensional coordinate system of the production line itself is a left-handed coordinate system;

[0027] S304, calculating the fusion image L of multiple cameras on the corresponding side of the m-th visual detection component m (x,y):

[0028] Among them, k n is the kth camera of the nth camera n The number of captured images, K n is the total number of images taken by the nth camera; ω m,n is the image fusion weight coefficient of the nth camera on the corresponding side of the mth visual inspection component, ∑ω m =1,∑ω m is the sum of the image fusion weight coefficients of the mth visual detection component; (X n ,Y n ) is the coordinate of the nth camera on the xy plane of the production line body coordinate system.

[0029] As a preferred embodiment of the present invention, the method in which the central control microcontroller controls the third visual inspection component to detect other defects includes the following steps:

[0030] S311, the central control single chip obtains the photographic image of the bottom of the workpiece by the first CCD camera of the first visual inspection component, the photographic image of the top of the workpiece by the second CCD camera of the second visual inspection component, and obtains the multiple camera fusion images L of the corresponding side of the mth visual inspection component calculated in step S304 m (x,y);

[0031] S312, construct the normal field vector N(x,y)=[N x (x,y),N y (x,y),N z (x,y)] T ; Among them, N x (x,y),N x (x,y) and N x(x, y) are the components of the three-dimensional image normal of the workpiece surface image point (x, y) on the x-axis, y-axis and z-axis respectively, N z (x,y) is normalized to 1;

[0032] S313, calculate the image point (x, y) in the normal field y-axis gradient G y (y,z) and the z-axis gradient G z (y,z), and integrate the two to get the 3D point cloud depth map Z(x,y):

[0033] Where x′ is the integration unit of Z(x,y) on the normal field x-axis, y′ is the integration unit of Z(x,y) on the normal field y-axis; Z(x0,y0) is the depth of the initial image point (x0,y0);

[0034] S314, repeating steps S311-S313 for the M image points of the workpiece surface image, optimizing the iteration based on the least squares method, calculating the error function E(Z) of the gradient field and the depth field of the M image points until it converges to the minimum value, stopping the iteration, and obtaining the three-dimensional point cloud set P = {p1, p2, ..., p i ,…,p M}, i = 1, 2, ..., M; p i is the i-th 3D point in the 3D point cloud set reconstructed from the i-th image point; p i The three-dimensional coordinates of the point are (x i ,y i ,Z i (x i ,y i )),Z i (x i ,y i ) is the 3D point cloud depth map of the i-th image point obtained by the least squares optimization iteration, and the 3D point cloud depth map obtained by the optimization iteration is used as the vertical coordinate of the 3D point cloud;

[0035] S315. Reconstruct the M three-dimensional points in the three-dimensional point cloud set P obtained in step S314 into a polygonal network using the Delaunay triangulation method based on the empty sphere rule, and then reconstruct a three-dimensional point cloud network image of the workpiece surface.

[0036] Furthermore, in the step S313, the x-axis gradient G of the image point (x, y) in the normal field is calculated. x (x,y) and the y-axis gradient G y The calculation formula for (x,y) is as follows:

[0037]

[0038] Among them, Ny =∑L m (x,y),∑L m (x, y) is the fusion image L of multiple cameras of the front side of the workpiece taken by the second visual inspection component (i.e., the fusion image L of multiple images of the front side of the workpiece taken by the third CCD camera and the fourth camera in the second visual inspection component). a (x, y)) and the fused images of multiple cameras of the rear side of the workpiece taken by the third visual inspection component (i.e., the fused image L of multiple images of the front side of the workpiece taken by the fifth CCD camera to the seventh camera in the second visual inspection component) b (x,y)) and N z =L u (x,y)+L d (x,y),L u (x, y) is the image of the top of the workpiece taken by the second CCD camera, L d (x, y) is the image of the bottom of the workpiece taken by the first CCD camera.

[0039] As a preferred embodiment of the present invention, the error function E(Z) between the gradient field and the depth field of the M image points in the step S314 is calculated as follows:

[0040]

[0041] As a preferred embodiment of the present invention, the step S315 includes the following steps:

[0042] S3151, select any three non-collinear three-dimensional points p in the three-dimensional point cloud set P. i 、p j and p k , p i The three-dimensional coordinates of the point are (x i ,y i ,Z i (x i ,y i )), p j The three-dimensional coordinates of the point are (x j ,y j ,Z j (x j ,y j )), p k The three-dimensional coordinates of the point are (x k ,y k ,Z k (x k ,y k ))

[0043] S3152, calculate the p of three 3D points i The vector between two other points

[0044]

[0045] S3153, further calculate the vertical to three three-dimensional points p i 、p j and p k The plane p i p j p k The normal vector Its module length

[0046] u x =(y j -y i )(z k -z i )-(z j -z i )(y k -y i );

[0047] u y =(z j -z i )(x k -x i )-(x j -x i )(z k -z i );

[0048] u z =(x j -x i )(y k -y i )-(y j -y i )(x k -x i );

[0049] S3153, calculate the coordinates of the center O of the circumscribed circle (O x ,O y ,O z );

[0050]

[0051] S3154. Calculate the radius R of the circumscribed circle O:

[0052] Get three three-dimensional points p i 、p j and p kThe polygonal network formed;

[0053] S3155. Repeat steps S3151-S3154 for all three non-collinear three-dimensional points in the three-dimensional point cloud set P to reconstruct a three-dimensional point cloud network image of the workpiece surface.

[0054] The present invention also provides a CCD-based visual inspection workpiece production line using the above method, the production line includes a first base and a second base, the production line also includes a central control single chip computer for executing steps S1-S5 of the above method, a vibrating material component arranged on the first base, and a workpiece transmission disc, a feeding conveyor belt, a first visual inspection component, a first blowing nozzle, a second visual inspection component, a second blowing nozzle, a third visual inspection component and a third blowing nozzle arranged on the workpiece transmission disc, arranged on the second base; the first blowing nozzle to the third blowing nozzle are arranged in sequence from left to right on the front side of the feeding conveyor belt, and corresponding first product recovery bins, second product recovery bins and third product recovery bins are respectively arranged on the rear side of the feeding conveyor belt;

[0055] The first visual detection component includes a first CCD camera and a first multi-spectral illumination disk; the second visual detection component includes a second CCD camera, a square multi-spectral illumination frame, a third CCD camera and a fourth CCD camera, and a first projection panel;

[0056] The third visual detection component includes a fifth CCD camera, a sixth CCD camera, a seventh CCD camera, a second projection panel and a second multi-spectral illumination disk;

[0057] The vibrating material assembly comprises a workpiece delivery port, an inner convex rotating screening material hopper, a screening material receiving track connected with the outlet of the inner convex rotating screening material hopper, and a pulse electromagnet device.

[0058] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0059] 1. The present invention sets a first visual inspection component to a third visual inspection component, photographs the workpiece from different angles, obtains images at different photographing angles, and then can gradually detect the bottom defects and the top defects of the workpiece respectively, fuses the photographed images of the third CCD camera and the fourth CCD camera to detect the front defects of the workpiece, fuses the photographed images of the fifth CCD camera to the seventh CCD camera and detects the back defects of the workpiece, and further restores the depth map of the bottom image, the top image and the two fused images to construct a three-dimensional point cloud, further adopts the Delaunay triangulation method to enhance the stereo features to obtain a reconstructed three-dimensional surface image of the workpiece, and detects defects in other parts of the workpiece. It can perform all-round three-dimensional defect detection on the workpiece, gradually perform defect risk identification, improve the accuracy of defect detection, avoid defect misjudgment (part of the workpiece that meets the finished product standard is misjudged as having a defect in a certain part and the finished product is placed in a defective product recycling bin) or low defect recognition (too many defective products are finally placed in the finished product collection basket) causing a decrease in the yield rate.

[0060] 2. The present invention fuses the images taken by multiple CCD cameras, firstly through the internal reference matrix J n and the rotation matrix R n Camera calibration and image registration are performed to align images taken by different cameras into the same coordinate system so that their overlapping parts can be accurately aligned. Different image fusion methods are then used to generate high-quality images or 3D models. m The relevant calculation formula of (x, y) is used to perform weighted average fusion of images taken at different angles. Image fusion based on multi-viewpoint can improve image quality and the accuracy of three-dimensional information.

[0061] 3. The present invention, through steps S311-S315, constructs an integral normal field and then restores the depth map of the workpiece. This step is a key step in the three-dimensional reconstruction technology based on the normal field. This process is based on the relationship between the gradient field and the depth field. The depth map is calculated by path integral, least squares method and other methods, and converted into a three-dimensional point cloud or mesh for three-dimensional stereo reconstruction. Then, through the Delaunay triangulation method, the points in the three-dimensional point cloud P of the workpiece can be connected into a triangular mesh, and then the three-dimensional point cloud network image of the workpiece surface is reconstructed to form a surface model of the workpiece. This method can accurately reconstruct the three-dimensional shape of the object surface, and the three-dimensional stereogram finally generated can be visualized by three-dimensional rendering software. The stereoscopic image can be used for quality control, shape analysis, and defect detection. By solving the surface normal distribution to restore the three-dimensional shape, the three-dimensional image fusion of the workpiece surface can enhance the representation of the three-dimensional features and improve the clarity and detectability of the surface features. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0063] Figure 1 is a flow chart of the method of the present invention;

[0064] Figure 2 It is a side view of the front part of the production line in an embodiment of the present invention; Figure 3 This is a schematic diagram of the rear part of the production line in an embodiment of the present invention; Figure 4 is a top view of a production line in an embodiment of the present invention;

[0065] Figure 5 It is a schematic diagram of the positional relationship between the three-dimensional coordinate system of the production line body of the present invention, the two-dimensional coordinate system of the image, and the three-dimensional coordinate system of the CCD camera and the image points being photographed;

[0066] Figure 6 is Z restored using the peaks function i (x i ,y i )’s three-dimensional surface depth map;

[0067] Figure 7 Z obtained by reduction in steps S311-S314 of the present invention i (x i ,y i )’s three-dimensional surface depth map;

[0068] Figure 8 The three-dimensional point cloud network diagram of the workpiece surface is reconstructed based on the empty sphere rule using the Delaunay triangulation method;

[0069] Fig. 9 A product image of a workpiece that needs to be inspected in an embodiment of the present invention;

[0070] Fig.10 In order to use the method of the present invention Fig. 9 A grayscale image obtained by performing a median filter on the overhead photographic image of the product and converting it into a grayscale image;

[0071] Fig.11 In order to use the method of the present invention Fig. 9 The three-dimensional point cloud image obtained after the overhead photographic image of the product in steps S312-S313;

[0072] Fig.12 For Fig.11Continue to perform steps S314-S315 to reconstruct the obtained three-dimensional point cloud network image of the attack surface. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] The present invention provides a CCD-based visual inspection workpiece production line and inspection method. The method provided by the present invention adopts a production line such as Figure 1-Figure 4 As shown, the method of the present invention is based on multiple CCD cameras taking pictures of the inspected workpiece from different angles to perform defect detection and product screening, and the method includes the following steps:

[0075] S1, the multiple workpieces produced are vibrated and transmitted in sequence by the vibrating material assembly 3 to the workpiece transmission disc 4;

[0076] S2, control the workpiece transmission disc 4 to rotate in situ, and make multiple workpieces fall into the upper part of the starting end of the feeding conveyor belt 5 after being photographed and inspected by the first visual inspection component 6 in turn; the photographed images of the first visual inspection component 6 are transmitted to the central control single chip computer for calculation to determine whether the workpiece has defects; when the inspection result obtained by the first visual inspection component 6 is that there is a problem with the bottom of the workpiece, control the first blowing nozzle 7 to open, and blow the defective workpiece at the first part into the corresponding first product recovery bucket 12;

[0077] S3, control the feeding conveyor belt 5 to inspect the workpieces dropped thereon through the second visual inspection component 8 and the third visual inspection component 10 in sequence; the second visual inspection component 8 takes a picture and transmits it to the central control single chip computer for calculation to determine whether the workpiece has defects; when the inspection result obtained by the second visual inspection component 8 is that there is a problem with the top and / or the front side of the workpiece, control the second blowing nozzle 9 to open, and blow the defective workpiece at the second part into the corresponding second product recovery bucket 13;

[0078] S4, after the workpiece is inspected by the third visual inspection component 8, the central single chip computer performs point cloud fusion on the three-dimensional images of the workpiece from the first visual inspection component 6 to the third visual inspection component 11, and performs defect detection on other parts of the workpiece;

[0079] S5. When the inspection result obtained by the third visual inspection component 11 shows that there is a problem on the rear side of the workpiece, or when the inspection result obtained by step S4 shows that other parts of the workpiece have defects, the third blowing nozzle 11 is controlled to open to blow the defective workpiece in the third part into the corresponding third product recovery bucket 14. Finally, after the defective products with defects in the corresponding parts detected by multiple CCD cameras on the production line of the present invention and calculated by the central control single chip are blown into the corresponding workpiece recovery bucket in advance, the workpieces that can continue to stay on the feeding conveyor belt and be transmitted to the tail end are all finished products without any defects on the three-dimensional solid surface, and are then conveyed to the finished product recovery basket.

[0080] In steps S1-S5 of the present invention, the first defect is a bottom defect; the second defect is a top defect and / or a front defect; the third defect is a back defect and / or other defects; other defects refer to any defects on the three-dimensional solid surface of the standard workpiece CAD drawing.

[0081] As a preferred embodiment of the present invention, the method in which the central control microcontroller controls the first visual detection component 6 to detect the first part defect, controls the second visual detection component 8 to detect the second part defect, and controls the third visual detection component 10 to detect the third part defect comprises the following steps:

[0082] Obtain a photographic image of the workpiece where the defect is located;

[0083] If it is necessary to calculate and obtain the detection calculation result of whether the workpiece has bottom defects, the central control microcontroller obtains the photographic image of the first CCD camera;

[0084] If it is necessary to calculate and obtain the detection calculation result of whether the workpiece has a top defect, the central control microcontroller obtains the photographic image of the second CCD camera; if it is necessary to calculate and obtain the detection calculation result of whether the workpiece has a front side defect, the central control microcontroller obtains the fused image obtained by processing the photographic image of the third CCD camera and the front side photographic image of the fourth CCD camera in steps S301-S304;

[0085] If it is necessary to calculate and obtain the detection calculation result of whether the workpiece has a rear side defect, the central control microcontroller obtains the fused image of the rear side photographs of the photographs taken by the fifth CCD camera to the seventh CCD camera in steps S301-S304;

[0086] If it is necessary to calculate and determine whether any part of the workpiece surface has defects, that is, it is necessary to detect other defect types in the third part defect, it is necessary to obtain the three-dimensional point cloud network image of the workpiece surface reconstructed in steps S311-S315;

[0087] Perform median filtering on the acquired image to reduce the interference of noise on subsequent processing, and perform grayscale processing to obtain a grayscale image I (x, y); the median filtering method adopts a common method in the prior art;

[0088] By calculating the gray level co-occurrence matrix of the image, the gray level probability P(i,j) of each pixel pair (i,j) in the gray level image in the matrix can be obtained, and the texture feature information of the image can be further extracted according to the gray level co-occurrence matrix; the feature information includes contrast, entropy and uniformity.

[0089] The steps to calculate the gray level co-occurrence matrix are as follows:

[0090] A) Select an image point (x, y) in the grayscale image, and select another image point (x+dcosθ, y+dsinθ) with an expected distance d and an angle θ.

[0091] B) Traverse each possible grayscale value i, find all image points with grayscale value j at distance d and angle θ, and get pixel pairs (i, j);

[0092] C) Calculate the number of occurrences of all pixel pairs (i, j) calculated in step B); normalize the number of occurrences of all image points (x, y) and their corresponding pixel pairs (i, j) to obtain the grayscale probability P(i, j):

[0093]

[0094] Apply the Ostu threshold method to the images of the corresponding parts, and compare the texture feature information of each image extracted with its threshold;

[0095] If none of the texture feature information of the image exceeds the threshold, the corresponding part does not have defects; if any two of the feature information exceed the threshold, it is concluded that the corresponding part has defects; if only one feature information exceeds the threshold, the workpiece is marked as having a defect risk, and the subsequent visual inspection components are continued to wait for repeating the above steps and making judgments based on the same rules; when only one feature of the image texture feature of the subsequent visual inspection component exceeds its threshold, if the inspected component is marked as having a defect risk, the workpiece is identified as having defects.

[0096] Defects include workpiece protrusions, workpiece depressions, missing solid parts of the workpiece, or bubbles on the workpiece surface.

[0097] The image texture feature information of each gray-level co-occurrence matrix includes contrast Con, entropy En, and uniformity Ho, and the calculation formulas are as follows:

[0098]

[0099] Where N is the number of gray levels. Contrast Con is an indicator that measures the total amount of local grayscale changes in an image, reflecting the clarity of the image and the depth of the texture grooves. Entropy En is an indicator that measures the randomness of the amount of information contained in an image, reflecting the complexity of the image texture; the larger the entropy value, the more complex the image texture. (ij) in the Ho formula 2 Represents the grayscale difference of pixel pair (i, j). Ho is an indicator to measure the local grayscale uniformity of an image, which reflects the uniformity of the image texture.

[0100] The above method is used to illustrate the defect determination and defect risk marking rules of the workpiece. For example, when the first visual inspection component 6 detects and obtains the photographed image of the first CCD camera 61, performs planting filtering to obtain the grayscale image I (x, y), and calculates the grayscale co-occurrence matrix, and the contrast, entropy and uniformity of the bottom image of the workpiece are calculated to be less than their corresponding thresholds, the workpiece continues to perform the second part defect inspection;

[0101] If the calculated contrast and uniformity both exceed their corresponding thresholds, when the workpiece is conveyed to the first blowing nozzle 7 by the feeding conveyor belt, the first blowing nozzle 7 is controlled to blow the workpiece into the first product recovery barrel 12;

[0102] If the contrast of the detected bottom image of the workpiece only exceeds the threshold value, the workpiece is at risk of defects. When the workpiece is transported to the first blowing nozzle 7 by the feeding conveyor belt, the first blowing nozzle 7 is not turned on, and the workpiece continues to be photographed by the second CCD camera 83 to the fourth CCD camera 84 of the second visual inspection component 8, and the second part of the workpiece is inspected for defects. If the contrast, entropy and uniformity of the second part image do not exceed the corresponding threshold values, the workpiece continues to be inspected for defects in the third part.

[0103] If the calculated contrast and entropy of the second part image of the workpiece exceed their corresponding thresholds, when the workpiece is conveyed to the second blowing nozzle 9 by the feeding conveyor belt, the second blowing nozzle 9 is controlled to blow the workpiece into the second product recovery bucket 13;

[0104] If only the calculated contrast of the image of the second part of the workpiece exceeds the corresponding threshold, the first visual inspection component 6 marks the workpiece as a defect risk, and therefore, the workpiece is marked as having a defect. Then, when the workpiece is conveyed to the second blowing nozzle 9 by the feeding conveyor belt, the second blowing nozzle 9 is controlled to blow the workpiece into the second product recovery bucket 13;

[0105] If a workpiece marked as having a defect risk by the first visual inspection component 6 is inspected by the second visual inspection component 8 and then continues to be inspected by the third visual inspection component 10, and the calculation result of the third visual inspection component 10 is that only one feature of the texture feature information of the third part image exceeds the threshold, the third blow nozzle 11 is controlled to open and blow the workpiece into the third workpiece recovery bucket 14.

[0106] Further, in the process of controlling the second visual inspection component to detect the front side defects and in the process of controlling the third visual inspection component to detect the rear side defects, after performing image fusion on the corresponding side multiple cameras, texture feature extraction of the image is performed to perform defect detection, and performing image fusion on the corresponding side multiple cameras includes the following steps:

[0107] S301, obtaining photographic images I of multiple cameras on the corresponding side m,n (x, y); where m is a or b, a represents the front side, b represents the back side; n = 3, 4, 5, 6, 7; such as I a,4 (x, y) represents the photographic image of the fourth CCD camera 84 of the second visual inspection component 8, which obtains the photographic image of the front side of the workpiece; b,6 (x, y) represents the photographic image of the sixth CCD camera 102 of the third visual inspection component 10, which obtains the photographic image of the rear side of the workpiece; x is the x-axis coordinate of the image point in the image two-dimensional coordinate system, and y is the y-axis coordinate of the image point in the image two-dimensional coordinate system; Figure 5 x in ima Coordinate axis and y ima The plane formed by the coordinate axes is the two-dimensional image coordinate system; Figure 5 The three-dimensional coordinate system composed of the x-axis, y-axis and z-axis is the three-dimensional coordinate system of the production line itself;

[0108] S302, calculate the internal parameter matrix J of the nth CCD camera n :

[0109] Among them, f x,n is the x-axis focal length of the intersection of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, f y,n The y-axis focal length of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system; Figure 5 As shown, c x,n is the x-axis coordinate position of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, c y,n is the y-axis coordinate position of the intersection of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system; that is, (c x,n ,c y,n) is the intersection point of the optical axis of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system and the collected image plane; W n and H n are the width and height of the image captured by the nth CCD camera respectively;

[0110] S303, calculate the rotation matrix R of the nth CCD camera n :

[0111] R n =R n,z (γ n )R n,y (β n )R n,x (α n ), where α n , β n and γ n are the x-axis rotation angle, y-axis rotation angle and z-axis rotation angle of the nth CCD camera around the three-dimensional coordinate system of the production line body; Figure 4 As shown, α3=60°, α4=120°, α5=-135°, α6=-90°, α7=-45°; β n =0°,γ n =0°; Figure 5 As shown, x c ,y c and z c The coordinate system composed is the camera 3D coordinate system of the nth CCD camera, x c The rotation angle of the x-axis relative to the three-dimensional coordinate system of the production line is α n ,y c The rotation angle of the y-axis relative to the three-dimensional coordinate system of the production line is β n , z c The z-axis rotation angle γ relative to the three-dimensional coordinate system of the production line n ;

[0112]

[0113] like Figure 1-Figure 3 The three-dimensional coordinate system of the production line body shown is a left-hand coordinate system;

[0114] S304, calculating the fusion image L of multiple cameras on the corresponding side of the m-th visual detection component m (x,y):

[0115] Among them, k n is the kth camera of the nth camera n The number of captured images, K nis the total number of images taken by the nth camera; ω m,n is the image fusion weight coefficient of the nth camera on the corresponding side of the mth visual inspection component, ∑ω m =1,∑ω m is the sum of the image fusion weight coefficients of the mth visual detection component; Figure 5 As shown, (X n ,Y n ) is the coordinate of the nth camera on the xy plane of the production line body coordinate system.

[0116] The second visual detection component 8 fuses the images of multiple cameras on the front side. ∑ω a =ω a,3 +ω a,4 =1;

[0117] The second visual detection component 8 fuses the images of multiple cameras on the front side. ∑ω b =ω b,5 +ω b,6 +ω b,7 =1;

[0118] The central control microcontroller fuses the third and fourth images at different angles taken by the third CCD camera 83 and the fourth CCD camera 84 in the second visual inspection component 8 through steps S301-S304; similarly, the central control microcontroller fuses the images at different angles taken by the fifth CCD camera 101 to the seventh CCD camera 103 in the third visual inspection component 10 through steps S301-S304, and then performs the above-mentioned median filtering, grayscale processing, calculation of grayscale co-occurrence matrix, and the use of Ostu threshold method to extract texture features of the image and further judge and mark the defect risk. Through steps S301-S304, the images taken by CCD cameras at different angles are fused to improve the clarity and detail performance of the image.

[0119] The present invention fuses the images taken by multiple CCD cameras, firstly through the internal reference matrix J n and the rotation matrix R n Camera calibration and image registration are performed to align images taken by different cameras into the same coordinate system so that their overlapping parts can be accurately aligned. Different image fusion methods are then used to generate high-quality images or 3D models. m The relevant calculation formula of (x, y) is used to perform weighted average fusion of images taken at different angles. Image fusion based on multi-viewpoint can improve image quality and the accuracy of three-dimensional information.

[0120] As another preferred embodiment of the present invention, the method process in which the central control microcontroller controls the third visual inspection component to detect other defects includes the following steps:

[0121] S311, the central control single chip obtains the photographic image of the bottom of the workpiece by the first CCD camera of the first visual inspection component, the photographic image of the top of the workpiece by the second CCD camera of the second visual inspection component, and obtains the multiple camera fusion images L of the corresponding side of the mth visual inspection component calculated in step S304 m (x, y) (i.e., obtaining the fusion image L of the multi-angle photographic images of the front side of the workpiece taken by the third CCD camera 83 and the fourth CCD camera 84 of the second visual inspection component) a (x, y) and obtain the fusion image L of the multi-angle photographic images of the rear side of the workpiece by the fifth CCD camera 101 to the seventh CCD camera 103 in the third visual inspection component 10 b (x,y));

[0122] S312. The present invention adopts a 3D reconstruction method based on a normal field. The normal field provides the normal direction of each point on the surface of the object. By integrating these normal fields, the depth information of the object, that is, the 3D shape, can be restored. Therefore, it is necessary to construct a normal field vector N(x, y)=[N x (x,y),N y (x,y),N z (x,y)] T ; Among them, N x (x,y),N x (x,y) and N z (x, y) are the components of the three-dimensional image normal of the workpiece surface image point (x, y) on the x-axis, y-axis and z-axis respectively, N z (x,y) is normalized to 1;

[0123] The three-dimensional point cloud depth map Z(x,y) of the image point (x,y) and the three-dimensional component of the normal field vector N(x,y) of the workpiece three-dimensional image have the following relationship:

[0124]

[0125] Where N y =∑L m (x,y),∑L m (x, y) is a fusion image L of multiple cameras of the front side of the workpiece taken by the second visual inspection component (i.e., a fusion image L of multiple images of the front side of the workpiece taken by the third CCD camera 83 and the fourth CCD camera 84 in the second visual inspection component 8). a(x, y)) and the fused images of multiple cameras on the rear side of the workpiece by the third visual inspection component 10 (i.e., the fused image L of multiple images of the front side of the workpiece taken by the fifth CCD camera 101 to the seventh CCD camera 103 in the third visual inspection component 8) b (x,y)) and N z =L u (x,y)+L d (x,y),L u (x, y) is the image of the top of the workpiece taken by the second CCD camera, L d (x, y) is the image of the bottom of the workpiece taken by the first CCD camera; N z (x,y) is normalized to 1, then N x (x,y)=1 / (L u (x,y)+L d (x,y));

[0126] ∑L m =L a (x,y)+L b (x, y);

[0127] In order to recover the depth map z(x,y) of the workpiece from the normal field, the gradient field needs to be integrated. Therefore, the gradient field of each normal component needs to be further calculated.

[0128] S313, calculate the image point (x, y) in the normal field y-axis gradient G y (y,z) and the z-axis gradient G z (y,z), and integrate the two to get the 3D point cloud depth map Z(x,y):

[0129] Where x′ is the integration unit of Z(x,y) in the normal field x-axis, y′ is the integration unit of Z(x,y) in the normal field y-axis; Z(x0,y0) is the depth of the initial image point (x0,y0), which can be set to 0 or a known value according to the specific application scenario;

[0130] Calculate the x-axis gradient G of the normal field at the image point (x, y) x (x,y) and the y-axis gradient G y The calculation formula for (x,y) is as follows:

[0131]

[0132] In the method provided by the present invention, the central control single chip microcomputer controls the third visual inspection component to detect other defects, constructs the normal field vector N(x, y) of the three-dimensional image of the workpiece, and then accumulates the gradient of the normal field along a fixed path to restore the depth. The depth map recovery method has a small amount of calculation. In order to reduce the sensitivity to the integral path during the depth map recovery process and reduce the cumulative error, it is necessary to further perform step S314 to optimize and iterate the acquisition of the depth map;

[0133] S314, repeating steps S311-S313 for the M image points of the workpiece surface image, optimizing the iteration based on the least squares method, calculating the error function E(Z) of the gradient field and the depth field of the M image points until it converges to the minimum value, stopping the iteration, and obtaining the three-dimensional point cloud set P = {p1, p2, ..., p i ,…,p M}, i = 1, 2, ..., M; p i is the i-th 3D point in the 3D point cloud set reconstructed from the i-th image point; p i The three-dimensional coordinates of the point are (x i ,y i ,Z i (x i ,y i )),Z i (x i ,y i ) is the 3D point cloud depth map of the i-th image point obtained by the least squares optimization iteration, and the 3D point cloud depth map obtained by the optimization iteration is used as the vertical coordinate of the 3D point cloud;

[0134] Among them, the error function E(Z) between the gradient field and the depth field of M image points is calculated as follows:

[0135]

[0136] The depth map is obtained by solving the global optimization problem in step S314. By minimizing the error function, a globally optimal depth distribution can be obtained. The least square method or other numerical optimization algorithms are usually used to solve this problem. Step S314 can effectively reduce the influence of noise and improve the robustness and accuracy of the three-dimensional point cloud.

[0137] S315. Reconstruct the M three-dimensional points in the three-dimensional point cloud set P obtained in step S314 into a polygonal network using the Delaunay triangulation method based on the empty sphere rule, and then reconstruct a three-dimensional point cloud network image of the workpiece surface.

[0138] Restoring the depth map of the workpiece by integrating the normal field is a key step in the 3D reconstruction technology based on the normal field. This process is based on the relationship between the gradient field and the depth field. The depth map is calculated by path integral, least squares method and other methods, and converted into a 3D point cloud or mesh for 3D reconstruction. Then, through the Delaunay triangulation method, the points in the 3D point cloud set P of the workpiece can be connected into a triangular mesh, and then the 3D point cloud network image of the workpiece surface can be reconstructed to form a surface model of the workpiece. This method can accurately reconstruct the 3D shape of the object surface. The final generated 3D stereogram can be visualized through 3D rendering software. The stereo image can be used for quality control, shape analysis, and defect detection.

[0139] Comparative Example 1

[0140] In order to verify the three-dimensional point cloud depth map Z of the i-th image point obtained by the least squares optimization iteration in steps S311-S314 provided by the present invention i (x i ,y i ), the present invention does not use the steps S311-S314 of the present invention for the normal field vector N(x, y) of the workpiece three-dimensional image, but uses the peaks function of Matlab based on the workpiece image point (x i ,y i ) obtains the depth map of the i-th image point as the control group of the present invention, such as Figure 6 As shown, Z is restored using the peaks function i (x i ,y i ), Figure 7 Z obtained by reduction in steps S311-S314 of the present invention i (x i ,y i ) of the three-dimensional surface depth map. Figure 6 and Figure 7 It can be seen from the comparison that the method provided by the present invention can effectively reduce the i (x i ,y i )’s three-dimensional surface depth map restoration process, and improves the robustness and accuracy of the final three-dimensional point cloud based on it.

[0141] The implementation of step S315 is based on any three 3D points p in the 3D point cloud set P. i 、p j and p k , i, j and k all belong to {1,2,…,i,…,M}, forming the circumscribed circle of tetrahedron T, and the coordinates of the center of the circumscribed circle are (O x ,O y ,Oz ); specifically comprising the following steps:

[0142] S3151. Select any three non-collinear three-dimensional points p in the three-dimensional point cloud set P. i 、p j and p k , p i The three-dimensional coordinates of the point are (x i ,y i ,Z i (x i ,y i )), p j The three-dimensional coordinates of the point are (x j ,y j ,Z j (x j ,y j )), p k The three-dimensional coordinates of the point are (x k ,y k ,Z k (x k ,y k ))

[0143] S3152, calculate the p of three 3D points i The vector between two other points

[0144]

[0145] S3153, further calculate the vertical to three three-dimensional points p i 、p j and p k The plane p i p j p k The normal vector Its module length

[0146]

[0147] u y =(z j -z i )(x k -x i )-(x j -x i )(z k -z i );

[0148] u z =(x j -x i )(y k -yi )-(y j -y i )(x k -x i ); like Figure 8 As shown, Figure 8 (a) is a schematic diagram of a plane formed by any three non-co-edge three-dimensional points in the three-dimensional point cloud set P through steps S3151-S3153;

[0149] S3153, calculate the coordinates of the center O of the circumscribed circle (O x ,O y ,O z );

[0150]

[0151] The center O of the circle is located at the normal vector in the direction of the plane p i p j p k The distance is equal to the radius R of the circumscribed circle;

[0152] S3154. Calculate the radius R of the circumscribed circle O:

[0153] Get three three-dimensional points p i 、p j and p k The polygonal network formed; Figure 8 (b) is a schematic diagram of multiple circumscribed circles obtained after the coordinates of the center O of the circumscribed circle of each plane are calculated in S3153;

[0154] S3155. Repeat steps S3151-S3154 for all three non-collinear three-dimensional points in the three-dimensional point cloud set P to reconstruct a three-dimensional point cloud network image of the workpiece surface.

[0155] The empty ball rule of the Delaunay triangulation method means that for any generated triangle T, its circumscribed circle does not contain other three-dimensional points in the three-dimensional point cloud set P. In addition, when generating a triangular network structure, the Delaunay triangulation method tends to maximize the size of the minimum internal angle of the triangle, thereby avoiding the generation of overly sharp triangles. The Delaunay triangulation method can be used to connect the points in the three-dimensional point cloud set P of the workpiece obtained in step S314 into a triangular mesh, thereby forming a surface model of the workpiece.

[0156] like Figure 1-4As shown, the present invention adopts a CCD-based visual inspection workpiece production line of the method as described above, the production line includes a first base 1 and a second base 2, the production line also includes a central control single chip computer for executing steps S1-S5 of the above method, a vibrating material component 3 arranged on the first base 1, and a workpiece transmission disc 4, a feeding conveyor belt 5, a first visual inspection component 6, a first blowing nozzle 7, a second visual inspection component 8 for detecting defects on the upper part and the front side of the workpiece, a second blowing nozzle 9, a third visual inspection component 10 for detecting defects on the rear side of the workpiece, and a third blowing nozzle 11. The first blowing nozzle 5 to the third blowing nozzle 11 are sequentially arranged on the front side of the feeding conveyor belt 5 from left to right, and corresponding first product recovery barrels 12, second product recovery barrels 13 and third product recovery barrels 14 are respectively arranged on the rear side of the feeding conveyor belt 5;

[0157] like Figure 2 As shown, the first visual detection component 6 includes a first CCD camera 61 and a first multi-spectral illumination disk 62 disposed on the upper side of the first CCD camera 61;

[0158] like Figure 2 , Figure 3 As shown, the second visual inspection component 8 includes a second CCD camera 81 disposed above the feeding conveyor belt 5, a square multi-spectral lighting frame 82 disposed below the second CCD camera 81 and located above the feeding conveyor belt 5, a third CCD camera 83 and a fourth CCD camera 84 disposed on the front side of the feeding conveyor belt 5, and a first projection panel 85 disposed on the rear side of the feeding conveyor belt 5;

[0159] The third visual inspection component 10 includes a fifth CCD camera 101, a sixth CCD camera 102, a seventh CCD camera 103, a second projection panel 104, which is arranged on the front side of the feeding conveyor 5, and a second multi-spectral lighting disk 105, which is arranged above the feeding conveyor. The projection panel is arranged to enclose the light emitted by the corresponding lighting disk or lighting frame at the corresponding visual inspection component position, so as to improve the brightness of the photographed image of this part of the visual inspection component, avoid the photographed image being too dark due to the divergence of light, and avoid the phenomenon that excessive processing affects the final workpiece screening result in the subsequent grayscale processing process.

[0160] like Figure 4As shown, the blue dotted line indicates the direction parallel to the x-axis, the red dotted line is the direction of the optical axis of each CCD camera, the direction of each CCD lens is the direction of the optical axis of the lens receiving light, the angle between the red dotted line and the blue dotted line is the rotation angle of the above CCD camera around the x-axis of the three-dimensional coordinate system of the production line body, the second multi-spectral illumination disk 105 is located at the position where the fifth CCD camera 101, the sixth CCD camera 102 and the seventh CCD camera 103 are located and are located directly above the feeding conveyor belt 5; the second CCD The lens direction of the D camera 81 is perpendicular to the plane where the feeding conveyor belt 5 is located. The lens directions of the third CCD camera 82 and the fourth CCD camera 83 form an obtuse angle α3 and an acute angle α4 = 60° with the feeding direction along the straight line where the blue dotted line is located from left to right; the lens directions of the fifth CCD camera 101, the sixth CCD camera 102 and the seventh CCD camera 103 form an obtuse angle α5 = -135°, a right angle α6 = -90° and an acute angle α7 = -45° with the feeding direction respectively;

[0161] The vibrating material assembly 3 includes a workpiece delivery port 31, an inner convex rotary screening hopper 32 arranged under the delivery port 31, a screening material receiving track 33 connected to the outlet 321 of the inner convex rotary screening hopper 32, and a pulse electromagnet device 34 arranged below the straight section of the screening material receiving track 33; the working principle of the electromagnet device 34 depends on the pulse electromagnet arranged inside it, and the force generated by the electromagnet causes the hopper to vibrate in the vertical direction. Due to the inclination of the spring sheet, the hopper performs torsional vibration around its vertical axis. This vibration causes the parts in the hopper to rise along the spiral track and finally reach the discharge port, completing the automatic feeding process.

[0162] like Figure 2 , Figure 3As shown, the initial end of the arc segment of the screening material receiving track 33 is arranged at the outlet 321 of the inwardly convex rotating screening hopper 32, and the terminal end of the arc segment of the screening material receiving track 33 is connected and fixed to the initial end of the straight segment of the screening material receiving track 33; the entire screening material receiving track 33 gradually decreases from left to right from the initial end of its arc segment to the terminal end of its straight segment, and then the terminal end of the straight segment of the screening material receiving track 33 is located at the lower end of the workpiece transmission disc 4, and the workpiece transmission disc 4 is provided with a plurality of screening components 41 symmetrically arranged on its circumferential edge with its center as the center point. When the workpiece gradually slides down from left to right along the screening material receiving track to the end of its straight segment At the end, the upper part of the workpiece can be picked up one by one by the screening component 41 which rotates to the upper end of the straight line segment in sequence. A first motor (not shown in the figure) is arranged at the bottom of the workpiece transfer disc 4 for driving the workpiece transfer disc to rotate in situ during operation. Further, driven by the rotation of the workpiece transfer disc, it passes by the first visual inspection component 6 located at the lower end of the circumferential edge of the workpiece transfer disc and the starting end of the feeding conveyor belt 5 in sequence. After the lower image of the workpiece is photographed by the first visual inspection component 6, it falls on the feeding conveyor belt 5. The feeding conveyor belt is driven by a second motor (not shown in the figure) to transfer the workpieces from left to right to its tail end in sequence.

[0163] A finished workpiece collection basket is arranged at the lower part of the terminal end of the feeding conveyor belt (5).

[0164] Example 1

[0165] When the above method of the present invention is used for Fig. 9 When testing the product picture shown:

[0166] 1) When the workpiece passes through the first CCD camera 61 of the first visual inspection component 6, the workpiece bottom defects are inspected;

[0167] 2) and obtaining the top photographic image of the workpiece by the second CCD camera 81 of the second visual inspection component 8, the following is obtained: Fig.10 The grayscale image shown is subjected to Ostu threshold method texture feature extraction to detect defects on the top of the workpiece. The contrast, entropy and uniformity of the top grayscale image do not exceed the threshold value. The central control microcontroller continues to acquire the photographic images of the third CCD camera 83 and the fourth CCD camera 84 of the second visual inspection component 8, and obtains the workpiece front side fusion image after fusing the photographic images of the two CCD cameras through steps S301-S304, and transmits it to the central control microcontroller again.

[0168] 2.1) The central control microcontroller processes the obtained front fusion image L a (x, y) is subjected to median filtering and grayscale processing to obtain a grayscale image I a (x,y);

[0169] 2.2) Calculate the gray level co-occurrence matrix of the image, obtain the gray level probability P(i,j) of each pixel pair (i,j) in the gray level image in the matrix, and further extract the texture feature information of the image according to the gray level co-occurrence matrix; the texture feature information of the image includes contrast Con, entropy En and uniformity Energy;

[0170] 2.3) Grayscale image I of the front fusion image a (x,y) Apply the Ostu threshold method to compare the texture feature information of each image extracted with its threshold;

[0171] 2.4) If only the entropy En texture feature of the front fusion image exceeds the threshold, the workpiece is marked as having a defect risk;

[0172] 3) The central control microcontroller continues to obtain the photographic images of the fifth CCD camera 101 to the seventh CCD camera 103 of the third visual inspection component 10, and fuses the three CCD camera images again through steps S301-S304 to obtain a fused image of the back side of the workpiece;

[0173] 3.1) The central control microcontroller processes the obtained front fusion image L b (x, y) is subjected to median filtering and grayscale processing to obtain a grayscale image I b (x,y)

[0174] 3.2) Calculate the gray level co-occurrence matrix of the image, obtain the gray level probability P(i,j) of each pixel pair (i,j) in the gray level image in the matrix, and further extract the texture feature information of the image according to the gray level co-occurrence matrix;

[0175] 3.3) Grayscale image I of the rear fusion image b (x,y) Apply the Ostu threshold method to compare the texture feature information of each image extracted with its threshold;

[0176] 3.4) Grayscale image I of the rear fusion image b The texture feature information of (x,y) does not exceed its threshold;

[0177] 4) The central control microcontroller obtains the bottom image of the first visual detection component 6, the top image of the second CCD camera 81 of the second visual detection component 8, the front fusion image obtained by S301-S304, and the back fusion image obtained by S301-S304 again, and executes steps S311-S315, and in step S315, executes steps S3151-S3155 to obtain the following: Fig.11 The 3D point cloud of the workpiece shown ( Fig.11 Only part of the 3D point cloud image of the top of the workpiece is shown), and the following Fig.12 The 3D point cloud network image of the workpiece shown ( Fig.12 Only the 3D point cloud network image of the top of the workpiece is shown);

[0178] 4.1) The central control microcontroller obtains the three-dimensional point cloud network image L t (x, y) is subjected to median filtering and grayscale processing to obtain a grayscale image I t (x,y)

[0179] 4.2) Calculate the gray level co-occurrence matrix of the image, and further extract the texture feature information of the image based on the gray level co-occurrence matrix;

[0180] 4.3) Grayscale image I of the 3D point cloud network image t (x,y) Apply the Ostu threshold method to extract the grayscale image I of the three-dimensional point cloud network image t The three texture feature information of (x, y) are compared with their corresponding thresholds;

[0181] 4.4) Grayscale image of 3D point cloud network image I t The contrast Con and entropy En of (x, y) do not exceed their thresholds, but their uniformity Ho exceeds the threshold;

[0182] Since it has been marked as having a defect risk, when the Ostu threshold method is used again to compare the thresholds of the texture feature information of the image, as long as one of the texture feature information exceeds the threshold, the central control microcontroller determines that the workpiece is a defective workpiece, and when it passes through the third blow nozzle 11, the third blow nozzle 11 is controlled to blow into the third product recovery barrel 13.

[0183] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0184] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0185] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for detecting a workpiece production line based on CCD vision inspection, wherein the method is based on multiple CCD cameras taking pictures of the inspected workpiece from different angles to perform defect detection and product screening, and is characterized in that: The method comprises the following steps: S1, the plurality of workpieces produced are vibrated and transmitted in sequence by the vibrating material assembly (3) to the workpiece transmission disc (4); S2, controlling the workpiece transmission disc (4) to rotate in situ, allowing a plurality of workpieces to be photographed and inspected by the first visual inspection component (6) in turn and then fall directly above the starting end of the feeding conveyor belt (5); when the inspection result obtained by the first visual inspection component (6) indicates that there is a problem with the bottom of the workpiece, controlling the first blowing nozzle (7) to open, blowing the defective workpiece at the first position into the corresponding first product recovery bucket (12); S3, controlling the feeding conveyor belt (5) to inspect the workpieces dropped thereon in turn through the second visual inspection component (8) and the third visual inspection component (10); when the inspection result obtained by the second visual inspection component (8) indicates that there is a problem with the top and / or the front side of the workpiece, controlling the second blowing nozzle (9) to open, and blowing the defective workpiece at the second position into the corresponding second product recovery bucket (13); S4, after the workpiece has been inspected by the third visual inspection component (8), the central control single chip computer performs point cloud fusion on the three-dimensional images of the workpiece from the first visual inspection component (6) to the third visual inspection component (11), and performs defect detection on other parts of the workpiece; S5. When the inspection result obtained by the third visual inspection component (11) indicates that there is a problem on the rear side of the workpiece, or when the inspection result obtained by step S4 shows that there are defects in other parts of the workpiece, the third blowing nozzle (11) is controlled to open to blow the defective workpiece at the third part into the corresponding third product recovery bucket (14).

2. The detection method for a workpiece production line based on CCD vision detection according to claim 1 is characterized in that: The first defect is the bottom defect; the second defect is the top defect and / or the front defect; the third defect is the back defect and / or other defects; other defects refer to any defects on the three-dimensional solid surface of the standard workpiece CAD drawing.

3. The detection method of a workpiece production line based on CCD vision detection according to claim 2 is characterized in that: The method in which the central control single chip computer controls the first visual detection component (6) to detect a first part defect, controls the second visual detection component (8) to detect a second part defect, and controls the third visual detection component (10) to detect a third part defect comprises the following steps: Obtain a photographic image of the workpiece where the defect is located; Perform median filtering on the acquired image and grayscale it to obtain a grayscale image I(x,y); Calculate the gray level co-occurrence matrix of the image, and further extract the texture feature information of the image based on the gray level co-occurrence matrix; Apply the Ostu threshold method to the images of the corresponding parts, and compare the texture feature information of each image extracted with its threshold; If none of the texture feature information of the image exceeds the threshold, the corresponding part does not have defects; if any two of the feature information exceed the threshold, it is concluded that the corresponding part has defects; if only one feature information exceeds the threshold, the workpiece is marked as having a defect risk, and the subsequent visual inspection components are continued to wait for repeating the above steps and making judgments based on the same rules; when only one feature of the image texture feature of the subsequent visual inspection component exceeds its threshold, if the inspected component is marked as having a defect risk, the workpiece is identified as having defects.

4. The detection method of a workpiece production line based on CCD vision detection according to claim 3 is characterized in that: In the process of controlling the second visual inspection component to detect the front side defects and in the process of controlling the third visual inspection component to detect the rear side defects, after performing image fusion on the multiple cameras on the corresponding side, texture feature extraction of the image is performed to perform defect detection, and the image fusion on the multiple cameras on the corresponding side includes the following steps: S301, obtaining photographic images I of multiple cameras on the corresponding side m,n (x, y); where m is a or b, a represents the front side, and b represents the back side; n = 3, 4, 5, 6, 7; x is the x-axis coordinate of the image point in the image two-dimensional coordinate system, and y is the y-axis coordinate of the image point in the image two-dimensional coordinate system; S302, calculate the internal parameter matrix J of the nth CCD camera n : Among them, f x,n is the x-axis focal length of the intersection of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, f y,n The y-axis focal length of the intersection of the nth CCD camera and the line connecting the image point in the two-dimensional image coordinate system; c x,n is the x-axis coordinate position of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system, c y,n is the y-axis coordinate position of the intersection point of the line connecting the nth CCD camera and the image point in the two-dimensional image coordinate system; S303, calculate the rotation matrix R of the nth CCD camera n : R n =R n,z (γ n )R n,y (β n )R n,x (α n ), where α n , β n and γ n They are the x-axis rotation angle, y-axis rotation angle and z-axis rotation angle of the nth CCD camera around the three-dimensional coordinate system of the production line body; α3=60°, α4=120°, α5=-135°, α6=-90°, α7=-45°;β n =0°,γ n =0°; The three-dimensional coordinate system of the production line itself is a left-handed coordinate system; S304, calculating the fusion image L of multiple cameras on the corresponding side of the m-th visual detection component m (x,y): Among them, k n is the kth camera of the nth camera n The number of captured images, K n is the total number of images taken by the nth camera; ω m,n is the image fusion weight coefficient of the nth camera on the corresponding side of the mth visual inspection component, ∑ω m =1,∑ω m is the sum of the image fusion weight coefficients of the mth visual detection component; (X n , Y n ) is the coordinate of the nth camera on the xy plane of the production line body coordinate system.

5. The detection method of a workpiece production line based on CCD vision detection according to claim 4 is characterized in that: The method in which the central control single chip microcomputer controls the third visual inspection component to detect other defects includes the following steps: S311, the central control single chip obtains the photographic image of the bottom of the workpiece by the first CCD camera of the first visual inspection component, the photographic image of the top of the workpiece by the second CCD camera of the second visual inspection component, and obtains the multiple camera fusion images L of the corresponding side of the mth visual inspection component calculated in step S304 m (x, y); S312, construct the normal field vector N(x, y)=[N x (x,y),N y (x,y),N z (x, y)] T ; Among them, N x (x, y), N x (x, y) and N x (x, y) are the components of the three-dimensional image normal of the workpiece surface image point (x, y) on the x-axis, y-axis and z-axis respectively, N z (x, y) is normalized to 1; S313, calculate the image point (x, y) on the y-axis gradient G of the normal field y (y, z) and the z-axis gradient G z (y, z), and integrate the two to obtain the three-dimensional point cloud depth map Z(x, y): Where x' is the integration unit of Z(x, y) in the normal field x-axis, y' is the integration unit of Z(x, y) in the normal field y-axis; Z(x0, y0) is the depth of the initial image point (x0, y0); S314, repeating steps S311-S313 for the M image points of the workpiece surface image, optimizing the iteration based on the least squares method, calculating the error function E(Z) of the gradient field and the depth field of the M image points until it converges to the minimum value, stopping the iteration, and obtaining the three-dimensional point cloud set P = {p1, p2, ..., p i , ..., p M }, i = 1, 2, ..., M; p i is the i-th 3D point in the 3D point cloud set reconstructed from the i-th image point; p i The three-dimensional coordinates of the point are (x i ,y i , Z i (x i ,y i )), Z i (x i ,y i ) is the 3D point cloud depth map of the i-th image point obtained by the least squares optimization iteration, and the 3D point cloud depth map obtained by the optimization iteration is used as the vertical coordinate of the 3D point cloud; S315. Reconstruct the M three-dimensional points in the three-dimensional point cloud set P obtained in step S314 into a polygonal network using the Delaunay triangulation method based on the empty sphere rule, and then reconstruct a three-dimensional point cloud network image of the workpiece surface.

6. The detection method of a workpiece production line based on CCD vision detection according to claim 5, characterized in that: In the step S313, the x-axis gradient G of the image point (x, y) in the normal field is calculated. x (x, y) and the y-axis gradient G y The calculation formula for (x, y) is as follows: Among them, N y =∑L m (x, y), ∑L m (x, y) is the fusion image L of multiple cameras of the front side of the workpiece taken by the second visual inspection component (i.e., the fusion image L of multiple images of the front side of the workpiece taken by the third CCD camera and the fourth camera in the second visual inspection component). a (x, y)) and the fused image L of multiple cameras of the rear side of the workpiece taken by the third visual inspection component (i.e., the fused image L of multiple images of the front side of the workpiece taken by the fifth CCD camera to the seventh camera in the second visual inspection component). b (x,y)) and N z =L u (x,y)+L d (x,y),L u (x, y) is the image of the top of the workpiece taken by the second CCD camera, L d (x, y) is the image of the bottom of the workpiece taken by the first CCD camera.

7. The detection method of a workpiece production line based on CCD vision detection according to claim 5 is characterized in that: The error function E(Z) between the gradient field and the depth field of the M image points in step S314 is calculated as follows:

8. The CCD-based visual inspection workpiece production line and inspection method according to claim 7, characterized in that: The step S315 includes the following steps: S3151, select any three non-collinear three-dimensional points p in the three-dimensional point cloud set P. i 、p j and p k , p i The three-dimensional coordinates of the point are (x i ,y i ,Z i (x i ,y i )), p j The three-dimensional coordinates of the point are (x j ,y j ,Z j (x j ,y j )), p k The three-dimensional coordinates of the point are (x k ,y k ,Z k (x k ,y k )) S3152, calculate the p of three 3D points i The vector between two other points S3153, further calculate the vertical to three three-dimensional points p i 、p j and p k The plane p i p j p k The normal vector Its module length u x =(y j -y i )(z k -z i )-(z j -z i )(y k -y i ); u y =(z j -z i )(x k -x i )-(x j -x i )(z k -z i ); or z (x) j -x i )(and k -and i )-(and j -and i )(x k -x i ); S3153, calculate the coordinates of the center O of the circumscribed circle (O x ,O y ,O z ); S3154. Calculate the radius R of the circumscribed circle O: Get three three-dimensional points p i 、p j and p k The polygonal network formed; S3155. Repeat steps S3151-S3154 for all three non-collinear three-dimensional points in the three-dimensional point cloud set P to reconstruct a three-dimensional point cloud network image of the workpiece surface.

9. A CCD-based visual inspection workpiece production line using the method according to any one of claims 1 to 8, the production line comprising a first base (1) and a second base (2), characterized in that: The production line further comprises a central control single chip computer for executing steps S1-S5 of any of the methods described in claims 1-8, a vibrating material assembly (3) arranged on the first base (1), and a workpiece transfer disc (4) arranged on the second base (2), a feeding conveyor belt (5), a first visual detection component (6) arranged on the workpiece transfer disc (4), a first blowing nozzle (7), a second visual detection component (8), a second blowing nozzle (9), a third visual detection component (10) and a third blowing nozzle (11); the first blowing nozzle (5) to the third blowing nozzle (11) are arranged in sequence from left to right on the front side of the feeding conveyor belt (5), and corresponding first product recovery bins (12), second product recovery bins (13) and third product recovery bins (14) are respectively arranged on the rear side of the feeding conveyor belt (5); The first visual detection component (6) comprises a first CCD camera (61) and a first multi-spectral illumination disk (62); the second visual detection component (8) comprises a second CCD camera (81), a square multi-spectral illumination frame (82), a third CCD camera (83) and a fourth CCD camera (84), and a first projection panel (85); The third visual detection component (10) comprises a fifth CCD camera (101), a sixth CCD camera (102), a seventh CCD camera (103), a second projection panel (104) and a second multi-spectral illumination disk (105); The vibrating material assembly (3) comprises a workpiece delivery port (31), an inwardly convex rotating screening material hopper (32), a screening material receiving track (33) connected to an outlet (321) of the inwardly convex rotating screening material hopper (32), and a pulse electromagnet device (34).

Images