Shoe pattern defect detection system based on visual technology

Abstract

The invention relates to the technical field of machine vision and detection, in particular to a shoe pattern defect detection system based on the visual technology, which comprises an acquisition processing unit, a three-dimensional model unit, a defect detection unit and a defect processing unit. The three-dimensional model unit reconstructs an accurate shoe pattern three-dimensional model in sequence, the defect detection unit divides the model into a plurality of areas, calculates the average curvature, and accurately analyzes and judges the flatness defect degree by adopting a plane fitting algorithm based on a least square method and a curvature difference algorithm based on a sliding window; and the defect processing unit automatically processes the shoes with serious defects, so that high efficiency, accuracy and automation of shoe pattern defect detection are realized, and the quality detection level of shoe production is powerfully improved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision and detection technology, in particular to a shoe pattern defect detection system based on vision technology. Background Art

[0002] Machine vision and inspection is a key technology. In the field of footwear manufacturing, the quality control of shoe patterns is of great significance to ensure the overall quality of the product. At present, the inspection of shoe patterns in practical applications mainly relies on manual visual inspection. This traditional inspection method can meet the quality monitoring needs within a certain range, but with the improvement of production efficiency and accuracy requirements, the ability of manual inspection in processing complex geometric shapes and texture features is limited to a certain extent.

[0003] Manual inspection methods rely on experience and visual observation, so it is often difficult to deeply analyze the complex geometric features and texture structures of shoe patterns. In addition, when faced with the demand for efficient production, manual inspection is inefficient and difficult to achieve comprehensive data analysis, which brings certain challenges to the optimization of production processes and the continuous improvement of quality.

[0004] Therefore, in order to achieve quantitative analysis of defect information in the field of footwear manufacturing on the one hand, and to provide data support for production improvement and quality management of enterprises on the other hand, we provide a shoe pattern defect detection system based on vision technology. Summary of the invention

[0005] The object of the present invention is to provide a shoe pattern defect detection system based on vision technology to solve the problems raised in the above background technology.

[0006] The present invention establishes a three-dimensional surface model based on the three-dimensional shoe pattern, divides the shoe into main areas according to the three-dimensional surface model, and determines the flatness defects of the shoe pattern by analyzing the average curvature value of the points in each sub-area in each main area.

[0007] To achieve the above-mentioned purpose, a shoe pattern defect detection system based on vision technology is provided, including an acquisition processing unit, a three-dimensional model unit, a defect detection unit and a defect processing unit;

[0008] The acquisition and processing unit uses binocular vision technology to acquire images of the shoes to obtain three-dimensional image data of the shoes, and pre-processes the acquired three-dimensional image data;

[0009] The three-dimensional model unit reconstructs a three-dimensional model of the shoe pattern based on the preprocessed three-dimensional image data, wherein when reconstructing based on binocular vision, a corner detection algorithm is used to extract image feature points, and the three-dimensional coordinates of the feature points are calculated by a triangulation algorithm, and then a surface fitting technology is used to construct a three-dimensional surface model of the shoe pattern;

[0010] The defect detection unit divides the three-dimensional surface model into multiple detection areas, calculates the average curvature of each area to preliminarily characterize its flatness, and uses a plane fitting algorithm based on the least squares method to perform local refinement analysis on the area with abnormal average curvature to determine whether the area has a flatness defect. For the area determined to have a flatness defect, the curvature change trend of its adjacent areas is analyzed, and the degree of the flatness defect is determined according to the analysis result;

[0011] When the defect processing unit detects that the shoes have serious flatness defects, it automatically controls the robotic arm to grab the defective shoes and place them in a specific defective product recycling area, and at the same time sends an alarm signal containing the shoe identification code, defect type and detection time to the production management center.

[0012] As a further improvement of the technical solution, the acquisition processing unit uses a contrast enhancement algorithm to pre-process the three-dimensional image data, as follows:

[0013] The contrast enhancement algorithm adopts a method based on combining histogram stretching with contrast-limited adaptive histogram equalization, specifically:

[0014] Traverse each pixel in the collected three-dimensional image data, determine its grayscale value, and count the frequency of occurrence of each grayscale value to obtain the original grayscale histogram of the image;

[0015] Determine the minimum grayscale value and the maximum grayscale value in the original image, and design a stretching transformation function, wherein the stretching function is used to map the original grayscale value to a new stretched grayscale range, traverse each pixel in the image again, replace its original grayscale value with the new grayscale value calculated by the stretching transformation function, and obtain an image after histogram stretching;

[0016] The stretched image is divided into non-overlapping sub-regions, and histogram equalization is performed on the sub-regions to obtain the final contrast-enhanced image.

[0017] As a further improvement of the technical solution, the corner point detection algorithm in the three-dimensional model unit is the Harris corner point detection algorithm, which is as follows:

[0018] Use the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions respectively, and calculate the autocorrelation matrix for each pixel in the image;

[0019] According to the autocorrelation matrix, the corner point response function of each pixel is calculated, and the pixels whose median value of the corner point response function is less than the preset threshold are excluded to screen out the preliminary corner point area. In the preliminary corner point area, for each pixel, its focus response value is compared with the corner point response value of the neighboring pixel points;

[0020] If the corner point response value of the pixel is not the maximum value in the neighborhood, it will be excluded. The remaining pixel points after non-maximum suppression are the final detected corner points.

[0021] As a further improvement of the technical solution, the triangulation algorithm in the three-dimensional model unit adopts an optimization algorithm based on epipolar constraints, which is as follows:

[0022] Use a calibration plate of known size and shape, and use a binocular camera to collect images of the calibration plate. For each calibration image, use a corner detection algorithm to extract the coordinates of the corner points on the calibration plate. According to the extracted corner point coordinates and the actual geometric dimensions of the calibration plate, the calibration algorithm is used to calculate the intrinsic and extrinsic matrix of the binocular camera.

[0023] For the shoe image to be reconstructed, feature points are extracted from the left and right camera images respectively and feature descriptors are generated. Feature points are matched in the left and right images according to the feature descriptors to find the same-name point pairs in the left and right images. For each pair of initially matched same-name points, epipolar constraints are used to screen them to obtain the final same-name point pairs.

[0024] For the final pairs of same-name points, a triangulation relationship is constructed based on the geometric model of the binocular camera, and the three-dimensional coordinates of the same-name points are calculated based on the principle of similar triangles and the intrinsic and extrinsic parameter matrices of the camera.

[0025] As a further improvement of the technical solution, the surface fitting technology in the three-dimensional model unit adopts non-uniform rational B-spline surface fitting, and the specific steps are as follows:

[0026] According to the shape characteristics of the shoe pattern, the number of control points required is calculated, and based on the edge points of the shoe pattern, the initial positions of the control points are determined by an automatic algorithm;

[0027] Construct the non-uniform rational B-spline surface expression, obtain the degree and parameters, and calculate the B-spline basis function according to the degree and parameters;

[0028] The data points in the reconstructed three-dimensional model of the shoe pattern are substituted into the non-uniform rational B-spline surface expression, the objective function is constructed, and the minimum value of the objective function is calculated using the least squares method. The least squares optimization process is repeated until the value of the objective function is less than the preset threshold, and the three-dimensional surface model is obtained.

[0029] As a further improvement of the technical solution, when the defect detection unit divides the shoe pattern three-dimensional model into multiple detection areas, an adaptive division method based on image geometric features and shoe pattern topological structure is adopted, which is as follows:

[0030] The 3D version of the Canny edge detection algorithm is used to extract the edges of the 3D model of the shoe pattern to obtain a set of edge points. The overall shape characteristics of the shoe are analyzed based on the edge points, and finally the geometric regions are divided according to the overall shape characteristics.

[0031] Then, in each region, the texture features of the shoe pattern and the average curvature of each point in the region are analyzed. The texture features are obtained by calculating the direction and density of the texture, and sub-regions of each geometric region are delineated according to the analysis results.

[0032] As a further improvement of the technical solution, the defect detection unit adopts a method based on the weighted sum of Gaussian curvature and average curvature when calculating the average curvature of each area, which is as follows:

[0033] For each point on the surface of the three-dimensional surface model of the shoe pattern, firstly calculate the coefficient of the first basic form, then calculate the coefficient of the second basic form, and then calculate the Gaussian curvature according to the coefficients of the first and second basic forms;

[0034] Calculate the mean curvature correlation vector, calculate the mean curvature based on the mean curvature correlation vector, and set the Gaussian curvature weight and mean curvature weight based on historical experience. For each point in the detection area, multiply its Gaussian curvature and mean curvature by the corresponding weights respectively, and then sum them up to get the mean curvature value of the area.

[0035] As a further improvement of the technical solution, the operation of the defect detection unit when using the plane fitting algorithm based on the least squares method is as follows:

[0036] The average curvature value of the region is compared with a preset threshold value to determine the region with abnormal average curvature, extract the coordinate data of all three-dimensional points in the region with abnormal average curvature, and randomly select several points in the region as the initial sample point set to calculate the initial fitting plane equation;

[0037] A linear equation system is constructed according to the selected points, and the singular value decomposition method is used to solve the linear equation system to obtain the initial plane equation parameters, and the initial fitting plane is determined according to the parameters;

[0038] Calculate the distance deviations from all points in the region to the initial fitting plane, add points whose distance deviations are greater than the first threshold to the sample point set to update the sample point set, reconstruct the linear equations based on the updated sample point set and solve them to obtain new plane equation parameters and determine the new fitting plane;

[0039] Repeat the above steps and iterate continuously until the error of the fitting plane is lower than the second threshold. The iteration stops and the final fitting plane is obtained for flatness defect determination.

[0040] As a further improvement of the technical solution, the defect detection unit adopts a curvature difference algorithm based on a sliding window when analyzing the curvature change trend of adjacent areas. The specific steps are as follows:

[0041] Set the size and step length of the sliding window. For each sliding window position, calculate the curvature difference between the central area and the adjacent area in the window to obtain the curvature difference vector in each adjacent direction.

[0042] The maximum value, minimum value and root mean square of the curvature difference vector are calculated, and the flatness defects are divided into three levels: slight defect, moderate defect and severe defect according to the comparison results between the maximum value, minimum value and root mean square of the curvature difference vector and the preset threshold.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] In the shoe pattern defect detection system based on vision technology, the three-dimensional model unit can accurately reconstruct the three-dimensional model of the shoe pattern through corner detection, triangulation and surface fitting algorithms, providing an accurate basis for subsequent defect detection. Based on this model, the defect detection unit uses regional average curvature calculation, plane fitting algorithm and curvature difference algorithm to accurately detect the flatness defects of the shoe pattern, effectively reduce the occurrence of missed detection and false detection, and improve the accuracy and consistency of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an overall block diagram of the present invention.

[0046] The meaning of each number in the figure is:

[0047] 1. Acquisition and processing unit; 2. Three-dimensional model unit; 3. Defect detection unit; 4. Defect processing unit. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a shoe pattern defect detection system based on visual technology, please refer to Figure 1 As shown, it includes an acquisition processing unit 1, a three-dimensional model unit 2, a defect detection unit 3 and a defect processing unit 4;

[0050] The acquisition and processing unit 1 uses binocular vision technology to acquire images of the shoes to obtain three-dimensional image data of the shoes, and uses a contrast enhancement algorithm to pre-process the acquired three-dimensional image data, as follows:

[0051] Traverse each pixel in the collected 3D image data, determine its grayscale value, and count the frequency of each grayscale value to obtain the original grayscale histogram of the image. ,in Representing the grayscale, accurately grasping the overall grayscale characteristics of the image, helps to formulate appropriate enhancement strategies for different grayscale distributions.

[0052] Get the minimum gray value in the original image by finding the minimum and maximum gray values ​​with non-zero frequency in the gray histogram and the maximum gray value to determine the start and end points of the stretch.

[0053] According to the grayscale range of the image, a mapping relationship is constructed to redistribute the original grayscale value to a wider range to improve the contrast. The stretching transformation function is set as , whose formula is , this function converts the original gray value Mapping to the new stretched grayscale range of 0 to 255 makes the originally low-contrast image clearer overall.

[0054] Apply the stretch transformation to each pixel to improve the overall contrast of the image, and then traverse each pixel in the image again to convert its original grayscale value Replace with the calculated new grayscale value , get the image after histogram stretching , enhances image details, making the outline and features of the shoe pattern more obvious, facilitating subsequent processing and analysis.

[0055] Will Divide into sub-regions of the same size and without overlap, to avoid the over-enhancement or loss of details in some regions caused by global equalization. , grayscale histogram in statistical sub-region , providing accurate data for local adaptive enhancement and calculating the cumulative distribution function in the sub-region ; Make the grayscale distribution more uniform, improve the contrast within the sub-area, and limit the contrast according to the preset parameters , the cumulative distribution function is restricted as follows:

[0056] Calculates the slope of the cumulative distribution function ; Accurately quantify the trend of grayscale value changes to provide a basis for reasonable contrast limitation and prevent local over-enhancement from causing detail loss or artifacts. , then adjust the cumulative distribution function so that ,Right now ; In order to effectively control the degree of local contrast enhancement and protect image details.

[0057] According to the processed cumulative distribution function, the equalized gray value of each pixel in the sub-area is calculated I e ( x ， y ) = [ CDF mn ( I new ( x ， y ) ) N mn × 255 ] ;in are the coordinates of the pixels in the subregion, It is a sub-region The total number of pixels in the area is increased to achieve local contrast equalization, optimize local contrast, and enhance the detail clarity of the shoe pattern in different areas.

[0058] After sub-region processing, pixels at the sub-region boundary may be discontinuous. In order to obtain a smooth image, bilinear interpolation is performed on the entire image. For pixels that are not on the sub-region boundary, their values ​​remain unchanged; for pixels on the sub-region boundary, bilinear interpolation is performed based on the equalized grayscale values ​​of the corresponding pixels in the adjacent sub-region to obtain the final contrast-enhanced image. .

[0059] The three-dimensional model unit 2 reconstructs the three-dimensional model of the shoe pattern based on the preprocessed three-dimensional image data. When reconstructing based on binocular vision, the Harris corner detection algorithm is used to extract image feature points, as follows:

[0060] Image gradient can reflect the degree and direction of change of pixel values ​​in an image, and is an important basis for detecting corner points. Sobel operator is used to calculate In the horizontal direction and vertical direction The gradient of each pixel in the image , by convolving the template of the Sobel operator in the horizontal and vertical directions with the image, the gradient values ​​in the horizontal and vertical directions are obtained, which are and , effectively captures the changes in edges and textures in images, has a certain degree of robustness to noise, and provides accurate gradient data for corner detection without increasing excessive computational complexity.

[0061] For each pixel in the image , calculate its autocorrelation matrix The autocorrelation matrix can describe the change relationship of the pixel grayscale value in the local area of ​​the image. By calculating the autocorrelation matrix, the degree of grayscale change in the image in different directions and the correlation between them can be quantified, thereby providing a basis for judging whether the pixel point is a corner point.

[0062] According to the autocorrelation matrix Then calculate the corner point response function of each pixel , exclude pixels whose corner point response function median is less than the preset threshold, and screen out the preliminary corner point area. By screening the preliminary corner point area by threshold, we can quickly focus on the area where corner points may exist, improve the speed of corner point detection, and at the same time ensure the accuracy of detection to a certain extent, avoiding too many irrelevant pixels from participating in subsequent complex calculations. In the shoe pattern image, it helps to improve the efficiency and accuracy of the entire corner point detection process, thereby providing important feature point support for building an accurate shoe pattern three-dimensional model.

[0063] In the preliminary corner point area, for each pixel, compare its corner point response value Compared with the corner point response values ​​of the surrounding neighborhood pixels, if the corner point response value of the pixel is not the maximum value in the neighborhood, it will be excluded, and the remaining pixels are the final detected corner points. These corner points are distributed in important positions such as the edge of the pattern and the place where the texture changes dramatically. They provide a reliable feature point basis for the subsequent triangulation algorithm, making the calculated three-dimensional coordinates more accurate.

[0064] Then use a calibration plate of known size and shape, and use the binocular camera to capture images of the calibration plate from different angles to ensure that the calibration plate is clearly visible in the image and covers different positions and postures. For each calibration image, use the above-mentioned corner detection algorithm to extract the coordinates of the corner points on the calibration plate. At the same time, according to the actual geometric dimensions of the calibration plate, the calibration algorithm is used to calculate the intrinsic parameter matrix and extrinsic parameter matrix of the binocular camera.

[0065] Camera calibration is a key step in determining the camera's internal parameters and external position and posture. The calibration plate of known size and shape provides an accurate geometric reference. The corner points are prominent features on the calibration plate, and their accurate coordinate extraction is crucial for calculating the camera parameters. The intrinsic parameter matrix describes the optical and geometric properties inside the camera, and the extrinsic parameter matrix determines the camera's position and posture in the world coordinate system. These parameters are the basis for the subsequent accurate calculation of the object's three-dimensional coordinates.

[0066] For the shoe image to be reconstructed, feature extraction algorithms are used in the left and right camera images to extract feature points and generate feature descriptors. Feature point matching is performed in the left and right images based on the feature descriptors. By calculating the similarity between feature descriptors, the same-name point pairs in the left and right images are found. The imaging of the same spatial point in the left and right camera images has a certain similarity. The corresponding points can be preliminarily found through matching feature descriptors.

[0067] Given the intrinsic and extrinsic parameters of the camera, the epipolar equation corresponding to a point in the left image in the right image can be calculated based on the epipolar geometry. The distance from the matching point in the right image to its corresponding epipolar line is used as the basis for judgment. If the distance is greater than a certain threshold, the matching point pair is considered to be a mismatch and is excluded. After epipolar constraint screening, more accurate pairs of points with the same name are obtained. When constructing a three-dimensional model, these accurate pairs of points with the same name can ensure that the calculated three-dimensional coordinates are more accurate, thereby improving the quality of the three-dimensional model.

[0068] For the pairs of points with the same name that have been filtered through the epipolar constraint, a triangulation relationship is constructed based on the geometric model of the binocular camera, as follows:

[0069] Assume the optical center of the left camera is , the optical center of the right camera is , the coordinates of the points with the same name in the left image are , whose coordinates in the right image are , convert image coordinates to normalized camera coordinates, for the left image, ;in is the intrinsic parameter matrix of the left camera, and for the right image, ;in is the intrinsic parameter matrix of the right camera.

[0070] Calculate the three-dimensional coordinates of points with the same name based on triangulation relationships The calculated three-dimensional coordinates of the same-name points accurately represent the position and shape of the shoe pattern in the three-dimensional space. In the shoe pattern detection, the three-dimensional model of the shoe pattern can be accurately constructed, so that the position of each feature point in the model is highly consistent with the actual shoe pattern, which provides a high-precision model basis for the subsequent flatness defect detection, can more accurately detect various defects in the shoe pattern, improve the performance and reliability of the detection system, and help to improve the quality control level of footwear products.

[0071] The three-dimensional coordinates of the feature points are calculated using an optimization algorithm based on epipolar constraints, as follows:

[0072] According to the shape characteristics of the shoe pattern and combined with historical experience, a number of control points are set for the decorative part of the toe and the pattern area of ​​the sole. A number of control points are determined, and the initial positions of the control points are determined by an automatic algorithm based on the edge points of the shoe pattern. The edge points are screened from the three-dimensional coordinates of the points with the same name.

[0073] The number and initial position of control points directly affect the accuracy and efficiency of surface fitting. It has good versatility for shoes of different styles and patterns, and there is no need to adjust parameters for each shoe type. The initial position of the control points is determined by an automatic algorithm, which reduces manual intervention and makes the subsequent surface fitting process more efficient and accurate.

[0074] After determining the number and initial position of control points, we start to construct the non-uniform rational B-spline surface expression ,but:

[0075] ;

[0076] in, is a parameter variable, with a value range of [ 0 , 1 ] between, They are The number of control points in the direction, They are Second and The B-spline basis function of order, is with control point The associated weight.

[0077] According to the selected number of , And parameters , calculate the B-spline basis function according to the recursive formula ,Constructing expressions and calculating basis functions are the core steps to achieve surface fitting.,By reasonably selecting parameters and calculating basis functions, a surface close to the real shape of the shoe pattern can be constructed according to control points and weights.,In the construction of the three-dimensional model of the shoe pattern, the accurately constructed surface expression and calculated B-spline basis function enable the surface to accurately fit the shoe pattern shape represented by the three-dimensional coordinates of the synonymous points.

[0078] Substitute the three-dimensional coordinates of the same-name points into the non-uniform rational B-spline surface expression to construct the objective function. The expression of the objective function is: ;in, It is the three-dimensional coordinate data of the same-name points. The least squares method is used to solve the minimum value of the objective function E. In surface fitting, the objective function measures the difference between the surface and the actual data points (the three-dimensional coordinates of the same-name points). By minimizing this difference, the surface can fit the data points as closely as possible, thereby improving the fitting accuracy. The three-dimensional surface model optimized by the least squares method can reproduce the shape of the shoe pattern with high accuracy.

[0079] The defect detection unit 3 divides the three-dimensional surface model into multiple detection areas using a three-dimensional version based on the Canny edge detection algorithm. In each divided geometric area, the direction and density of the texture are calculated to analyze the texture characteristics of the shoe pattern. At the same time, the average curvature of each point in the area is calculated. For each point on the surface of the three-dimensional surface model of the shoe pattern, the coefficient of the first basic form is first calculated, as follows:

[0080] set up is a parameterized surface representation, ;but ; ; ; is represented as a dot product operation, followed by the coefficients of the second fundamental form, and the surface normal vector ,Then ; and calculate the Gaussian curvature based on the coefficients of the first and second fundamental forms ,Right now ; Calculate the mean curvature related vector, that is: ; ; Calculate the mean curvature based on the mean curvature correlation vector ; Gaussian curvature weight and mean curvature weight are set based on historical experience. For each point in the detection area, its Gaussian curvature and mean curvature are multiplied by the corresponding weights respectively, and then the sum is obtained to obtain the mean curvature value of the area. Finally, sub-areas of each geometric area are delineated based on texture features and mean curvature analysis results.

[0081] Screening abnormal areas based on the average curvature can quickly focus on areas where flatness defects may exist, reduce the amount of calculation, improve detection efficiency, and have good adaptability to complex shoe pattern shapes.

[0082] Edge extraction and geometric area division are based on the overall structural features of the shoe pattern, which helps to grasp the shape of the shoe pattern from a macro perspective and provide a framework for subsequent local analysis. Analysis based on the three-dimensional model can accurately reflect the true shape and characteristics of the shoe pattern, and has higher accuracy than two-dimensional image analysis. In the decorative pattern area of ​​the shoe upper, texture and curvature analysis can be used to find out whether the pattern is deformed or uneven. Reasonable sub-area delineation makes subsequent defect detection more detailed and accurate.

[0083] The average curvature value calculated for each area is compared with the preset threshold to determine the area with abnormal average curvature. The coordinate data of all three-dimensional points in the area with abnormal average curvature are extracted. Several points in the area are randomly selected as the initial sample point set to calculate the initial fitting plane equation, as follows:

[0084] Assume the plane equation is ax+by+cz+d=0, for the selected sample point (x i ,y i , z i ), and write it in matrix form, namely The singular value decomposition method is used to solve the above linear equations to obtain the initial plane equation parameters (a0, b0, c0, d0), and the initial fitting plane P0 is determined based on these parameters: a0x+b0y+c0z+d0=0.

[0085] Calculate the distance deviation from all points in the region to the initial fitting plane, add points whose distance deviation is greater than the first threshold to the sample point set for updating the sample point set, and reconstruct the linear equations and solve them according to the updated sample point set to obtain the new plane equation parameters. , determine the new fitting plane Repeat the above steps and iterate continuously until the error of the fitting plane is lower than the second threshold value, then the iteration stops and the final fitting plane is obtained for flatness defect determination.

[0086] The least squares method is used for plane fitting because it can find the plane equation that minimizes the error between the plane and the data points. It has a good mathematical basis for describing the flatness of the region. The random selection and iterative update of the initial sample point set can gradually optimize the fitting plane to better adapt to the distribution of points in the region, especially when dealing with complex shapes and noisy areas. It can improve the accuracy of fitting. By setting a threshold to control the update of the sample point set and the iterative stop condition, it can avoid over-calculation while ensuring the fitting accuracy.

[0087] The singular value decomposition method for solving linear equations has the characteristics of good numerical stability and high computational efficiency. It can quickly and accurately obtain the parameters of the plane equation. The iterative optimization process can continuously improve the accuracy of the fitting plane, making defect judgment more reliable. In shoe pattern detection, accurate plane fitting can accurately evaluate the flatness of the area.

[0088] Set the size of the sliding window and the sliding step size to ensure that adjacent windows have overlapping areas. For each sliding window position, calculate the curvature difference between the central area in the window and the adjacent area to obtain the curvature difference vector in each adjacent direction. , and calculate the maximum value of the curvature difference vector , minimum and the RMS value According to the comparison results of the maximum value, minimum value and root mean square of the curvature difference vector with the preset threshold, the flatness defects are divided into three levels: slight defects, moderate defects and severe defects.

[0089] The setting of sliding window can comprehensively analyze the curvature change trend of different positions without missing any area of ​​the shoe pattern. Windows of different sizes are suitable for shoe pattern feature analysis of different scales. Small windows can capture detailed changes, and large windows can evaluate the overall trend. Curvature differential analysis based on sliding windows can comprehensively and meticulously detect the degree of flatness defects of shoe patterns. It has good detection capabilities for various complex shapes and defects of different degrees. In shoe pattern detection, it can accurately judge the degree of flatness defects in different areas.

[0090] like , it is determined that there are serious flatness defects in this area. is the preset severe defect threshold, if and , it is judged as a moderate flatness defect. is the moderate defect threshold, is the moderate RMS value threshold, if and , it is judged as a slight flatness defect. is the mild defect threshold, is the mild RMS threshold.

[0091] Calculating the curvature difference vector can quantify the curvature difference between the central area and the adjacent area, thereby reflecting the change in flatness. The degree of defect is determined by comparing with the preset threshold. This is based on the quantitative definition of the quality standard for footwear products, which makes the defect judgment objective and consistent. It can take corresponding treatment measures according to the different degrees of defects, effectively discover areas with large local flatness changes, and also evaluate the overall flatness trend, which helps to improve the overall quality of footwear products.

[0092] When the defect processing unit 4 detects that the shoes have serious flatness defects, it automatically controls the robotic arm to grab the defective shoes and place them in a specific defective product recycling area, and at the same time sends an alarm signal containing the shoe identification code, defect type and detection time to the production management center.

[0093] In the present invention, the acquisition and processing unit uses binocular vision to acquire and pre-process the three-dimensional image data of the shoe, and the three-dimensional model unit reconstructs an accurate three-dimensional model of the shoe pattern accordingly. The defect detection unit divides the model into multiple areas, calculates the average curvature, and adopts a plane fitting algorithm based on the least squares method and a curvature difference algorithm based on a sliding window to accurately analyze and determine the degree of flatness defects. The defect processing unit automatically processes shoes with serious defects, thereby achieving efficient, accurate and automated shoe pattern defect detection, and effectively improving the quality inspection level of footwear production.

[0094] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only preferred examples of the present invention, and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. The shoe pattern defect detection system based on vision technology is characterized by: It comprises a collection and processing unit (1), a three-dimensional model unit (2), a defect detection unit (3) and a defect processing unit (4); The acquisition and processing unit (1) acquires images of the shoe using binocular vision technology to obtain three-dimensional image data of the shoe, and pre-processes the acquired three-dimensional image data; The three-dimensional model unit (2) reconstructs a three-dimensional model of the shoe pattern based on the pre-processed three-dimensional image data, wherein when reconstructing based on binocular vision, a corner point detection algorithm is used to extract image feature points, and the three-dimensional coordinates of the feature points are calculated by a triangulation algorithm, and then a surface fitting technology is used to construct a three-dimensional surface model of the shoe pattern; The defect detection unit (3) divides the three-dimensional surface model into a plurality of detection areas, calculates the average curvature of each area to preliminarily characterize its flatness, and uses a plane fitting algorithm based on the least squares method to perform local refinement analysis on the area with abnormal average curvature to determine whether the area has a flatness defect, and for the area determined to have a flatness defect, analyzes the curvature change trend of its adjacent areas, and determines the degree of the flatness defect based on the analysis result; When the defect processing unit (4) detects that a shoe has a serious flatness defect, it automatically controls the robot arm to grab the defective shoe and place it in a specific defective product recycling area, and at the same time sends an alarm signal containing the shoe identification code, defect type and detection time to the production management center.

2. The shoe pattern defect detection system based on vision technology according to claim 1 is characterized in that: The acquisition and processing unit (1) uses a contrast enhancement algorithm to pre-process the three-dimensional image data, specifically as follows: The contrast enhancement algorithm adopts a method based on combining histogram stretching with contrast-limited adaptive histogram equalization, specifically: Traverse each pixel in the collected three-dimensional image data, determine its grayscale value, and count the frequency of occurrence of each grayscale value to obtain the original grayscale histogram of the image; Determine the minimum grayscale value and the maximum grayscale value in the original image, and design a stretching transformation function, wherein the stretching function is used to map the original grayscale value to a new stretched grayscale range, traverse each pixel in the image again, replace its original grayscale value with the new grayscale value calculated by the stretching transformation function, and obtain an image after histogram stretching; The stretched image is divided into non-overlapping sub-regions, and histogram equalization is performed on the sub-regions to obtain the final contrast-enhanced image.

3. The shoe pattern defect detection system based on vision technology according to claim 1 is characterized in that: The corner point detection algorithm in the three-dimensional model unit (2) is the Harris corner point detection algorithm, which is as follows: Use the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions respectively, and calculate the autocorrelation matrix for each pixel in the image; According to the autocorrelation matrix, the corner point response function of each pixel is calculated, and the pixels whose median value of the corner point response function is less than the preset threshold are excluded to screen out the preliminary corner point area. In the preliminary corner point area, for each pixel, its focus response value is compared with the corner point response value of the neighboring pixel points; If the corner point response value of the pixel is not the maximum value in the neighborhood, it will be excluded. The remaining pixel points after non-maximum suppression are the final detected corner points.

4. The shoe pattern defect detection system based on vision technology according to claim 3 is characterized in that: The triangulation algorithm in the three-dimensional model unit (2) adopts an optimization algorithm based on epipolar constraints, which is as follows: Use a calibration plate of known size and shape, and use a binocular camera to capture images of the calibration plate, extract the actual geometric dimensions of the calibration plate in each calibration image, and extract the corner point coordinates based on the detected corner points, and calculate the intrinsic and extrinsic matrix of the binocular camera through the calibration algorithm; For the shoe image to be reconstructed, feature points are extracted from the left and right camera images respectively and feature descriptors are generated. Feature points are matched in the left and right images according to the feature descriptors to find the same-name point pairs in the left and right images. For each pair of initially matched same-name points, epipolar constraints are used to screen them to obtain the final same-name point pairs. For the final pairs of same-name points, a triangulation relationship is constructed based on the geometric model of the binocular camera, and the three-dimensional coordinates of the same-name points are calculated based on the principle of similar triangles and the intrinsic and extrinsic parameter matrices of the camera.

5. The shoe pattern defect detection system based on vision technology according to claim 1 is characterized in that: The surface fitting technology in the three-dimensional model unit (2) adopts non-uniform rational B-spline surface fitting, and the specific steps are as follows: According to the shape characteristics of the shoe pattern, the number of control points required is calculated, and based on the edge points of the shoe pattern, the initial positions of the control points are determined by an automatic algorithm; Construct the non-uniform rational B-spline surface expression, obtain the degree and parameters, and calculate the B-spline basis function according to the degree and parameters; The data points in the reconstructed three-dimensional model of the shoe pattern are substituted into the non-uniform rational B-spline surface expression, the objective function is constructed, and the minimum value of the objective function is calculated using the least squares method. The least squares optimization process is repeated until the value of the objective function is less than the preset threshold, and the three-dimensional surface model is obtained.

6. The shoe pattern defect detection system based on vision technology according to claim 1, characterized in that: When the defect detection unit (3) divides the shoe pattern three-dimensional model into a plurality of detection areas, an adaptive division method based on image geometric features and shoe pattern topological structure is adopted, which is specifically as follows: The 3D version of the Canny edge detection algorithm is used to extract the edges of the 3D model of the shoe pattern to obtain a set of edge points. The overall shape characteristics of the shoe are analyzed based on the edge points, and finally the geometric regions are divided according to the overall shape characteristics. Then, in each region, the texture features of the shoe pattern and the average curvature of each point in the region are analyzed. The texture features are obtained by calculating the direction and density of the texture, and sub-regions of each geometric region are delineated according to the analysis results.

7. The shoe pattern defect detection system based on vision technology according to claim 6 is characterized in that: The defect detection unit (3) calculates the average curvature of each area by using a method based on the weighted sum of Gaussian curvature and average curvature, as follows: For each point on the surface of the three-dimensional surface model of the shoe pattern, firstly calculate the coefficient of the first basic form, then calculate the coefficient of the second basic form, and then calculate the Gaussian curvature according to the coefficients of the first and second basic forms; Calculate the mean curvature correlation vector, calculate the mean curvature based on the mean curvature correlation vector, and set the Gaussian curvature weight and mean curvature weight based on historical experience. For each point in the detection area, multiply its Gaussian curvature and mean curvature by the corresponding weights respectively, and then sum them up to get the mean curvature value of the area.

8. The shoe pattern defect detection system based on vision technology according to claim 7 is characterized in that: The operation of the defect detection unit (3) when using the plane fitting algorithm based on the least squares method is as follows: The average curvature value of the region is compared with a preset threshold value to determine the region with abnormal average curvature, extract the coordinate data of all three-dimensional points in the region with abnormal average curvature, and randomly select several points in the region as the initial sample point set to calculate the initial fitting plane equation; A linear equation system is constructed according to the selected points, and the singular value decomposition method is used to solve the linear equation system to obtain the initial plane equation parameters, and the initial fitting plane is determined according to the parameters; Calculate the distance deviations from all points in the region to the initial fitting plane, add points whose distance deviations are greater than the first threshold to the sample point set to update the sample point set, reconstruct the linear equations based on the updated sample point set and solve them to obtain new plane equation parameters and determine the new fitting plane; Repeat the above steps and iterate continuously until the error of the fitting plane is lower than the second threshold. The iteration stops and the final fitting plane is obtained for flatness defect determination.

9. The shoe pattern defect detection system based on vision technology according to claim 1, characterized in that: The defect detection unit (3) adopts a curvature difference algorithm based on a sliding window when analyzing the curvature change trend of adjacent areas. The specific steps are as follows: Set the size and step length of the sliding window. For each sliding window position, calculate the curvature difference between the central area and the adjacent area in the window to obtain the curvature difference vector in each adjacent direction. The maximum value, minimum value and root mean square of the curvature difference vector are calculated, and the flatness defects are divided into three levels: slight defect, moderate defect and severe defect according to the comparison results between the maximum value, minimum value and root mean square of the curvature difference vector and the preset threshold.

Images