A printing and dyeing defect detection method and system

Abstract

The application relates to the field of machine vision, and discloses a printing and dyeing defect detection method and system. The method comprises the following steps: an image of a printing and dyeing object surface is collected by a multi-angle imaging device arranged on a production line to obtain an original image set containing multiple visual angles; alignment processing is performed on the original image set to obtain a registration image set in a unified coordinate system; a three-dimensional point cloud model is generated according to the registration image set to obtain the geometric structure features of the printing and dyeing object surface; color information in the registration image set is projected to the three-dimensional point cloud model to obtain a three-dimensional color distribution model; pixel points in the three-dimensional color distribution model are grouped to obtain boundary features of a color difference area; according to the boundary features of the color difference area, the coordinate range of the color difference area in the three-dimensional color distribution model is analyzed to determine the spatial position coordinates of the defect. Through the technical scheme, color information projection distortion caused by geometric undulations is avoided, and the recognition accuracy of the color difference area is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, and in particular to a printing and dyeing defect detection method and system. BACKGROUND

[0002] In the production process of the printing and dyeing industry, dyeing color difference is an important reason affecting the quality of printing and dyeing products. In order to ensure product consistency, online detection technology is needed to identify the dyeing color difference on the surface of the printing and dyeing object in real time, so as to timely adjust the production process parameters and reduce the generation of unqualified products.

[0003] At present, a single-view or double-view imaging device arranged on the production line is mainly used to collect the 2D image of the surface of the printing and dyeing object, and the color space conversion and color difference value calculation are performed on the image, and then the color difference area is located and its coordinates in the 2D image are determined.

[0004] However, during the transmission process of the printing and dyeing object on the production line, the surface of the printing and dyeing object is prone to geometric undulations such as wrinkles and local protrusions due to factors such as tension changes and equipment extrusion; the 2D image can only reflect the planar projection information of the surface of the printing and dyeing object, and cannot obtain the three-dimensional geometric shape thereof, and the color information of the geometric undulation area may be distorted due to the perspective projection, which affects the accurate identification of the color difference area.

[0005] How to solve this technical problem is a technical problem that needs to be overcome by those skilled in the art. SUMMARY

[0006] The embodiment of the present application provides a printing and dyeing defect detection method to at least partially solve the above technical problems.

[0007] In order to achieve the above purpose, according to the first aspect of the present application, a printing and dyeing defect detection method is provided, comprising:

[0008] An image collection of the surface of the printing and dyeing object is performed by a multi-angle imaging device arranged on the production line to obtain a raw image set containing multiple views;

[0009] An alignment processing is performed on the raw image set to obtain a registration image set in a unified coordinate system;

[0010] A three-dimensional point cloud model is generated according to the registration image set and the geometric structure feature of the surface of the printing and dyeing object is obtained;

[0011] The color information in the registration image set is projected to the three-dimensional point cloud model to obtain a three-dimensional color distribution model;

[0012] The pixel points in the three-dimensional color distribution model are grouped to obtain the boundary feature of the color difference area;

[0013] According to the boundary feature of the color difference region, a coordinate range of the color difference region in the three-dimensional color distribution model is analyzed, and a spatial position coordinate of a defect is determined.

[0014] Specifically, the original image set is aligned to obtain a registered image set in a unified coordinate system, including:

[0015] Key feature points are extracted from the original image set, and feature descriptors corresponding to the key feature points are generated by using a scale-invariant feature transform algorithm;

[0016] An initial matching point pair is obtained by matching the feature descriptors, and if the number of the initial matching point pair exceeds a preset matching number threshold, an abnormal matching point in the initial matching point pair is removed by using a random sample consensus algorithm to obtain a reliable matching point pair.

[0017] An affine transformation matrix is calculated according to the reliable matching point pair, and a coordinate transformation parameter is extracted therefrom;

[0018] The original image set is geometrically corrected based on the coordinate transformation parameter to obtain a corrected image set;

[0019] A scale difference of the corrected image set is detected, and if the scale difference exceeds a preset scale threshold, a scale adjustment operation is performed on the corrected image set to obtain a scale consistent image set;

[0020] A rotation angle deviation of the scale consistent image set is detected, and if the rotation angle deviation is greater than a preset angle threshold, a rotation matrix is applied to the scale consistent image set for rotation correction to obtain a rotation corrected image set;

[0021] A translation vector of the rotation corrected image set and a preset reference coordinate system is calculated to obtain a translation deviation, and if the translation deviation is greater than a preset translation threshold, the rotation corrected image set is translated by fusing the translation vector to obtain a registered image set in a unified coordinate system.

[0022] Specifically, a three-dimensional point cloud model is generated according to the registered image set, and a geometric structure feature of the printing surface is obtained, including:

[0023] The registered image set is obtained, corresponding pixel point matching is performed on images of different viewing angles in the registered image set, and an initial three-dimensional point cloud model containing spatial coordinate information of the printing surface is generated;

[0024] The initial three-dimensional point cloud model is subjected to point cloud registration by using an iterative closest point algorithm to obtain an aligned three-dimensional point cloud model;

[0025] Calculate a local curvature value of each point cloud point based on the aligned three-dimensional point cloud model, and generate a curvature distribution map of the printing and dyeing surface;

[0026] If the local curvature value of a point cloud point in the curvature distribution map exceeds a preset curvature threshold, mark a region composed of the point and its adjacent points as a convex region, and obtain a convex region set;

[0027] Extract a boundary point of each convex region from the convex region set, and perform quadratic curve fitting on the boundary point to obtain boundary curve parameters;

[0028] Calculate the shortest distance between two adjacent convex regions according to the boundary curve parameters, and generate a distance matrix; if the shortest distance between any two adjacent convex regions in the distance matrix is less than a preset distance threshold, determine that the two convex regions are the same geometric structure, and merge the two convex regions to obtain merged convex region information;

[0029] Obtain the geometric structure features of the printing and dyeing surface based on the merged convex region information, the spatial coordinate data of the aligned three-dimensional point cloud model, and the local curvature information of the curvature distribution map.

[0030] Specifically, obtain the registered image set, perform corresponding pixel point matching on images with different viewing angles in the registered image set, and generate an initial three-dimensional point cloud model containing printing and dyeing surface spatial coordinate information, including:

[0031] Obtain the registered image set; select one image from the registered image set as a reference image, and select the remaining images in the registered image set as matching images;

[0032] Perform dense feature point recognition on the reference image to generate a dense feature point coordinate set containing the spatial coordinates of all dense feature points;

[0033] Generate a corresponding feature vector for each dense feature point in the dense feature point coordinate set; integrate all feature vectors to establish a reference image feature library;

[0034] Perform sliding window matching between each matching image and the reference image: based on the feature vectors of the pixel points in the matching image and the feature vectors of the dense feature points in the reference image feature library, calculate the Euclidean distance between the two to determine the similarity; filter out the point pairs with a similarity higher than a preset matching threshold as candidate matching point pairs;

[0035] According to the candidate matching point pairs between each matching image and the reference image, calculate the corresponding disparity map between the two;

[0036] Based on the internal parameter matrix, the external parameter matrix and the disparity map obtained by the multi-angle imaging device, the coordinates of each pair of candidate matching points in three-dimensional space are calculated respectively;

[0037] The three-dimensional space coordinates of all candidate matching points are integrated to form an initial point cloud data set containing preliminary space coordinate information of the printing surface.

[0038] Specifically, the color information in the registration image set is projected onto the three-dimensional point cloud model to obtain a three-dimensional color distribution model, including:

[0039] The RGB color data of each image is extracted from the registration image set to construct an initial color data set containing the correspondence between pixel coordinates and RGB color values;

[0040] The initial color data set is matched one by one with the point cloud points of the three-dimensional point cloud model, and the point cloud vacancy points that are not matched to the color data are identified;

[0041] The point cloud vacancy points are labeled with a null identifier;

[0042] The color points that are not labeled with the null identifier are assigned with corresponding RGB color values to obtain a point cloud color assignment result containing real color information;

[0043] The point cloud color assignment result is fused with the geometric structure features of the three-dimensional point cloud model to generate a three-dimensional color distribution model.

[0044] Specifically, the pixel points in the three-dimensional color distribution model are grouped to obtain the boundary features of the color difference region, including:

[0045] The spatial coordinate data and RGB color data of all pixel points in the three-dimensional color distribution model are obtained, and the valid color points that are not labeled with the null identifier are screened out and the point cloud vacancy points with the null identifier are removed based on the null identifier;

[0046] Based on the spatial coordinates and RGB color values, a two-dimensional point set matrix is constructed for the screened valid color points: each valid color point contains spatial coordinate parameters and RGB color parameters, the spatial coordinate parameters and RGB color parameters of each valid color point are taken as a row vector, all row vectors are arranged in order to form a two-dimensional point set matrix, and all row vectors included in the two-dimensional point set matrix collectively constitute a valid pixel point set;

[0047] The valid pixel point set is grouped to obtain a grouping cluster set; wherein the spatial Euclidean distance and the RGB color difference value between two points are calculated, the spatial Euclidean distance and the RGB color difference value are weighted summed according to a preset weight as a clustering measurement value, and the pixel points with a clustering measurement value less than a preset initial threshold value are divided into the same cluster to obtain a grouping cluster set.

[0048] calculating an average inter-cluster color difference value between the cluster sets; if the average inter-cluster color difference value exceeds a preset color difference threshold, marking a corresponding cluster class as a color difference region and obtaining a color difference region set;

[0049] extracting a pair of adjacent pixel points of an edge of each color difference region from the color difference region set, performing edge recognition on the pair of adjacent pixel points, and obtaining a continuous boundary point sequence;

[0050] calculating a curvature change value of each point in the boundary point sequence to determine a boundary type to obtain a boundary type set; if the curvature change value exceeds a preset curvature change range, determining that the boundary is a sharp boundary; if the curvature change value does not exceed the preset curvature change range, determining that the boundary is a smooth boundary;

[0051] fusing the boundary type set and spatial distribution information of the cluster set, performing dimension reduction processing on the fused data by using a principal component analysis algorithm to obtain N groups of feature vectors after dimension reduction;

[0052] performing cosine similarity matching on the N groups of feature vectors after dimension reduction and a preset boundary template to obtain a color difference region morphology corresponding to the boundary and obtain a boundary feature of the color difference region.

[0053] Specifically, according to the boundary feature of the color difference region, analyzing a coordinate range of the color difference region in the three-dimensional color distribution model to determine a spatial position coordinate of the defect, including:

[0054] obtaining the boundary feature of the color difference region and extracting the boundary point sequence and the boundary type set;

[0055] respectively counting a maximum value and a minimum value of a coordinate of the boundary point sequence on an X axis, a Y axis and a Z axis to obtain a preliminary coordinate range of the color difference region;

[0056] taking a center point coordinate of the preliminary coordinate range as a defect center coordinate and combining the preliminary coordinate range to obtain the spatial position coordinate of the defect.

[0057] Specifically, the method further includes:

[0058] obtaining a three-dimensional point cloud coordinate corresponding to each boundary point in the boundary point sequence of the color difference region;

[0059] querying a local curvature value and convex region information of a corresponding coordinate in the geometric structure feature;

[0060] if a boundary point is located in a convex region and a local curvature value of the boundary point is greater than a preset structure sensitivity threshold, marking the boundary point as a structure sensitive boundary point;

[0061] statistically obtaining a proportion of structure-sensitive boundary points in the color difference region; if the proportion exceeds a preset proportion threshold, performing geometric deformation compensation on spatial position coordinates of the color difference region; projecting a preliminary coordinate range of the original color difference region to a reference plane without deformation by a mapping function based on boundary curve parameters of the convex region, to obtain a compensated color difference region coordinate range; wherein the mapping function is constructed based on the boundary curve parameters of the convex region;

[0062] taking a center point of the compensated color difference region coordinate range as a corrected defect center coordinate, and obtaining a final spatial position coordinate of the defect in combination with the compensated coordinate range.

[0063] Specifically, before the cluster is marked as a color difference region, the method further comprises:

[0064] obtaining a design template image corresponding to the printed product, and extracting a theoretical color distribution map and a texture structure map of the design template image;

[0065] projecting the three-dimensional color distribution model to a two-dimensional plane and performing spatial alignment with the design template image;

[0066] for each candidate color difference cluster, querying an expected color value and an allowable color difference range of a corresponding region in the design template image;

[0067] if a color difference between the actual color value and the expected color value of the cluster is less than the allowable color difference range, determining that the cluster is a design-allowed color change, and not marking it as a color difference region.

[0068] According to a second aspect of the present application, a printed defect detection system is provided, comprising:

[0069] a first processing module configured to: acquire images of a printed product surface by a multi-angle imaging device arranged on a production line, to obtain an original image set containing multiple views;

[0070] a second processing module configured to: perform alignment processing on the original image set to obtain a registered image set in a unified coordinate system;

[0071] a third processing module configured to: generate a three-dimensional point cloud model and obtain geometric structure features of the printed product surface according to the registered image set;

[0072] a fourth processing module configured to: project color information in the registered image set to the three-dimensional point cloud model to obtain a three-dimensional color distribution model;

[0073] a fifth processing module configured to: group pixel points in the three-dimensional color distribution model to obtain boundary features of a color difference region;

[0074] The sixth processing module is configured to: according to the boundary feature of the color difference region, analyze a coordinate range of the color difference region in the three-dimensional color distribution model, and determine a spatial position coordinate of the defect.

[0075] To sum up, the embodiment of the present application avoids color information projection distortion caused by geometric undulations and improves the recognition accuracy of the color difference region.

[0076] Other features and advantages of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0078] Figure 1 is a step flow chart of a printing and dyeing defect detection method provided in an exemplary embodiment of the present application;

[0079] Figure 2 is a system schematic diagram of a printing and dyeing defect detection system provided in an exemplary embodiment of the present application;

[0080] Label explanation: 201, first processing module; 202, second processing module; 203, third processing module; 204, fourth processing module; 205, fifth processing module; 206, sixth processing module. DETAILED DESCRIPTION

[0081] The technical solutions in the embodiments of the present application will be described clearly and completely in the following by combining the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0082] The present application provides a printing and dyeing defect detection method, please refer to Figure 1 The printing and dyeing defect detection method provided in the embodiment of the present application comprises the following steps:

[0083] Step 101, image acquisition of the surface of the printing and dyeing object is performed by a multi-angle imaging device arranged on the production line to obtain a raw image set containing multiple views.

[0084] Step 102, alignment processing is performed on the raw image set to obtain a registration image set under a unified coordinate system.

[0085] Step 103, generating a three-dimensional point cloud model according to the registered image set and obtaining geometric structure features of the printing surface.

[0086] Step 104, projecting color information in the registered image set to the three-dimensional point cloud model to obtain a three-dimensional color distribution model.

[0087] Step 105, grouping pixel points in the three-dimensional color distribution model to obtain boundary features of the color difference region.

[0088] Step 106, according to the boundary features of the color difference region, analyzing the coordinate range of the color difference region in the three-dimensional color distribution model, and determining the spatial position coordinates of the defect.

[0089] Through the above scheme, the full-range original image of the printing surface is collected through multi-view imaging, which makes up for the lack of single-view information; the original image is aligned to eliminate the perspective offset and scale difference; the three-dimensional point cloud model is generated based on the registered image and the geometric structure features are extracted; the three-dimensional color distribution model is formed by fusing color information and three-dimensional point cloud model; the color difference region and its boundary features are identified through pixel grouping and boundary extraction, which improves the color difference region recognition accuracy; the three-dimensional spatial coordinates of the defect are determined based on the boundary features, which solves the limitation of two-dimensional detection that can only output plane coordinates, and effectively improves the color difference recognition accuracy.

[0090] In some embodiments, the original image set is aligned to obtain a registered image set in a unified coordinate system, including:

[0091] Extracting key feature points from the original image set; using a scale-invariant feature transform algorithm to generate feature descriptors corresponding to the key feature points;

[0092] Matching the feature descriptors to obtain an initial matching point pair; if the number of the initial matching point pair exceeds a preset matching number threshold, using a random sample consensus algorithm to remove abnormal matching points in the initial matching point pair to obtain a reliable matching point pair;

[0093] Calculating an affine transformation matrix according to the reliable matching point pair and extracting coordinate transformation parameters therefrom;

[0094] Performing geometric correction on the original image set based on the coordinate transformation parameters to obtain a corrected image set;

[0095] Detecting the scale difference of the corrected image set, if the scale difference exceeds a preset scale threshold, performing a scale adjustment operation on the corrected image set to obtain a scale consistent image set;

[0096] detecting a rotation angle deviation of the scale-consistent image set, and if the rotation angle deviation is greater than a preset angle threshold, applying a rotation matrix to rotate and correct the scale-consistent image set to obtain a rotation-corrected image set;

[0097] calculating a translation vector of the rotation-corrected image set and a preset reference coordinate system to obtain a translation deviation, and if the translation deviation is greater than a preset translation threshold, fusing the translation vector to perform translation correction on the rotation-corrected image set to obtain a registration image set in a unified coordinate system.

[0098] Specifically, from the original image set, key feature points capable of stably representing image structures are extracted frame by frame. The key feature points refer to pixel points with sharp gray changes in the image, such as intersection points of printing surface textures and edge end points, reflecting local structural information of the image. A scale-invariant feature transform (SIFT) algorithm is used to generate feature descriptors corresponding to each key feature point. The SIFT algorithm is an algorithm that can maintain feature stability under image scale scaling, rotation and illumination changes. By constructing a scale space, detecting extreme points and calculating direction gradients, a 128-dimensional high-dimensional feature vector can be generated, which can uniquely represent the structural properties of the key feature points.

[0099] By calculating the Euclidean distance of the key feature point descriptors between different view images, the smaller the Euclidean distance, the higher the structural similarity of the two feature points. Feature point pairs with a distance less than a preset similarity threshold are selected as initial matching point pairs. If the number of the initial matching point pairs exceeds a preset matching number threshold, a random sample consensus algorithm is used to remove abnormal matching points in the initial matching point pairs, that is, mis-matching points deviating from the overall image transformation rule.

[0100] According to the reliable matching point pairs, an affine transformation matrix capable of describing a linear transformation relationship between images is solved by using a least square method; the affine transformation matrix is a 2*3 matrix, contains three transformation parameters of translation, rotation and scaling, and can completely represent a geometric transformation rule of mapping one image to another image; a translation amount, a rotation angle and a scaling coefficient are extracted from the solved affine transformation matrix as subsequent coordinate transformation parameters. Based on the coordinate transformation parameters, geometric correction is performed on each image in the original image set; the geometric correction is performed by adjusting spatial coordinates of image pixels to eliminate image deformation caused by lens distortion and view angle deviation. Scale differences of the images in the corrected image set are detected; if the scale differences exceed a preset scale threshold, a bilinear interpolation algorithm is used to perform a scale adjustment operation on the corrected image set to achieve smooth scaling of image sizes and avoid image blurring caused by scale adjustment, and finally a scale consistent image set is obtained. The angles of the edges of the printed matter in the scale consistent image set are detected to determine rotation angle deviations of the images from a preset reference direction; if the rotation angle deviations are greater than a preset angle threshold, a rotation matrix is applied to perform rotation correction on the scale consistent image set; the rotation matrix is a 2*2 trigonometric function matrix, new coordinates of rotated pixels are calculated by multiplying pixel coordinates and sine and cosine values of the rotation angle, and the images can be rotated around a preset reference point to be parallel to the reference direction to obtain a rotation corrected image set.

[0101] One of the images is taken as a reference, preferably an image collected from the front, which has the smallest geometric deviation and the most similar view angle to the normal projection of the printed matter surface, and a preset reference coordinate system is established; an X axis of the preset reference coordinate system is parallel to a conveying direction of a production line, a Y axis is perpendicular to the conveying direction, and an origin is set as a lower left corner of the image; translation vectors of each image in the rotation corrected image set from the preset reference coordinate system are calculated to obtain translation deviations; if the translation deviations are greater than a preset translation threshold, the rotation corrected image set is subjected to translation correction by fusing the translation vectors, and finally a registration image set under a unified coordinate system is obtained.

[0102] In order to eliminate the problems of view angle deviation, scale inconsistency, rotation inclination and the like caused by the difference in collection angles and the installation deviation of equipment, the original image set is subjected to alignment processing to unify all the images to the same preset reference coordinate system, and finally a registration image set is obtained.

[0103] In some embodiments, a three-dimensional point cloud model is generated according to the registration image set and geometric structure features of the printed matter surface are obtained, including:

[0104] The registration image set is acquired, corresponding pixel point matching is performed on the images with different view angles in the registration image set, and an initial three-dimensional point cloud model containing spatial coordinate information of the printed matter surface is generated;

[0105] aligning the initial three-dimensional point cloud model to obtain an aligned three-dimensional point cloud model by using an iterative closest point algorithm;

[0106] calculating a local curvature value of each point cloud point based on the aligned three-dimensional point cloud model to generate a curvature distribution map of the printing and dyeing surface;

[0107] If the local curvature value of a point cloud point in the curvature distribution map exceeds a preset curvature threshold, marking a region composed of the point cloud point and adjacent point cloud points as a convex region to obtain a convex region set.

[0108] extracting a boundary point of each convex region from the convex region set, and performing quadratic curve fitting on the boundary point to obtain boundary curve parameters.

[0109] calculating a shortest distance between two adjacent convex regions according to the boundary curve parameters to generate a distance matrix; if the shortest distance between any two adjacent convex regions in the distance matrix is less than a preset distance threshold, determining that the two convex regions are the same geometric structure, and merging the two convex regions to obtain merged convex region information.

[0110] obtaining geometric structure features of the printing and dyeing surface based on the merged convex region information, spatial coordinate data of the aligned three-dimensional point cloud model, and local curvature information of the curvature distribution map.

[0111] Specifically, the registration image set is obtained, and corresponding pixel point matching is performed on images with different angles in the registration image set by using a multi-view stereo matching algorithm. The multi-view stereo matching algorithm is an algorithm for finding corresponding pixels of a same physical point in different images by calculating the similarity of pixels between different angle images, and then recovering three-dimensional coordinates by combining camera parameters. The internal parameters such as focal length, principal point coordinates, and distortion coefficients, and the external parameters such as camera spatial position and attitude angle are calibrated in advance by combining a multi-angle imaging device, and the three-dimensional spatial coordinates of each matching pixel point are calculated by triangulation principle, the three-dimensional coordinates of all pixel points are integrated, and an initial three-dimensional point cloud model containing spatial coordinate information of the printing and dyeing surface is generated.

[0112] The initial three-dimensional point cloud model is aligned by using an iterative closest point algorithm; the iterative closest point algorithm is a registration algorithm for finding the optimal transformation matrix between two point clouds by iteration to minimize the distance between the point clouds, and is commonly used to eliminate the positional deviation of multi-angle point clouds; the optimal transformation matrix between different angle point clouds is calculated by multiple iterations to eliminate the positional deviation of the initial point cloud caused by the difference in angle, and an aligned three-dimensional point cloud model is obtained.

[0113] Based on the aligned three-dimensional point cloud model, a local curvature value is calculated for each point cloud point; wherein the local curvature value is a physical quantity representing the bending degree of the region where the point cloud point is located, and the greater the curvature value, the more obvious the bending of the surface of the region; the local curvature value of all point cloud points is associated with its spatial coordinates (X, Y), and the color gradient is divided according to the curvature value to generate a curvature distribution map of the printing surface; the curvature distribution map is a two-dimensional image that displays the local curvature value of the point cloud in a color mapping manner. If the local curvature value of a certain point cloud point in the curvature distribution map exceeds a preset curvature threshold, the region composed of the point and its adjacent points is marked as a convex region; all point cloud points are traversed, and the above marking operation is repeated to obtain a convex region set containing multiple convex regions.

[0114] The boundary points of each convex region are extracted from the convex region set; wherein the boundary point refers to the point cloud point at the edge of the convex region, and the determination standard is that the neighborhood of the point contains both convex region points and non-convex region points; for the extracted boundary points, a quadratic curve fitting is performed using the least squares method to obtain boundary curve parameters.

[0115] According to the boundary curve parameters, the shortest distance between two adjacent convex regions is calculated; specifically, all points of the boundary curves of the two convex regions are extracted, the Euclidean distance between any two points is calculated, and the minimum value is taken as the shortest distance between the two regions; the shortest distances of all adjacent convex regions are sorted to generate a distance matrix; the above distance matrix is a two-dimensional matrix, and the rows and columns correspond to the convex region numbers, and the matrix elements are the shortest distances of the corresponding two regions. If the shortest distance of any two adjacent convex regions in the distance matrix is less than a preset distance threshold, it is determined that the two convex regions are the same geometric structure, such as the same convex split caused by the difference in point cloud density, and the point cloud and boundary curve parameters of the two convex regions are integrated to obtain the information of the merged convex region, which contains the spatial coordinate range, boundary curve parameters, and average curvature value of the merged region.

[0116] Based on the merged convex region information, the spatial coordinate data of the aligned three-dimensional point cloud model, and the local curvature information of the curvature distribution map, the geometric structure features of the printing surface are integrated; the geometric structure features specifically include: the spatial position of the convex region, the convex size, the edge morphology, and the wrinkle formed by the continuous distribution of the local low curvature region.

[0117] In this step, the initial point cloud is registered by using the iterative closest point algorithm, the position deviation of the multi-view point cloud is eliminated by iteratively optimizing the transformation matrix, and the stereo shape distortion problem caused by inaccurate point cloud alignment is solved; by distance matrix calculation and merging of similar regions, the same protrusion splitting problem caused by the difference in point cloud density is eliminated; by determining the position and shape of the protrusion and wrinkle, color distortion in these areas can be avoided during color projection, and the detection accuracy is improved.

[0118] In some embodiments, the registered image set is obtained, the corresponding pixel points of the images with different viewing angles in the registered image set are matched, and an initial three-dimensional point cloud model containing surface spatial coordinate information of the printing and dyeing object is generated, including:

[0119] The registered image set is obtained; one image in the registered image set is selected as a reference image, and the remaining images in the registered image set are all taken as matching images;

[0120] The dense feature points of the reference image are identified to generate a dense feature point coordinate set containing the spatial coordinates of all dense feature points;

[0121] A corresponding feature vector is generated for each dense feature point in the dense feature point coordinate set; and all feature vectors are integrated to establish a reference image feature library;

[0122] Each matching image and the reference image are matched by using a sliding window: based on the feature vectors of the pixel points in the matching image and the feature vectors of the dense feature points in the reference image feature library, the Euclidean distance between them is calculated to determine the similarity; the point pairs with a similarity higher than a preset matching threshold are selected as candidate matching point pairs;

[0123] According to the candidate matching point pairs between each matching image and the reference image, a corresponding disparity map is calculated;

[0124] Based on the intrinsic parameter matrix and the extrinsic parameter matrix obtained by pre-calibration of the multi-angle imaging device and the disparity map, the coordinates of each pair of candidate matching point pairs in the three-dimensional space are calculated;

[0125] The three-dimensional space coordinates of all candidate matching point pairs are integrated to form an initial point cloud data set containing preliminary surface spatial coordinate information of the printing and dyeing object.

[0126] Specifically, the set of registration images is acquired, an image with a view closest to the normal projection of the printing surface is selected from the set as a reference image, and the remaining images in the set of registration images are all taken as to-be-matched images. Dense feature point recognition is performed on the reference image, and a sub-pixel corner detection algorithm can be used to extract dense feature points in the image. The above-mentioned dense feature points refer to feature points with high density distribution in the image, which can cover the surface of the printing material, such as texture and edge details. All dense feature points meeting the conditions in the reference image are identified through the algorithm, the (u, v) coordinates of each feature point in the image coordinate system are recorded, and a dense feature point coordinate set containing the spatial coordinates of all dense feature points is generated. (Here, the spatial coordinates refer to two-dimensional coordinates in the image coordinate system, which will be converted to three-dimensional coordinates later.

[0127] A corresponding feature vector is generated for each dense feature point in the dense feature point coordinate set, which can be implemented by using a BRIEF descriptor. The above-mentioned BRIEF descriptor is a lightweight binary feature descriptor, which generates a fixed-length binary vector by randomly selecting multiple pairs of pixel points in the neighborhood of the feature point and comparing the pixel gray values. For each dense feature point, a 16x16 neighborhood window is constructed with the dense feature point as the center, 256 pairs of non-repeated pixel points are randomly selected, the gray values of each pair of pixel points are compared, and a 256-bit binary feature vector is generated. The correspondence between the spatial coordinates of all dense feature points and the feature vectors is integrated, and a reference image feature library is established.

[0128] Each to-be-matched image is matched with the reference image in a sliding window manner. The sliding window matching refers to a matching manner in which the image is traversed in units of a pre-set size window, and the feature vectors of the pixels in the window are compared with the feature vectors in the reference image feature library. The specific process can be as follows: for each to-be-matched image, a 5x5 window size and a 1-pixel step are used for sliding traversal, and the feature vector of the center pixel of each window is extracted. Based on the feature vector of the pixel in the to-be-matched image and the feature vectors of the dense feature points in the reference image feature library, the Euclidean distances of the two are calculated, and the point pairs with an Euclidean distance less than a pre-set matching threshold are selected as candidate matching point pairs.

[0129] According to the candidate matching point pairs between each to-be-matched image and the reference image, a disparity map corresponding to each to-be-matched image and the reference image is calculated. The above-mentioned disparity map is an image reflecting the difference in the projection positions of the same physical point in different view images.

[0130] Based on the intrinsic parameter matrix and the extrinsic parameter matrix obtained by pre-calibration of the multi-angle imaging device and the disparity map, the coordinates of each candidate matching point pair in three-dimensional space are calculated. The three-dimensional space coordinates corresponding to the candidate matching point pairs of all to-be-matched images and the reference image are integrated, and the coordinates calculated repeatedly are removed to form an initial point cloud data set containing preliminary spatial coordinate information of the surface of the printing material.

[0131] In some embodiments, projecting color information in the set of registered images to the three-dimensional point cloud model obtains a three-dimensional color distribution model, comprising:

[0132] extracting RGB color data of each image from the set of registered images to construct an initial color dataset containing a one-to-one correspondence between pixel coordinates and RGB color values;

[0133] one-to-one matching the initial color dataset with point cloud points of the three-dimensional point cloud model and identifying point cloud vacancy points that are not matched to color data;

[0134] labeling the point cloud vacancy points with a null identifier;

[0135] assigning corresponding RGB color values to color points that are not labeled with the null identifier to obtain a point cloud color assignment result containing true color information;

[0136] fusing the point cloud color assignment result with geometric structural features of the three-dimensional point cloud model to generate a three-dimensional color distribution model.

[0137] Specifically, RGB color data of each image is extracted from the set of registered images; RGB color data is a three-channel numerical value representing the color of an image pixel; for each registered image, the RGB color values of the corresponding pixels are extracted one by one according to the pixel coordinates (u, v) to construct an initial color dataset containing a one-to-one correspondence between pixel coordinates (u, v) and RGB color values (R, G, B). The initial color dataset is one-to-one matched with point cloud points of the three-dimensional point cloud model, each point cloud point containing a unique three-dimensional world coordinate (X, Y, Z); the matching process is based on a preset mapping relationship established by the intrinsic matrix and the extrinsic matrix of the multi-angle imaging device, which can convert the two-dimensional coordinates (u, v) of the image pixels into three-dimensional world coordinates (X, Y, Z); through coordinate mapping, the point cloud point corresponding to each pixel coordinate in the initial color dataset is found, the association between color data and point cloud points is realized, and point cloud vacancy points that are not matched to color data are identified; point cloud vacancy points are point cloud points in the three-dimensional point cloud model that cannot be matched to the color information of the initial color dataset due to reasons such as visual angle blind area and image acquisition missing, such points lack effective color information and need to be marked separately to avoid subsequent color misjudgment. The point cloud vacancy points are labeled with a null identifier; the above-mentioned null identifier is a marker symbol used to clearly distinguish point cloud vacancy points from effective color points, used to avoid misjudging vacancy points without color information as effective color regions, or color difference detection deviation caused by missing color information; after labeling, the points in the point cloud model are divided into two categories: vacancy points with a null identifier and effective color points without a null identifier.

[0138] Assign corresponding RGB color values to color point positions without null identifier; in the assignment process, find the initial color data corresponding to the color point position through coordinate mapping, and directly assign the RGB color value in the data to the corresponding point cloud position to obtain a point cloud color assignment result containing real color information.

[0139] Fuse the point cloud color assignment result with the geometric structure features of the three-dimensional point cloud model; the geometric structure features include convex region, wrinkle and other three-dimensional morphological information of the printing surface; in the fusion process, the RGB color value in the point cloud color assignment result is associated and stored with the geometric information of the corresponding point position in the three-dimensional point cloud model, and finally a three-dimensional color distribution model is generated.

[0140] The method establishes a mapping relationship based on camera calibration parameters to realize the matching of color data and point cloud positions; the null identifier is used to clearly mark the point cloud vacancy position, avoiding the interference of point positions without color information on detection; the color assignment result is deeply fused with the geometric structure features, so that the three-dimensional color distribution model contains spatial position, color attribute and geometric attribute.

[0141] In some embodiments, grouping the pixel points in the three-dimensional color distribution model to obtain boundary features of a color difference region includes:

[0142] Obtain spatial coordinate data and RGB color data of all pixel points in the three-dimensional color distribution model, filter out effective color point positions without null identifier based on the null identifier, and eliminate point cloud vacancy positions with null identifier;

[0143] Based on the spatial coordinates and RGB color values, a two-dimensional point set matrix is constructed for the filtered effective color point positions: each effective color point position contains spatial coordinate parameters and RGB color parameters, the spatial coordinate parameters and RGB color parameters of each effective color point position are taken as a row vector, all row vectors are arranged in order to form a two-dimensional point set matrix, and all row vectors included in the two-dimensional point set matrix collectively constitute an effective pixel point set;

[0144] Group the effective pixel point set to obtain a grouping cluster set; wherein, the spatial Euclidean distance and the RGB color difference value between two points are calculated, the spatial Euclidean distance and the RGB color difference value are weighted summed according to a preset weight as a clustering measurement value, the pixel points with a clustering measurement value less than a preset initial threshold value are divided into the same cluster class, and the grouping cluster set is obtained;

[0145] Calculate the average inter-cluster color difference value between the grouping cluster set; if the average inter-cluster color difference value exceeds a preset color difference threshold value, the corresponding cluster class is marked as a color difference region and a color difference region set is obtained;

[0146] Extracting a pair of adjacent pixel points of each color difference region edge from the color difference region set, performing edge recognition on the pair of adjacent pixel points, and obtaining a continuous boundary point sequence;

[0147] Calculating a curvature change value of each point in the boundary point sequence to determine a boundary type to obtain a boundary type set, and determining that the boundary is a sharp boundary if the curvature change value exceeds a preset curvature change range, and determining that the boundary is a smooth boundary if the curvature change value does not exceed the preset curvature change range;

[0148] Fusing the boundary type set and spatial distribution information of the grouping cluster set, performing dimension reduction processing on the fused data by using a principal component analysis algorithm to obtain N groups of feature vectors after dimension reduction;

[0149] Performing cosine similarity matching on the N groups of feature vectors after dimension reduction and a preset boundary template to obtain a color difference region morphology corresponding to the boundary and obtain a boundary feature of the color difference region.

[0150] Specifically, spatial coordinate data of all pixel points in the three-dimensional color distribution model is obtained; effective color points without a null value are screened out based on the null identifier, and point cloud missing points with the null identifier are removed. Based on the spatial coordinates and the RGB color values, a two-dimensional point set matrix is constructed for the screened effective color points: each effective color point includes 3 spatial coordinate parameters (X, Y, Z) and 3 RGB color parameters (R, G, B), a total of 6 feature parameters; the 6 parameters of each effective color point are combined in the order of X-Y-Z-R-G-B as a row vector, and all row vectors are arranged in the order of the X-axis coordinates of the point cloud points in the three-dimensional space to form a two-dimensional point set matrix; the two-dimensional point set matrix described above is a data matrix with a row number equal to the number of effective color points and a column number of 6, which can store the spatial position and color attribute of the point simultaneously.

[0151] The set of effective pixel points is grouped by using a clustering algorithm, and the specific process is as follows: the spatial Euclidean distance and the RGB color difference value between any two effective pixel points are calculated, the spatial Euclidean distance and the RGB color difference value are weighted and summed according to a preset weight, such as a spatial distance weight of 0.3 and a color difference weight of 0.7, to obtain a clustering measurement value; the clustering measurement value described above is a quantitative index that comprehensively reflects the spatial proximity and color similarity of two points, and a smaller value indicates that the two points are more likely to belong to the same dyeing region; the pixel points with a clustering measurement value less than the threshold value are divided into the same cluster, and the clustering center is optimized through iteration to finally obtain a grouping cluster set; the grouping cluster set is a set composed of multiple cluster classes, each cluster class includes effective pixel points with similar spatial positions and small color differences, and represents a continuous dyeing region on the surface of the printing and dyeing object.

[0152] Calculate the average inter-cluster color difference value between the group clusters, and if the average inter-cluster color difference value between two cluster classes exceeds the preset color difference threshold, it indicates that the two cluster classes represent the staining regions with significant color difference. The cluster class with color deviating from the normal staining range is marked as a color difference region. Repeat the above marking operation by traversing all group clusters to finally obtain a color difference region set.

[0153] Extract the adjacent pixel pairs of the edge of each color difference region from the color difference region set. The adjacent pixel pairs refer to the adjacent effective pixel combinations at the edge of the color difference region, one belonging to the color difference region and the other belonging to the normal staining region, which are used to locate the boundary of the color difference region. Use the Canny edge detection algorithm to perform edge recognition on the adjacent pixel pairs, filter out continuous edge pixels through edge recognition, and arrange them in order of spatial position to form a boundary point sequence. Calculate the curvature change value of each point in the boundary point sequence. The curvature change value can be calculated by selecting a point and its adjacent points before and after it, fitting the tangent lines of the three points, and calculating the angle difference between the two adjacent tangent lines, which is used to represent the bending degree of the boundary point. If the curvature change value of a boundary point exceeds the range, it is determined that the boundary is a sharp boundary. Sharp boundary refers to a boundary type with sharp bending and large curvature change, commonly found in the edges of spot-shaped and block-shaped color difference regions. If the curvature change value does not exceed the preset range, it is determined to be a smooth boundary. Smooth boundary refers to a boundary type with gentle bending and small curvature change, commonly found in the edges of strip-shaped and band-shaped color difference regions. Traverse all points in the boundary point sequence, record the boundary type corresponding to each point, and form a boundary type set.

[0154] Fuse the boundary type set with the spatial distribution information of the group clusters to form a fused data set, and the data dimension after fusion is boundary type, spatial range, and pixel density. Use principal component analysis algorithm to reduce the dimension of the fused data, retain the principal component with the largest variance in the data, and eliminate redundant information. The dimension reduction algorithm reduces the subsequent matching calculation while retaining the core features. Extract the principal components of the fused data through the PCA algorithm to generate N groups of feature vectors after dimension reduction, where N is equal to the number of color difference regions in the color difference region set, and each feature vector group corresponds to the core features of a color difference region.

[0155] Match the N groups of feature vectors after dimension reduction with the preset boundary templates using cosine similarity. The closer the cosine value is to 1, the higher the vector similarity. If the cosine similarity between a feature vector group and a preset boundary template is higher than the preset matching threshold, it is determined that the shape of the color difference region corresponding to the feature vector group is consistent with the template. Integrate the boundary type, shape, and spatial range of each color difference region to finally obtain the boundary features of the color difference region.

[0156] In some embodiments, according to the boundary feature of the color difference region, the coordinate range of the color difference region in the three-dimensional color distribution model is analyzed to determine the spatial position coordinates of the defect, including:

[0157] The boundary feature of the color difference region is obtained, and a boundary point sequence and a boundary type set are extracted;

[0158] The maximum and minimum values of the coordinate of the boundary point sequence on the X-axis, Y-axis and Z-axis are respectively counted to obtain the preliminary coordinate range of the color difference region;

[0159] The center point coordinates of the preliminary coordinate range are taken as the defect center coordinates, and the spatial position coordinates of the defect are obtained in combination with the preliminary coordinate range.

[0160] In some embodiments, the method further includes:

[0161] The three-dimensional point cloud coordinates corresponding to each boundary point in the boundary point sequence of the color difference region are obtained;

[0162] The local curvature value and the convex region information of the corresponding coordinates in the geometric structure feature are queried;

[0163] If a boundary point is located in a convex region and its local curvature value is greater than a preset structure sensitivity threshold, the boundary point is marked as a structure-sensitive boundary point;

[0164] The proportion of the structure-sensitive boundary points in the color difference region is counted; if the proportion exceeds a preset proportion threshold, the spatial position coordinates of the color difference region are compensated for geometric deformation: based on the boundary curve parameters of the convex region, the preliminary coordinate range of the original color difference region is projected to a reference plane without deformation through a mapping function to obtain the compensated color difference region coordinate range; wherein the mapping function is constructed based on the boundary curve parameters of the convex region;

[0165] The center point of the compensated color difference region coordinate range is taken as the corrected defect center coordinates, and the final spatial position coordinates of the defect are obtained in combination with the compensated coordinate range.

[0166] Specifically, the traditional method usually only identifies defects based on color difference, ignoring the influence of the surface three-dimensional structure on the imaging color, for example, the change of light reflection at the convex position may be misjudged as color difference. This step analyzes the boundary feature and the geometric structure feature in combination to determine whether the color difference is caused by real uneven dyeing or visual illusion caused by physical deformation, and compensates for the spatial coordinates accordingly, thereby improving the detection accuracy.

[0167] In some embodiments, before the cluster is labeled as a color difference region, the method further includes:

[0168] An image of a design template corresponding to the current printing and dyeing object is acquired, and a theoretical color distribution map and a texture structure map of the image of the design template are extracted;

[0169] The three-dimensional color distribution model is projected to a two-dimensional plane and spatially aligned with the image of the design template;

[0170] For each candidate color difference cluster, the expected color value and the allowable color difference range of the corresponding region in the image of the design template are queried;

[0171] If the color difference between the actual color value and the expected color value of the cluster is less than the allowable color difference range, it is determined that the cluster is a design-allowed color change, and is not marked as a color difference region.

[0172] Specifically, in actual printing and dyeing production, the cloth may have patterns, jacquard, gradient, and these designs themselves contain color changes, which are easy to be misjudged as color difference defects; through the above steps, design-allowed color changes can be effectively screened, and such non-defect color differences are avoided from being mislabeled as color difference regions.

[0173] Reference Figure 2 The second embodiment of the present application provides a printing and dyeing defect detection system, comprising:

[0174] The first processing module 201 is configured to acquire images of the surface of the printing and dyeing object through a multi-angle imaging device arranged on a production line to obtain a set of original images containing multiple views;

[0175] The second processing module 202 is configured to perform alignment processing on the set of original images to obtain a set of registration images in a unified coordinate system;

[0176] The third processing module 203 is configured to generate a three-dimensional point cloud model according to the set of registration images and obtain geometric structure features of the surface of the printing and dyeing object;

[0177] The fourth processing module 204 is configured to project color information in the set of registration images to the three-dimensional point cloud model to obtain a three-dimensional color distribution model;

[0178] The fifth processing module 205 is configured to group pixel points in the three-dimensional color distribution model to obtain boundary features of a color difference region;

[0179] The sixth processing module 206 is configured to analyze the coordinate range of the color difference region in the three-dimensional color distribution model according to the boundary features of the color difference region, and determine the spatial position coordinates of the defect.

[0180] It should be noted that the printing and dyeing defect detection system provided by the embodiment of the present application is used to execute all process steps of the printing and dyeing defect detection method of the above-mentioned embodiment, and the working principles and beneficial effects of the two are one-to-one correspondence, thus not repeated.

[0181] In the description of the present application, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0182] In the above-mentioned embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0183] The embodiments, implementation manners and related technical features of the present application can be combined and replaced with each other without conflict.

[0184] The above is only the preferred embodiment of the present application, and does not limit the present application in any form, but any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present application without departing from the technical solution content of the present application, still belongs to the scope of the technical solution of the present application.

Claims

1. A method for detecting defects in printing and dyeing, characterized in that, include: Images of the printed and dyed materials are acquired by multi-angle imaging devices deployed on the production line, resulting in a set of original images containing multiple perspectives. The original image set is aligned to obtain a registered image set in a unified coordinate system; A three-dimensional point cloud model is generated based on the registered image set, and the geometric structural features of the printed and dyed material surface are obtained; The color information in the registered image set is projected onto the three-dimensional point cloud model to obtain a three-dimensional color distribution model; The boundary features of the color difference region are obtained by grouping the pixels in the three-dimensional color distribution model. Based on the boundary characteristics of the color difference region, analyze the coordinate range of the color difference region in the three-dimensional color distribution model to determine the spatial location coordinates of the defect; Projecting the color information from the registered image set onto the 3D point cloud model to obtain a 3D color distribution model includes: Extract the RGB color data of each image from the registered image set to construct an initial color dataset containing the correspondence between pixel coordinates and RGB color values; The initial color dataset is matched one by one with the point cloud points of the 3D point cloud model, and the empty point cloud points that do not match the color data are identified. The point cloud empty space bits are labeled with empty value identifiers; Assign corresponding RGB color values ​​to color points that are not labeled with null value identifiers to obtain point cloud color assignment results containing real color information; The point cloud color assignment results are fused with the geometric structural features of the 3D point cloud model to generate a 3D color distribution model; The boundary features of color difference regions are obtained by grouping the pixels in the three-dimensional color distribution model, including: Obtain the spatial coordinate data and RGB color data of all pixels in the three-dimensional color distribution model, filter out the valid color points without null values ​​based on the null value identifier, and remove the point cloud null points with null value identifiers. For the filtered valid color points, a two-dimensional point set matrix is ​​constructed based on spatial coordinates and RGB color values: each valid color point contains spatial coordinate parameters and RGB color parameters. The spatial coordinate parameters and RGB color parameters of each valid color point are used as a row vector. All row vectors are arranged in order to form a two-dimensional point set matrix. All row vectors contained in the two-dimensional point set matrix together constitute the set of valid pixels. The effective pixel set is grouped to obtain a grouped cluster set; wherein, the spatial Euclidean distance and RGB color difference between two points are calculated, and the spatial Euclidean distance and RGB color difference are weighted and summed according to a preset weight as a clustering metric. Pixels with a clustering metric value less than a preset initial threshold are divided into the same cluster to obtain the grouped cluster set. Calculate the average inter-cluster color difference value among the grouped clusters; if the average inter-cluster color difference value exceeds a preset color difference threshold, mark the corresponding cluster as a color difference region and obtain a color difference region set; Extract adjacent pixel pairs from the edge of each color difference region from the set of color difference regions, perform edge recognition on the adjacent pixel pairs, and obtain a continuous sequence of boundary points; The curvature change value of each point in the boundary point sequence is calculated to determine the boundary type and obtain a boundary type set. If the curvature change value exceeds the preset curvature change range, the boundary is determined to be a sharp boundary; if the curvature change value does not exceed the preset curvature change range, the boundary is determined to be a smooth boundary. The spatial distribution information of the boundary type set and the grouped cluster set is fused, and the dimensionality reduction of the fused data is performed by the principal component analysis algorithm to obtain N sets of dimensionality-reduced feature vectors. The N sets of feature vectors after dimensionality reduction are matched with the preset boundary template by cosine similarity to obtain the shape of the color difference region corresponding to the boundary and the boundary features of the color difference region.

2. The method according to claim 1, characterized in that, The original image set is aligned to obtain a registered image set in a unified coordinate system, including: Key feature points are extracted from the original image set; feature descriptors corresponding to the key feature points are generated using a scale-invariant feature transformation algorithm. Initial matching point pairs are obtained by matching the feature descriptors; if the number of initial matching point pairs exceeds a preset matching number threshold, an abnormal matching point in the initial matching point pairs is removed by a random sampling consensus algorithm to obtain reliable matching point pairs. Calculate the affine transformation matrix based on the reliable matching point pairs and extract the coordinate transformation parameters from it; Based on the coordinate transformation parameters, the original image set is geometrically corrected to obtain a corrected image set; The scale difference of the corrected image set is detected. If the scale difference exceeds a preset scale threshold, a scale adjustment operation is performed on the corrected image set to obtain a scale-consistent image set. The rotation angle deviation of the scale-consistent image set is detected. If the rotation angle deviation is greater than a preset angle threshold, a rotation matrix is ​​applied to the scale-consistent image set to perform rotation correction, thereby obtaining a rotation-corrected image set. The translation vector between the rotation-corrected image set and the preset reference coordinate system is calculated to obtain the translation deviation; if the translation deviation is greater than the preset translation threshold, the translation vector is fused to perform translation correction on the rotation-corrected image set to obtain a registered image set under a unified coordinate system.

3. The method according to claim 2, characterized in that, A three-dimensional point cloud model is generated based on the registered image set, and the geometric structural features of the printed material surface are obtained, including: The registered image set is obtained, and the corresponding pixel points of the images from different perspectives in the registered image set are matched to generate an initial three-dimensional point cloud model containing the spatial coordinate information of the surface of the dyed material. The initial 3D point cloud model is registered using the iterative nearest point algorithm to obtain the aligned 3D point cloud model. Based on the aligned 3D point cloud model, the local curvature value of each point cloud location is calculated to generate a curvature distribution map of the printed material surface. If the local curvature value of a point cloud location in the curvature distribution map exceeds a preset curvature threshold, then the area formed by that point and its adjacent points is marked as a convex region, thus obtaining a set of convex regions. Extract the boundary points of each protruding region from the set of protruding regions, and perform quadratic curve fitting on the boundary points to obtain the boundary curve parameters; The shortest distance between two adjacent raised regions is calculated based on the boundary curve parameters to generate a distance matrix. If the shortest distance between any two adjacent raised regions in the distance matrix is ​​less than a preset distance threshold, the two raised regions are determined to be of the same geometric structure, and the two raised regions are merged to obtain information on the merged raised region. Based on the merged raised area information, the spatial coordinate data of the aligned three-dimensional point cloud model, and the local curvature information of the curvature distribution map, the geometric structural features of the printed and dyed material surface are obtained.

4. The method according to claim 3, characterized in that, Obtain the registered image set, perform corresponding pixel matching on images from different perspectives in the registered image set, and generate an initial 3D point cloud model containing the spatial coordinate information of the printed material surface, including: Obtain the registration image set; select one image from the registration image set as the reference image, and use the remaining images in the registration image set as images to be matched; Dense feature point identification is performed on the reference image to generate a dense feature point coordinate set containing the spatial coordinates of all dense feature points; For each dense feature point in the set of dense feature point coordinates, generate a corresponding feature vector; integrate all feature vectors to establish a benchmark image feature library; Each image to be matched is matched with a reference image using a sliding window: based on the feature vectors of pixels in the image to be matched and the feature vectors of dense feature points in the feature library of the reference image, the Euclidean distance between the two is calculated to determine the similarity; point pairs with similarity higher than a preset matching threshold are selected as candidate matching point pairs. Calculate the disparity map between each candidate matching point pair between the image to be matched and the reference image. Based on the pre-calibrated intrinsic and extrinsic matrix of the multi-angle imaging device and the disparity map, the coordinates of each candidate matching point pair in three-dimensional space are calculated respectively. Integrate the three-dimensional spatial coordinates of all candidate matching point pairs to form an initial point cloud dataset containing preliminary spatial coordinate information of the surface of the printed material.

5. The method according to claim 4, characterized in that, Based on the boundary characteristics of the color difference region, the coordinate range of the color difference region in the three-dimensional color distribution model is analyzed to determine the spatial location coordinates of the defect, including: Obtain the boundary features of the color difference region and extract the boundary point sequence and boundary type set; By statistically analyzing the maximum and minimum coordinates of the boundary point sequence on the X, Y, and Z axes, the preliminary coordinate range of the color difference region can be obtained. The spatial coordinates of the defect are obtained by taking the center point coordinates of the preliminary coordinate range as the defect center coordinates and combining them with the preliminary coordinate range.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the three-dimensional point cloud coordinates corresponding to each boundary point in the color difference region boundary point sequence; Query the local curvature values ​​and convex region information of the corresponding coordinates in the geometric structure features; If a boundary point is located in a convex region and its local curvature value is greater than a preset structural sensitivity threshold, then the boundary point is marked as a structurally sensitive boundary point. The proportion of structurally sensitive boundary points within the color difference region is statistically analyzed. If the proportion exceeds a preset threshold, geometric deformation compensation is performed on the spatial coordinates of the color difference region: based on the boundary curve parameters of the raised region, the initial coordinate range of the original color difference region is projected onto the undeformed reference plane through a mapping function to obtain the compensated coordinate range of the color difference region; wherein, the mapping function is constructed based on the boundary curve parameters of the raised region. The center point of the compensated color difference area coordinate range is used as the corrected defect center coordinate, and the final spatial position coordinate of the defect is obtained by combining the compensated coordinate range.

7. The method according to claim 6, characterized in that, Before labeling clusters as color difference regions, the method further includes: Obtain the design template image corresponding to the current printed material, and extract the theoretical color distribution map and texture structure map of the design template image; The three-dimensional color distribution model is projected onto a two-dimensional plane and spatially aligned with the design template image; For each candidate color difference cluster, query the expected color value and allowed color difference range of the corresponding region in the design template image; If the color difference between the actual color value and the expected color value of the cluster is less than the allowable color difference range, then the cluster is determined to be a color change allowed by the design and is not marked as a color difference area.

8. A dyeing and printing defect detection system, characterized in that, include: The first processing module is used to: acquire images of the surface of the printed and dyed material through a multi-angle imaging device arranged on the production line, and obtain a set of original images containing multiple perspectives; The second processing module is used to: perform alignment processing on the original image set to obtain a registered image set under a unified coordinate system; The third processing module is used to: generate a three-dimensional point cloud model based on the registered image set and obtain the geometric structural features of the surface of the printed material; The fourth processing module is used to: project the color information in the registered image set onto the three-dimensional point cloud model to obtain a three-dimensional color distribution model; The fifth processing module is used to: group the pixels in the three-dimensional color distribution model to obtain the boundary features of the color difference region; The sixth processing module is used to: analyze the coordinate range of the color difference region in the three-dimensional color distribution model based on the boundary characteristics of the color difference region, and determine the spatial location coordinates of the defect; Projecting the color information from the registered image set onto the 3D point cloud model to obtain a 3D color distribution model includes: Extract the RGB color data of each image from the registered image set to construct an initial color dataset containing the correspondence between pixel coordinates and RGB color values; The initial color dataset is matched one by one with the point cloud points of the 3D point cloud model, and the empty point cloud points that do not match the color data are identified. The point cloud empty space bits are labeled with empty value identifiers; Assign corresponding RGB color values ​​to color points that are not labeled with null value identifiers to obtain point cloud color assignment results containing real color information; The point cloud color assignment results are fused with the geometric structural features of the 3D point cloud model to generate a 3D color distribution model; The boundary features of color difference regions are obtained by grouping the pixels in the three-dimensional color distribution model, including: Obtain the spatial coordinate data and RGB color data of all pixels in the three-dimensional color distribution model, filter out the valid color points without null values ​​based on the null value identifier, and remove the point cloud null points with null value identifiers. For the filtered valid color points, a two-dimensional point set matrix is ​​constructed based on spatial coordinates and RGB color values: each valid color point contains spatial coordinate parameters and RGB color parameters. The spatial coordinate parameters and RGB color parameters of each valid color point are used as a row vector. All row vectors are arranged in order to form a two-dimensional point set matrix. All row vectors contained in the two-dimensional point set matrix together constitute the set of valid pixels. The effective pixel set is grouped to obtain a grouped cluster set; wherein, the spatial Euclidean distance and RGB color difference between two points are calculated, and the spatial Euclidean distance and RGB color difference are weighted and summed according to a preset weight as a clustering metric. Pixels with a clustering metric value less than a preset initial threshold are divided into the same cluster to obtain the grouped cluster set. Calculate the average inter-cluster color difference value among the grouped clusters; if the average inter-cluster color difference value exceeds a preset color difference threshold, mark the corresponding cluster as a color difference region and obtain a color difference region set; Extract adjacent pixel pairs from the edge of each color difference region from the set of color difference regions, perform edge recognition on the adjacent pixel pairs, and obtain a continuous sequence of boundary points; The curvature change value of each point in the boundary point sequence is calculated to determine the boundary type and obtain a boundary type set. If the curvature change value exceeds the preset curvature change range, the boundary is determined to be a sharp boundary; if the curvature change value does not exceed the preset curvature change range, the boundary is determined to be a smooth boundary. The spatial distribution information of the boundary type set and the grouped cluster set is fused, and the dimensionality reduction of the fused data is performed by the principal component analysis algorithm to obtain N sets of dimensionality-reduced feature vectors. The N sets of feature vectors after dimensionality reduction are matched with the preset boundary template by cosine similarity to obtain the shape of the color difference region corresponding to the boundary and the boundary features of the color difference region.

Images