Wood surface defect detection method and system based on image recognition

Abstract

The invention relates to the field of image recognition, and discloses a wood surface defect detection method and system based on image recognition, and the method comprises the following steps: collection and registration: synchronously obtaining the image data of the wood surface through a collection device, and carrying out the external parameter calibration of the collection device through a Zhang Zhengyou calibration method, establishing a conversion relationship between the world coordinate system and the coordinate system of each acquisition device, and registering the image data set by using thin-plate spline interpolation based on the conversion relationship to generate a time-space consistent registration data set; and processing modeling: performing multi-modal preprocessing on the registration data set to form a preprocessed data set including spectrum, polarization and depth information, and based on the preprocessed data set. Hyperspectral, polarization and depth image data of the wood surface are synchronously obtained, space-time alignment is achieved through Zhang Zhengyou calibration and thin plate spline interpolation, asynchronous errors of multiple sensors are eliminated, and reflection is removed through polarization to separate diffuse reflection components.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a wood surface defect detection method and system based on image recognition. BACKGROUND

[0002] Under the development trend of industry and intelligent manufacturing, the demand for automatic detection of surface defects in the wood processing industry is increasingly urgent. As the core means of industrial visual detection, image recognition technology needs to integrate hyperspectral, polarization, depth and other multi-modal data, break through the efficiency and accuracy bottleneck of traditional manual detection, and support the whole-process quality control of wood processing.

[0003] Traditional wood surface defect detection methods mostly use hand-designed feature extraction methods, such as gray scale statistics, edge detection or texture descriptor, and realize defect recognition by setting fixed thresholds or rules. However, since such methods only rely on single modal gray scale or texture information, in the light variation scene, the specular reflection on the wood surface will interfere with the gray scale feature extraction of the defect area, causing the defect features to be covered by the reflection, and thus causing a large fluctuation in detection accuracy. SUMMARY

[0004] To overcome the shortcomings of the prior art, the present application provides a wood surface defect detection method and system based on image recognition, which solves the problem of traditional wood surface defect detection methods that, in the light variation scene, the specular reflection on the wood surface will interfere with the gray scale feature extraction of the defect area, causing the defect features to be covered by the reflection, and thus causing a large fluctuation in detection accuracy.

[0005] To achieve the above purpose, the present application realizes the following technical scheme: a wood surface defect detection method based on image recognition, comprising the following steps: Acquisition and registration: synchronously acquiring image data of the wood surface through a collection device, calibrating the collection device using Zhang Zhengyou calibration method, establishing the conversion relationship between the world coordinate system and the coordinate system of each collection device, and using thin plate spline interpolation to register the image data set based on the conversion relationship to generate a spatio-temporally consistent registration data set; Processing and modeling: multi-modal preprocessing is performed on the registration data set to form a preprocessing data set including spectral, polarization and depth information, and based on the preprocessing data set, texture structure tensors are calculated, eigenvalue decomposition is performed, and a normalized metric tensor is defined to construct a Riemannian manifold model of the wood surface and extract differential geometric features of the wood surface; Tensor fusion: the preprocessing data set and the differential geometric features are dimensionally aligned, and cross-modal feature orthogonalization processing is performed through regularized Tucker decomposition to generate a fusion feature vector containing multi-domain associated features; Defect detection: a geodesic active contour model is constructed based on a Riemannian manifold model, the contour evolution is driven by a fusion feature vector, the defect edges on the wood surface are extracted, and the detection result is optimized through curvature weighted focus loss and Riemannian geometric constraint.

[0006] By adopting the technical scheme, the hyperspectral, polarization and depth image data of the wood surface are synchronously acquired, the space-time alignment is realized through Zhang Zhengyou calibration and thin plate spline interpolation, the multi-sensor asynchronous error is eliminated, the diffuse reflection component is separated from the polarization anti-reflection, the texture geometric features are extracted by combining structure tensor and Riemannian manifold modeling, the natural texture interference is inhibited, the orthogonal fusion of spectral-polarization-geometric features is realized by using regularized Tucker decomposition, the weak defect signal is enhanced, the defect edges are accurately extracted based on the Riemannian manifold constructed geodesic active contour model combined with curvature weighted focus loss and geometric constraint optimization, thereby the complementarity of multi-modal data and the geometric characteristics of wood texture are combined, the illumination robust defect feature extraction is realized, and the problem that the traditional wood surface defect detection method is interfered by the specular reflection of the wood surface in the illumination variation scene, the gray feature extraction of the defect area is affected, the defect features are covered by the reflection, and the detection accuracy fluctuates greatly is solved.

[0007] Preferably, the acquisition device comprises a hyperspectral camera, a polarization camera and a structured light 3D scanner, the image data comprises hyperspectral data, polarization images and depth maps, and the conversion relationship is P w=RPc+t , wherein P w is a three-dimensional point coordinate in a world coordinate system, P c is a three-dimensional point coordinate in an acquisition device coordinate system, is a rotation matrix, is a translation vector, and the calibration error is less than 0.3 pixels.

[0008] Preferably, the mathematical expression of the thin plate spline interpolation is , wherein x=(x,y) is a pixel coordinate before registration, x'=(x',y') is a pixel coordinate after registration, is an affine transformation matrix, c k is the kth control point coordinate, w k is an interpolation weight, and the base function is , and the registration accuracy is less than 0.1 pixel.

[0009] Preferably, the calculation method of the texture structure tensor is to calculate the diffuse reflection component image I d in the preprocessed data set, and , generate the second moment matrix , after smoothing by a Gaussian kernel g σ . where σ is the standard deviation of the Gaussian kernel, and * denotes the convolution operation; constructing a symmetric positive definite texture structure tensor performing eigenvalue decomposition on T(x) to obtain eigenvalues λ1≥λ2 and corresponding eigenvectors e1,e2, where λ1 and λ2 represent the anisotropy strength of the texture region, and e1 is the main direction vector of the texture.

[0010] Preferably, the normalized metric tensor is defined as where T ij (x) is the component of the texture structure tensor, and ε=10 -6 is a regularization term used to avoid zero denominator; constructing a Riemannian manifold model based on the metric tensor, and calculating the geodesic distance between two points on the manifold by the Fast Marching algorithm, with a time complexity of O(NIogN), N being the total number of image pixels.

[0011] Preferably, the optimization objective of the regularized Tucker decomposition is where is a four-dimensional data tensor, including the reduced hyperspectral data, polarization parameters and differential geometric features, where H is the height pixel number of the wood surface image, W is the width pixel number of the wood surface image, λ ’ is the effective band number of the hyperspectral data after dimensionality reduction by minimum noise separation transformation, p is the channel number of the polarization image, is the core tensor, , is the spatial dimension factor matrix, , is the modal dimension factor matrix, and α=0.1 is the regularization weight, I is the unit matrix, and the alternating least squares method is used for solving, with a convergence threshold of 10 -4 .

[0012] Preferably, the evolution equation of the geodesic active contour model is where φ is a level set function used to represent the implicit representation of the contour, is the curvature of the contour, representing the bending degree of the contour, is an edge indicator function, and Gσ is a Gaussian kernel with a standard deviation σ=1.5, used to suppress texture noise; the evolution equation is discretized by using the semi-Lagrangian method, with a time step ∆t=0.1.

[0013] Preferably, the curvature weighted focal loss is where N is the total number of samples, α t is a class balance weight calculated by the reciprocal of the frequency of occurrence of the defect class, and γ=2 is a focal parameter used to reduce the weight of easily classified samples, K iThe cross-sectional curvature corresponding to the i-th sample is calculated by the eigenvalue of the texture structure tensor ; The loss function of the Riemannian geometry constraint is , wherein N d is the number of pixels of the defect area, Ω d is the defect area, d g (x) is the geodesic distance on the Riemannian manifold, and d e (x) is the Euclidean space distance.

[0014] Preferably, the multi-modal preprocessing includes dimension reduction on the hyperspectral data in the registered data set by using the minimum noise separation transform, and the first λ' effective bands are reserved; the diffuse reflection component is separated based on the Mueller matrix model for the polarization image , wherein I u is the non-polarized light intensity, I p is the polarized light intensity, and k is the polarization reflection coefficient; the median filter is used for smoothing and denoising the depth map to remove random noise points; The differential geometric properties include the cross-sectional curvature and the geodesic curvature, the cross-sectional curvature is calculated by the eigenvalue of the texture structure tensor , and the geodesic curvature is calculated by the tangent derivative of the contour on the Riemannian manifold, which is used to describe the bending characteristics of the defect edge along the main direction of the texture.

[0015] The wood surface defect detection system based on image recognition is used for the wood surface defect detection method based on image recognition, and includes: The acquisition registration module is used for synchronously acquiring the image data of the wood surface by using the acquisition device, calibrating the external parameters of the acquisition device by using the Zhang Zhengyou calibration method, establishing the conversion relationship between the world coordinate system and the coordinate system of each acquisition device, and registering the image data set by using the thin plate spline interpolation based on the conversion relationship, to generate the registered data set consistent in time and space; The processing modeling module is used for multi-modal preprocessing of the registered data set, to form the preprocessing data set including spectral, polarization and depth information, and calculating the texture structure tensor based on the preprocessing data set, performing eigenvalue decomposition, defining the normalized metric tensor, constructing the Riemannian manifold model of the wood surface, and extracting the differential geometric features of the wood surface; The tensor fusion module is used for dimension alignment of the preprocessing data set and the differential geometric features, and cross-modal feature orthogonalization processing by regularized Tucker decomposition, to generate the fusion feature vector containing multi-domain associated features; The defect detection module is used for constructing the geodesic active contour model based on the Riemannian manifold model, driving the contour evolution by using the fusion feature vector, extracting the defect edge of the wood surface, and optimizing the detection result by the curvature weighted focus loss and the Riemannian geometry constraint.

[0016] The application provides a wood surface defect detection method and system based on image recognition. The application has the following advantages: 1. The application synchronously acquires hyperspectral, polarization and depth image data of the wood surface, realizes space-time alignment through Zhang Zhengyou calibration and thin plate spline interpolation, eliminates multi-sensor asynchronous errors, separates diffuse reflection components through polarization anti-reflection, extracts texture geometric features by combining structure tensor and Riemannian manifold modeling, suppresses natural texture interference, realizes orthogonalization fusion of spectral-polarization-geometric features through regularized Tucker decomposition, enhances weak defect signals, constructs a geodesic active contour model based on Riemannian manifold, and accurately extracts defect edges by combining curvature weighted focal loss and geometric constraint optimization, so that the complementarity of multi-modal data and the geometric characteristics of wood texture are combined, and illumination-robust defect feature extraction is realized.

[0017] 2. The application calculates texture structure tensor and performs eigenvalue decomposition to obtain texture anisotropy parameters, maps the texture to Riemannian manifold by constructing a normalized metric tensor, describes the texture directionality by using differential geometric features, quantitatively expresses the geometric difference between natural texture and defects, and reduces the false detection probability in the texture dense area.

[0018] 3. The application excavates the cross-modal correlation of spectral-polarization-geometric features through regularized Tucker decomposition, enhances the representation ability of weak defect signals, and drives the evolution of the geodesic active contour model based on Riemannian manifold measurement. Sub-pixel level edge positioning is realized through a semi-Lagrangian method, the defect contour is ensured to be orthogonal to the main direction of the texture, the feature dilution problem caused by traditional multi-modal simple splicing is improved, and the detection ability of small defects is obviously improved.

[0019] 4. The application drives the evolution of the contour based on Riemannian manifold measurement, and the contour is orthogonal to the main direction of the texture by focusing on the defect area through curvature weighted focal loss, so that the geodesic active contour model and the curvature weighted loss are optimized cooperatively, the problems of fuzzy and broken defect edges in the traditional method are improved, the accurate extraction of the defect edge is realized, and the integrity and accuracy of the detection result are improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The method flowchart of the wood surface defect detection method based on image recognition provided by the application is provided. Figure 2 The system architecture diagram of the wood surface defect detection system based on image recognition provided by the application is provided. DETAILED DESCRIPTION

[0021] The technical solutions of the present application will be described clearly and completely below in combination with the drawings of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the present application.

[0022] Please refer to the drawings of the present application Figure 1 The embodiment of the present application provides a wood surface defect detection method based on image recognition, comprising the following steps: Collecting and registering: synchronously acquiring image data of the wood surface by a collecting device, calibrating the external parameters of the collecting device by Zhang Zhengyou calibration method, establishing the conversion relationship between the world coordinate system and the coordinate system of each collecting device, and registering the image data set based on the conversion relationship by using thin plate spline interpolation to generate a registered data set consistent in time and space; the collecting device comprises a hyperspectral camera, a polarization camera and a structured light 3D scanner, the image data comprises hyperspectral data, polarization images and depth maps, and the conversion relationship is P w=RPc+t , wherein P w is a three-dimensional point coordinate in the world coordinate system, P c is a three-dimensional point coordinate in the coordinate system of the collecting device, is a rotation matrix, is a translation vector, and the calibration error is less than 0.3 pixels.

[0023] The mathematical expression of the thin plate spline interpolation is , wherein x=(x,y) is the pixel coordinate before registration, x'=(x',y') is the pixel coordinate after registration, is an affine transformation matrix, c k is the kth control point coordinate, w k is the interpolation weight, and the basis function is less than 0.1 pixel.

[0024] Specifically, the collecting and registering step realizes the time and space consistency through multi-dimensional collaborative calibration: first, synchronously control the hyperspectral camera, the polarization camera and the structured light 3D scanner to trigger the collection, so as to ensure that the hyperspectral data, the polarization images and the depth maps of the same area of the wood surface are synchronously acquired in the time dimension; then, the external parameter calibration is performed by using Zhang Zhengyou calibration method: a three-dimensional chessboard with 10x10 corner points is used to shoot multiple groups of chessboard images with different poses in the field of view of the collecting device, and the rotation matrix and the translation vector The conversion error between the world coordinate system and the coordinate system of each acquisition device is less than 0.3 pixels; based on the conversion relationship, when the image data set is registered, first, the feature points of the multi-modal image are extracted through the FAST corner point detection algorithm, and 20 evenly distributed feature points are selected as the control points c of the thin plate spline interpolation k , and then the registered pixel coordinates are calculated according to the formula , wherein the affine transformation matrix A is fitted by the least square method, and the interpolation weight w k is derived from the distance between the control points and the basis function , and the registration accuracy is less than 0.1 rotation matrix pixels; the space-time consistent data set generated after registration provides a spatial correspondence basis for the fusion analysis of spectral, polarization and depth information in subsequent multi-modal preprocessing, and guarantees the accuracy of texture structure tensor construction and Riemannian manifold modeling.

[0025] In this embodiment, for the wood surface data synchronously acquired by the hyperspectral camera, the polarization camera and the structured light 3D scanner, the three-dimensional point coordinates Pc in the coordinate system of each acquisition device are taken as the input, the rotation matrix R and the translation vector t obtained by the Zhang Zhengyou calibration method are used, and the three-dimensional point coordinates P w in the world coordinate system are calculated by the formula P c = RP w +t to realize the unified association of the multi-acquisition devices in the spatial dimension; on this basis, for the multi-modal image data set, the image feature points are extracted as the control points c of the thin plate spline interpolation k , the pixel coordinates x before registration are taken as the input, the affine transformation matrix A, the interpolation weight w k and the basis function fitted are combined, and the pixel coordinates x' after registration are calculated by the formula to complete the sub-pixel level alignment of the multi-modal images in the pixel dimension, so that the multi-source heterogeneous data is fused into the space-time consistent registration data set, thereby providing the basis data with consistent spatial correspondence relationship for the texture structure tensor construction and the Riemannian manifold feature extraction in subsequent processing modeling.

[0026] Processing modeling: the multi-modal preprocessing is performed on the registration data set to form a preprocessing data set including spectral, polarization and depth information, and based on the preprocessing data set, the texture structure tensor is calculated, the eigenvalue decomposition is performed, the normalized metric tensor is defined, the Riemannian manifold model of the wood surface is constructed, and the differential geometric features of the wood surface are extracted; the calculation method of the texture structure tensor is to calculate the spatial gradients and of the diffuse reflection component image Id in the preprocessing data set, and then generate the second moment matrix after smoothing by the Gaussian kernel g σ . , where σ is the standard deviation of the Gaussian kernel, * denotes the convolution operation; construct the symmetric positive definite texture structure tensor Eigenvalue decomposition of T(x) gives eigenvalues λ1≥ λ2 and corresponding eigenvectors e1, e2, where λ1and λ2characterize the anisotropy strength of the texture region, and e1is the texture principal direction vector.

[0027] The normalized metric tensor is defined as where Tij(x) is the component of the texture structure tensor, and ϵ = 10 -6 is the regularization term to avoid zero denominator; based on the metric tensor, a Riemannian manifold model is constructed, and the geodesic distance between two points on the manifold is calculated by the Fast Marching algorithm, with a time complexity of O(NIogN), where N is the total number of image pixels.

[0028] Multimodal preprocessing includes dimensionality reduction of hyperspectral data in the registered dataset using minimum noise fraction transformation, retaining the first λ ’ effective bands; for polarized images, the diffuse reflection component is separated based on the Mueller matrix model , where I u is the non-polarized light intensity, I p is the polarized light intensity, and k is the polarization reflection coefficient; for depth maps, median filtering is used for smoothing and denoising to remove random noise points. Differential geometric properties include cross-sectional curvature and geodesic curvature, cross-sectional curvature is calculated by the eigenvalues of the texture structure tensor , and geodesic curvature is calculated by the tangent derivative of the contour on the Riemannian manifold, which is used to describe the bending characteristics of the defect edge along the texture principal direction.

[0029] Specifically, the modeling step is processed to collect the registered spatiotemporal consistent dataset as input, and first, the registered dataset is preprocessed for multimodal: for hyperspectral data, dimensionality reduction is performed using minimum noise fraction transformation, retaining the first λ' effective bands to reduce data; for polarized images, based on the Mueller matrix model, the diffuse reflection component I is separated by the formula d , eliminating the interference of specular reflection; for depth maps, median filtering is used for smoothing and denoising to remove random noise points, and the preprocessed dataset containing spectral, polarized diffuse reflection, and depth information is formed. Subsequently, based on the diffuse reflection component image Id in the preprocessed dataset, the texture structure tensor is calculated: the spatial gradient is calculated by the Sobel operator and , and the Gaussian kernel g σ with a standard deviation of σ is used to smooth the gradient square and cross terms to generate the second moment matrix , , , and the symmetric positive definite texture structure tensor is constructed Eigenvalue decomposition of T(x) is performed to obtain eigenvalues λ1≥λ2 and corresponding eigenvectors e1, e2, λ1 and λ2 represent the texture anisotropy strength, and e1 is the texture principal direction vector. To construct the Riemannian manifold model, a normalized metric tensor is defined as , ϵ = 10 -6 is a regularization term. Based on the tensor, the geodesic distance between two points on the manifold is calculated by the Fast-Marching algorithm with a time complexity of O(NLogN), N is the total number of image pixels, and the manifold-based description of the texture geometric characteristics is realized. Finally, the differential geometric features are extracted: the cross-sectional curvature is calculated by the eigenvalues of the texture structure tensor, and the formula is The geodesic curvature is calculated by the tangent derivative of the profile on the Riemannian manifold, which is used to describe the bending characteristics of the defect edge along the texture principal direction, and provides geometric feature support for subsequent defect detection.

[0030] In this embodiment, the non-polarized light intensity I u , the polarized light intensity I p and the calibrated polarized reflection coefficient k of the pre-processed polarized image are taken as inputs, and the formula is used to calculate the output diffuse reflection component image I d , which eliminates the interference of specular reflection and retains the intrinsic information of the texture; then I d is taken as input, and the spatial gradient and of the Sobel operator are calculated to capture the edge changes of the texture; subsequently, the gradient I x and I y are taken as inputs, and the convolution operation is performed on the gradient square items I 2 x , I 2 y and the cross item I x I y by the Gaussian kernel g σ with a standard deviation of σ to output the second moment matrix , , , which smooths the noise and aggregates the local texture information; based on the above second moment matrix, the texture structure tensor is constructed, which describes the local direction and intensity distribution of the texture; after eigenvalue decomposition of T(x) to obtain eigenvalues λ1≥λ2, the tensor components T ij (x), the eigenvalues λ1(x), λ2(x) and the regularization term ϵ = 10 -6 are taken as inputs, and the formula is used to calculate the output normalized metric tensor, which maps the texture structure to the metric of the Riemannian manifold, realizes the geometric modeling of the texture, and finally takes the eigenvalues λ1(x), λ2(x) as inputs, and the formula The curvature of the output section is calculated, and the curvature features of the texture are extracted to highlight the geometric differences between the defect area and the normal texture, providing feature support for subsequent defect detection.

[0031] Tensor fusion: The preprocessed dataset is dimensionally aligned with differential geometric features, and cross-modal feature orthogonalization is performed using regularized Tucker decomposition to generate a fused feature vector containing multi-domain related features; the optimization objective of regularized Tucker decomposition is... ,in, The tensor is a four-dimensional data tensor, including the dimensionality-reduced hyperspectral data, polarization parameters, and differential geometric features, where H is the height in pixels of the wood surface image, W is the width in pixels of the wood surface image, and λ... ’ denoted by , represents the number of effective bands retained after dimensionality reduction of the hyperspectral data using minimum noise separation transform, and p represents the number of channels in the polarization image. For the core tensor, , For spatial dimension factor matrix, , The modal dimension factor matrix is ​​given, α = 0.1 is the regularization weight, and I is the identity matrix. The alternating least squares method is used to solve the problem, with a convergence threshold of 10. -4 .

[0032] Specifically, the tensor fusion step takes the preprocessed dataset obtained from the modeling process and the extracted differential geometric features as input. In one embodiment, the preprocessed dataset contains dimensionality-reduced hyperspectral data and polarization parameters. First, the dimensions of each modal data are aligned: the hyperspectral data is regularized into a three-dimensional structure that matches the image spatial dimension. In one embodiment, the image spatial dimension includes height H and width W. The polarization image and differential geometric features are also adapted to the same spatial dimension. In one embodiment, the differential geometric features include cross-sectional curvature and geodesic curvature, with a total of 2 channels. Then, a four-dimensional data tensor is constructed by concatenating the channels. , where λ ’ Let be the number of effective bands after hyperspectral dimensionality reduction, and p be the number of polarization channels. Then, a regularized Tucker decomposition is performed on this tensor, with the optimization objective being... Core Tensor Characterizing the correlation patterns among multimodal features, spatial dimension factor matrix , Dimensionality reduction is performed on the features in the height and width dimensions of the image, and orthogonal constraints are applied. and Avoid spatial feature redundancy; modal dimension factor matrix , Dimensionality reduction was performed on the features of the hyperspectral and polarization-geometric modes respectively; a regularization weight α=0.1 was used to balance the fitting error and the orthogonality constraint strength, and an iterative solution using alternating least squares method was adopted, with a convergence threshold set to 10.-4 Through this decomposition, the feature vector fusing the multi-domain correlation information of hyperspectral, polarization and differential geometry is generated, the redundancy of cross-modal features is removed and complementary enhancement is achieved, and more discriminative feature expression is provided for subsequent defect detection.

[0033] In this embodiment, the preprocessed data set and the differential geometric features obtained by processing modeling are taken as inputs, which are first spliced into a four-dimensional data tensor , and the tensor is taken as the input of the regularization Tucker decomposition optimization target formula; by minimizing , the core tensor is driven to fit the distribution rule of the multi-modal data with the factor matrix space height dimension A, space width dimension B, hyperspectral modal C and polarization-geometry modal D, while A and B are constrained to be orthogonal by , so as to eliminate the feature redundancy of the image space dimension; the optimized core tensor and factor matrix are output by iterative solution through the alternating least squares method, and the feature vector fusing the multi-domain correlation information of hyperspectral, polarization and differential geometry is extracted from these matrices, the redundancy of cross-modal features is removed and complementary enhancement is achieved, and more discriminative feature support is provided for subsequent defect detection.

[0034] Defect detection: a geodesic active contour model is constructed based on the Riemannian manifold model, the contour evolution is driven by the fused feature vector, the defect edge on the wood surface is extracted, and the detection result is optimized through curvature weighted focal loss and Riemannian geometry constraint. The evolution equation of the geodesic active contour model is , wherein ɸ is a level set function used to represent the implicit representation of the contour, is the curvature of the contour, representing the bending degree of the contour, is an edge indicator function, and Gσ is a Gaussian kernel with a standard deviation σ=1.5, which is used to suppress texture noise; the evolution equation is discretized by using a semi-Lagrangian method, and the time step ∆t=0.1.

[0035] The curvature weighted focal loss is , wherein N is the total number of samples, α t is a class balance weight, which is calculated by the reciprocal of the frequency of defect categories, γ=2 is a focal parameter, which is used to reduce the weight of easily classified samples, K i is the cross-sectional curvature corresponding to the i-th sample, which is calculated by the eigenvalue of the texture structure tensor . The loss function of the Riemannian geometry constraint is , wherein N d is the number of pixels in the defect area, Ω d is the defect area, d g (x) is the geodesic distance on the Riemannian manifold, and d e (x) is the Euclidean space distance.

[0036] Specifically, the defect detection step takes the Riemannian manifold model constructed through processing and the fused feature vector generated by tensor fusion as input. In one embodiment, the Riemannian manifold model includes a normalized metric tensor and differential geometric features. First, the level set function ɸ of the geodesic activity contour model is initialized based on the discriminative information of the fused feature vector. This function implicitly represents the initial distribution of the defect contour. Subsequently, the contour curvature is calculated using the normalized metric tensor of the Riemannian manifold. Meanwhile, the diffuse reflection component image I obtained from the processing model is... d An edge indicator function is constructed by convolving a Gaussian kernel Gσ with a standard deviation σ=1.5 and extracting the gradient. To suppress texture noise interference, the curvature k and edge indicator function g are substituted into the geodesic profile evolution equation. The semi-Lagrange method is used for discretization, with a time step of ∆t=0.1, driving the contour to evolve along the geodesic direction of the Riemannian manifold to extract the edges of defects on the wood surface. To optimize the detection results, two types of loss functions are introduced: curvature-weighted focus loss is based on the eigenvalues ​​λ1 and λ2 of the texture structure tensor in the modeling, and is calculated using the formula... Calculate the curvature K of the sample section i Combined with category balancing weight α t With focus parameter γ=2, construct This reduces the weight of easily classified samples and increases the focus on defective regions; the Riemann geometric constraint loss utilizes the Riemann manifold geodesic distance d calculated by the Fast-Marching algorithm in the processing model. g (x) and Euclidean distance d g (x), through the formula Constrain the geometric consistency of the defect region. The two are optimized synergistically to improve the accuracy and robustness of defect detection, making the results more closely resemble the actual defect morphology.

[0037] In this embodiment, the fused feature vector generated by tensor fusion is first used as input to initialize the level set function ɸ of the geodesic activity contour model, outputting a function for the initial distribution of the hidden defect contour, providing a starting point for contour evolution; then, the normalized metric tensor obtained from the processing model is used as input, and the function is processed using the formula... The output contour curvature k is calculated to characterize the degree of contour bending; then the diffuse component image I obtained from the processing model is used. d Using the input as input, an edge indicator function is constructed through Gaussian kernel Gσ convolution with standard deviation σ=1.5 and gradient extraction. The output shows the edge response after suppressing texture noise, providing guidance for contour evolution; the curvature k and edge indicator function g are substituted into the contour evolution equation for geodesy activities. , the output evolution of the level set function is obtained, and the dynamic extraction of the defect contour along the Riemannian manifold direction is realized; the eigenvalues λ1 and λ2 of the texture structure tensor in the processing modeling are taken as inputs, and the formula is used to calculate the sample cross-sectional curvature K i , the category balance weight α t is combined, the focal parameter γ=2 is taken, the curvature weighted focal loss is constructed , the loss value after reducing the weight of the easily classified sample is output, and the attention degree to the defect area is enhanced; the Riemannian manifold geodesic distance d g (x) calculated by the Fast-Marching algorithm in the processing modeling and the Euclidean distance d g (x) are combined, the formula is used to construct the Riemannian geometry constraint loss, the loss value of the geometric consistency of the constraint defect area is output, and the detection result is more in line with the true defect morphology; through the cooperative optimization of the two types of loss functions, the accurate defect detection result is finally output, and the accurate identification and positioning of the wood surface defects are realized.

[0038] The multi-modal data of the same wood region is time-synchronized by triggering the acquisition of the hyperspectral camera, the polarization camera and the structured light scanner synchronously, and the conversion relationship between the world coordinate system and the device coordinate system is solved by the Zhang Zhengyou calibration method. The multi-modal images are sub-pixel registered by using thin-plate spline interpolation, and a spatio-temporally consistent data set is generated, laying a spatial alignment foundation for subsequent anti-light interference processing. When processing modeling, the registered data is pre-processed in multiple modes: the hyperspectral data is dimensionally reduced to remove redundant bands, the polarization image is separated based on the Mueller matrix model to eliminate the influence of specular reflection, and after smoothing and denoising the depth map, the texture structure tensor is calculated combined with the diffuse reflection component, the texture anisotropy parameters are obtained by eigenvalue decomposition, the normalized metric tensor is defined to construct the Riemannian manifold model, and the wood texture is mapped to a manifold space with geometric properties such as curvature and geodesic distance, thereby suppressing the confusion between natural texture and defects from a geometric perspective. In the tensor fusion stage, the pre-processed spectral, polarization parameters and differential geometric features extracted from the Riemannian manifold are dimensionally aligned to construct a four-dimensional data tensor, and the cross-modal features are orthogonalized by regularized Tucker decomposition to eliminate feature redundancy caused by light variation and enhance the expression of weak defect signals in multi-domain associated features. When detecting defects, a geodesic active contour model is constructed based on the Riemannian manifold model, the contour distribution is initialized using the fusion feature vector, and the contour is driven to expand along the manifold direction by the geodesic active contour evolution equation. The curvature weighted focal loss and Riemannian geometric constraint optimization are combined to make the contour evolution process focus on the defect area with curvature mutation and suppress the light reflection interference, and finally realize the extraction of the defect edge in the strong light environment. Through the complementary fusion of multi-modal data and the geometric modeling of wood texture, the problem of defect feature covered by reflection caused by light variation in traditional methods is solved, and the detection accuracy in strong light environment is improved to a stable level.

[0039] Please refer to the attached Figure 2 The wood surface defect detection system based on image recognition is used for the wood surface defect detection method based on image recognition, and comprises: The acquisition and registration module is used for synchronously acquiring image data of the wood surface by using the acquisition device, calibrating the external parameters of the acquisition device by using the Zhang Zhengyou calibration method, establishing the conversion relationship between the world coordinate system and the coordinate system of each acquisition device, and registering the image data set based on the conversion relationship by using thin-plate spline interpolation to generate a spatio-temporally consistent registered data set. The processing and modeling module is used for pre-processing the registered data set in multiple modes to form a pre-processed data set comprising spectral, polarization and depth information, and based on the pre-processed data set, calculating a texture structure tensor, performing eigenvalue decomposition, defining a normalized metric tensor, constructing a Riemannian manifold model of the wood surface, and extracting differential geometric features of the wood surface. Tensor fusion module: used to align the dimensions of the preprocessed dataset with differential geometric features, and to perform cross-modal feature orthogonalization through regularized Tucker decomposition to generate a fused feature vector containing multi-domain related features; Defect detection module: It is used to construct a geodesic activity contour model based on the Riemannian manifold model, drive contour evolution by using fused feature vectors, extract defect edges on the wood surface, and optimize the detection results by curvature-weighted focal loss and Riemannian geometric constraints.

[0040] Specifically, the acquisition and registration module integrates a hyperspectral camera, a polarization imager, and a depth scanner. A hardware-triggered synchronization mechanism ensures temporal consistency of multi-sensor data acquisition. After establishing a unified coordinate system using the Zhang Zhengyou calibration method, pixel-level registration is achieved using a thin-plate spline interpolation algorithm, outputting a spatiotemporally aligned multimodal dataset. The processing and modeling module performs hierarchical processing on the registered data: first, it compresses the dimensionality of the hyperspectral data through minimum noise separation transform; then, it separates the diffuse reflection components of the polarization image based on the Mueller matrix model; next, it calculates the texture gradient using the Sobel operator; and finally, it constructs a texture structure tensor using Gaussian kernel smoothing. Eigenvalue decomposition is then used to obtain texture anisotropy parameters. Finally, a normalized metric tensor maps the wood surface to a Riemannian manifold space, extracting differential geometric features such as cross-sectional curvature and geodesic curvature. The tensor fusion module reconstructs the preprocessed multimodal data and geometric features into a four-dimensional tensor. Regularized Tucker decomposition achieves orthogonal constraints and correlation mining of cross-modal features, generating a fused feature vector. The defect detection module constructs a geodesic activity contour model based on the geometric properties of the Riemannian manifold, extracts defect edges through the evolution of the level set function, optimizes the detection results by combining curvature-weighted focal loss and Riemannian geometric constraint loss function, and finally outputs a detection report containing defect location, type and geometric parameters.

[0041] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting surface defects in wood based on image recognition, characterized in that, Includes the following steps: Acquisition and Registration: Image data of the wood surface is acquired synchronously through acquisition devices. The external parameters of the acquisition devices are calibrated using the Zhang Zhengyou calibration method. The transformation relationship between the world coordinate system and the coordinate system of each acquisition device is established. Based on the transformation relationship, thin plate spline interpolation is used to register the image dataset, generating a spatiotemporally consistent registration dataset. Modeling: The registration dataset is preprocessed in a multimodal manner to form a preprocessed dataset including spectral, polarization and depth information. Based on the preprocessed dataset, the texture structure tensor is calculated, eigenvalue decomposition is performed, and a normalized metric tensor is defined to construct a Riemannian manifold model of the wood surface and extract the differential geometric features of the wood surface. Tensor fusion: Aligns the preprocessed dataset with differential geometric features in terms of dimensions, and performs cross-modal feature orthogonalization through regularized Tucker decomposition to generate a fused feature vector containing multi-domain related features; Defect detection: A geodesic activity contour model is constructed based on the Riemannian manifold model. The contour evolution is driven by fused feature vectors to extract the defect edges on the wood surface. The detection results are optimized by curvature-weighted focal loss and Riemannian geometric constraints.

2. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The acquisition equipment includes a hyperspectral camera, a polarization camera, and a structured light 3D scanner. The image data includes hyperspectral data, polarization images, and depth maps. The conversion relationship is P. w=RPc+t , where P w P represents the coordinates of a three-dimensional point in the world coordinate system. c To acquire the three-dimensional point coordinates in the device coordinate system, Let be a rotation matrix. It is a translation vector, and the calibration error is less than 0.3 pixels.

3. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The mathematical expression for the thin-plate spline interpolation is: Where x=(x,y) are the pixel coordinates before registration, and x'=(x',y') are the pixel coordinates after registration. Let c be the affine transformation matrix. k Let w be the coordinates of the kth control point. k For interpolation weights, basis functions The registration accuracy is less than 0.1 pixels.

4. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The method for calculating the texture structure tensor is to calculate the diffuse component image I in the preprocessed dataset. d Calculate spatial gradient and Through Gaussian kernel g σ After smoothing, a second-order moment matrix is ​​generated. , , Where σ is the standard deviation of the Gaussian kernel, and * denotes convolution operation; construct a symmetric positive definite texture structure tensor. Eigenvalue decomposition of T(x) yields eigenvalues ​​λ1≥λ2 and corresponding eigenvectors e1,e2, where λ1 and λ2 characterize the anisotropy intensity of the texture region, and e1 is the principal direction vector of the texture.

5. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The normalized metric tensor is defined as follows: , among which, T ij (x) represents the components of the texture structure tensor, ϵ=10 -6 The regularization term is used to avoid the denominator being zero. A Riemannian manifold model is constructed based on the metric tensor, and the geodesic distance between two points on the manifold is calculated using the FastMarching algorithm, with a time complexity of O(NIogN), where N is the total number of image pixels.

6. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The optimization objective of the regularized Tucker decomposition is: ,in, The tensor is a four-dimensional data tensor, including the dimensionality-reduced hyperspectral data, polarization parameters, and differential geometric features, where H is the height in pixels of the wood surface image, W is the width in pixels of the wood surface image, and λ... ’ denoted by , represents the number of effective bands retained after dimensionality reduction of the hyperspectral data using minimum noise separation transform, and p represents the number of channels in the polarization image. For the core tensor, , For spatial dimension factor matrix, , The modal dimension factor matrix is ​​given, α = 0.1 is the regularization weight, and I is the identity matrix. The alternating least squares method is used to solve the problem, with a convergence threshold of 10. -4 .

7. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The evolution equation of the geodesic activity profile model is: , where ɸ is the level set function used to represent the implicit representation of the contour. The curvature of the contour represents the degree of bending of the contour. Gσ is the edge indicator function, and Gσ is a Gaussian kernel with standard deviation σ=1.5, used to suppress texture noise. The evolution equation is discretized using a semi-Lagrangian method with a time step of ∆t=0.

1.

8. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The curvature-weighted focus loss is Where N is the total number of samples, α t The category balancing weights are calculated using the reciprocal of the frequency of the defect category. γ=2 is the focus parameter, used to reduce the weight of easily classified samples. K i The cross-sectional curvature corresponding to the i-th sample is calculated using the eigenvalues ​​of the texture structure tensor. ; The loss function of the Riemannian geometric constraint is , where N d Ω represents the number of pixels in the defect area. d For the defect area, d g (x) is the geodesic distance on the Riemannian manifold, d e (x) represents the Euclidean distance.

9. The method for detecting wood surface defects based on image recognition according to claim 1, characterized in that: The multimodal preprocessing includes dimensionality reduction of the hyperspectral data in the registration dataset using a minimum noise separation transform, retaining the first λ' effective bands; and separating the diffuse reflection component of the polarization image based on the Mueller matrix model. , among which, I u I represents the intensity of unpolarized light. p Here, k represents the polarized light intensity, and k is the polarization reflection coefficient. Median filtering is applied to the depth map for smoothing and denoising to remove random noise points. The differential geometric properties include cross-sectional curvature and geodesic curvature, with the cross-sectional curvature calculated using the eigenvalues ​​of the texture structure tensor. The geodesic curvature is calculated by the tangential derivative of the profile on the Riemannian manifold and is used to describe the bending characteristics of the defect edge along the main texture direction.

10. A wood surface defect detection system based on image recognition, characterized in that: The image recognition-based wood surface defect detection method according to any one of claims 1-9 includes: The acquisition and registration module is used to synchronously acquire image data of the wood surface through the acquisition device, use the Zhang Zhengyou calibration method to calibrate the external parameters of the acquisition device, establish the transformation relationship between the world coordinate system and the coordinate system of each acquisition device, and use thin plate spline interpolation based on the transformation relationship to register the image dataset and generate a spatiotemporally consistent registration dataset. The processing and modeling module is used to perform multimodal preprocessing on the registration dataset to form a preprocessed dataset including spectral, polarization and depth information. Based on the preprocessed dataset, it calculates the texture structure tensor, performs eigenvalue decomposition, defines the normalized metric tensor, constructs the Riemannian manifold model of the wood surface, and extracts the differential geometric features of the wood surface. Tensor fusion module: used to align the dimensions of the preprocessed dataset with differential geometric features, and to perform cross-modal feature orthogonalization through regularized Tucker decomposition to generate a fused feature vector containing multi-domain related features; Defect detection module: It is used to construct a geodesic activity contour model based on the Riemannian manifold model, drive contour evolution by using fused feature vectors, extract defect edges on the wood surface, and optimize the detection results by curvature-weighted focal loss and Riemannian geometric constraints.

Images