A method and system for predicting the internal knot morphology of wood based on surface defect images

Abstract

The present application is a kind of wood internal knot morphology prediction method and system based on surface defect image, related to the field of log knot defect detection, to solve the problem that the oval knot on the upper and lower surfaces of wood will be distorted, resulting in the difficulty of image inversion, the existing knot defect inversion method of wood-based panel is complex and high in cost. Utilize WGAN to expand the data set, improve the effectiveness of the data set, and enhance the model recognition ability; use YOLO network to detect the defect of the panel, only need to process the picture simply, can complete the identification of the image; obtain the vertex coordinate set, for the problem that the edge straight line does not intersect and the vertex is difficult to determine caused by the proportion deviation of the long and short axes of the ellipse, a density clustering method is proposed to locate the vertex by excluding interference points; finally, according to the bottom surface ellipse and the vertex position information, the morphology of the knot in the solid wood panel is inverted, which has the characteristics of low cost and easy to realize.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of log knot defect detection, in particular to a wood internal knot morphology prediction method and system based on surface defect images. BACKGROUND

[0002] The comprehensive utilization rate of wood is the main goal of log optimization. Due to the existence of log knot defects, not only the processing difficulty is increased, but also the visual effect and mechanical properties of the board after cutting are affected. At present, researchers have carried out detection based on ultrasonic wave, X-ray, stress wave and other methods, trying to establish a correlation between electrical signals and internal defects, and to inverse the internal defects of the log. However, the acquisition of spatial data related to the internal knot defects of the log is still the primary task and difficult problem of modeling analysis.

[0003] At present, knot detection mainly includes laser imaging, X-ray scanning and near-infrared spectrum analysis methods, and the literature [1] applies laser imaging technology to calculate the surface fiber angle and then describes the internal defects of the board; the literature [2] applies X-ray technology to describe the knot defect morphology of the solid wood board, and calculates the cross-sectional area and spatial depth of the knot defect to predict the mechanical parameters of the board; the literature [3] uses a laser scanner to find the internal projection of the defect on four surfaces to estimate the defect volume and then predict the mechanical properties of the solid wood board; the literature [4] uses near-infrared spectrum technology to construct the relationship between edge angle and spectral characteristics, and uses the inclination value to describe the position of the knot. Although these methods have detected the knot morphology, the equipment cost is high and the technical difficulty is great.

[0004] Therefore, image processing is applied to knot detection, which has the advantages of simple system composition, strong information acquisition ability and flexible detection. The image processing process can realize effective segmentation of the defect area through the organic integration of algorithms. The literature [5] proposes an image binarization optimization algorithm based on local threshold value for the problem of uneven background of wood defect images; the literature [6] proposes a region growing segmentation method based on image fusion to realize fast and accurate segmentation of the defect area; the literature [7] uses median filtering and edge detector to detect the accurate position, size and shape of wood defects in the image; the literature [8] proposes a defect segmentation method based on texture filling for board defect detection, acquires a texture mask image, then uses the background color of the board to fade the texture, and finally applies a weighted threshold method to complete the defect segmentation.

[0005] In recent years, the level set method overcomes the shortcomings of complex steps and tedious calculation in the process of fusing various methods, naturally changes the topological structure with continuous evolution curves, and obtains more detailed features, and can segment complex target objects [9] For example, the region-based Chan-Vese (C-V) level set model and the morphological method are used in succession in the document

[10] to segment a color tree image, global optimization segmentation effect is obtained, but the model iteration takes a long time. The distance regularized level set evolution (DRLSE) model based on the boundary is proposed in the document

[11] , on the basis of the level set, the internal energy of the distance regularization term is used to reduce the deviation between the level set function and the distance sign function, accurate image segmentation is realized, and the calculation efficiency is high compared with the C-V model based on region segmentation.

[0006] Although the defect segmentation algorithm of solid wood board can capture the defect contour on the plane, how to use image processing technology to inverse the internal shape of the defect has not been deeply studied. It is generally believed in the existing view that knots exist in the form of three-dimensional cones in wood

[12] , in the processing process of solid wood veneer, knots usually appear in the form of ellipses on the upper and lower surfaces of the board. However, as a kind of biological material with non-homogeneous and anisotropic properties, the ellipses on the upper and lower surfaces of the wood are distorted, which brings certain difficulties to the image inversion of the cone. SUMMARY

[0007] The technical problem to be solved by the present application is:

[0008] In order to solve the problem that the ellipses of knots on the upper and lower surfaces of wood as a kind of biological material with non-homogeneous and anisotropic properties are distorted, the straight line with the edge of the knot as the vertex cannot intersect, the vertex is difficult to determine, and the existing knot defect inversion method of wood board is complex and high in cost.

[0009] The technical scheme adopted by the present application to solve the above technical problem is:

[0010] The present application provides a wood internal knot shape prediction method based on surface defect image, comprising the following steps:

[0011] S1, select a board with knots on the upper and lower surfaces, collect the defect images of the upper and lower surfaces of the board, perform traditional data expansion by flipping, affine transformation, brightness and protection degree adjustment and noise increase on the collected images, then train the WGAN network, further expand the data set by using the data generated by the WGAN, and then extract the edge contour and position coordinates of the knot defect of the solid wood board by using the YOLO algorithm.

[0012] S2, obtain the fitting ellipse parameters, and calculate the angle deviation between the upper and lower fitting ellipses;

[0013] S3, use the ellipse parameters obtained in S2 to construct an ellipse parameter setting neural network model, and the network output is the upper and lower ellipse long axis coplanar information coordinates;

[0014] S4, compare the upper and lower ellipse long axis information, and select the large ellipse as the base of the conical body shape;

[0015] S5, use the method of intersecting the straight line of the corresponding points of the upper and lower ellipses with the long axis surface to obtain the model vertex coordinate data set;

[0016] S6, use a density clustering algorithm to cluster the multiple intersection points formed in S5, and consider the class with a small number of clusters as a noise point and remove it;

[0017] S7, process the vertex data set, and use weighted average to obtain a more accurate conical vertex coordinate;

[0018] S8, use the base ellipse and the conical point data to predict the location and shape of the knot in the solid wood board.

[0019] Further, the step S2 comprises:

[0020] Take the ellipse long axis point as the starting point, connect the edge points of the two ellipses corresponding to the angle every 10°-15° to form an edge line, a total of 22 edge lines, but not including the two long axis points; After the ellipse angle correction, the intersection point of the edge line and the long axis surface is the conical vertex, and the vertex coordinate data set is obtained.

[0021] Further, in the process of obtaining the model vertex coordinate data set in S5, take one end point of the ellipse long axis as the starting point, and connect the edge points of the two ellipses corresponding to the angle every 15° to form an edge line.

[0022] Further, in S6, the specific steps of the density clustering algorithm used are as follows: select a point in the vertex coordinate data set in S5, then take this point as the center to draw a circle, find the average value of the vectors from the center to all points, and the sum of the center and the average value of the vectors is the new center. Then iterate this process until the end condition is met.

[0023] Further, the loss function in the clustering algorithm in S6 is

[0024]

[0025] Wherein, K is the number of clusters; N is the number of elements contained in each class; r kn represents the data point, n is 1 when the data point n is classified into class k, otherwise it is 0; |xn -μ k | represents the Manhattan distance of the element from the cluster center.

[0026] Further, in S8, when the defect morphology is predicted, the correctness of the model and the superiority of the clustering method are measured by three indexes of model base area deviation rate, cross-sectional area deviation rate and cross-sectional center point distance deviation.

[0027] Further, in S8, the correctness of the model and the superiority of the clustering method are measured by three indexes of model base area deviation rate, cross-sectional area deviation rate and cross-sectional center point distance deviation, and the weight coefficients of the three indexes are the same, and the calculation formula is as follows,

[0028]

[0029]

[0030]

[0031] Wherein, S1' is the model base area, S2' is the model cross-sectional area, O2' is the model cross-sectional center point, S1 is the original image base area, S2 is the original image cross-sectional area, O2 is the original image cross-sectional center point, f S1 is the model base area deviation rate, f S2 is the cross-sectional area deviation rate, and l2 is the cross-sectional center point distance deviation.

[0032] Further, in S1, the defect image uses JHUM130m-E type camera and HM5014MP5 type lens to collect the image of the plate.

[0033] Further, in S1, the size of the defect in the plate image is measured by the number of pixels.

[0034] A wood internal knot morphology prediction system based on surface defect image, comprising:

[0035] An image acquisition and knot defect edge contour extraction module is used to image the knots on the plate, expand, and then extract the edge contour and position coordinates of the plate knot defect;

[0036] A knot vertebra morphology construction module is used to obtain a plurality of knot vertebra models with different vertices.

[0037] A knot unique vertex acquisition module is used to obtain more accurate knot vertebra vertex coordinates through clustering processing.

[0038] A plate internal knot defect position and morphology prediction module is used to predict the defect position and morphology of the knot in the plate according to the knot vertebra and knot vertex coordinates.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] This invention discloses a method and system for predicting the morphology of knots inside wood based on surface defect images. First, it utilizes WGAN to expand the dataset, solving the problem of learning instability and further improving the effectiveness of the dataset and enhancing the model's recognition ability. Then, it uses the YOLOv3 network to detect defects in solid wood boards, requiring only simple image processing to complete image recognition and achieve ideal results. To address the problem of non-intersecting edge lines and difficulty in determining vertices due to deviations in the major and minor axes of the ellipse, a density clustering method to eliminate interference points is proposed to locate vertices. Finally, the morphology of knots in solid wood boards is inverted based on the information of the base ellipse and vertex positions, thus providing data support for the inversion of defect morphology within logs.

[0041] This invention uses 10 defective larch slabs for experiments, obtaining images of the slabs at a resolution of 762x176. Fitting accuracy is measured in pixels. Experimental results show that the average deviation rate of the base area of ​​the defective cone model is... The average deviation rate of cross-sectional area is 2.43%. Average distance deviation of the center point of the cross section Based on the above data, the pixel length shows that this method can accurately reflect the shape of defects inside solid wood boards, and has the characteristics of low cost and easy implementation.

[0042] The above experiments show that by clustering the model vertex dataset to eliminate the influence of interference points, the defect vertex can be accurately located and the defect morphology can be inverted, thus providing spatial data of the defect within the log. Overall, the entire calculation method used in this invention is simpler than existing algorithms, and it ensures inversion accuracy while improving the calculation speed. Attached Figure Description

[0043] Figure 1 This is a flowchart of a method for predicting the internal knot morphology of wood based on surface defect images, as described in an embodiment of the present invention.

[0044] Figure 2 This is a diagram showing defects on the upper and lower surfaces of some of the sheet metal in an embodiment of the present invention;

[0045] Figure 3 This is a three-dimensional view of the vertebral body in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram illustrating the process of forming a spatial vertex dataset in an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the vertex dataset density clustering algorithm in an embodiment of the present invention;

[0048] Figure 6 A schematic diagram for model verification in the embodiments of the present application. DETAILED DESCRIPTION

[0049] In the description of the present application, it should be noted that the terms such as "upper", "lower", "front", "rear", "left", "right" and the like in the embodiments indicate the position relationship based on the drawings of the specification, and do not mean that the elements and devices referred to must be operated according to the specific position and the defined operation and method, structure, and the like. Such positional terms do not constitute a limitation on the present application.

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings.

[0051] Specific implementation scheme one: combined Figures 1 to 6 As shown in the drawings, the present application provides a wood internal knot shape prediction method based on surface defect image, comprising the following steps:

[0052] S1, select a board with knots on the upper and lower surfaces, collect the upper and lower surface defect images of the board in an environment with sufficient light, if the size of the board is inconsistent, the fitting accuracy can be measured by pixel size, through traditional data expansion by flipping, affine transformation, brightness and protection degree adjustment and noise increase, training with WGAN network, further expanding the data set with WGAN generated data, and extracting the solid wood board knot defect edge profile and position coordinates with YOLO algorithm,

[0053] The step of further expanding the data set with WGAN includes dividing the original data set into board images with and without defects; then the two parts of images are sent into the WGAN network for training as training sets, and generated images are obtained;

[0054] In the process of extracting the solid wood board knot defect edge profile and position coordinates with YOLO algorithm, the data set expanded with WGAN can be used for defect detection of solid wood board with YOLOv3 network, and then the knot defect edge profile and position coordinates are obtained; the above method is prior art;

[0055] S2, obtain the fitting ellipse parameters, calculate the angle deviation between the upper and lower fitting ellipses, construct an ellipse parameter setting neuron network model with the ellipse parameters, the network output is the upper and lower ellipse long axis coplanar information coordinates, and the large ellipse is selected as the cone bottom surface;

[0056] S3, use the method of intersecting the straight line of the corresponding points of the upper and lower ellipses with the long axis surface to obtain the model vertex coordinate data set, specifically,

[0057] Starting from the major axis point of the ellipse, connect the edge points of the two ellipses at corresponding angles every 10°-15° to form edge lines, forming a total of 22 edge lines, excluding the two major axis points; after ellipse angle correction, the intersection of the edge lines and the major axis plane is the vertex of the cone, and the vertex coordinate dataset is obtained.

[0058] S4. Use density clustering algorithm to cluster the multiple intersections formed in S5. Treat the categories with fewer clusters as noise points and remove them.

[0059] The specific steps of the density clustering algorithm used are as follows: select a point in the vertex coordinate dataset in S5, draw a circle with this point as the center, calculate the average value of the vectors from the center to all points, and the sum of the center and the average value of the vectors is the new center. Then iterate this process until the condition of a point is met.

[0060] The loss function in the clustering algorithm is:

[0061]

[0062] Where K is the number of clusters; N is the number of elements in each cluster; r kn Representing data points, n is 1 if it is classified into category k, and 0 otherwise; |x n -μ k | Represents the Manhattan distance of an element from its cluster center.

[0063] S5. Process the vertex dataset and use a weighted average to obtain more accurate cone vertex coordinates;

[0064] S5. Invert the defect shape using the base ellipse and cone point data, specifically:

[0065] Since the cone shape is related to three parameters—the model's base area S1', the model's cross-sectional area S2', and the model's cross-sectional midpoint O2' (assuming the model's base midpoint coincides with the midpoint of the actual defect image)—the three parameters S1', S2', and O2' of the cone model are compared with the three parameters S1, S2, and O2' of the original image. The deviation rate of the model's base area (f) is used as the criterion. S1 ), cross-sectional area deviation rate (f S2 The correctness of the model and the superiority of the clustering method are measured by three indicators: (1) the distance deviation between the cross-section center points and (l2). The weight coefficients of the three indicators are the same, and the calculation formulas are shown in (2), (3) and (4).

[0066]

[0067]

[0068]

[0069] The application firstly expands the data set by using the WGAN, solves the problem of unstable learning, further improves the effectiveness of the data set, and enhances the recognition ability of the model; the defects of solid wood boards are detected by using the YOLOv3 network, only simple processing is needed for the pictures, the image recognition can be completed, and the ideal effect can be achieved; and the defect boundary is fitted by using an elliptic equation; in view of the problem that the edge straight lines are not intersected and the vertexes are difficult to determine due to the proportion deviation of the major axis and the minor axis of the ellipse, a density clustering method for excluding interference points is proposed to position the vertexes; finally, the shape of the knot in the solid wood board is inversed according to the bottom surface ellipse and the vertex position information, and then data support is provided for the defect shape inversion in the log;

[0070] The application adopts 10 spruce defect boards for experiment, the board image resolution is 762x176, the fitting accuracy is measured in pixels, and the experimental results show that the average deviation rate of the bottom area of the defect cone model is 2.43%, the average deviation rate of the cross-sectional area is 2.43%, and the average distance deviation of the cross-sectional center point is 2.43 pixels in length. Based on the above data, it is concluded that the method can accurately invert the shape of the defect in the solid wood board, has the characteristics of low cost and being beneficial to implementation.

[0071] The above experiment shows that by clustering the vertex data set of the model and excluding the influence of interference points, the defect vertex can be accurately positioned and the defect shape can be inversed, and then the spatial data of the defect in the log can be given; compared with the existing algorithm, the defect inversion method adopted by the application is simpler, and the calculation speed is improved while the inversion accuracy is ensured.

[0072] Preferably, in S1, a JHUM130m-E type global exposure camera and a 50mm HM5014MP5 type lens are used for image acquisition of the board, the maximum frame rate of the JHUM130m-E type global exposure camera is 60fps, the pixel is 1.3 million colors, the lens aperture of the M5014MP5 type lens is F1.4, and a 2 / 3 inch imaging photosensitive element is adapted, and a separate 1 / 2.5 inch, 640*480 pixel photosensitive chip should be used at the minimum imaging distance, and the minimum image distance is 60mm. When the image acquisition is carried out, the camera is stationary and located directly above the board, and the board is moved in two axes.

[0073] Preferably, in the process of acquiring the model vertex coordinate data set in S3, the elliptic major axis point is taken as the starting point, every 15°, the edge points corresponding to the angles of the two ellipses are connected to form edge lines, and a total of 22 edge lines are formed.

[0074] Specific implementation scheme two: in combination with Figures 1 to 6As shown, the present application provides a wood internal knot morphology prediction system based on surface defect images, comprising:

[0075] An image acquisition and knot defect edge profile extraction module is used for image acquisition and expansion of the knots on the board, and then the edge profile and position coordinates of the knot defects on the board are extracted.

[0076] A knot cone morphology construction module is used for obtaining a plurality of knot cone models with different vertexes.

[0077] A knot unique vertex acquisition module is used for obtaining more accurate knot cone vertex coordinates through clustering processing.

[0078] A board internal knot defect position and morphology prediction module is used for predicting the defect position and morphology of the knot in the board according to the knot cone vertex coordinates and the knot cone vertex coordinates. The other combinations and connection relationships of the present embodiment are the same as those of the specific embodiment.

[0079] Although the present application discloses as above, the protection scope of the present application is not limited to this. The person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

[0080] Reference:

[0081] [1] Olsson, A., Oscarsson, J., Serrano, E. et al. Prediction of timber bending strength and in-member cross-sectional stiffness variation on the basis of local wood fibre orientation [J]. European Journal of Wood and Wood Products. 2013, 71: 319-333.

[0082] [2] Hittawe M M, Sidibe D, Meriaudeau F. A machine vision based approach for timber knots detection. [C]. Proc. SPIE 9534, Twelfth International Conference on Quality Control by Artificial Vision 2015, 95340L (30 April 2015); doi: 10.1117 / 12.2182770.

[0083] [3] Roblot G, Bleron L, Meriaudeau F, Marchal R. Automatic computation of the knot area ratio for machine strength grading of douglas-fir and spruce timber [J]. European Journal of Environmental and Civil Engineering, 2010, 14(10): 1317-1332.

[0084] [4] Yu HL, Zhang M, Hou HY, et al. Inversion of knot morphology of solid wood board based on near-infrared spectroscopy analysis [J]. Spectroscopy and Spectral Analysis, 2019, 39(08): 2618-2623.

[0085] [5] Luo W, Sun LP. An improved binarization algorithm of wood image defect segmentation based on non-uniform background [J]. Journal of Forestry Research, 2019, 30(4): 1527-1533.

[0086] [6] Zhang YZ, Xu L, Ding L, et al. Surface defect detection method of solid wood floor based on image fusion segmentation [J]. Electric Machines and Control, 2014, 18(7): 113-118.

[0087] [7] Mu H. Study of wood defects detection based on image processing. IEEE Conference on Fuzzy Systems and Knowledge Discovery (FSKD), 2010.

[0088] [8] Li C, Lv X W, Tu W J, et al. Design of intelligent sorting system for solid wood surface based on computer vision[J]. Journal of Beijing Forestry University, 2016, 38(03): 102-109.

[0089] [9] Li C M, Xu C Y, Gui C F, et al. Level set evolution with outreinitialization: a new variational formulation[C]. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE Computer Society Press, 2005: 430-436.

[0090]

[10] Bai X B, Guo J Q, Chen K, et al. Color tree image segmentation based on C-V model level set and morphology[J]. Journal of Northwest Forestry University, 2015, 30(02): 196-200.

[0091]

[11] Li C M, Xu C, Gui C, M D Fox. Distance Regularized Level Set Evolution and Its Application to Image Segmentation[J]. IEEE Trans. Image Processing, 2010, 19(12): 3243-3254.

[0092]

[12] Pablo G, Manuel G. A three-dimensional wood material model to simulate the behavior of wood with any type of knot at the macro-scale[J]. Wood Science and Technology, 2013. 47: 585-599.

Claims

1. A method for predicting the internal knot morphology of wood based on surface defect images, characterized by: The method comprises the following steps: S1, selecting a plate material with knots on the upper and lower surfaces, collecting defect images of the upper and lower surfaces of the plate material, performing traditional data expansion on the collected images through image flipping, affine transformation, brightness and protection adjustment, and noise increase, then training the images using a WGAN network, further expanding the data set using the data generated by the WGAN, and extracting the edge profile and position coordinates of the knot defects of the solid wood plate using a YOLO algorithm; S2, obtaining the parameters of the fitted ellipse, and calculating the angle deviation between the upper and lower fitted ellipses; S3, using the ellipse parameters obtained in step S2 to construct an ellipse parameter setting neural network model, and the network output is the coplanar information coordinates of the upper and lower ellipse major axes; S4, comparing the upper and lower ellipse major axis information, and selecting the large ellipse as the base of the knot cone shape; S5, using the method of intersecting the straight line of the corresponding points of the upper and lower ellipses with the major axis plane to obtain the model vertex coordinate data set; including, taking the ellipse major axis point as the starting point, connecting the edge points of the corresponding angles of the two ellipses every 10°-15° to form edge lines, a total of 22 edge lines, but not including the two major axis points; the intersection point of the edge line and the major axis plane after the ellipse angle correction is the cone vertex, and the vertex coordinate data set is obtained; S6, using a density clustering algorithm to cluster the multiple intersection points formed in S5, regarding the categories with less clustering number as noise points and removing them; S7, processing the vertex data set, and using weighted average to obtain more accurate cone vertex coordinates; S8, predicting the defect position and shape of the knot in the solid wood plate using the bottom ellipse and the cone point data.

2. The method according to claim 1, wherein: In the process of obtaining the model vertex coordinate data set in S5, taking one end point of the ellipse major axis as the starting point, connecting the edge points of the corresponding angles of the two ellipses every 15° to form edge lines.

3. The method according to claim 2, wherein: In S6, the specific steps of the density clustering algorithm used are as follows: selecting a point in the vertex coordinate data set in S5, then taking this point as the center to draw a circle, calculating the average value of the vectors from the center to all points, and the sum of the center and the average value of the vectors is a new center, then iterating this process until the condition of a point is met.

4. The method according to claim 3, wherein: The loss function in the clustering algorithm in S6 is (1) where K is the number of clusters; N is the number of elements contained in each cluster; denotes a data point, 1 if n is classified into category k, otherwise 0; denotes the Manhattan distance of an element from the cluster center.

5. The method according to claim 4, wherein: In S8, when predicting the defect shape, the correctness of the model and the superiority of the clustering method are measured by three indexes of model bottom area deviation rate, cross-sectional area deviation rate and cross-sectional center point distance deviation.

6. The method according to claim 5, wherein: In S8, the correctness of the model and the superiority of the clustering method are measured by three indexes of model bottom area deviation rate, cross-sectional area deviation rate and cross-sectional center point distance deviation, and the weight coefficients of the three indexes are the same, and the calculation formula is as follows, (2) (3) (4) wherein, is the model base area, is the model cross-sectional area, is the model cross-sectional midpoint, is the original image base area, is the original image cross-sectional area, is the original image cross-sectional midpoint, f S1 is the model base area deviation rate, f S2 is the cross-sectional area deviation rate, is the cross-sectional center point distance deviation.

7. The method according to claim 6, wherein: In S1, the defect images are collected by using a JHUM130m-E camera and a HM5014MP5 lens on the plate material.

8. The method according to claim 7, wherein: In S1, the defect size in the plate image is measured by the number of pixels.

Images