Multi-modal intelligent wood defect detection and performance evaluation system and method

Abstract

The invention discloses a multi-modal intelligent wood defect detection and performance evaluation system and a detection and evaluation method, which integrate multiple sensing technologies and deep learning algorithms to realize wood defect detection and quality evaluation. The system comprises a wood transmission module, a multi-modal data acquisition module, an information processing module and an intelligent grading module, and surface and internal characteristics of wood are obtained through visible light, infrared, near-infrared, ultrasonic and microwave sensors. A feature pyramid network (FPN) and an instance segmentation technology are utilized to accurately identify defects such as cracks, wormholes, knot scars and the like, and ARIMA and LSTM models are combined to predict the moisture content, so that the quality evaluation precision is improved. The method can intelligently classify wood, improves the detection efficiency, and is suitable for the fields of wood processing, furniture manufacturing and the like.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a multi-modal intelligent wood defect detection and performance evaluation system and method. Background Art

[0002] As an important material widely used in the construction, furniture and decoration industries, the quality of wood directly affects the performance and market value of the product. However, wood is prone to various defects such as cracks, wormholes, and knots during production and processing. At the same time, the moisture content and density of wood are also important indicators that affect its mechanical properties. Therefore, how to achieve efficient detection of wood defects and physical properties has always been a technical problem that needs to be solved in the industry.

[0003] Traditional wood defect detection mainly relies on manual visual inspection, which is limited by problems such as slow detection speed, manual fatigue, strong subjectivity and low recognition rate of minor defects, making it difficult to meet the requirements of modern industrial production for high efficiency and high precision. In recent years, with the rapid development of computer vision, sensor technology and deep learning algorithms, automatic wood defect detection schemes based on image processing technology have gradually been proposed. However, these schemes usually only focus on single modal data and fail to comprehensively utilize wood surface images, internal structure information and physical property data, resulting in limited accuracy and reliability of detection results.

[0004] Therefore, there is an urgent need for a multimodal intelligent wood defect detection and performance evaluation system and method that can effectively combine multiple information such as visible light images, infrared thermal imaging, near-infrared spectra, ultrasonic signals and microwave density, so as to comprehensively improve the accuracy of wood defect detection and the automation level of the detection process, and provide strong support for wood processing and quality control.

[0005] Chinese patent document CN202411707363A discloses a "wood surface defect detection method, system, medium and device". It includes: a preprocessing module for acquiring eucalyptus veneer images and performing preprocessing; a first feature map acquisition module for extracting feature maps of eucalyptus veneer images and performing feature aggregation through an inverse depth-separable stem module; a frequency feature enhancement module for extracting key frequency information in feature maps and restoring local details through a rectangular self-calibrated adaptive frequency attention network and a frequency-enhanced channel attention module; a bark defect area display module for inputting the final feature map into a segmentation head to output the identified bark defect area. This method improves the detection accuracy of bark defects in combination with frequency feature enhancement technology, but it is mainly aimed at a single defect type (such as bark defects) and lacks the comprehensive detection capability of multiple complex defects and wood physical properties (such as moisture content, density, etc.). Summary of the invention

[0006] The object of the present invention is to provide a multi-modal intelligent wood defect detection and performance evaluation method aiming at the deficiencies existing in the prior art. By fusing visible light images, infrared thermal imaging images, near-infrared spectral data, ultrasonic signals, and microwave density data, and through multi-modal feature alignment and dynamic weighted fusion, accurate detection of wood surface defects, internal defects, and moisture content can be comprehensively achieved.

[0007] Technical solution: The technical solution adopted by the present invention to solve the problem is as follows: A multi-modal intelligent wood defect detection and performance evaluation system includes a housing, a wood transmission module, a multi-modal data acquisition module, a multi-modal information processing module, and a wood grading and sorting module; multi-modal information of wood is collected through a visible light camera, an infrared thermal imager, a near-infrared spectral sensor, an ultrasonic detector, and a microwave density sensor; the multi-modal information processing module is connected to the multi-modal data acquisition module and is used for processing the multi-modal information; the wood transmission module uses a black rubber track to convey the wood to the multi-modal data acquisition module.

[0008] Preferably, the multi-modal data acquisition module includes multiple sensors, where the visible light camera collects the surface texture information of the wood, the infrared thermal imager is used to detect internal defects and temperature distribution of the wood, the near-infrared spectral sensor is used to analyze the chemical composition of the wood, and the ultrasonic detector and the microwave density sensor are used to measure the moisture content and density of the wood. Each sensor is installed in a modular design to ensure the diversity and comprehensiveness of information collection.

[0009] Preferably, the multi-modal data acquisition module adopts a method of alternately installing several LED light sources and sensor brackets to ensure uniform light distribution and symmetric distribution on both sides of the sensors. Each LED light source can be adjusted according to needs to optimize the acquisition effect of different modal data.

[0010] Preferably, the black rubber track is made of high-strength frosted PVC material, which not only ensures good friction and grip, but also can effectively absorb stray light in each spectral band, avoiding interference from light pollution, thereby improving the accuracy of the sensor for wood defect detection.

[0011] The present invention also provides a detection method for multi-modal intelligent wood defect detection and performance evaluation. By collecting visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data, and comprehensively utilizing the complementary characteristics of multi-modal data, comprehensive analysis of wood surface defects, internal moisture content, and thickness can be achieved. Using high-precision data synchronization and preprocessing technology to ensure the consistency of multi-modal data, extracting features through a multi-branch convolutional neural network and fusing them to generate a unified feature vector, and constructing an accurate defect detection and quality evaluation model, thereby realizing automatic grading and intelligent processing of wood; it specifically includes the following steps:

[0012] S1: Read the visible light image, infrared thermal image, near-infrared spectrum, ultrasonic signal, and microwave density data into the image processing module.

[0013] S2: Input the multi-modal data into a multi-branch convolutional neural network (CNN), extract features by combining with a feature pyramid network (FPN), and generate a unified high-dimensional feature vector through feature alignment and fusion.

[0014] S3: Based on the unified high-dimensional feature vector, construct a defect detection model to locate, classify, segment, and generate instance boundaries for wood surface defects, improving the defect recognition accuracy.

[0015] S4: Collect the moisture content of the outer and inner regions of the wooden board and weightedly integrate it into the overall moisture content feature. Use ARIMA and LSTM models to model and predict the historical trend of the overall moisture content.

[0016] S5: Generate a comprehensive quality score according to the defect detection results and moisture content prediction, and complete the wood grade classification. Send woods of different grades to designated storage or processing areas.

[0017] Preferably, for the step S1, the method of "reading the visible light image, infrared thermal image, near-infrared spectrum, ultrasonic signal, and microwave density data into the image processing module" is specifically implemented as follows:

[0018] S11: Use LabVIEW to control an industrial RGB camera, an infrared thermal imager, a hyperspectral camera, an ultrasonic detector, and a microwave density sensor simultaneously, and combine with a hardware trigger to achieve synchronous data acquisition in the hard trigger mode.

[0019] S12: Perform preprocessing steps on the collected data, such as denoising, normalization, coordinate registration, and modal mask processing. Among them, for the "modal mask processing", for each pixel value F in the single-modal feature map F output by the neural network m Calculate two pixel scoring metrics respectively: absolute value / response quality and entropy value. Compare the scores with a preset threshold. Pixel points exceeding the threshold retain their original values, and pixel points not reaching the threshold are assigned a value of 0. Finally, generate a masked feature map. The scoring formula for the absolute value / response quality is as follows: m,i Calculate two pixel scoring metrics respectively: absolute value / response quality and entropy value. Compare the scores with a preset threshold. Pixel points exceeding the threshold retain their original values, and pixel points not reaching the threshold are assigned a value of 0. Finally, generate a masked feature map. The scoring formula for the absolute value / response quality is as follows:

[0020] Q m,i = F m,i

[0021] The scoring formula for the entropy value is as follows:

[0022] Q m,i = -F m,i · log(F m,i )

[0023] Preferably, the specific implementation method of "inputting multi-modal data into a multi-branch convolutional neural network (CNN), extracting features by combining with a Feature Pyramid Network (FPN), and generating a unified high-dimensional feature vector through feature alignment and fusion" in step S2 is as follows:

[0024] S21: The feature map output from S1 is first passed through a convolutional neural network (CNN) and then input into a Feature Pyramid Network (FPN). Through the top-down feature propagation and lateral connection mechanism, the high-resolution and low-resolution feature maps are fused step by step to generate a unified feature map with multi-scale information.

[0025] S22: The feature sub-map of each modality is separated from the unified feature map F i . A pixel-level dynamic weighting mechanism is introduced, and by calculating the weight W i,j of each pixel point of the feature sub-map F i,j , the modality features are optimized. The weight calculation formula is as follows: i,k,j In the formula, F

[0026]

[0027] represents the value of the k-th modality at the pixel point j=(x,y) in the unified feature map F i,k,j ; i is used to amplify the significant feature response; represents the local gradient information of the pixel point (x,y), which is used to measure the feature change; α, β, γ are weight coefficients dynamically learned through training. Finally, the optimized feature map is obtained:

[0028] F

[0029] F i ' ,k,j = W i,k,j · F i,k,j

[0030] Finally, the optimized single-modal feature sub-map F i ' ,k is generated, and the optimized multi-modal fusion feature sub-map F i ' is generated through channel fusion operation.

[0031] S23: The multi-modal fusion feature sub-map F i ' generates a high-dimensional feature tensor through feature splicing operation, and then successively undergoes batch normalization, layer normalization, and dimensionality reduction processing, and finally generates a unified high-dimensional feature vector.

[0032] Preferably, the specific implementation method of "constructing a defect detection model based on a unified high-dimensional feature vector to locate, classify, segment, and generate instance boundaries for wood surface defects, and improve defect recognition accuracy" in step S3 is as follows:

[0033] S31: Input the unified high-dimensional feature vector into the defect detection model, and use the YOLO algorithm to simultaneously generate candidate boxes and classify defects, calibrate the areas where defects such as cracks, wormholes, and knots may exist, and assign class labels (such as cracks, wormholes, knots, etc.) to each candidate box.

[0034] S32: On the basis of classification, use the ResNet classification network to further perform pixel-level segmentation on the defect areas within the candidate boxes, accurately divide the shapes and boundaries of the defects, and generate defect masks.

[0035] S33: Perform instance segmentation on the segmented defect areas to distinguish different defect instances of the same category, generate independent segmentation masks and boundary information for each instance; in the process of the instance segmentation task, after generating a fixed-size feature map through ROI Align, before entering the classification, regression, and segmentation branches, optimize the modal features through a multi-modal information dynamic weighted fusion mechanism. First, split the feature submaps F of each modality from the unified feature map F k , and calculate the global dynamic weight W of each modality k . Then, combine the calculated weight W k with the corresponding modal feature submap F k to perform weighted optimization on the modal features and generate an optimized modal feature submap F k '. Finally, fuse all the optimized modal feature submaps into a unified optimized feature map F fused , and use it as the input for the subsequent classification, regression, and segmentation branches. The global dynamic weight calculation formula is as follows:

[0036]

[0037] In the formula: F k represents the k-th modality feature map after ROI Align; ||F k || represents the global average intensity of the modal features; Var(F k ) represents the global variance of the modal features; represents the global gradient mean of the modal features; α, β, γ are weighted coefficients dynamically learned through training

[0038] S34: Generate class labels, position coordinates, and instance boundary information for each defect, and store them as detection outputs.

[0039] Preferably, in step S4, "collect the moisture content of the outer and inner regions of the wooden board, weight and integrate them into the overall moisture content feature, and use the ARIMA and LSTM models to model and predict the historical trend of the overall moisture content." The specific implementation method is as follows:

[0040] S41: Collect the moisture content of the outer and inner regions of the wooden board respectively. By assigning weights to each sampling point, calculate the weighted average value to obtain the overall moisture content of the outer and inner sides, and finally integrate it into the input feature of the overall moisture content.

[0041] S42: The input data includes the moisture content of the outer and inner sides of the wood and the current environmental temperature and humidity. These data constitute the basic input vector X of the model t =[H 外侧，t ,H 内侧，t ,T,H env .

[0042] H 外侧，t ,H 内侧，t respectively represent the weighted moisture content of the outer and inner sides at the current moment. T represents the current environmental temperature, and H env represents the current environmental humidity.

[0043] S43: The encoder maps the input vector X t to the mean μ and standard deviation σ of the latent feature distribution, and generates the latent variable z = μ + σ·ε through the reparameterization trick. The VAE decoder receives the latent variable z and the input vector X t , and generates the pseudo-historical time series of the moisture content of the outer and inner sides: The set of preliminary pseudo-historical data on the outer side is: and the set of preliminary pseudo-historical data on the inner side is: After that, generate data by introducing the prior constraint of the physical model to make the pseudo-historical sequence more in line with the actual law of moisture absorption and desorption of wood. After introducing the physical model formula, the moisture content of the inner and outer sides is as follows:

[0044]

[0045] In the formula: H env is the current environmental humidity; α is the moisture absorption / desorption rate; β is the moisture diffusion rate; λ and γ are the constraint weights of the outer and inner side equilibrium VAE generated data and the physically corrected data respectively.

[0046] S44: Integrate the pseudo-historical data generated by VAE into the time series of the overall moisture content and input it into the ARIMA model. Capture the linear trend and periodic changes through difference and regression analysis, and output the preliminary prediction value. The calculation method of the time series of the overall moisture content is as follows:

[0047] H 整体,t-1 =αH外侧,t-1 +βH 内侧,t-1

[0048] Wherein: α and β are the weights of the moisture content on the outer and inner sides respectively.

[0049] S45: Combine the current detection data X t with the pseudo-historical time series generated by VAE to form a multi-dimensional input feature sequence. The input is expressed as X = [H 外侧,t , H 内侧,t , H 外侧,t-1 , H 内侧,t-1 , ……]. Input it into the LSTM model for time series modeling, and finally output the predicted value H t+1 , H t+2 , H t+3 , …… to form a predicted moisture content map.

[0050] S46: Integrate the linear trend prediction value output by the ARIMA model and the non-linear dynamic prediction value output by the LSTM model, and generate the final prediction result through the weighted average method. The integration formula is as follows:

[0051] H final = w ARIMA ·H ARIMA + w LSTM ·H LSTM

[0052] Wherein, w ARIMA and w LSTM respectively represent the weight coefficients of the prediction values of the ARIMA and LSTM models.

[0053] Preferably, in step S5, "generate a comprehensive quality score according to the defect detection result and the moisture content prediction index, complete the wood grade classification, and send the wood of different grades to the designated storage or processing area.

[0054] S51: Through the output of the defect detection model, score the defects on the wood surface and inside. The defects are divided into different types, and each type is given a weight W k . The calculation formula for defect scoring is as follows:

[0055]

[0056] Wherein, W k represents the weight of the kth type of defect; λ represents the weight factor of the internal defect (λ > 1);

[0057] A surface,k represents the surface area of the kth type of defect; A internal,k represents the internal projected area of the kth type of defect; A totalRepresents the total surface area of the wood.

[0058] S52: Based on the moisture content detection data, combined with the ideal moisture content range of the wood, calculate the moisture content score. The moisture content score is as follows:

[0059]

[0060] In the formula, H represents the actual moisture content of the wood; H opt represents the ideal moisture content determined by the wood material; [H min , H max represents the allowable moisture content range;

[0061] S53: Combine the defect score and the moisture content score to generate the final comprehensive score of the wood, which is used to reflect the overall quality of the wood. The scoring formula is as follows:

[0062] S final =(1 - S defect )·S moisture ·100

[0063] S54: According to the comprehensive score S final classify the wood grades:

[0064] Superior grade: S final ≥90; Good grade: 80 ≤ S final <90; Qualified grade: 60 ≤ S final <80; Defective grade: S final <60

[0065] Advantageous effects: Compared with the prior art, the present invention has the following advantages:

[0066] (1) The present invention integrates a visible light camera, an infrared thermal imager, a near-infrared spectroscopy sensor, an ultrasonic detector, and a microwave density sensor through a multi-modal information acquisition module, uses a hardware trigger to achieve synchronous data acquisition, and combines a modal mask mechanism to eliminate low-quality pixel points, improving the reliability of the data and providing high-quality input for subsequent analysis;

[0067] (2) The present invention combines a multi-branch convolutional neural network (CNN) and a feature pyramid network (FPN) to extract and fuse features from multi-modal data, enhancing modal complementarity and significantly improving the positioning and segmentation accuracy of defects such as wood cracks, wormholes, and knots;

[0068] (3) The present invention uses a pseudo-historical time series generation mechanism, combines ARIMA and LSTM models, accurately measures the moisture content of the wood, and achieves high-precision prediction of future changes in the moisture content, improving the accuracy of wood performance evaluation;

[0069] (4) The present invention realizes intelligent grading and diversion of wood based on comprehensive scoring, optimizing production efficiency and automation level to meet the actual industrial requirements;

[0070] (5) Through the multi-modal information processing module and instance segmentation technology, the present invention generates category labels, position coordinates and instance boundaries for wood defects, realizing fine-grained analysis of wood defects and improving the comprehensiveness and accuracy of quality inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a schematic diagram of the overall structure of the present invention;

[0072] Figure 2 is a schematic diagram of the internal structure of the present invention;

[0073] Figure 3 is a flowchart of the defect detection algorithm of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] The present invention will be further clarified below with reference to the drawings and specific embodiments. These embodiments are implemented on the premise of the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0075] As shown in the figure, this embodiment provides a multi-modal intelligent wood defect detection and performance evaluation system, which includes a housing 1, a wood conveying module, a multi-modal data acquisition module, a multi-modal information processing module, and a wood grading and sorting module. The housing 1 is composed of a metal frame and a protective panel, and several LED light sources and sensor brackets are alternately installed on the inner top surface; the inner surface of the housing 1 is coated with black baking paint with a high light absorption rate. The wood conveying module includes a black rubber track 5, a motor, and an encoder drive. The black rubber track 5 has a width of 1.5 meters and is made of high-strength frosted PVC material. Anti-slip side strips are provided on both sides of the track, which can effectively absorb stray light in each spectral band and avoid interference from light pollution. In the multi-modal data acquisition module, several LED light sources are evenly distributed on the inner top surface of the housing 1. The first LED light source 41 is located exactly in the middle between the first bracket 11 and the second bracket 12; the second LED light source 42 is located exactly in the middle between the second bracket 12 and the third bracket 13; the third LED light source 43 is located exactly in the middle between the third bracket 13 and the fourth bracket 14; the fourth LED light source 44 is located exactly in the middle between the fourth bracket 14 and the fifth bracket 15; the first LED light source 41 and the second LED light source 42 are installed to rotate 2 to 6 degrees in opposite directions with the second bracket 12 as the symmetry axis; the second LED light source 42 and the third LED light source 43 are installed to rotate 2 to 6 degrees in opposite directions with the third bracket 13 as the symmetry axis; the third LED light source 43 and the fourth LED light source 44 are installed to rotate 2 to 6 degrees in opposite directions with the fourth bracket 14 as the symmetry axis. The multi-modal information processing module includes an industrial control computer 3. The industrial control computer 3 is installed on the side of the housing 3 and is connected to each sensor through a 10-gigabit Ethernet. LabVIEW is used to simultaneously control the visible light camera 21, the infrared thermal imager 22, the near-infrared spectral sensor 23, the ultrasonic detector 24, and the microwave density sensor 25, and a hardware trigger is combined to achieve synchronous data acquisition in the hard trigger mode; the feature processing unit includes an image acquisition card and a GPU. The wood grading and sorting module includes a diversion chute 61, a diversion conveyor belt 62, and a storage box 63. The chute 61 guides the wood to different grades of conveyor belts 62 through angle adjustment and finally stores it in the storage box 63. The conveyor belt 62 has a width of 1.5 meters and is provided with anti-slip treatment on its surface.

[0076] The method used during detection is as follows: Place the wood to be detected on the black rubber track 5, and the wood conveying module moves the wood forward along the track. When the wood reaches the specified position, the multi-modal data acquisition module starts to work. The visible light camera 21, infrared thermal imager 22, near-infrared spectroscopy sensor 23, ultrasonic detector 24, and microwave density sensor 25 respectively collect data on the wood. Use LabVIEW to control the camera, thermal imager, detector, and sensor simultaneously, and combine with a hardware trigger to achieve synchronous data acquisition in the hard trigger mode. These devices are all fixed on the corresponding brackets, and several LED light sources are on both sides of the sensor bracket. Centered on the sensor bracket, they are installed to rotate 2 to 6 degrees in opposite directions perpendicular to the direction of the black rubber track, which can effectively supplement the light source at the defective parts of the wood surface and ensure that each acquisition device can accurately collect data on the wood. Then, perform preprocessing steps such as denoising, normalization, coordinate registration, and modal mask processing on the collected data; among them, "modal mask processing" is performed on the single-modal feature map F output by the neural network m for each pixel value F m,i in it, and calculate two pixel scoring metrics respectively: absolute value / response quality and entropy value. Compare the scores with the preset threshold. The pixel points exceeding the threshold retain the original value, and the pixel points not reaching the threshold are assigned a value of 0, and finally generate a feature map after mask processing. The scoring calculation formula for absolute value / response quality is as follows:

[0077] Q m,i =F m,i

[0078] The scoring calculation formula for entropy value is as follows:

[0079] Q m,i =-F m,i ·log(F m,i )

[0080] First, pass the feature map output after preprocessing and modal mask processing through a convolutional neural network (CNN), and then input it into a feature pyramid network (FPN). Through the top-down feature propagation and lateral connection mechanism, fuse the high-resolution and low-resolution feature maps level by level to generate a unified feature map with multi-scale information, and separate the feature sub-maps F of each modality from it i,j . Introduce a pixel-level dynamic weighting mechanism, and optimize the modal features by calculating the weight W i,j of each pixel point in the feature sub-map F i,k,j . The weight calculation formula is as follows:

[0081]

[0082] In the formula, F i,k,j represents the unified feature map F iThe value of the k-th mode at pixel point j=(x,y); Used to amplify the significant feature response; Represents the local gradient information of pixel point (x,y), used to measure feature changes; α, β, γ are weight coefficients dynamically learned through training.

[0083] Finally, the optimized feature map is obtained:

[0084] F′ i,k,j =W i,k,j ·F i,k,j

[0085] Finally, the optimized single-modal feature sub-map F′ is generated i,k And through channel fusion operation, the optimized multi-modal fusion feature sub-map F i ' is generated; then through feature splicing operation, a high-dimensional feature tensor is generated, and then successively through batch normalization, layer normalization and dimensionality reduction processing, finally a unified high-dimensional feature vector is generated.

[0086] The generated unified high-dimensional feature vector is input into the defect detection model, and the YOLO algorithm is used to simultaneously complete candidate box generation and defect classification, calibrate the areas where defects such as cracks, insect eyes, and knots may exist, and assign class labels (such as cracks, insect eyes, knots, etc.) to each candidate box; then use the ResNet classification network to further perform pixel-level segmentation on the defect areas within the candidate boxes, accurately divide the shapes and boundaries of the defects, and generate defect masks; then perform instance segmentation on the segmented defect areas, distinguish different defect instances of the same category, generate independent segmentation masks and boundary information for each instance; finally, generate class labels, position coordinates, instance boundary information for each type of defect according to the results of instance segmentation, and store them as detection outputs.

[0087] In the process of the instance segmentation task, after generating a fixed-size feature map through ROIAlign and before entering the classification, regression and segmentation branches, the modal features are optimized through a multi-modal information dynamic weighted fusion mechanism. First, the feature sub-maps F k of each modality are split from the unified feature map F, and the global dynamic weight W k of each modality is calculated respectively. Then the calculated weight W k is combined with the corresponding modal feature sub-map F k to perform weighted optimization on the modal features and generate the optimized modal feature sub-map F k '. Finally, all the optimized modal feature sub-maps are re-fused into a unified optimized feature map F fused , and used as the input for the subsequent classification, regression and segmentation branches. The global dynamic weight calculation formula is as follows:

[0088]

[0089] In the formula: F k represents the k-th modal feature map after ROIAlign; ||F k || represents the global average intensity of the modal feature; Var(F k ) represents the global variance of the modal feature; represents the global gradient mean of the modal feature; α, β, γ are weighted coefficients dynamically learned through training

[0090] Predict the moisture content of the wooden board: Collect the moisture content of the outer and inner regions of the wooden board respectively. By assigning weights to each sampling point, calculate the weighted average value to obtain the overall moisture content of the outer and inner sides, and finally integrate it into the input feature of the overall moisture content; The input data includes the moisture content of the outer and inner sides of the wood and the current environmental temperature and humidity. These data constitute the basic input vector X t =[H 外侧，t ,H 内侧，t ,T,H env . H 外侧，t ,H 内侧，t respectively represent the outer and inner weighted moisture contents at the current moment. T represents the current environmental temperature, and H env represents the current environmental humidity. The encoder maps the input vector X t to the mean μ and standard deviation σ of the latent feature distribution, and generates the latent variable z = μ + σ·ε through the reparameterization trick. The VAE decoder receives the latent variable z and the input vector X t , and generates the pseudo-historical time series of the outer and inner moisture contents: The set of preliminary pseudo-historical data for the outer side is: and the set of preliminary pseudo-historical data for the inner side is: After that, generate data by introducing the prior constraint of the physical model to make the pseudo-historical sequence more in line with the actual laws of wood moisture absorption and desorption. After introducing the physical model formula, the inner and outer moisture contents are as follows:

[0091]

[0092] In the formula: H env is the current environmental humidity; α is the moisture absorption / desorption rate; β is the moisture diffusion rate; λ and γ are the constraint weights for balancing the VAE-generated data and the physically corrected data on the outer and inner sides respectively.

[0093] Then, integrate the pseudo-historical data generated by the VAE into the time series of the overall moisture content and input it into the ARIMA model. Capture the linear trend and periodic changes through differencing and regression analysis, and output the preliminary prediction value. The calculation method of the time series of the overall moisture content is as follows:

[0094] H整体,t-1 = αH 外侧,t-1 + βH 内侧,t-1

[0095] where α and β are the weights of the moisture content on the outer and inner sides, respectively.

[0096] Then, combine the current detection data X t with the pseudo-historical time series generated by the VAE to form a multi-dimensional input feature sequence. The input is represented as X = [H 外侧,t , H 内侧,t , H 外侧,t-1 , H 内侧,t-1 , ……]. Input it into the LSTM model for time series modeling, and finally output the predicted values of the moisture content H t+1 , H t+2 , H t+3 , …… to form a predicted moisture content map. Finally, integrate the linear trend predicted value output by the ARIMA model and the non-linear dynamic predicted value output by the LSTM model, and generate the final prediction result through the weighted average method. The integration formula is as follows:

[0097] H final = w ARIMA ·H ARIMA + w LSTM ·H LSTM

[0098] where w ARIMA and w LSTM represent the weight coefficients of the predicted values of the ARIMA and LSTM models, respectively.

[0099] Score the defects on the wood surface and inside through the output of the defect detection model. Classify the defects into different types, and assign a weight W k to each type. The calculation formula for the defect score is as follows:

[0100]

[0101] where W k represents the weight of the k-th type of defect; λ represents the weight factor of the internal defect (λ > 1);

[0102] A surface,k represents the surface area of the k-th type of defect; A internal,k represents the internal projected area of the k-th type of defect; A total represents the total surface area of the wood.

[0103] Based on the moisture content detection data, combine with the ideal moisture content range of the wood to calculate the moisture content score. The moisture content score is as follows:

[0104]

[0105] In the formula, H represents the actual moisture content of the wood; H opt represents the ideal moisture content determined by the wood material; [H min ,H max represents the allowable moisture content range.

[0106] Combining the defect score and the moisture content score to generate the final comprehensive score of the wood, which is used to reflect the overall quality of the wood. The scoring formula is as follows:

[0107] S final =(1 - S defect )·S moisture ·100

[0108] According to the comprehensive score S final classify the wood grades:

[0109] Superior grade: S final ≥90; Good grade: 80 ≤ S final <90; Qualified grade: 60 ≤ S final <80; Defective grade: S final <60

[0110] Finally, according to the wood grades classified by the comprehensive score, guide the woods of different grades to the designated storage areas through the shunt conveyor belt.

[0111] The above specific implementation manners are only a preferred embodiment of the present invention, and are not used to limit the implementation and the scope of the claims of the present invention. Any equivalent changes and modifications made according to the content of the patent protection scope of the present invention shall be included within the scope of the patent application of the present invention.

Claims

1. A multimodal intelligent wood defect detection and performance evaluation system, characterized in that: The invention comprises a housing (1), a wood conveying module, a multimodal data acquisition module, a multimodal information processing module, and a wood grading and sorting module; the housing (1) is composed of a metal frame and a protective panel, and a plurality of LED light sources and sensor brackets are alternately installed on the internal top surface; the wood conveying module comprises a black rubber track (5), a motor and an encoder drive; the multimodal data acquisition module comprises a visible light camera (21) fixed to a first bracket (11), an infrared thermal imager (22) fixed to a second bracket (12), and a near infrared spectrometer (23) fixed to a third bracket (13). The multimodal information processing module comprises an industrial control computer (3); the industrial control computer (3) is connected to a visible light camera (21), an infrared thermal imager (22), a near infrared spectrum sensor (23), an ultrasonic detector (24) and a microwave density sensor (25); the wood grading and sorting module comprises a diversion slideway (61), a diversion conveyor belt (62) and a storage box (63).

2. The multimodal intelligent wood defect detection and performance evaluation system according to claim 1 is characterized in that: The black rubber track (5) is made of high-strength frosted PVC material, and is provided with anti-slip edge strips on the surface for absorbing stray light and improving the sensor detection accuracy; each LED light source is symmetrically distributed with the sensor bracket as the center and the angle is adjusted to optimize the light source coverage.

3. The multimodal intelligent wood defect detection and performance evaluation system according to claim 1 is characterized in that: The multimodal information processing module controls the hardware trigger through LabVIEW, synchronously collects visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals and microwave density data, and performs denoising, normalization, coordinate registration and modal mask processing on the collected data.

4. A multimodal intelligent wood defect detection and performance evaluation method, characterized in that: The multimodal intelligent wood defect detection and performance evaluation system according to any one of claims 1 to 3 is used, comprising the following steps: S1: Use multimodal sensors to synchronously collect visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data of wood, and perform data preprocessing; S2: The preprocessed multimodal data is input into a multi-branch convolutional neural network, combined with a feature pyramid network to extract multi-scale features, and a unified high-dimensional feature vector is generated through dynamic weighted fusion; S3: Based on the high-dimensional feature vector, a defect detection model is constructed, and pixel-level segmentation and instance segmentation are performed to output defect category, location and boundary information; S4: Collect moisture content data of the outer and inner sides of wood, generate pseudo historical time series through variational autoencoder, and combine ARIMA and LSTM models to predict future moisture content; S5: Generate a comprehensive quality score based on the defect score and moisture content score, divide the wood into grades and complete the sorting through the diversion device.

5. The multimodal intelligent wood defect detection and performance evaluation method according to claim 4 is characterized in that: The step S1 further comprises: S11: Use LabVIEW to simultaneously control industrial RGB cameras, infrared thermal imagers, hyperspectral cameras, ultrasonic detectors, and microwave density sensors, and combine hardware triggers to achieve synchronous data acquisition in hard trigger mode; S12: Preprocessing the collected data including denoising, normalization, coordinate registration and modal mask processing, wherein the modal mask processing is performed on the single modal feature map F output by the neural network. m The value of each pixel in F m,i Two pixel scoring indicators are calculated separately: absolute value / response quality and entropy value. The score is compared with the preset threshold. Pixels exceeding the threshold retain the original value, and pixels below the threshold are assigned a value of 0. Finally, a masked feature map is generated. The absolute value / response quality score calculation formula is as follows: Absolute value / response quality score: Q m,i =F m,i Entropy Rating: Q m,i =-F m,i ·log(F m,i )。 6. The multimodal intelligent wood defect detection and performance evaluation method according to claim 4 is characterized in that: The step S2 further comprises: S21: The feature map output from S1 is first passed through the convolutional neural network and then input into the feature pyramid network. Through the top-down feature propagation and lateral connection mechanism, the high-resolution and low-resolution feature maps are fused step by step to generate a unified feature map with multi-scale information. S22: From the unified feature map F i The characteristic subgraph F of each mode is separated i,j , introduces a pixel-level dynamic weighting mechanism, by calculating the feature subgraph F i,j The weight of each pixel W i,k,j , the modal features are optimized, and the weight calculation formula is: In the formula, F i,k,j Denotes the unified feature map F i The value of the kth mode at pixel j = (x, y), Used to amplify significant feature responses, ▽F i,k,j Represents the local gradient information of the pixel point (x, y), which is used to measure feature changes. α, β, and γ are weight coefficients obtained through dynamic learning in training; Get the optimized feature map: F′ i,k,j =W i,k,j ·F i,k,j ; Generate optimized unimodal feature subgraph F′ i,k And generate the optimized multi-modal fusion feature subgraph F through channel fusion operation i '; S23: Feature subgraph F of multimodal fusion i 'A high-dimensional feature tensor is generated through feature concatenation, followed by batch normalization, layer normalization and dimensionality reduction, to finally generate a unified high-dimensional feature vector.

7. The multimodal intelligent wood defect detection and performance evaluation method according to claim 4 is characterized in that: The step S3 further comprises: S31: Input the unified high-dimensional feature vector into the defect detection model, use the YOLO algorithm to simultaneously complete candidate box generation and defect classification, calibrate areas where defects such as cracks, wormholes, and knots may exist, and assign a category label to each candidate box; S32: Based on the classification, the defect area in the candidate box is further segmented at the pixel level using the ResNet classification network to accurately divide the shape and boundary of the defect and generate a defect mask; S33: Perform instance segmentation on the defective area after segmentation, distinguish different defect instances of the same category, and generate independent segmentation masks and boundary information for each instance; in the process of instance segmentation task, after generating a fixed-size feature map through ROIAlign, before entering the classification, regression and segmentation branches, the modal features are optimized through the dynamic weighted fusion mechanism of multimodal information, and the feature sub-maps F of each modality are split from the unified feature map F k , calculate the global dynamic weight W of each mode separately k , the calculated weight W k And the corresponding modal feature subgraph F k Combined with the modal features, the modal features are weighted optimized to generate the optimized modal feature subgraph F′ k , all optimized modal feature sub-graphs are re-fused into a unified optimized feature graph F fused , and used as the input of subsequent classification, regression and segmentation branches. The global dynamic weight calculation formula is as follows: Where: F k represents the k-th modal feature map after ROIAlign, ||F k || represents the global average strength of the modal feature, Var(F k ) represents the global variance of the modal feature, ▽||F k || represents the global gradient mean of the modal feature; α, β, γ are weighting coefficients dynamically learned through training; S34: Generate category labels, location coordinates, and instance boundary information for each defect and store them as detection output.

8. The multimodal intelligent wood defect detection and performance evaluation method according to claim 4 is characterized in that: The step S4 further comprises: S41: The moisture contents of the outer and inner areas of the wooden board are collected separately, and a weight is assigned to each sampling point, and the weighted average is calculated to obtain the overall moisture contents of the outer and inner sides, which are finally integrated into the input features of the overall moisture content; S42: Input data includes the moisture content of the outer and inner sides of the wood and the temperature and humidity of the current environment, which constitute the basic input vector X of the model. t =[H 外侧，t ,H 内侧，t , T,H env ], H 外侧，t ,H 内侧，t They represent the weighted moisture content of the outer and inner sides at the current moment, T represents the current ambient temperature, H env Indicates the current ambient humidity; S43: The encoder takes the input vector X t Mapped to the mean μ and standard deviation σ of the potential feature distribution, and generate the latent variable z = μ + σ · ε through the reparameterization technique. The VAE decoder receives the latent variable z and the input vector X t , generate pseudo-historical time series of outer and inner water contents: The set of preliminary pseudo-historical data for the outer side is: The set of preliminary pseudo-historical data inside is: By introducing the physical model prior constraints to generate data, the pseudo-history sequence is made more consistent with the actual law of wood moisture absorption and dehumidification. After introducing the physical model formula, the moisture content of the inside and outside is as follows: Where: H env is the current ambient humidity, α is the moisture absorption / dehumidification rate, β is the moisture diffusion rate, λ and γ are the constraint weights of the outer and inner balanced VAE generated data and the physical correction data respectively; S44: Integrate the pseudo historical data generated by VAE into the time series of the overall moisture content and input it into the ARIMA model. Capture the linear trend and cyclical changes through difference and regression analysis, and output the preliminary prediction value. The time series calculation method of the overall moisture content is as follows: H 整体,t-1 =αH 外侧,t-1 +βH 内侧,t-1 Where: α and β are the weights of the moisture content of the outer side and the inner side respectively; S45: The current detection data X t Combined with the pseudo historical time series generated by VAE, a multi-dimensional input feature sequence is formed. The input is represented as X = [H 外侧,t ,H 内侧,t ,H 外侧,t-1 ,H 内侧,t-1 ,……], input it into the LSTM model for time series modeling, and finally output the predicted value H of water content at the future moment t+1 ,H t+2 ,H t+3 ,……Composition prediction moisture content diagram; S46: Integrate the linear trend prediction value output by the ARIMA model with the nonlinear dynamic prediction value output by the LSTM model, and generate the final prediction result through the weighted average method. The integration formula is as follows: H final =w ARIMA ·H ARIMA +w LSTM ·H LSTM In the formula, w ARIMA and w LSTM Represent the weight coefficients of the predicted values ​​of ARIMA and LSTM models respectively.

9. The multimodal intelligent wood defect detection and performance evaluation method according to claim 4 is characterized in that: The step S5 further comprises: S51: The defects on the surface and inside of the wood are scored through the output of the defect detection model, and the defects are divided into different types, each type is assigned a weight W k , the defect score is calculated as follows: Where W k represents the weight of the k-th defect; λ represents the weight factor of internal defects (λ>1); A surface,k A represents the surface area of ​​the kth type of defect; internal,k Represents the internal projection area of ​​the kth type of defect; A total represents the total surface area of ​​wood; S52: Based on the moisture content test data and the ideal moisture content range of the wood, the moisture content score is calculated; the moisture content score is as follows: In the formula, H represents the actual moisture content of the wood; H opt Indicates the ideal moisture content determined by the wood material; [H min ,H max ] indicates the allowable moisture content range; S53: The defect score and moisture content score are combined to generate the final comprehensive score of the wood, which is used to reflect the overall quality of the wood. The scoring formula is as follows: S final =(1-S defect )·S moisture ·100。 10. The multimodal intelligent wood defect detection and performance evaluation method according to claim 9, It is characterized in that The timber grading standards are: Superior product: S final ≥90; Good: 80≤S final <90; Qualified products: 60≤S final <80; Defective: S final <60.

Images