Multimodal intelligent wood defect detection and performance evaluation system and method

Abstract

The application discloses a kind of multi-modal intelligent wood defect detection and performance evaluation system and detection evaluation method, integrates a variety of sensing technologies and deep learning algorithm, realize wood defect detection and quality evaluation.System includes wood transmission, multi-modal data acquisition, information processing and intelligent grading module, obtains wood surface and internal characteristics by visible light, infrared, near infrared, ultrasonic and microwave sensor.Utilize feature pyramid network (FPN) and instance segmentation technology accurately identify crack, borer eye, knot and other defects, and combine ARIMA and LSTM model to predict moisture content, improve quality evaluation accuracy.The application can intelligently classify wood, improve detection efficiency, applicable to wood processing, furniture manufacturing and other fields.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a multimodal intelligent wood defect detection and performance evaluation system and method. Background Technology

[0002] Wood, a crucial material widely used in the construction, furniture, and decoration industries, directly impacts product performance and market value. However, wood is prone to various defects during production and processing, such as cracks, wormholes, and knots. Furthermore, the moisture content and density of wood are important indicators affecting its mechanical properties. Therefore, achieving efficient detection of wood defects and physical properties has always been a pressing technical challenge for the industry.

[0003] Traditional wood defect detection primarily relies on manual visual inspection, which is limited by slow inspection speed, human fatigue, strong subjectivity, and low recognition rate of minute defects, making it difficult to meet the high efficiency and high precision requirements of modern industrial production. In recent years, with the rapid development of computer vision, sensor technology, and deep learning algorithms, automatic wood defect detection schemes based on image processing technology have gradually emerged. However, these schemes typically focus only on single-modal data, failing to comprehensively utilize wood surface images, internal structural information, and physical property data, resulting in limited accuracy and reliability of the detection results.

[0004] Therefore, there is an urgent need for a multimodal intelligent wood defect detection and performance evaluation system and method that can effectively combine various information such as visible light images, infrared thermal imaging, near-infrared spectroscopy, ultrasonic signals and microwave density to comprehensively improve the accuracy of wood defect detection and the level of automation in the detection process, and provide strong support for wood processing and quality control.

[0005] Chinese patent document CN202411707363A discloses a "method, system, medium, and device for detecting surface defects in wood." It employs the following modules: a preprocessing module for acquiring and preprocessing eucalyptus veneer images; a first feature map acquisition module for extracting feature maps from the eucalyptus veneer images using an inverse depth-separable stem module and performing feature aggregation; a frequency feature enhancement module for extracting key frequency information from the feature maps and restoring local details using a rectangular self-calibrating adaptive frequency attention network and a frequency enhancement channel attention module; and a bark defect region display module for inputting the final feature map into a segmentation head to output the identified bark defect regions. This method, combined with frequency feature enhancement technology, improves the detection accuracy of bark defects, but it mainly targets single defect types (such as bark defects) and lacks the comprehensive detection capability for multiple complex defects and wood physical properties (such as moisture content and density). Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a multimodal intelligent wood defect detection and performance evaluation method. This method integrates visible light images, infrared thermal imaging images, near-infrared spectral data, ultrasonic signals, and microwave density data. Through multimodal feature alignment and dynamic weighted fusion, it comprehensively achieves accurate detection of wood surface defects, internal defects, and moisture content.

[0007] Technical Solution: The technical solution adopted by this invention to solve the problem is as follows: A multimodal intelligent wood defect detection and performance evaluation system includes a shell, a wood transport module, a multimodal data acquisition module, a multimodal information processing module, and a wood grading and sorting module; multimodal information of wood is acquired through a visible light camera, an infrared thermal imager, a near-infrared spectral sensor, an ultrasonic detector, and a microwave density sensor; the multimodal information processing module is connected to the multimodal data acquisition module and is used to process the multimodal information; the wood transport module uses a black rubber track to transport the wood to the multimodal data acquisition module.

[0008] Preferably, the multimodal data acquisition module includes multiple sensors, wherein a visible light camera acquires surface texture information of the wood, an infrared thermal imager is used to detect internal defects and temperature distribution in the wood, a near-infrared spectral sensor is used to analyze the chemical composition of the wood, and an ultrasonic detector and a microwave density sensor are used to measure the moisture content and density of the wood. The sensors are installed in a modular design to ensure the diversity and comprehensiveness of the information acquired.

[0009] Preferably, the multimodal data acquisition module employs an alternating installation of several LED light sources and sensor brackets to ensure uniform light distribution and symmetrical distribution on both sides of the sensor. The angle of each LED light source can be adjusted as needed to optimize the acquisition effect of different modal data.

[0010] Preferably, the black rubber track is made of high-strength frosted PVC material, which not only ensures good friction and grip, but also effectively absorbs stray light in various spectral bands, avoiding light pollution interference, thereby improving the accuracy of the sensor in detecting wood defects.

[0011] This invention also provides a detection method for multimodal intelligent wood defect detection and performance evaluation. By acquiring visible light images, infrared thermograms, near-infrared spectroscopy, ultrasonic signals, and microwave density data, and comprehensively utilizing the complementary characteristics of multimodal data, a comprehensive analysis of wood surface defects, internal moisture content, and thickness is achieved. High-precision data synchronization and preprocessing techniques ensure the consistency of multimodal data. Features are extracted and fused using a multi-branch convolutional neural network to generate a unified feature vector, constructing an accurate defect detection and quality assessment model, thereby realizing automatic grading and intelligent processing of wood. The specific steps include:

[0012] S1: Read visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data into the image processing module.

[0013] S2: Input multimodal data into a multi-branch convolutional neural network (CNN), combine it with a feature pyramid network (FPN) to extract features, and generate a unified high-dimensional feature vector through feature alignment and fusion.

[0014] S3: Based on a unified high-dimensional feature vector, a defect detection model is constructed to locate, classify, segment, and generate instance boundaries for wood surface defects, thereby improving the accuracy of defect recognition.

[0015] S4: Collect and weight the moisture content of the outer and inner areas of the wooden board to form the overall moisture content feature, and use ARIMA and LSTM models to model and predict the historical trend of the overall moisture content.

[0016] S5: Generate a comprehensive quality score based on defect detection results and moisture content prediction, and complete the timber grading, then distribute timber of different grades to designated storage or processing areas.

[0017] Preferably, the specific implementation method of "reading visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data into the image processing module" in step S1 is as follows:

[0018] S11: Using LabVIEW to simultaneously control an industrial RGB camera, infrared thermal imager, hyperspectral camera, ultrasonic detector, and microwave density sensor, and combining hardware triggers to achieve synchronous data acquisition in hard-triggered mode.

[0019] S12: Perform preprocessing steps on the acquired data, including denoising, normalization, coordinate registration, and modal masking; among which, "modal masking" involves processing the single-modal feature map output by the neural network. The value of each pixel in Two pixel scoring metrics are calculated: absolute value / response quality and entropy value. The scores are compared with preset thresholds; pixels exceeding the threshold retain their original values, while pixels below the threshold are assigned a value of 0. The final result is a masked feature map. The formula for calculating the absolute value / response quality score is as follows:

[0020]

[0021] The formula for calculating the entropy value is as follows:

[0022]

[0023] As a preferred embodiment, the specific implementation method of "inputting multimodal data into a multi-branch convolutional neural network (CNN), extracting features by combining it with a feature pyramid network (FPN), and generating a unified high-dimensional feature vector through feature alignment and fusion" in step S2 is as follows: S21: The feature map output from S1 is first passed through a convolutional neural network (CNN), and then input into a feature pyramid network (FPN). Through top-down feature propagation and lateral connection mechanisms, the high-resolution and low-resolution feature maps are fused step by step to generate a unified feature map with multi-scale information.

[0024] S22: From the unified feature map The feature sub-graphs of each mode are extracted from the middle. A pixel-level dynamic weighting mechanism is introduced, which calculates feature sub-maps. Weight of each pixel Modal features are then optimized. The weight calculation formula is as follows:

[0025]

[0026] In the formula, Representing a unified feature map The k-th mode at pixel point The value at; Used to amplify salient feature responses; Represents pixels Local gradient information is used to measure feature changes; These are the weight coefficients obtained through dynamic learning during training.

[0027] The final optimized feature map is obtained:

[0028]

[0029] Finally, the optimized single-modal feature map is generated. And an optimized multimodal fusion feature map is generated through channel fusion operation. .

[0030] S23: Feature sub-map of multimodal fusion A high-dimensional feature tensor is generated by feature concatenation, and then subjected to batch normalization, layer normalization and dimensionality reduction processing to finally generate a unified high-dimensional feature vector.

[0031] As a preferred embodiment, the specific implementation method of "constructing a defect detection model based on a unified high-dimensional feature vector to locate, classify, segment, and generate instance boundaries for wood surface defects, thereby improving defect recognition accuracy" in step S3 is as follows:

[0032] S31: Input a unified high-dimensional feature vector into the defect detection model, and use the YOLO algorithm to simultaneously generate candidate boxes and classify defects, identify areas that may have defects such as cracks, wormholes, and knots, and assign a category label (such as cracks, wormholes, knots, etc.) to each candidate box.

[0033] S32: Using the ResNet classification network, further pixel-level segmentation is performed on the defect region within the candidate box to accurately delineate the shape and boundary of the defect and generate a defect mask.

[0034] S33: Perform instance segmentation on the segmented defect region, distinguishing different defect instances of the same category, and generating an independent segmentation mask and boundary information for each instance; in the instance segmentation task process, after generating a fixed-size feature map through ROI Align, before entering the classification, regression, and segmentation branches, optimize the modal features through a dynamic weighted fusion mechanism of multimodal information. First, from the unified feature map... Feature sub-graphs of each mode are extracted from the middle. Calculate the global dynamic weights for each mode. Next, the calculated weights will be... With the corresponding modal feature subgraph By combining these methods, the modal features are weighted and optimized to generate optimized modal feature subgraphs. Finally, all optimized modal feature submaps are re-fused into a unified optimized feature map. This weight is then used as input for subsequent classification, regression, and segmentation branches. The global dynamic weight calculation formula is as follows:

[0035]

[0036] In the formula: This represents the k-th modal feature map after ROI Align; Represents the global average intensity of modal features; The global variance representing modal features; The global gradient mean representing modal features; Weighting coefficients learned through training dynamically

[0037] S34: Generate category labels, location coordinates, and instance boundary information for each defect, and store them as detection output.

[0038] Preferably, step S4, which involves "collecting and weighting the moisture content data from the outer and inner regions of the wooden board to form an overall moisture content feature, and then using ARIMA and LSTM models to model and predict the historical trend of the overall moisture content," is implemented as follows:

[0039] S41: Collect the moisture content of the outer and inner areas of the wooden board separately, assign weights to each sampling point, calculate the weighted average value to obtain the overall moisture content of the outer and inner areas, and finally integrate them into the input feature of the overall moisture content.

[0040] S42: The input data includes the moisture content of the outer and inner surfaces of the wood, as well as the current ambient temperature and humidity. This data forms the basic input vector of the model. . These represent the outer and inner weighted water content at the current moment, respectively. Indicates the current ambient temperature. This indicates the current ambient humidity.

[0041] S43: The encoder will input vector Mapped to the mean of the latent feature distribution and standard deviation And generate latent variables through reparameterization techniques. The VAE decoder receives latent variables. and input vector The pseudo-historical time series of water content on the outer and inner sides are generated: the preliminary set of pseudo-historical data on the outer side is as follows: The set of preliminary pseudo-historical data from the inner side is as follows: Then, by introducing prior constraints from the physical model, data was generated to make the pseudo-historical sequence more consistent with the actual laws of moisture absorption and desiccation in wood. The moisture content of the inner and outer surfaces after introducing the physical model formulas is as follows:

[0042]

[0043]

[0044] In the formula: Current ambient humidity; Moisture absorption / dehumidification rate; This represents the rate of water diffusion. and These are the constraint weights for the outer and inner balancing VAE generated data and the physical correction data, respectively.

[0045] S44: Integrate the pseudo-historical data generated by VAE into a time series of overall moisture content, input it into the ARIMA model, and capture linear trends and periodic changes through differencing and regression analysis to output preliminary predicted values. The time series calculation method for overall moisture content is as follows:

[0046]

[0047] In the formula: and These are the weights for the moisture content on the outer and inner sides, respectively.

[0048] S45: Transfer current detection data Combined with the pseudo-historical time series generated by VAE, a multi-dimensional input feature sequence is formed. The input is represented as... The data is then input into an LSTM model for time series modeling, ultimately outputting the predicted water content at future time points. Composition and predicted moisture content diagram.

[0049] S46: Integrate the linear trend predictions from the ARIMA model with the nonlinear dynamic predictions from the LSTM model, and generate the final prediction result using a weighted average method. The integration formula is as follows:

[0050]

[0051] In the formula, and These represent the weight coefficients of the ARIMA and LSTM model predictions, respectively.

[0052] Preferably, step S5 involves generating a comprehensive quality score based on the defect detection results and moisture content prediction index, completing the timber grading, and distributing timber of different grades to designated storage or processing areas.

[0053] S51: Based on the output of the defect detection model, scores are given for defects on the surface and inside of the wood. Defects are categorized into different types, and each type is assigned a weight. The formula for calculating the defect score is as follows:

[0054]

[0055] In the formula, Indicates the first Weights of class defects; Weighting factors representing internal defects ;

[0056] Indicates the first Surface area of ​​the defect type; Indicates the first Internal projected area of ​​the defect type; This represents the total surface area of ​​the timber.

[0057] S52: Based on moisture content test data and the ideal moisture content range for wood, calculate the moisture content score. The moisture content score is as follows:

[0058]

[0059] In the formula, This indicates the actual moisture content of the wood; This indicates the ideal moisture content, which is determined by the type of wood. Indicates the permissible moisture content range;

[0060] S53: Combines the defect score and moisture content score to generate a final comprehensive score for the timber, reflecting its overall quality. The scoring formula is as follows:

[0061]

[0062] S54: Based on the overall score Timber grading: Superior grade: ;Good product: Qualified products: Defective products:

[0063] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0064] (1) This invention integrates a visible light camera, an infrared thermal imager, a near-infrared spectral sensor, an ultrasonic detector, and a microwave density sensor through a multimodal information acquisition module. It uses hardware triggers to achieve synchronous data acquisition and combines a modal masking mechanism to remove low-quality pixels, thereby improving the reliability of the data and providing high-quality input for subsequent analysis.

[0065] (2) This invention combines multi-branch convolutional neural network (CNN) and feature pyramid network (FPN) to extract and fuse features of multimodal data, enhance modal complementarity, and significantly improve the localization and segmentation accuracy of defects such as wood cracks, wormholes, and knots;

[0066] (3) This invention uses a pseudo-historical time series generation mechanism, combined with ARIMA and LSTM models, to accurately measure the moisture content of wood and achieve high-precision prediction of future changes in moisture content, thereby improving the accuracy of wood performance evaluation.

[0067] (4) The present invention realizes intelligent grading and diversion of wood based on comprehensive scoring, optimizes production efficiency and automation level, and meets the actual needs of industry;

[0068] (5) This invention generates category labels, location coordinates and instance boundaries for wood defects through a multimodal information processing module and instance segmentation technology, thereby realizing fine-grained analysis of wood defects and improving the comprehensiveness and accuracy of quality inspection. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of the overall structure of the present invention;

[0070] Figure 2 This is a schematic diagram of the internal structure of the present invention;

[0071] Figure 3 This is a flowchart of the defect detection algorithm of the present invention. Detailed Implementation

[0072] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solutions of the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0073] As shown in the figure, this embodiment provides a multimodal intelligent wood defect detection and performance evaluation system, including a housing 1, a wood conveying module, a multimodal data acquisition module, a multimodal information processing module, and a wood grading and sorting module. The housing 1 consists of a metal frame and a protective panel, with several LED light sources and sensor supports alternately installed on the internal top surface; the internal surface of the housing 1 is coated with a high-absorption black paint. The wood conveying module includes a black rubber track 5, a motor, and an encoder drive. The black rubber track 5 is 1.5 meters wide and made of high-strength frosted PVC material. Anti-slip strips are provided on both sides of the track to effectively absorb stray light in various spectral bands and avoid light pollution interference. In the multimodal data acquisition module, several LED light sources are evenly distributed on the top surface inside the housing 1. The first LED light source 41 is located between the first support 11 and the second support 12; the second LED light source 42 is located between the second support 12 and the third support 13; the third LED light source 43 is located between the third support 13 and the fourth support 14; and the fourth LED light source 44 is located between the fourth support 14 and the fifth support 15. The first LED light source 41 and the second LED light source 42 are installed with the second support 12 as the axis of symmetry, rotated 2 to 6 degrees in opposite directions. The second LED light source 42 and the third LED light source 43 are installed with the third support 13 as the axis of symmetry, rotated 2 to 6 degrees in opposite directions. The third LED light source 43 and the fourth LED light source 44 are installed with the fourth support 14 as the axis of symmetry, rotated 2 to 6 degrees in opposite directions. The multimodal information processing module includes an industrial control computer 3. The industrial control computer 3 is installed on the side of the housing 3 and is connected to each sensor via a 10 Gigabit Ethernet connection. LabVIEW is used to simultaneously control a visible light camera 21, an infrared thermal imager 22, a near-infrared spectral sensor 23, an ultrasonic detector 24, and a microwave density sensor 25, combined with hardware triggers to achieve synchronous data acquisition in hard-trigger mode; the feature processing unit includes an image acquisition card and a GPU. The wood grading and sorting module includes a diversion chute 61, a diversion conveyor belt 62, and a storage bin 63. The chute 61 guides the wood to different grades of the conveyor belt 62 by angle adjustment, and finally stores it in the storage bin 63. The conveyor belt 62 is 1.5 meters wide and has an anti-slip surface.

[0074] The detection method is as follows: The wood to be detected is placed on the black rubber track 5, and the wood conveying module moves the wood forward along the track. When the wood reaches the designated position, the multimodal data acquisition module starts working, and the visible light camera 21, infrared thermal imager 22, near-infrared spectral sensor 23, ultrasonic detector 24, and microwave density sensor 25 respectively acquire data from the wood. LabVIEW is used to control the camera, thermal imager, detector, and sensor simultaneously, and hardware triggers are used to achieve synchronous data acquisition in hard-triggered mode. These devices are all fixed on corresponding brackets, and several LED light sources are installed on both sides of the sensor bracket, rotated 2 to 6 degrees in opposite directions perpendicular to the direction of the black rubber track, with the sensor bracket as the center. This effectively supplements the light source at the defects on the wood surface, ensuring that each acquisition device achieves accurate data acquisition from the wood. Then, the acquired data is subjected to preprocessing steps such as denoising, normalization, coordinate registration, and modal masking; among them, "modal masking" processes the single-modal feature map output by the neural network. The value of each pixel in Two pixel scoring metrics are calculated: absolute value / response quality and entropy value. The scores are compared with preset thresholds; pixels exceeding the threshold retain their original values, while pixels below the threshold are assigned a value of 0. The final result is a masked feature map. The formula for calculating the absolute value / response quality score is as follows:

[0075]

[0076] The formula for calculating the entropy value is as follows:

[0077]

[0078] The feature maps output after preprocessing and modality masking are first passed through a convolutional neural network (CNN), and then input into a feature pyramid network (FPN). Through top-down feature propagation and lateral connection mechanisms, high-resolution and low-resolution feature maps are fused step by step to generate a unified feature map with multi-scale information. From this, feature sub-maps for each modality are extracted. A pixel-level dynamic weighting mechanism is introduced, which calculates feature sub-maps. Weight of each pixel Modal features are then optimized. The weight calculation formula is as follows:

[0079]

[0080] In the formula, Representing a unified feature map The k-th mode at pixel point The value at; Used to amplify salient feature responses; Represents pixels Local gradient information is used to measure feature changes; These are the weight coefficients obtained through dynamic learning during training.

[0081] The final optimized feature map is obtained:

[0082]

[0083] Finally, the optimized single-modal feature map is generated. And an optimized multimodal fusion feature map is generated through channel fusion operation. Next, a high-dimensional feature tensor is generated through feature concatenation, followed by batch normalization, layer normalization, and dimensionality reduction to finally generate a unified high-dimensional feature vector.

[0084] The generated unified high-dimensional feature vector is input into the defect detection model. The YOLO algorithm is used to simultaneously generate candidate boxes and classify defects, identifying areas where defects such as cracks, wormholes, and knots may exist. Each candidate box is assigned a category label (e.g., crack, wormhole, knot, etc.). Then, based on the classification, the ResNet classification network further performs pixel-level segmentation on the defect regions within the candidate boxes, accurately delineating the shape and boundaries of the defects and generating defect masks. Next, the segmented defect regions are segmented into instances to distinguish different defect instances of the same category, generating independent segmentation masks and boundary information for each instance. Finally, based on the instance segmentation results, category labels, location coordinates, and instance boundary information are generated for each type of defect and stored as detection output.

[0085] In the instance segmentation task process, after generating a fixed-size feature map using ROI Align, and before proceeding to the classification, regression, and segmentation branches, the modal features are optimized through a dynamic weighted fusion mechanism of multimodal information. This optimization begins with a unified feature map. Feature sub-graphs of each mode are extracted from the middle. Calculate the global dynamic weights for each mode. Next, the calculated weights will be... With the corresponding modal feature subgraph By combining these methods, the modal features are weighted and optimized to generate optimized modal feature subgraphs. Finally, all optimized modal feature sub-maps are re-fused into a unified optimized feature map. This weight is then used as input for subsequent classification, regression, and segmentation branches. The global dynamic weight calculation formula is as follows:

[0086]

[0087] In the formula: This represents the k-th modal feature map after ROI Align; Represents the global average intensity of modal features; The global variance representing modal features; The global gradient mean representing modal features; Weighting coefficients learned through training dynamically

[0088] To predict the moisture content of the wood planks: Moisture content data is collected from the outer and inner sides of the planks separately. Each sampling point is weighted, and a weighted average is calculated to obtain the overall moisture content of the outer and inner sides. This average is then integrated into the overall moisture content input feature. The input data includes the moisture content of the outer and inner sides of the wood, as well as the current ambient temperature and humidity. This data constitutes the basic input vector of the model. . These represent the outer and inner weighted water content at the current moment, respectively. Indicates the current ambient temperature. This represents the current ambient humidity. The encoder will input the vector. Mapped to the mean of the latent feature distribution and standard deviation And generate latent variables through reparameterization techniques. The VAE decoder receives latent variables. and input vector The pseudo-historical time series of water content on the outer and inner sides are generated: the preliminary set of pseudo-historical data on the outer side is as follows: The set of preliminary pseudo-historical data from the inner side is as follows: Then, by introducing prior constraints from the physical model, data was generated to make the pseudo-historical sequence more consistent with the actual laws of moisture absorption and desiccation in wood. The moisture content of the inner and outer surfaces after introducing the physical model formulas is as follows:

[0089]

[0090]

[0091] In the formula: Current ambient humidity; Moisture absorption / dehumidification rate; This represents the rate of water diffusion. and These are the constraint weights for the outer and inner balancing VAE generated data and the physical correction data, respectively.

[0092] Next, the pseudo-historical data generated by VAE is integrated into a time series of overall moisture content and input into the ARIMA model. Linear trends and periodic changes are captured through differencing and regression analysis, and preliminary predicted values ​​are output. The time series calculation method for overall moisture content is as follows:

[0093]

[0094] In the formula: and These are the weights for the moisture content on the outer and inner sides, respectively.

[0095] Then the current detection data Combined with the pseudo-historical time series generated by VAE, a multi-dimensional input feature sequence is formed. The input is represented as... The data is then input into an LSTM model for time series modeling, ultimately outputting the predicted water content at future time points. A predicted moisture content map is constructed. Finally, the linear trend predictions from the ARIMA model and the nonlinear dynamic predictions from the LSTM model are integrated, and a weighted average method is used to generate the final prediction result. The integration formula is as follows:

[0096]

[0097] In the formula, and These represent the weight coefficients of the ARIMA and LSTM model predictions, respectively.

[0098] The defect detection model outputs scores defects on the wood surface and internally. Defects are categorized into different types, and each type is assigned a weight. The formula for calculating the defect score is as follows:

[0099]

[0100] In the formula, Indicates the first Weights of class defects; Weighting factors representing internal defects ;

[0101] Indicates the first Surface area of ​​the defect type; Indicates the first Internal projected area of ​​the defect type; This represents the total surface area of ​​the timber.

[0102] Based on moisture content test data and the ideal moisture content range for wood, a moisture content score is calculated. The moisture content score is as follows:

[0103]

[0104] In the formula, This indicates the actual moisture content of the wood; This indicates the ideal moisture content, which is determined by the type of wood. This indicates the permissible range of moisture content.

[0105] The defect score and moisture content score are combined to generate a final comprehensive score for the timber, which reflects the overall quality of the wood. The scoring formula is as follows:

[0106]

[0107] Based on the overall score Timber grading: Superior grade: ;Good product: Qualified products: Defective products:

[0108] Finally, based on the comprehensive score and the resulting timber grade, different grades of timber are guided to designated storage areas via diversion conveyor belts.

[0109] The above-described specific embodiments are merely preferred embodiments of the present invention and are not intended to limit the implementation of the present invention or the scope of the claims. All equivalent changes and modifications made in accordance with the scope of patent protection of the present invention should be included within the scope of the present invention patent application.

Claims

1. A multi-modal intelligent wood defect detection and performance evaluation method, characterized in that, The following steps are implemented using a multi-modal intelligent wood defect detection and performance evaluation system: S1: Synchronously collect visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data of the wood through multi-modal sensors, and perform data preprocessing; S2: Input the preprocessed multi-modal data into a multi-branch convolutional neural network, combine a feature pyramid network for multi-scale feature extraction, and generate a unified high-dimensional feature vector through dynamic weighted fusion; S3: Based on the high-dimensional feature vector, construct a defect detection model, and perform pixel-level segmentation and instance segmentation to output defect categories, locations, and boundary information; S4: Collect the moisture content data of the outer and inner sides of the wood, generate pseudo historical time series through a variational autoencoder, and predict future moisture content by combining ARIMA and LSTM models, including the following steps: S41: Collect the moisture content of the outer and inner sides of the wood, respectively, calculate the weighted average value by assigning weights to each sampling point to obtain the overall moisture content of the outer and inner sides, and finally integrate it into the input features of the overall moisture content; S42: the input data contains the moisture content of the outer side and inner side of the wood, and the temperature and humidity of the current environment, constituting the model basic input vector , respectively represent the outer side and inner side weighted moisture content at the current time, represents the current environmental temperature, represents the current environmental humidity; S43: The encoder maps the input vector to a mean and standard deviation of a latent feature distribution and generates latent variables by a reparameterization trick The VAE decoder receives the latent variables and the input vector and generates pseudo-historical time series of the outer and inner water fractions: The set of preliminary pseudo historical data for the outer side is: ， The set of preliminary pseudo historical data for the inner side is: ， By introducing physical model prior constraints to generate data, the pseudo historical sequence is more consistent with the actual rules of wood moisture absorption and desorption. After introducing the physical model formula, the inner and outer moisture contents are as follows: ， ， wherein: is the current ambient humidity, is the hygro- / de-humidification rate, is the moisture diffusion rate, and are the constraint weights for the outer and inner side equilibrium VAE generation data and physical correction data, respectively; S44: Integrate the pseudo historical data generated by the VAE into the time series input of the overall moisture content into the ARIMA model, capture linear trends and periodic changes through difference and regression analysis, and output preliminary prediction values. The time series calculation method of the overall moisture content is as follows: ， wherein: and are the weights for the outer and inner moisture content, respectively; S45: combine the current detection data and the pseudo historical time series generated by the VAE to form a multi-dimensional input feature sequence, and the input is represented as Input the LSTM model for time series modeling, and finally output the water cut prediction value at the future time to form a predicted water cut graph; S46: Integrate the linear trend prediction value output by the ARIMA model and the nonlinear dynamic prediction value output by the LSTM model to generate the final prediction result through weighted averaging. The integration formula is as follows: ， wherein, and denote the weight coefficients of the ARIMA and LSTM model prediction values, respectively; S5: Generate a comprehensive quality score based on the defect score and moisture content score, divide the wood grade, and complete the sorting through a sorting device.

2. The multi-modal intelligent wood defect detection and performance evaluation method according to claim 1, wherein, The step S1 further includes: S11: Use LabVIEW to control industrial RGB cameras, infrared thermal imagers, hyperspectral cameras, ultrasonic detectors, and microwave density sensors simultaneously, and combine hardware triggers to realize synchronous data acquisition in hard trigger mode; S12: performing preprocessing on the collected data, including denoising, normalization, coordinate registration, and modal mask processing, wherein the modal mask processing is performed on the value of each pixel point in the single-modal feature map output by the neural network Two pixel score indicators, absolute value / response quality and entropy value, are calculated respectively, the score is compared with a preset threshold, the pixel points exceeding the threshold retain the original value, and the pixel points not reaching the threshold are assigned a value of 0, and finally a feature map processed by the mask is generated, and the score calculation formula of the absolute value / response quality is as follows:​ Absolute value / response quality score: ， Entropy score: 。 3. The multi-modal intelligent wood defect detection and performance evaluation method according to claim 2, wherein, The step S2 further includes: S21: The feature map output from S1 is first passed through a convolutional neural network and then input into a feature pyramid network. Through top-down feature propagation and horizontal connection mechanisms, high-resolution and low-resolution feature maps are gradually fused to generate a unified feature map with multi-scale information; S22: extract feature sub-graphs of each modality from the unified feature graph , introduce a pixel-level dynamic weighting mechanism, optimize the modality features by calculating the weight of each pixel point of the feature sub-graph , the weight calculation formula is:​​ ， In the formula, denotes the unified feature map The value of the kth modality in the pixel point , for amplifying the significant feature response, denotes the local gradient information of the pixel point , for measuring the feature change, is a weight coefficient obtained by dynamic learning through training; The optimized feature map is obtained: ， Generating an optimized single-modal feature subgraph and generating an optimized multi-modal fused feature subgraph through a channel fusion operation ; S23: multi-modal fused feature sub-graph The high-dimensional feature tensor is generated through the feature splicing operation, and then sequentially undergoes batch normalization, layer normalization and dimension reduction processing, and finally a unified high-dimensional feature vector is generated.

4. The multi-modal intelligent wood defect detection and performance evaluation method according to claim 1, wherein, The step S3 further includes: S31: Input the unified high-dimensional feature vector into the defect detection model, use the YOLO algorithm to simultaneously complete candidate box generation and defect classification, and label the regions that may have cracks, insect eyes, and knot scars. Assign a class label to each candidate box; S32: further pixel-level segmentation of the defect region in the candidate frame is performed based on the classification by using the ResNet classification network, to accurately divide the shape and boundary of the defect, and generate a defect mask; S33: instance segmentation is performed on the segmented defect area to distinguish different defect instances of the same category, and independent segmentation masks and boundary information are generated for each instance; in the process of the instance segmentation task, after a fixed-size feature map is generated through ROIAlign, before entering the classification, regression and segmentation branches, the modal features are optimized through a multi-modal information dynamic weighting fusion mechanism, and each modal feature submap is split from the unified feature map and the global dynamic weight calculation formula is as follows:​​​​​​ ， wherein: denotes the k-th modality feature map after ROIAlign, denotes the global mean intensity of the modality feature, denotes the global variance of the modality feature, denotes the global gradient mean of the modality feature; is a weighting coefficient dynamically learned through training. S34: a class label, position coordinates, and instance boundary information are generated for each defect, and stored as a detection output.

5. The multi-modal intelligent wood defect detection and performance evaluation method based on claim 1, characterized in that, The step S5 further comprises: S51: Score the defects on the surface and inside of the wood through the output of the defect detection model, and classify the defects into different types, each type is assigned a weight The calculation formula of the defect score is as follows: ， wherein represents the weight of the class of defects; represents the weight factor of the internal defects ; represents the surface area of the first class of defects; represents the internal projected area of the first class of defects; represents the total surface area of the wood; S52: based on the moisture content detection data, and in combination with the ideal moisture content range of wood, a moisture content score is calculated; the moisture content score is as follows: ， In the formula, represents the actual moisture content of the wood; represents the ideal moisture content determined by the wood material; represents the allowable moisture content range; S53: the defect score and the moisture content score are combined to generate a final comprehensive score of the wood, which is used to reflect the overall quality of the wood; the score formula is as follows: 。 6. The multi-modal intelligent wood defect detection and performance evaluation method based on claim 5, characterized in that, The wood grading standard is: Good Quality: ; Good product: ; Good product: ; Defective: .

7. The multi-modal intelligent wood defect detection and performance evaluation method based on claim 1, characterized in that, The multi-modal intelligent wood defect detection and performance evaluation system comprises a shell (1), a wood conveying module, a multi-modal data acquisition module, a multi-modal information processing module, and a wood grading and sorting module; the shell (1) is composed of a metal frame and a protective panel, and a plurality of LED light sources and sensor supports are alternately mounted on the top surface inside the shell (1); the wood conveying module comprises a black rubber track (5), a motor and an encoder drive; the multi-modal data acquisition module comprises a visible light camera (21) fixed to a first support (11), an infrared thermal imager (22) fixed to a second support (12), a near-infrared spectrum sensor (23) fixed to a third support (13), an ultrasonic detector (24) fixed to a fourth support (14), and a microwave density sensor (25) fixed to a fifth support (15); the multi-modal information processing module comprises an industrial control computer (3); the industrial control computer (3) is connected with the visible light camera (21), the infrared thermal imager (22), the near-infrared spectrum sensor (23), the ultrasonic detector (24), and the microwave density sensor (25); the wood grading and sorting module comprises a diversion chute (61), a diversion conveying belt (62), and a storage box (63).

8. The multi-modal intelligent wood defect detection and performance evaluation method based on claim 7, characterized in that, The black rubber track (5) of the multi-modal intelligent wood defect detection and performance evaluation system is made of high-strength frosted PVC material, and a non-slip edge is arranged on the surface of the black rubber track (5) to absorb stray light and improve the detection accuracy of the sensor; the LED light sources are symmetrically distributed around the sensor support and are adjusted in angle to optimize the coverage of the light sources.

9. The multi-modal intelligent wood defect detection and performance evaluation method based on claim 7, characterized in that, The multi-modal information processing module of the multi-modal intelligent wood defect detection and performance evaluation system controls the hardware trigger through LabVIEW, synchronously acquires visible light images, infrared thermal images, near-infrared spectra, ultrasonic signals, and microwave density data, and performs denoising, normalization, coordinate registration, and modal mask processing on the acquired data.

Images