Steel-wood combined component surface defect detection system and method based on image sensing

Abstract

The invention discloses a steel-wood combined component surface defect detection system and method based on image sensing, and relates to the technical field of defect detection.The system comprises an area identification module used for collecting a multi-modal image data set and determining a steel area subset and a wood area subset; the overlap analysis module is used for performing double-layer mapping and determining steel-wood combination interface information; the surface anomaly analysis module is used for performing surface anomaly analysis; the defect feature extraction module is used for extracting steel and wood defect features; the defect distribution diagram drawing module is used for drawing a surface defect distribution diagram; and the defect influence analysis module is used for carrying out defect influence analysis and locking surface defects. The technical problem that in the prior art, steel-wood combined component surface defect detection is not accurate enough, and consequently structure safety evaluation is not comprehensive enough is solved, and the technical effects that accurate detection of the steel-wood combined component surface defects is achieved, and the comprehensiveness of component structure safety evaluation is improved are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and in particular to a steel-wood combined component surface defect detection system and method based on image sensing. BACKGROUND

[0002] In the field of construction engineering, steel-wood combined components are widely used due to their strength of steel and aesthetic characteristics of wood. However, surface defects such as cracks and rust of steel, dryness and insect holes of wood, and debonding of the steel-wood combined interface will affect the structural safety and service life of the components. Existing surface defect detection technologies often cannot comprehensively and accurately detect defects in different material regions and the combined interface of steel-wood combined components, resulting in low detection efficiency, poor positioning accuracy, insufficient comprehensive evaluation of composite defects, and other problems, which leads to the inability to timely and accurately grasp the true condition of the components.

[0003] The existing technology has the technical problem of insufficient precision in detecting surface defects of steel-wood combined components, resulting in insufficient comprehensive evaluation of structural safety. SUMMARY

[0004] The present application provides a steel-wood combined component surface defect detection system and method based on image sensing, which is used to solve the technical problem of insufficient precision in detecting surface defects of steel-wood combined components in the prior art, resulting in insufficient comprehensive evaluation of structural safety.

[0005] In view of the above problems, the present application provides a steel-wood combined component surface defect detection system and method based on image sensing.

[0006] In a first aspect of the present application, a steel-wood combined component surface defect detection system based on image sensing is provided, which comprises: The area recognition module is configured to collect a multi-modal image dataset of a surface of a steel-wood combined component, perform area recognition based on the multi-modal image dataset, determine a steel region subset and a wood region subset; the overlap analysis module is configured to perform double-layer mapping by traversing the steel region subset and the wood region subset, perform overlap analysis according to a mapping result, and determine steel-wood combined interface information; the surface anomaly analysis module is configured to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract steel defect features and wood defect features; the defect feature extraction module is configured to perform combined surface anomaly analysis based on the steel-wood combined interface information, and extract steel-wood defect features; the defect distribution map drawing module is configured to verify the steel-wood defect features based on the steel defect features and the wood defect features, construct a defect detection result of the steel-wood combined component according to a verification result, and draw a component surface defect distribution map; and the defect influence analysis module is configured to perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock surface defects of the steel-wood combined component.

[0007] In a possible implementation manner, the steel spatial grid layer generation unit is configured to map the steel region subset to a first layer to generate a steel spatial grid layer; the wood spatial grid layer generation unit is configured to map the wood region subset to a second layer to generate a wood spatial grid layer; the mapping result generation unit is configured to perform double-layer projection on the steel spatial grid layer and the wood spatial grid layer to generate a mapping result, and extract a projection overlap region according to the mapping result; the overlap effectiveness analysis unit is configured to traverse the projection overlap region to determine a plurality of grid points, perform overlap effectiveness analysis based on the plurality of grid points, and generate a plurality of effective overlap labels; and the steel-wood combined interface information determination unit is configured to match the plurality of effective overlap labels with the plurality of grid points to determine the steel-wood combined interface information.

[0008] In a possible implementation manner, the interface normal projection subunit is configured to perform interface normal projection on the projection overlap region to determine an interface normal projection plane; the grid point determination subunit is configured to traverse the projection overlap region in a spiral path according to the interface normal projection plane to determine a plurality of grid points; and the multi-modal feature set determination subunit is configured to perform multi-modal feature analysis based on the plurality of grid points to determine a multi-modal feature set, perform third-order effectiveness verification according to the multi-modal feature set, and generate the plurality of effective overlap labels.

[0009] In possible implementation manners, the gradient vector field construction unit is configured to traverse the steel material region subset to perform image edge flow analysis and construct a gradient vector field; the change monitoring unit is configured to perform change monitoring based on the gradient vector field to determine a gradient change direction and a gradient change amplitude; the average gradient amplitude acquisition unit is configured to calculate an average value according to the gradient change amplitude to obtain an average gradient amplitude; the target pixel chain determination unit is configured to take the average gradient amplitude as a limiting constraint, track according to the gradient change direction, and determine a target pixel chain; and the record result analysis unit is configured to record the target pixel chain as surface anomaly data of the steel material region subset, analyze according to a record result, and obtain the steel defect feature.

[0010] In possible implementation manners, the surface anomaly data set acquisition subunit is configured to map the target pixel chain to the steel material region subset to perform analysis and obtain a surface anomaly data set, the surface anomaly data set including target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data; the first verification result generation subunit is configured to perform deep step verification based on the target pixel chain coordinate data and the target pixel chain gradient data to generate a first verification result, mark according to the first verification result, and obtain a step height parameter; the second verification result generation subunit is configured to perform thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data to generate a second verification result, mark according to the second verification result, and obtain a temperature gradient direction parameter; the feature analysis subunit is configured to perform feature analysis according to the step height parameter and the temperature gradient direction parameter to determine a physical crack defect feature; the third verification result generation subunit is configured to extract a non-pixel chain region according to the target pixel chain to perform near-infrared reflection verification to generate a third verification result, mark according to the third verification result to obtain a rust area parameter, and perform feature analysis according to the rust area parameter to determine a chemical rust defect feature; and the steel defect feature adding subunit is configured to add the physical crack defect feature and the chemical rust defect feature to a steel defect feature.

[0011] In a possible implementation, the steel material crack verification unit is configured to verify the steel material crack according to the steel material defect feature, and obtain a steel material defect verification result; the wood dry crack verification unit is configured to verify the wood dry crack according to the wood defect feature, and obtain a wood defect verification result; the wood defect layer construction unit is configured to construct a steel material defect layer based on the steel material defect feature, construct a wood defect layer based on the wood defect feature, and construct a steel-wood defect layer based on the steel material defect verification result and the wood defect verification result; the composite defect atlas construction unit is configured to map the steel material defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional space grid for fusion, and construct a composite defect atlas; and the composite defect atlas projection unit is configured to perform confidence analysis based on the composite defect atlas, project the composite defect atlas by using pseudo-color coding according to a plurality of confidence degrees, and draw the component surface defect distribution map.

[0012] In a possible implementation, the confidence extraction subunit is configured to extract an initial steel material crack confidence degree, an initial wood hole confidence degree, and an initial interface debonding confidence degree based on the composite defect atlas; the weighted analysis subunit is configured to perform weighted analysis according to the initial steel material crack confidence degree, the initial wood hole confidence degree, and the initial interface debonding confidence degree, and obtain a spatio-temporal weighted comprehensive confidence degree; the pseudo-color layer construction subunit is configured to perform color space mapping on the composite defect atlas according to the spatio-temporal weighted comprehensive confidence degree, and construct a pseudo-color layer; and the adaptive projection subunit is configured to perform curved surface adaptive projection based on the pseudo-color layer, and draw the component surface defect distribution map.

[0013] In a possible implementation, the grid coordinate system establishment unit is configured to establish a grid coordinate system based on the component surface defect distribution map, and traverse the grid coordinate system to extract a plurality of grid units; the influence factor determination unit is configured to traverse the plurality of grid units to perform local influence calculation, determine a plurality of local influence factors, and set a plurality of defect levels according to the plurality of local influence factors; the structure safety index acquisition unit is configured to map the plurality of defect levels to the component surface defect distribution map to perform structure safety analysis, and obtain a structure safety index; and the warning box determination unit is configured to project the component surface defect distribution map to a steel-wood combined component entity according to the structure safety index, determine a warning box, and lock the surface defect of the steel-wood combined component through the warning box.

[0014] In a possible implementation manner, the influence factor analysis subunit is configured to analyze and determine a plurality of steel local influence factors, a plurality of wood local influence factors and a plurality of combination local influence factors based on the plurality of local influence factors; the first defect influence level setting subunit is configured to perform defect influence determination on the component surface defect distribution map according to the plurality of steel local influence factors, and set a first defect influence level; the second defect influence level setting subunit is configured to perform defect influence determination on the component surface defect distribution map according to the plurality of wood local influence factors, and set a second defect influence level; the third defect influence level setting subunit is configured to perform defect influence determination on the component surface defect distribution map according to the plurality of combination local influence factors, and set a third defect influence level; and the influence level integration subunit is configured to integrate the first defect influence level, the second defect influence level and the third defect influence level, and construct the plurality of defect levels.

[0015] In a second aspect, the application provides a steel-wood combination component surface defect detection method based on image sensing, which comprises the following steps: A multi-modal image dataset of a steel-wood combination component surface is collected, a region identification is performed based on the multi-modal image dataset, a steel region subset and a wood region subset are determined, a double-layer mapping is performed by traversing the steel region subset and the wood region subset, an overlap analysis is performed according to the mapping result, steel-wood combination interface information is determined, a surface anomaly analysis is respectively performed on the steel region subset and the wood region subset, steel defect features and wood defect features are extracted, a combination surface anomaly analysis is performed based on the steel-wood combination interface information, steel-wood defect features are extracted, the steel-wood defect features are verified based on the steel defect features and the wood defect features, a defect detection result of the steel-wood combination component is constructed according to the verification result, and a component surface defect distribution map is drawn, a defect influence analysis is performed according to the component surface defect distribution map, a plurality of defect levels are set to be mapped to the component surface defect distribution map, and surface defects of the steel-wood combination component are locked.

[0016] The one or more technical solutions provided in the application have at least the following technical effects or advantages: The area recognition module is used to collect a multi-modal image data set of a steel-wood combined component surface, determine a steel area subset and a wood area subset; the overlap analysis module is used to traverse the steel area subset and the wood area subset for double-layer mapping, and determine steel-wood combined interface information; the surface anomaly analysis module is used to respectively perform surface anomaly analysis on the steel area subset and the wood area subset, and extract steel defect features and wood defect features; the defect feature extraction module is used to perform combined surface anomaly analysis based on the steel-wood combined interface information, and extract steel-wood defect features; the defect distribution map drawing module is used to verify the steel-wood defect features, and draw a component surface defect distribution map; and the defect influence analysis module is used to perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock the surface defects of the steel-wood combined component. The technical effect of achieving accurate detection of the surface defects of the steel-wood combined component and improving the comprehensiveness of the component structure safety evaluation is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The structure schematic diagram of the steel-wood combined component surface defect detection system based on image sensing provided by the embodiments of the present application is shown in the figure. Figure 2 The flow schematic diagram of the steel-wood combined component surface defect detection method based on image sensing provided by the embodiments of the present application is shown in the figure.

[0019] The figure mark explanation: area recognition module 10, overlap analysis module 20, surface anomaly analysis module 30, defect feature extraction module 40, defect distribution map drawing module 50, and defect influence analysis module 60. DETAILED DESCRIPTION

[0020] The present application provides a steel-wood combined component surface defect detection system and method based on image sensing, which is used to solve the technical problem that the steel-wood combined component surface defect detection is not accurate enough in the prior art, resulting in that the structure safety evaluation is not comprehensive enough.

[0021] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0022] As shown in Embodiment One, Figure 1 The present application provides an image sensor-based steel-wood combined component surface defect detection system, which comprises: A region identification module 10 is configured to collect a multi-modal image dataset of the surface of a steel-wood combined component, perform region identification based on the multi-modal image dataset, and determine a steel region subset and a wood region subset.

[0023] Specifically, a multi-modal image dataset of the surface of a steel-wood combined component is collected from different angles and under different lighting conditions by using various image collection devices such as industrial cameras, 3D cameras, and infrared sensors, covering visible light, near-infrared, thermal imaging, and other image information. Subsequently, a semantic segmentation algorithm based on deep learning, such as Mask R-CNN, is used. In the training stage, a large number of labeled steel-wood combined component image samples are used to enable the model to learn the differences between steel and wood in terms of texture, color, and spectral features. In the inference process, the collected multi-modal image is input into the trained model, and the model classifies each pixel in the image. Based on the classification results, the steel region subset and the wood region subset are accurately determined, and effective division of different material regions on the surface of the steel-wood combined component is achieved.

[0024] An overlap analysis module 20 is configured to traverse the steel region subset and the wood region subset for double-layer mapping, perform overlap analysis based on the mapping results, and determine steel-wood combined interface information.

[0025] Specifically, when traversing the steel region subset and the wood region subset for double-layer mapping and determining the steel-wood combined interface information, the steel region subset is first mapped to a first layer to generate a steel spatial grid layer, and the wood region subset is simultaneously mapped to a second layer to generate a wood spatial grid layer. Then, double-layer projection is performed on the two layers to generate a mapping result and extract a projection overlap region therefrom. Next, interface normal projection is performed on the projection overlap region to determine a projection plane, a plurality of grid points are determined in the region by traversing the region in a spiral path, multi-modal feature analysis is performed based on the grid points to form a feature set, and an effective overlap label is generated through third-order effectiveness verification. Finally, the effective overlap label is matched with the grid points to determine the steel-wood combined interface information.

[0026] The surface anomaly analysis module 30 is configured to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract steel defect features and wood defect features.

[0027] Specifically, the image edge flow analysis is performed on the steel region subset, the gradient vector field is constructed, and the gradient change direction and amplitude are monitored. The average gradient amplitude is calculated as a constraint, and the target pixel chain is tracked along the gradient change direction. The target pixel chain is mapped to the steel region subset to obtain a surface anomaly data set containing coordinates, length, and gradient data. The depth step verification generates a step height parameter, and the thermal field verification generates a temperature gradient direction parameter, thereby determining the physical crack defect features. At the same time, the non-pixel chain region is verified by near-infrared reflection to generate a rust area parameter and determine the chemical rust defect features. For the wood region subset, the wavelet texture decomposition is performed to extract high-frequency components, the morphological opening operation is performed to filter out wood grain noise, and the connected domain with an area greater than 5 mm² is retained as a candidate area of wormhole. Then, the curvature mutation of the depth point cloud is calculated, and when the local curvature is greater than 0.3 mm⁻¹ and presents a linear distribution, it is marked as dry cracking, thereby extracting the wood defect features.

[0028] The defect feature extraction module 40 is configured to perform combined surface anomaly analysis based on the steel-wood joint interface information, and extract steel-wood defect features.

[0029] Specifically, when performing combined surface anomaly analysis based on the steel-wood joint interface information, the multi-modal image data of the steel-wood joint interface region is analyzed according to the determined steel-wood joint interface information. By identifying the texture mutation, color difference, and three-dimensional topographic features at the interface, combined with the grid point distribution of the interface normal projection plane, the connection tightness of the interface region is analyzed. When detecting that there is a gap with a width greater than 0.2 mm or a near-infrared reflectivity difference greater than 15% or a point cloud curvature mutation region presents a continuous distribution at the interface, it is judged that there is an abnormal situation such as debonding or cracking at the steel-wood joint interface, and then the steel-wood defect features are extracted, such as the position, size, and severity of the interface debonding.

[0030] The defect distribution map drawing module 50 is configured to verify the steel-wood defect features based on the steel defect features, the wood defect features, and construct a defect detection result of the steel-wood joint component according to the verification result, and draw a component surface defect distribution map.

[0031] Specifically, based on the steel defect features (covering steel corrosion, welding cracks, porosity, coating peeling, etc.) and wood defect features (including wood cracking, decay, insect damage, coating peeling, etc.), the steel-wood combination defect features (such as glue failure, loose connectors, and joint surface cracking, etc.) are cross-verified. First, the steel defect features are used to verify whether the joint interface anomaly is caused by the expansion of steel corrosion or the extension of welding cracks. For example, when the distance between the steel crack endpoint and the debonding area of the joint interface is less than 0.5 mm, it is determined that there is an extension correlation. At the same time, the wood defect features are used to verify whether the joint anomaly is caused by wood dry cracking penetrating to the interface or insect damage to the interface structure. If the coincidence degree of the wood insect hole cluster area and the joint interface gap is more than 30%, it is determined that there is a causal relationship. After verification, the steel defect layer (annotating parameters such as corrosion area and crack length), the wood defect layer (annotating information such as insect hole position and dry cracking depth), and the steel-wood defect layer (recording data such as interface debonding range and connector loosening torque value) are constructed. The three layers of defect data are mapped to a three-dimensional space grid. The D-S evidence theory is used to fuse the confidence of multiple sources (steel crack confidence, wood insect hole confidence, and interface debonding confidence). The HSV color space is used to map the comprehensive confidence to a pseudo-color code (for example, set confidence 0.0~0.2 to blue, 0.2~0.5 to yellow, 0.5~0.8 to orange, and 0.8~1.0 to red). Finally, based on the curvature of the component surface, an adaptive projection is performed to generate a three-dimensional defect distribution map containing defect type, size, and risk level. Each defect point in the map is associated with multiple modal feature parameters (such as steel corrosion spectrum reflectivity, wood dry cracking point cloud curvature, and interface debonding infrared temperature difference value).

[0032] The defect impact analysis module 60 is used to perform defect impact analysis based on the component surface defect distribution map. A plurality of defect levels are mapped to the component surface defect distribution map, and the surface defects of the steel-wood combination component are locked.

[0033] Specifically, when performing defect impact analysis based on the component surface defect distribution map, a grid coordinate system is established on the distribution map. A plurality of grid elements are extracted. Local impact factors are calculated for each grid element, including steel local impact factors (such as corrosion area ratio and crack depth), wood local impact factors (such as insect hole density and dry cracking length), and combination local impact factors (such as interface debonding width and connector loosening degree). Based on these factors, the first, second, and third defect impact levels are set and integrated into a plurality of defect levels. The defect levels are mapped to the distribution map for structural safety analysis. The structural safety index is calculated. The distribution map is projected onto the steel-wood combination component entity according to the index. Warning boxes of different colors (such as red for high risk and yellow for medium risk) are determined. The surface defects of the component are precisely locked through the warning boxes. The defect position and damage degree are intuitively displayed.

[0034] In a possible implementation, the overlap analysis module 20 further includes: a steel material spatial mesh layer generation unit configured to map the steel material region subset to a first layer to generate a steel material spatial mesh layer.

[0035] a wood material spatial mesh layer generation unit configured to map the wood material region subset to a second layer to generate a wood material spatial mesh layer.

[0036] a mapping result generation unit configured to perform double-layer projection on the steel material spatial mesh layer and the wood material spatial mesh layer to generate a mapping result, and extract a projection overlap region according to the mapping result.

[0037] an overlap validity analysis unit configured to traverse the projection overlap region to determine a plurality of mesh points, perform overlap validity analysis based on the plurality of mesh points, and generate a plurality of valid overlap labels.

[0038] a steel-wood combination interface information determination unit configured to match the plurality of valid overlap labels with the plurality of mesh points to determine the steel-wood combination interface information.

[0039] Specifically, when the steel material region subset is mapped to the first layer, based on pixel coordinates and spatial position information of the steel material region subset, a regular spatial mesh is constructed in the first layer at a preset mesh precision (for example, 0.5 mm*0.5 mm), so that each mesh unit corresponds to an actual physical region of a steel material surface one by one, each pixel point in the steel material region subset is accurately positioned into a corresponding mesh unit of the mesh layer by a coordinate mapping algorithm, and thus a steel material spatial mesh layer containing spatial distribution information of the steel material surface is generated, providing a structured spatial data basis for subsequent double-layer projection and combination interface analysis.

[0040] When the wood material region subset is mapped to the second layer, a regular spatial mesh system is established in the second layer according to a preset mesh density (for example, 0.5 mm*0.5 mm) based on pixel coordinates and three-dimensional spatial coordinate information of the wood material region subset, each pixel point in the wood material region subset is one by one corresponded to a specific mesh unit of the mesh layer by a coordinate transformation algorithm, so that each mesh unit accurately maps an actual physical region of a wood material surface, and thus a wood material spatial mesh layer representing spatial distribution characteristics of the wood material surface is generated, providing a structured data support for subsequent double-layer projection analysis and accurate positioning of a steel-wood combination interface.

[0041] According to the double-layer projection of the steel spatial grid layer and the wood spatial grid layer, first, the two layers are unified to the same three-dimensional coordinate system to ensure the consistency of the spatial position. Then, by using the orthogonal projection or perspective projection method, the steel spatial grid layer and the wood spatial grid layer are superimposed and projected on the specified projection plane, so that the grid units of the two layers form overlapping or non-overlapping area distribution on the projection plane, thereby generating a mapping result. Then, the mapping result is analyzed by using an image recognition algorithm, and the overlapping parts of the steel grid and the wood grid on the projection plane are identified, and these overlapping parts are extracted as the projection overlapping area, which is the area where the steel and wood materials may be combined on the surface of the component, providing a key spatial position reference for subsequent determination of the steel-wood combination interface information.

[0042] When traversing the projection overlapping area to determine multiple grid points and performing overlapping effectiveness analysis, first, the projection overlapping area is projected on the interface normal, and the interface normal projection plane is determined, and then the projection overlapping area is traversed in a spiral path according to the projection plane to determine multiple grid points. Then, based on these grid points, multi-modal feature analysis is performed to extract the features of each grid point in the multi-modal data such as visible light images and near-infrared images to form a multi-modal feature set, and then the three-order effectiveness verification is performed according to the feature set, that is, the first-order verification calculates the reflectivity gradient of adjacent grid points to judge the material continuity, the second-order verification detects the thermal conductivity mutation to verify the thermal conduction consistency, and the third-order verification determines the geometric fit degree by calculating the local curvature difference of the depth point cloud. According to the third-order verification result, multiple effective overlapping labels are generated to determine the overlapping effectiveness of each grid point.

[0043] When matching the multiple effective overlapping labels with the multiple grid points, each effective overlapping label is associated with its corresponding grid point spatial coordinates by using a coordinate mapping algorithm, and a set of grid points marked as "effective" is selected. Based on the spatial distribution characteristics of the effective grid point set, the Delaunay triangulation algorithm or the least squares method is used to fit the steel-wood material interface curve / surface, thereby determining the specific position, trend and geometric form of the steel-wood combination interface. In this process, the effective overlapping label is used to verify whether the grid point belongs to the true combination interface, and the continuous area formed by the matched effective grid points is the steel-wood combination interface, and the coordinate data and geometric features can be directly used for subsequent combination surface anomaly analysis.

[0044] In one possible implementation manner, the overlapping effectiveness analysis unit further includes: An interface normal projection sub-unit is configured to perform interface normal projection on the projection overlapping area to determine an interface normal projection plane.

[0045] A grid point determination sub-unit is configured to traverse the projection overlapping area in a spiral path according to the interface normal projection plane to determine multiple grid points.

[0046] A multi-modal feature set determination subunit is configured to perform multi-modal feature analysis based on the plurality of grid points, determine a multi-modal feature set, perform third-order validity verification according to the multi-modal feature set, and generate the plurality of valid overlap labels.

[0047] Specifically, when performing interface normal projection on the projection overlap region to determine the interface normal projection plane, principal component analysis (PCA) is first performed on the point cloud data in the projection overlap region. By calculating the variance distribution of the point cloud data in each dimension, the direction with the largest variance is determined as the normal direction of the interface, which is perpendicular to the steel-wood combination interface. Then, taking the normal direction as the reference, a plane perpendicular to the normal direction is constructed, which is the interface normal projection plane. The projection plane is used for subsequent feature analysis and grid point traversal of the projection overlap region, and provides a projection reference for accurately determining the steel-wood combination interface information.

[0048] When traversing the projection overlap region in a spiral path according to the interface normal projection plane, the interface normal projection plane is taken as the reference plane, and the traversal starts from the geometric center of the projection overlap region and expands outward along the Archimedean spiral trajectory. During the traversal process, position points are uniformly selected on the spiral path at a predetermined spatial sampling interval (such as 0.1 mm), and each position point corresponds to a spatial coordinate in the projection overlap region, thereby determining a plurality of grid points. This spiral traversal method can ensure that every part of the projection overlap region is covered in an orderly and non-missing manner, thereby obtaining uniformly distributed grid points, which provide a basis for subsequent multi-modal feature analysis and overlap validity verification based on the grid points.

[0049] When performing multi-modal feature analysis and completing third-order validity verification based on the plurality of grid points, visible light images, near-infrared spectra, and thermal imaging data are first synchronously collected for each grid point. Steel features (such as iron element spectral reflectance peak at 680 nm, thermal conductivity threshold of 15 W / (m·K)) and wood features (such as cellulose spectral absorption valley at 1450 nm, thermal conductivity threshold of 0.15 W / (m·K)) are extracted, and a multi-modal feature set containing spectral reflectance, thermal conductivity, and point cloud three-dimensional coordinates is constructed. First-order verification is performed by calculating the near-infrared reflectance gradient (threshold set to 0.2 / μm) of adjacent grid points, and if the gradient suddenly changes, it is determined that the material is discontinuous. Second-order verification compares the measured thermal conductivity of the grid point with the standard thermal conductivity of steel and wood materials, and when the difference exceeds 20%, it is marked as thermal conduction abnormality. Third-order verification uses deep point cloud to calculate the local curvature difference (threshold 0.15 mm⁻¹), and the curvature mutation area is determined as geometric mismatch. Finally, according to the third-order verification result, an "effective overlap" or "invalid overlap" label is generated for each grid point, and only when the third-order verification passes, it is marked as valid.

[0050] The third-order validity verification evaluation criteria of the multi-modal feature set are shown in Table 1: Table 1 third-order effectiveness verification evaluation criteria table

[0051] In one possible implementation manner, the surface anomaly analysis module further comprises: A gradient vector field construction unit is configured to traverse the steel material region subset to perform image edge flow analysis and construct a gradient vector field.

[0052] A change monitoring unit is configured to perform change monitoring based on the gradient vector field, and determine a gradient change direction and a gradient change amplitude.

[0053] An average gradient amplitude acquisition unit is configured to perform average value calculation according to the gradient change amplitude, and obtain an average gradient amplitude.

[0054] A target pixel chain determination unit is configured to take the average gradient amplitude as a limiting constraint, track according to the gradient change direction, and determine a target pixel chain.

[0055] A record result analysis unit is configured to record the target pixel chain as surface anomaly data of the steel material region subset, analyze according to a record result, and obtain the steel defect feature.

[0056] Specifically, when traversing the steel material region subset to perform image edge flow analysis, an edge detection algorithm such as a Canny operator or a Sobel operator is used to calculate the gradient amplitude and direction of each pixel point in the region, thereby constructing a gradient vector field. In this process, by calculating the gradient components of the pixel point in the x and y directions, the gradient value and gradient direction of each pixel point are obtained, and these gradient vectors together constitute a gradient vector field describing the edge features of the steel material region subset, providing a basis for subsequent change monitoring and defect feature extraction based on the gradient vector field.

[0057] When change monitoring is performed based on the gradient vector field, the gradient vectors of adjacent pixel points are compared point by point to analyze the spatial change trend of the gradient, thereby determining the gradient change direction, i.e., the direction in which the pixel intensity changes fastest; at the same time, the difference between the gradient amplitudes of adjacent pixel points is calculated to obtain the gradient change amplitude, thereby quantifying the degree of change of the gradient. This process can effectively identify the edge and defect features of the steel surface, providing key data support for subsequent extraction of the steel defect feature.

[0058] When average value calculation is performed according to the gradient change amplitude, the data of all gradient change amplitudes in the gradient vector field are first collected, then these data are added together and divided by the number of data, thereby obtaining the average gradient amplitude. This average gradient amplitude can reflect the overall level of the surface gradient change of the steel material region subset, and provide an important reference for subsequent determination of the target pixel chain.

[0059] When tracking in the gradient change direction with the average gradient amplitude as a limiting constraint, the pixel points with gradient amplitudes greater than or equal to the average gradient amplitude in the gradient change direction are selected from the gradient vector field as starting points, and the adjacent pixel points are sequentially connected along the gradient change direction to form a continuous pixel chain. In the tracking process, it is continuously judged whether the gradient amplitudes of the adjacent pixel points meet the constraint condition of being greater than or equal to the average gradient amplitude, and if not, the tracking is terminated. The finally determined continuous pixel chain is the target pixel chain, which can effectively represent the abnormal feature area on the surface of the steel material.

[0060] When the target pixel chain is recorded as the surface abnormal data of the steel material region subset, the endpoint coordinates of the target pixel chain are automatically extracted by a computer vision algorithm, the accurate positions of the two endpoints are determined by using a contour detection and edge fitting algorithm, and the maximum pixel spacing of the pixel chain in the direction perpendicular to the strike direction is calculated and converted into the actual physical size to obtain the maximum width data. When analyzing the recording results, if the target pixel chain presents a continuous and steep step feature and the temperature gradient direction is consistent with the crack propagation direction, it is determined as a physical crack defect, and the depth and strike of the crack are determined in combination with the step height parameter and the temperature gradient direction parameter. If the near-infrared reflectivity of a non-pixel chain region is lower than the standard reflectivity threshold (such as 0.6) of the steel material, it is determined as a chemical corrosion defect according to the corrosion area parameter, so as to obtain the steel defect features including the defect type, position, size and morphology.

[0061] In a possible implementation manner, the recording result analysis unit further includes: A surface abnormal data set acquisition subunit is configured to map the target pixel chain to the steel material region subset for analysis, and obtain a surface abnormal data set. The surface abnormal data set includes target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data.

[0062] A first verification result generation subunit is configured to perform depth step verification based on the target pixel chain coordinate data and the target pixel chain gradient data, generate a first verification result, and obtain a step height parameter according to the marking of the first verification result.

[0063] A second verification result generation subunit is configured to perform a thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data, generate a second verification result, and obtain a temperature gradient direction parameter according to the marking of the second verification result.

[0064] A feature analysis subunit is configured to perform feature analysis according to the step height parameter and the temperature gradient direction parameter, and determine a physical crack defect feature.

[0065] The third verification result generation subunit is configured to perform near-infrared reflection verification on a non-pixel chain area extracted from the target pixel chain to generate a third verification result, mark according to the third verification result to obtain a rust area parameter, perform feature analysis according to the rust area parameter, and determine a chemical rust defect feature.

[0066] The steel defect feature adding subunit is configured to add the physical crack defect feature and the chemical rust defect feature to a steel defect feature.

[0067] Specifically, when the target pixel chain is mapped to a steel area subset for analysis, the pixel coordinates of the target pixel chain are converted into coordinate data in an actual physical coordinate system through a conversion matrix of image coordinates and physical coordinates, the number of continuous pixel points of the target pixel chain is calculated and converted into an actual length by using a contour tracking algorithm, the gradient amplitude and direction data of each pixel point are extracted, and thus a surface anomaly data set containing the coordinate data, length data and gradient data of the target pixel chain is obtained, thereby providing basic data support for subsequent depth step verification, thermal field verification and the like.

[0068] When the depth step verification is performed based on the coordinate data and gradient data of the target pixel chain, the three-dimensional coordinate values of the points on the target pixel chain are obtained by associating the coordinate data with the height information in a three-dimensional point cloud model, and the positions where the steps are likely to exist are determined in combination with the gradient data. When the height difference between adjacent point clouds exceeds a preset threshold (such as 0.1 mm), it is determined that there is a depth step, and a first verification result is generated. Then, according to the verification result, the depth step area is marked, and the maximum height difference is extracted as a step height parameter, thereby representing the depth feature of the physical crack on the steel surface.

[0069] When the thermal field verification is performed based on the coordinate data and length data of the target pixel chain, the temperature values of the positions of the target pixel chain are obtained by associating the coordinate data with the infrared thermal imaging data, and the temperature sampling path is determined according to the length data. Temperature data is collected at a fixed interval (such as every 0.5 mm) along the direction of the target pixel chain, the temperature difference value of adjacent sampling points is calculated, the direction with the maximum temperature change rate is determined, and the temperature gradient direction is determined. When the temperature gradient direction is consistent with the extension direction of the target pixel chain and the temperature change rate exceeds a preset threshold (such as 5℃ / mm), a second verification result is generated, the temperature gradient direction is marked according to the second verification result, and the direction is extracted as a temperature gradient direction parameter, thereby representing the heat conduction abnormal feature of the physical crack on the steel surface.

[0070] When the step height parameter and the temperature gradient direction parameter are analyzed, if the step height parameter exceeds a preset threshold (such as 0.1 mm), it indicates that there is a significant height mutation on the surface of the steel material, and the temperature gradient direction parameter shows that the direction with the maximum temperature change rate is consistent with the extension direction of the target pixel chain and the change rate exceeds a threshold (such as 5 ℃ / mm), it can be determined that the target pixel chain is a physical crack defect. The step height parameter reflects the depth characteristics of the crack, and the temperature gradient direction parameter reflects the influence direction of the crack on heat conduction, and the combination of the two can accurately represent the defect characteristics such as the position, depth and expansion trend of the physical crack.

[0071] When the target pixel chain is extracted to verify the near-infrared reflection of the non-pixel chain area, the non-pixel chain area is first demarcated by extending a certain range (such as 5 pixels on both sides of the pixel chain) outward based on the target pixel chain, the reflection spectrum data of the area is collected by the near-infrared camera, and compared with the standard near-infrared reflection spectrum of the steel material (such as a reflectance threshold of 0.6 at a wavelength of 680 nm). When the reflectance is lower than the threshold, it is determined to be a rust area, and a third verification result is generated. Then, the rust area boundary is marked according to the verification result, the proportion of the rust area in the non-pixel chain area is calculated, and the actual rust area is converted to obtain a rust area parameter. If the rust area parameter exceeds a preset threshold (such as 10%), the shape and distribution characteristics of the rust area are combined to determine the characteristics of the chemical rust defect, such as the rust type (overall rust or local rust) and the severity.

[0072] When the physical crack defect characteristics and the chemical rust defect characteristics are added to the steel defect characteristics, the step height, temperature gradient direction and other characteristic parameters of the physical crack are first integrated, and the rust area, reflectance and other characteristic parameters of the chemical rust are then classified and recorded in the steel defect characteristic database. The physical crack defect characteristics are stored according to the position, depth, expansion trend and other dimensions, and the chemical rust defect characteristics are classified according to the type, area, severity and other dimensions, and finally a complete steel defect characteristic set containing physical cracks and chemical rust is formed, which provides comprehensive feature data support for subsequent defect detection and evaluation of steel-wood combined components.

[0073] In one possible implementation manner, the defect distribution mapping module 50 further includes: The steel crack verification unit is configured to perform steel crack verification on the steel-wood defect characteristics according to the steel defect characteristics, and obtain a steel defect verification result.

[0074] The wood dryness verification unit is configured to perform wood dryness verification on the steel-wood defect characteristics according to the wood defect characteristics, and obtain a wood defect verification result.

[0075] The wood defect layer construction unit is configured to construct a steel defect layer based on the steel defect feature, construct a wood defect layer based on the wood defect feature, and construct a steel-wood defect layer based on the steel defect verification result and the wood defect verification result.

[0076] The composite defect map construction unit is configured to map the steel defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional space grid for fusion to construct a composite defect map.

[0077] The composite defect map projection unit is configured to perform confidence analysis based on the composite defect map, project the composite defect map using pseudo-color coding according to a plurality of confidence levels, and draw a component surface defect distribution map.

[0078] Specifically, when the steel-wood defect feature is verified according to the steel defect feature, the depth point cloud data of the steel part in the steel-wood defect feature is extracted first, and is compared with the step height parameter recorded in the steel defect feature to calculate the step height difference of the depth point cloud at the crack. If the step height difference of the depth point cloud is consistent with the step height parameter in the steel defect feature and exceeds a preset threshold (such as 0.1 mm), the verification is passed, it is determined that the steel crack defect in the steel-wood defect feature is valid, a steel defect verification result is generated, and it is indicated that the steel part of the steel-wood combined component has a crack defect conforming to the feature. If the conditions are not met, the verification is not passed, and it is indicated that the defect feature does not conform to the steel crack feature.

[0079] When the steel-wood defect feature is verified according to the wood defect feature, the texture data and humidity distribution data of the wood part in the steel-wood defect feature are extracted first, and are compared with the dry crack feature parameters (such as texture fracture degree and humidity change threshold) recorded in the wood defect feature. By analyzing the continuity and integrity of the wood surface texture, if it is found that the texture is obviously fractured and has a gap, and the humidity of the corresponding area is lower than the humidity threshold of the wood dry crack (such as lower than 12%), the verification is passed, it is determined that the wood dry crack defect in the steel-wood defect feature is valid, a wood defect verification result is generated, and it is indicated that the wood part of the steel-wood combined component has a dry crack defect conforming to the feature. If the conditions are not met, the verification is not passed, and it is indicated that the defect feature does not conform to the wood dry crack feature.

[0080] When the steel defect layer is constructed based on the defect characteristics of the steel, the physical crack defect characteristics (such as the step height parameter and the temperature gradient direction parameter) and the chemical corrosion defect characteristics (such as the corrosion area parameter) of the steel are mapped to a two-dimensional plane according to the spatial position and the characteristic type, and a steel defect layer reflecting the defect distribution on the surface of the steel is formed; when the wood defect layer is constructed based on the defect characteristics of the wood, the defect characteristics such as the dry crack of the wood are mapped according to the corresponding rules, and a wood defect layer is generated; then, based on the steel defect verification result and the wood defect verification result, the defect characteristics at the steel-wood combination interface are fused to construct a steel-wood defect layer, so as to represent the defect condition of the steel-wood combination area.

[0081] When the steel defect layer, the wood defect layer and the steel-wood defect layer are mapped to a three-dimensional space grid for fusion, first, the defect characteristic data of each layer (including the physical crack and chemical corrosion characteristics of the steel, the dry crack characteristics of the wood and the defect characteristics of the steel-wood combination interface) are converted into three-dimensional space coordinates, and then are mapped according to a unified grid coordinate system (such as taking the geometric center of the component as the origin, and the XYZ axes corresponding to the length, width and height directions), so that different types of defect characteristics are accurately positioned in the three-dimensional space. Then, a weighted fusion algorithm is used, different weights are given according to the type and severity of the defect characteristics, the defect characteristics in the overlapping area are fused and processed, the conflict data is eliminated, and the typical defect characteristics are retained, so as to finally construct a composite defect map containing the defect information of the steel, the wood and the combination interface, and realize the three-dimensional visualization integration of multi-dimensional defect information.

[0082] When the composite defect map is subjected to confidence analysis, first, multiple confidence parameters such as the initial steel crack confidence, the initial wood hole confidence and the initial interface debonding confidence are extracted from the composite defect map, then different weights are given according to the influence degree of each confidence on the safety of the component structure, and the spatio-temporal weighted comprehensive confidence is obtained through weighted calculation. Then, the composite defect map is subjected to color space mapping according to the comprehensive confidence, different confidence intervals are corresponded to different color channels, a pseudo-color layer is constructed, and finally the pseudo-color layer is subjected to surface adaptive projection, so that the defect map is matched with the surface of the component, and a defect distribution map directly reflecting the defect distribution and severity on the surface of the component is drawn.

[0083] In a possible implementation manner, the composite defect map projection unit further includes: A confidence extraction sub-unit configured to extract an initial steel crack confidence, an initial wood hole confidence and an initial interface debonding confidence based on the composite defect map.

[0084] A weighted analysis sub-unit configured to perform weighted analysis according to the initial steel crack confidence, the initial wood hole confidence and the initial interface debonding confidence, and obtain a spatio-temporal weighted comprehensive confidence.

[0085] The pseudo-color layer construction subunit is configured to perform color space mapping on the composite defect atlas according to the spatiotemporal weighted comprehensive confidence to construct a pseudo-color layer.

[0086] The adaptive projection subunit is configured to perform curved surface adaptive projection based on the pseudo-color layer to draw the component surface defect distribution map.

[0087] Specifically, when extracting the initial steel crack confidence based on the composite defect atlas, the initial steel crack confidence is calculated by using a fuzzy logic algorithm based on analysis of the step height parameter distribution density of the steel region in the atlas and the proportion exceeding the 0.1 mm threshold, combined with the consistency degree of the temperature gradient direction and the crack extension direction; when extracting the initial wood wormhole confidence, the initial wood wormhole confidence is output by using a neural network model based on statistics of the humidity distribution (the proportion of the area below the 12% humidity threshold) and the wormhole morphological characteristics (such as hole diameter and depth) of the texture fracture of the wood region; when extracting the initial interface debonding confidence, the initial interface debonding confidence is determined by using an analytic hierarchy process based on evaluation of the debonding area proportion of the steel-wood interface region and the number of overlapping invalid grid points in the interface normal projection, and the three initial confidences respectively quantify the possibility and severity of the existence of different types of defects.

[0088] When performing weighted analysis based on the initial steel crack confidence, the initial wood wormhole confidence, and the initial interface debonding confidence, the weight coefficients of each confidence in the time and space dimensions are first determined, for example, the time weight of the steel crack confidence is set to 0.2 and the space weight is set to 0.1, the time weight of the wood wormhole confidence is set to 0.2 and the space weight is set to 0.1, and the time weight of the interface debonding confidence is set to 0.2 and the space weight is set to 0.2, ensuring that the sum of all weight coefficients is 1. Then, the time weight and the space weight of each initial confidence are added to obtain the spatiotemporal comprehensive weight of each confidence, and the initial confidence is multiplied by the corresponding spatiotemporal comprehensive weight. Finally, the products are added, that is, the spatiotemporal weighted comprehensive confidence is calculated by the formula: spatiotemporal weighted comprehensive confidence = initial steel crack confidence × (0.2+0.1) + initial wood wormhole confidence × (0.2+0.1) + initial interface debonding confidence × (0.2+0.2), which considers the influence degree of different types of defects on the component in the time and space dimensions.

[0089] When the color space of the composite defect map is mapped according to the spatiotemporal weighted comprehensive confidence, a mapping relationship between the confidence and the color space is established first, for example, confidence 0.0-0.2 is set to correspond to blue, confidence 0.2-0.5 is set to correspond to yellow, confidence 0.5-0.8 is set to correspond to orange, and confidence 0.8-1.0 is set to correspond to red. Then, the spatiotemporal weighted comprehensive confidence of each grid point in the composite defect map is substituted into the mapping relationship, and a corresponding color value is assigned to each grid point to form a false color layer, so as to realize visual expression of the defect confidence and intuitively present defect regions with different confidences in different colors.

[0090] When the surface of the steel-wood combined component is adaptively projected based on the false color layer, surface curvature data of the steel-wood combined component is obtained through three-dimensional modeling first, then each pixel point of the false color layer is mapped and matched with a triangular mesh model of the component surface, the projection angle of the pixel point is adjusted according to the surface normal vector, the color transition of the concave-convex area of the surface is processed by using bicubic spline interpolation, the false color layer is deformed and fitted with the curvature of the component surface, and finally a visual distribution map containing information of defect position, type and severity is generated in the component entity surface coordinate system, so as to realize accurate superposition of the defect information and the geometric morphology of the component.

[0091] In a possible implementation manner, the defect influence analysis module 60 further includes: A grid coordinate system establishing unit is configured to establish a grid coordinate system based on the component surface defect distribution map, and traverse the grid coordinate system to extract a plurality of grid units.

[0092] An influence factor determining unit is configured to traverse the plurality of grid units to perform local influence calculation, determine a plurality of local influence factors, and set a plurality of defect levels according to the plurality of local influence factors.

[0093] A structure safety index obtaining unit is configured to map the plurality of defect levels to the component surface defect distribution map to perform structure safety analysis, and obtain a structure safety index.

[0094] A warning box determining unit is configured to project the component surface defect distribution map to an entity of the steel-wood combined component according to the structure safety index, determine a warning box, and lock the surface defect of the steel-wood combined component through the warning box.

[0095] Specifically, when establishing the grid coordinate system based on the component surface defect distribution map, the geometric center of the component is taken as the origin, and the grid lines are equally divided along the length and width directions to form a regular grid coordinate system. The grid line spacing can be set according to the component size and detection accuracy requirements, such as 10 mm x 10 mm. After completing the coordinate system establishment, the grid coordinate system is traversed in the order from top to bottom and from left to right. Each region formed by the intersection of grid lines is extracted as a grid cell to ensure that all areas of the component surface defect distribution map are covered, thereby obtaining a plurality of grid cells as basic analysis units for subsequent local influence calculation.

[0096] When traversing the plurality of grid cells for local influence calculation, a local influence calculation model is established for the defect type, size, number, and confidence level in each grid cell. For example, for a grid cell in a steel region, the weakening ratio of crack length and depth on steel strength is calculated, for a grid cell in a wood region, the influence degree of wormhole or dry crack on wood bearing capacity is evaluated, and the influence of steel-wood interface area debonding on the overall structure connection strength is considered, thereby determining a plurality of local influence factors. Then, a plurality of defect levels are set according to the numerical range of the local influence factor, for example, local influence factor ≤ 0.3 is set as low level, 0.3 < local influence factor ≤ 0.6 is set as medium level, and local influence factor > 0.6 is set as high level, to realize the grading and definition of the defect severity of different grid cells.

[0097] When mapping the plurality of defect levels to the component surface defect distribution map for structure safety analysis, each defect level is assigned a corresponding safety influence coefficient, such as low level defect corresponding to safety influence coefficient 0.2, medium level corresponding to 0.5, and high level corresponding to 0.8. Then, the defect level and its safety influence coefficient of each grid cell are superimposed on the component surface defect distribution map. According to the location, type, and severity of the defect, the influence on the overall structure strength and stiffness of the component is analyzed, and the structure safety index is calculated by weighted summation, for example, structure safety index = 1 - Σ (defect level safety influence coefficient x defect distribution area proportion), thereby quantitatively evaluating the structure safety condition of the component.

[0098] According to the structural safety index, when projecting the component surface defect distribution map to the steel-wood combined component entity, a structural safety index threshold is first set, such as 0.6. When the structural safety index of a certain area is lower than the threshold, it is determined that the defect area needs to be warned. Then, based on the three-dimensional model of the component, the low safety index area on the defect distribution map is mapped to the corresponding position on the entity surface. A warning box is generated with the area as the center. The size and color of the warning box are adjusted according to the defect level, such as a red large box for high-level defects, a yellow medium box for medium-level defects, and a blue small box for low-level defects. In this way, the two-dimensional defect distribution map is accurately projected onto the three-dimensional entity, realizing the visualization locking of the surface defects of the steel-wood combined component, and facilitating the rapid positioning and processing of the defect area.

[0099] In one possible implementation manner, the influence factor determination unit further includes: An influence factor analysis subunit configured to analyze based on the plurality of local influence factors to determine a plurality of steel local influence factors, a plurality of wood local influence factors, and a plurality of combination local influence factors.

[0100] A first defect influence level setting subunit configured to perform defect influence determination on the component surface defect distribution map according to the plurality of steel local influence factors, and set a first defect influence level.

[0101] A second defect influence level setting subunit configured to perform defect influence determination on the component surface defect distribution map according to the plurality of wood local influence factors, and set a second defect influence level.

[0102] A third defect influence level setting subunit configured to perform defect influence determination on the component surface defect distribution map according to the plurality of combination local influence factors, and set a third defect influence level.

[0103] An influence level integration subunit configured to integrate the first defect influence level, the second defect influence level, and the third defect influence level to construct the plurality of defect levels.

[0104] Specifically, when analyzing based on the plurality of local influence factors, the local influence factors are divided into steel local influence factors, wood local influence factors, and combination local influence factors according to the material type and position of the defects. The steel local influence factors include the influence degree of physical crack characteristics (such as step height parameters and temperature gradient direction parameters) and chemical corrosion characteristics (such as corrosion area parameters) on the performance of steel. The wood local influence factors include the influence parameters of the width of wood drying cracks, the size and density of insect holes, and the like on the strength of wood. The combination local influence factors involve the influence indicators of the steel-wood interface debonding area ratio and the interface normal projection characteristics on the steel-wood combination strength. Through the analysis of these factors, the classification and quantification of the influence of defects on different materials and interfaces are realized.

[0105] When the component surface defect distribution map is judged according to the plurality of steel local influence factors, the step height parameter, the temperature gradient direction parameter and the corrosion area parameter in the steel local influence factor are analyzed first. When the step height is more than 0.3 mm and the corrosion area ratio is greater than 5%, it is determined as high influence; when the step height is between 0.1 and 0.3 mm and the corrosion area ratio is between 2% and 5%, it is determined as medium influence; when the step height is less than 0.1 mm and the corrosion area ratio is less than 2%, it is determined as low influence, thereby setting the first defect influence level and realizing the classification of the influence degree of the steel area defect.

[0106] When the component surface defect distribution map is judged according to the plurality of wood local influence factors, the dry crack width, the wormhole size and the density parameters in the wood local influence factor are analyzed first. When the dry crack width is more than 1.5 mm and the wormhole diameter is greater than 3 mm and the density is more than 3 / 10 cm², it is determined as high influence; when the dry crack width is between 0.5 and 1.5 mm and the wormhole diameter is between 1 and 3 mm and the density is between 1 and 3 / 10 cm², it is determined as medium influence; when the dry crack width is less than 0.5 mm and the wormhole diameter is less than 1 mm and the density is less than 1 / 10 cm², it is determined as low influence, thereby setting the second defect influence level and realizing the classification of the influence degree of the wood area defect.

[0107] When the component surface defect distribution map is judged according to the plurality of combination local influence factors, the steel-wood interface debonding area ratio and the interface normal projection features are mainly analyzed. When the debonding area ratio is more than 10% and the number of overlapping invalid grid points in the interface normal projection is relatively large, it is determined as high influence; when the debonding area ratio is between 5% and 10% and the number of overlapping invalid grid points is moderate, it is determined as medium influence; when the debonding area ratio is less than 5% and the number of overlapping invalid grid points is relatively small, it is determined as low influence, thereby setting the third defect influence level and realizing the classification of the influence degree of the steel-wood combination area defect.

[0108] When the first defect influence level, the second defect influence level and the third defect influence level are integrated, a three-dimensional defect influence matrix is established first, taking the defect influence levels of the steel, the wood and the steel-wood combination area as three dimensions, and an integration rule is set. When any one of the three areas is of high influence level, the integrated defect level is determined as high level; if two areas are of medium influence level and the other is of low influence level, it is determined as medium-high level; if all the three areas are of low influence level or only one area is of medium influence level, it is determined as low level, and through this multi-dimensional weighted integration method, a plurality of defect levels covering the comprehensive influence of defects of different materials and interfaces are constructed, and the severity of the component surface defect is comprehensively reflected.

[0109] Embodiment two, based on the same inventive concept as the image sensor-based steel-wood joint member surface defect detection system in the preceding embodiment, as shown in Figure 2 The present application provides an image sensor-based steel-wood joint member surface defect detection method, and the method and system embodiments in the present application are based on the same inventive concept. The method comprises: Step S100: Collecting a multi-modal image dataset of the surface of a steel-wood joint member, performing region identification based on the multi-modal image dataset, and determining a steel region subset and a wood region subset.

[0110] Step S200: Performing double-layer mapping on the steel region subset and the wood region subset, performing overlap analysis based on the mapping result, and determining steel-wood joint interface information.

[0111] Step S300: Performing surface anomaly analysis on the steel region subset and the wood region subset respectively, and extracting steel defect features and wood defect features.

[0112] Step S400: Performing joint surface anomaly analysis based on the steel-wood joint interface information, and extracting steel-wood defect features.

[0113] Step S500: Verifying the steel-wood defect features based on the steel defect features and the wood defect features, constructing a defect detection result of the steel-wood joint member according to the verification result, and drawing a member surface defect distribution map.

[0114] Step S600: Performing defect impact analysis according to the member surface defect distribution map, setting a plurality of defect levels mapped to the member surface defect distribution map, and locking the surface defects of the steel-wood joint member.

[0115] Further, the method further comprises: mapping the steel region subset to a first layer to generate a steel spatial grid layer, mapping the wood region subset to a second layer to generate a wood spatial grid layer, performing double-layer projection based on the steel spatial grid layer and the wood spatial grid layer to generate a mapping result, extracting a projection overlap region based on the mapping result, determining a plurality of grid points by traversing the projection overlap region, performing overlap effectiveness analysis based on the plurality of grid points to generate a plurality of effective overlap labels, and matching the plurality of effective overlap labels with the plurality of grid points to determine the steel-wood joint interface information.

[0116] Further, the method further comprises: Interface normal projection is performed on the projection overlap region to determine an interface normal projection plane; the projection overlap region is traversed in a spiral path according to the interface normal projection plane to determine a plurality of grid points; multi-modal feature analysis is performed based on the plurality of grid points to determine a multi-modal feature set, and third-order effectiveness verification is performed according to the multi-modal feature set to generate the plurality of effective overlap labels.

[0117] Further, the method further comprises: Image edge flow analysis is performed on the steel material region subset to construct a gradient vector field; change monitoring is performed based on the gradient vector field to determine a gradient change direction and a gradient change amplitude; average value calculation is performed according to the gradient change amplitude to obtain an average gradient amplitude; the average gradient amplitude is taken as a limiting constraint, and tracking is performed according to the gradient change direction to determine a target pixel chain; the target pixel chain is recorded as surface anomaly data of the steel material region subset, and analysis is performed according to the recording result to obtain the steel defect feature.

[0118] Further, the method further comprises: The target pixel chain is mapped to the steel material region subset for analysis to obtain a surface anomaly data set, the surface anomaly data set including target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data; depth step verification is performed based on the target pixel chain coordinate data and the target pixel chain gradient data to generate a first verification result, and step height parameters are obtained according to the first verification result; thermal field verification is performed based on the target pixel chain coordinate data and the target pixel chain length data to generate a second verification result, and temperature gradient direction parameters are obtained according to the second verification result; physical crack defect features are determined according to the step height parameters and the temperature gradient direction parameters; non-pixel chain regions are extracted according to the target pixel chain for near-infrared reflection verification to generate a third verification result, and rust area parameters are obtained according to the third verification result; chemical rust defect features are determined according to the rust area parameters; and the physical crack defect features and the chemical rust defect features are added to the steel defect feature.

[0119] Further, the method further comprises: The steel material defect feature is verified according to the steel material crack of the steel-wood defect feature, and a steel material defect verification result is obtained; the wood defect feature is verified according to the wood crack of the steel-wood defect feature, and a wood defect verification result is obtained; a steel material defect layer is constructed based on the steel material defect feature, a wood defect layer is constructed based on the wood defect feature, and a steel-wood defect layer is constructed based on the steel material defect verification result and the wood defect verification result; the steel material defect layer, the wood defect layer and the steel-wood defect layer are mapped to a three-dimensional space grid for fusion to construct a composite defect map; confidence analysis is performed based on the composite defect map, the composite defect map is projected using pseudo-color coding according to a plurality of confidence degrees, and a component surface defect distribution map is drawn.

[0120] Further, the method further comprises: Based on the composite defect map, an initial steel material crack confidence, an initial wood hole confidence and an initial interface debonding confidence are extracted; weighted analysis is performed according to the initial steel material crack confidence, the initial wood hole confidence and the initial interface debonding confidence, and a spatio-temporal weighted comprehensive confidence is obtained; color space mapping is performed on the composite defect map according to the spatio-temporal weighted comprehensive confidence, and a pseudo-color layer is constructed; surface adaptive projection is performed based on the pseudo-color layer, and the component surface defect distribution map is drawn.

[0121] Further, the method further comprises: Based on the component surface defect distribution map, a grid coordinate system is established, and a plurality of grid elements are extracted by traversing the grid coordinate system; local influence calculation is performed by traversing the plurality of grid elements, a plurality of local influence factors are determined, and a plurality of defect levels are set according to the plurality of local influence factors; the plurality of defect levels are mapped to the component surface defect distribution map for structure safety analysis, and a structure safety index is obtained; according to the structure safety index, the component surface defect distribution map is projected to an entity of a steel-wood combined component, an alert box is determined, and the surface defect of the steel-wood combined component is locked through the alert box.

[0122] Further, the method further comprises: Based on the analysis of the plurality of local influence factors, a plurality of steel local influence factors, a plurality of wood local influence factors, and a plurality of combination local influence factors are determined. According to the plurality of steel local influence factors, defect influence determination is performed on the component surface defect distribution map, and a first defect influence level is set. According to the plurality of wood local influence factors, defect influence determination is performed on the component surface defect distribution map, and a second defect influence level is set. According to the plurality of combination local influence factors, defect influence determination is performed on the component surface defect distribution map, and a third defect influence level is set. The first defect influence level, the second defect influence level, and the third defect influence level are integrated to construct the plurality of defect levels.

[0123] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. Moreover, the above-mentioned embodiments of the present application are described. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0124] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0125] The present specification and drawings are only exemplary of the present application, and any and all modifications, changes, combinations, or equivalents within the scope of the present application are considered to be covered by the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the scope of the present application. Thus, if these modifications and changes of the present application belong to the scope of the present application and its equivalents, the present application is intended to include these modifications and changes.

Claims

1. Steel-wood combined member surface defect detection system based on image sensing, characterized by, The system comprises: A region identification module is configured to collect a multi-modal image dataset of a surface of a steel-wood combined component, perform region identification based on the multi-modal image dataset, determine a steel region subset and a wood region subset, and perform surface anomaly analysis on the steel region subset and the wood region subset respectively to extract steel defect features and wood defect features. A defect feature extraction module is configured to perform combined surface anomaly analysis based on the steel-wood combined interface information to extract steel-wood defect features. A defect distribution map drawing module is configured to verify the steel-wood defect features based on the steel defect features and the wood defect features, construct a defect detection result of the steel-wood combined component according to a verification result, and draw a component surface defect distribution map. A defect influence analysis module is configured to perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock surface defects of the steel-wood combined component. The overlap analysis module further comprises: A steel space grid layer generation unit is configured to map the steel region subset to a first layer to generate a steel space grid layer.

2. The image sensor-based surface defect detection system for steel-wood composite members according to claim 1, wherein A wood space grid layer generation unit is configured to map the wood region subset to a second layer to generate a wood space grid layer. A mapping result generation unit is configured to perform double-layer projection on the steel space grid layer and the wood space grid layer to generate a mapping result, extract a projection overlap region according to the mapping result, perform overlap effectiveness analysis on a plurality of grid points in the projection overlap region, and generate a plurality of effective overlap labels. A steel-wood combined interface information determination unit is configured to match the plurality of effective overlap labels with the plurality of grid points to determine the steel-wood combined interface information. The overlap effectiveness analysis unit further comprises: An interface normal projection sub-unit is configured to perform interface normal projection on the projection overlap region to determine an interface normal projection plane. A grid point determination sub-unit is configured to traverse the projection overlap region in a spiral path according to the interface normal projection plane to determine a plurality of grid points.

3. The image sensor-based surface defect detection system for steel-wood composite members as claimed in claim 2, wherein A multi-modal feature set determination sub-unit is configured to perform multi-modal feature analysis based on the plurality of grid points to determine a multi-modal feature set, perform third-order effectiveness verification according to the multi-modal feature set, and generate the plurality of effective overlap labels. The surface anomaly analysis module further comprises: A gradient vector field construction unit is configured to perform image edge flow analysis on the steel region subset to construct a gradient vector field. A change monitoring unit is configured to perform change monitoring based on the gradient vector field to determine a gradient change direction and a gradient change amplitude.

4. The image sensor-based surface defect detection system for steel-wood composite members according to claim 1, wherein An average gradient amplitude acquisition unit is configured to perform average value calculation according to the gradient change amplitude to obtain an average gradient amplitude. ​ ​ ​ The target pixel chain determination unit is configured to determine a target pixel chain by tracking the gradient change direction according to the average gradient amplitude as a limited constraint; The record result analysis unit is configured to record the target pixel chain as surface anomaly data of the steel material region subset, analyze the record result, and obtain the steel material defect feature.

5. The image sensor-based surface defect detection system for steel-wood composite members as claimed in claim 4, wherein, The record result analysis unit further includes: A surface anomaly data set acquisition subunit is configured to map the target pixel chain to the steel material region subset for analysis, obtain a surface anomaly data set, and acquire target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data. A first verification result generation subunit is configured to perform deep step verification based on the target pixel chain coordinate data and the target pixel chain gradient data, generate a first verification result, mark the first verification result, and obtain a step height parameter. A second verification result generation subunit is configured to perform thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data, generate a second verification result, mark the second verification result, and obtain a temperature gradient direction parameter. A feature analysis subunit is configured to perform feature analysis according to the step height parameter and the temperature gradient direction parameter, and determine a physical crack defect feature. A third verification result generation subunit is configured to extract a non-pixel chain region from the target pixel chain, perform near-infrared reflection verification, generate a third verification result, mark the third verification result, obtain a rust area parameter, perform feature analysis according to the rust area parameter, and determine a chemical rust defect feature. A steel material defect feature addition subunit is configured to add the physical crack defect feature and the chemical rust defect feature to a steel material defect feature.

6. The image sensor-based surface defect detection system for steel-to-wood junction members as claimed in claim 1, wherein, The defect distribution map drawing module further includes: A steel material crack verification unit is configured to perform steel material crack verification on the steel-wood defect feature according to the steel material defect feature, and obtain a steel material defect verification result. A wood dry crack verification unit is configured to perform wood dry crack verification on the steel-wood defect feature according to the wood defect feature, and obtain a wood defect verification result. A wood defect layer construction unit is configured to construct a steel defect layer based on the steel material defect feature, construct a wood defect layer based on the wood defect feature, and construct a steel-wood defect layer based on the steel material defect verification result and the wood defect verification result. A composite defect map construction unit is configured to map the steel defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional space grid for fusion, and construct a composite defect map. A composite defect map projection unit is configured to perform confidence analysis based on the composite defect map, project the composite defect map using pseudo-color coding according to multiple confidences, and draw a component surface defect distribution map.

7. The image sensor-based surface defect detection system for steel-to-wood junction members as claimed in claim 6, wherein, The composite defect map projection unit further includes: A confidence extraction subunit is configured to extract an initial steel material crack confidence, an initial wood wormhole confidence, and an initial interface debonding confidence based on the composite defect map. The weighted analysis subunit is configured to perform weighted analysis according to the initial steel material crack confidence, the initial wood worm hole confidence and the initial interface debonding confidence, and obtain a spatiotemporal weighted comprehensive confidence; The pseudo-color layer construction subunit is configured to perform color space mapping on the composite defect atlas according to the spatiotemporal weighted comprehensive confidence, and construct a pseudo-color layer; The adaptive projection subunit is configured to perform curved surface adaptive projection based on the pseudo-color layer, and draw the component surface defect distribution map.

8. The image sensor-based surface defect detection system for steel-to-wood junction members as claimed in claim 1, wherein, The defect influence analysis module further includes: The grid coordinate system establishment unit is configured to establish a grid coordinate system based on the component surface defect distribution map, and traverse the grid coordinate system to extract a plurality of grid units; The influence factor determination unit is configured to traverse the plurality of grid units to perform local influence calculation, determine a plurality of local influence factors, and set a plurality of defect levels according to the plurality of local influence factors; The structure safety index acquisition unit is configured to map the plurality of defect levels to the component surface defect distribution map to perform structure safety analysis, and obtain a structure safety index; The warning box determination unit is configured to project the component surface defect distribution map to a steel-wood combined component entity according to the structure safety index, determine a warning box, and lock the surface defect of the steel-wood combined component through the warning box.

9. The image sensor-based surface defect detection system for steel-to-wood junction members as claimed in claim 8, wherein, The influence factor determination unit further includes: The influence factor analysis subunit is configured to analyze the plurality of local influence factors to determine a plurality of steel local influence factors, a plurality of wood local influence factors and a plurality of combination local influence factors; The first defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of steel local influence factors, and set a first defect influence level; The second defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of wood local influence factors, and set a second defect influence level; The third defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of combination local influence factors, and set a third defect influence level; The influence level integration subunit is configured to integrate the first defect influence level, the second defect influence level and the third defect influence level to construct the plurality of defect levels.

10. A method for detecting surface defects of a steel-wood combined member based on image sensing, characterized by, The method is implemented by the steel-wood combined component surface defect detection system based on image sensing according to any one of claims 1-9, and the method includes: Collecting a multi-modal image data set of a steel-wood combined component surface, performing region identification based on the multi-modal image data set, and determining a steel region subset and a wood region subset; Traversing the steel region subset and the wood region subset to perform double-layer mapping, performing overlap analysis according to the mapping result, and determining steel-wood combined interface information; Performing surface anomaly analysis on the steel region subset and the wood region subset respectively, and extracting steel defect features and wood defect features; Performing combined surface anomaly analysis based on the steel-wood combined interface information, and extracting steel-wood defect features; Verify the steel-wood defect feature based on the steel defect feature and the wood defect feature, construct a defect detection result of the steel-wood combined component according to a verification result, and draw a component surface defect distribution map; Perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock the surface defect of the steel-wood combined component.

Images