Image sensor-based steel-wood combined member surface defect detection system and method

Abstract

The application discloses a steel-wood combined component surface defect detection system and method based on image sensing, relates to the technical field of defect detection, and comprises the following modules: a region identification module, which is used for collecting a multi-modal image data set, determining a steel region subset and a wood region subset; an overlap analysis module, which is used for carrying out double-layer mapping and determining steel-wood combined interface information; a surface anomaly analysis module, which is used for carrying out surface anomaly analysis; a defect feature extraction module, which is used for extracting steel-wood defect features; a defect distribution map drawing module, which is used for drawing a surface defect distribution map; and a defect influence analysis module, which is used for carrying out defect influence analysis and locking surface defects. The application solves the technical problem that steel-wood combined component surface defect detection is not accurate enough in the prior art, leading to insufficient comprehensive structure safety evaluation, achieves accurate steel-wood combined component surface defect detection, and improves the comprehensiveness of component structure safety evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of defect detection, and in particular to a steel-wood combined component surface defect detection system and method based on image sensing. BACKGROUND

[0002] In the field of construction engineering, steel-wood combined components are widely used due to their strength of steel and aesthetic characteristics of wood. However, surface defects such as cracks and rust of steel, dryness and worm holes of wood, and debonding of the steel-wood combined interface will affect the structural safety and service life of the components. Existing surface defect detection technologies often cannot comprehensively and accurately detect defects in different material regions and the combined interface of steel-wood combined components, resulting in low detection efficiency, poor positioning accuracy, insufficient comprehensive evaluation of composite defects, and other problems, which leads to the inability to timely and accurately grasp the true condition of the components.

[0003] The existing technology has the technical problem of insufficient precision in detecting surface defects of steel-wood combined components, resulting in insufficient comprehensive evaluation of structural safety. SUMMARY

[0004] The present application provides a steel-wood combined component surface defect detection system and method based on image sensing, which is used to solve the technical problem of insufficient precision in detecting surface defects of steel-wood combined components in the prior art, resulting in insufficient comprehensive evaluation of structural safety.

[0005] In view of the above problems, the present application provides a steel-wood combined component surface defect detection system and method based on image sensing.

[0006] In a first aspect of the present application, a steel-wood combined component surface defect detection system based on image sensing is provided, which comprises:

[0007] The area recognition module is configured to collect a multi-modal image dataset of a surface of a steel-wood combined component, perform area recognition based on the multi-modal image dataset, determine a steel region subset and a wood region subset; the overlap analysis module is configured to perform double-layer mapping by traversing the steel region subset and the wood region subset, perform overlap analysis according to a mapping result, and determine steel-wood combined interface information; the surface anomaly analysis module is configured to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract steel defect features and wood defect features; the defect feature extraction module is configured to perform combined surface anomaly analysis based on the steel-wood combined interface information, and extract steel-wood defect features; the defect distribution map drawing module is configured to verify the steel-wood defect features based on the steel defect features and the wood defect features, construct a defect detection result of the steel-wood combined component according to a verification result, and draw a component surface defect distribution map; and the defect influence analysis module is configured to perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock surface defects of the steel-wood combined component.

[0008] In a possible implementation manner, the steel spatial grid layer generation unit is configured to map the steel region subset to a first layer to generate a steel spatial grid layer; the wood spatial grid layer generation unit is configured to map the wood region subset to a second layer to generate a wood spatial grid layer; the mapping result generation unit is configured to perform double-layer projection on the steel spatial grid layer and the wood spatial grid layer to generate a mapping result, and extract a projection overlap region according to the mapping result; the overlap effectiveness analysis unit is configured to traverse the projection overlap region to determine a plurality of grid points, perform overlap effectiveness analysis based on the plurality of grid points, and generate a plurality of effective overlap labels; and the steel-wood combined interface information determination unit is configured to match the plurality of effective overlap labels with the plurality of grid points to determine the steel-wood combined interface information.

[0009] In a possible implementation manner, the interface normal projection subunit is configured to perform interface normal projection on the projection overlap region to determine an interface normal projection plane; the grid point determination subunit is configured to traverse the projection overlap region in a spiral path according to the interface normal projection plane to determine a plurality of grid points; and the multi-modal feature set determination subunit is configured to perform multi-modal feature analysis based on the plurality of grid points to determine a multi-modal feature set, perform third-order effectiveness verification according to the multi-modal feature set, and generate the plurality of effective overlap labels.

[0010] In possible implementation manners, the gradient vector field construction unit is configured to traverse the steel material region subset to perform image edge flow analysis and construct a gradient vector field; the change monitoring unit is configured to perform change monitoring based on the gradient vector field to determine a gradient change direction and a gradient change amplitude; the average gradient amplitude acquisition unit is configured to calculate an average value according to the gradient change amplitude to obtain an average gradient amplitude; the target pixel chain determination unit is configured to take the average gradient amplitude as a limiting constraint, track according to the gradient change direction, and determine a target pixel chain; and the record result analysis unit is configured to record the target pixel chain as surface anomaly data of the steel material region subset, analyze according to a record result, and obtain the steel defect feature.

[0011] In possible implementation manners, the surface anomaly data set acquisition subunit is configured to map the target pixel chain to the steel material region subset to perform analysis and obtain a surface anomaly data set, the surface anomaly data set including target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data; the first verification result generation subunit is configured to perform deep step verification based on the target pixel chain coordinate data and the target pixel chain gradient data to generate a first verification result, mark according to the first verification result, and obtain a step height parameter; the second verification result generation subunit is configured to perform thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data to generate a second verification result, mark according to the second verification result, and obtain a temperature gradient direction parameter; the feature analysis subunit is configured to perform feature analysis according to the step height parameter and the temperature gradient direction parameter to determine a physical crack defect feature; the third verification result generation subunit is configured to extract a non-pixel chain region according to the target pixel chain to perform near-infrared reflection verification to generate a third verification result, mark according to the third verification result to obtain a rust area parameter, and perform feature analysis according to the rust area parameter to determine a chemical rust defect feature; and the steel defect feature adding subunit is configured to add the physical crack defect feature and the chemical rust defect feature to a steel defect feature.

[0012] In a possible implementation, the steel material crack verification unit is configured to verify the steel material crack according to the steel material defect feature, and obtain a steel material defect verification result; the wood dry crack verification unit is configured to verify the wood dry crack according to the wood defect feature, and obtain a wood defect verification result; the wood defect layer construction unit is configured to construct a steel material defect layer based on the steel material defect feature, construct a wood defect layer based on the wood defect feature, and construct a steel-wood defect layer based on the steel material defect verification result and the wood defect verification result; the composite defect atlas construction unit is configured to map the steel material defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional space grid for fusion, and construct a composite defect atlas; and the composite defect atlas projection unit is configured to perform confidence analysis based on the composite defect atlas, project the composite defect atlas by using pseudo-color coding according to a plurality of confidence degrees, and draw the component surface defect distribution map.

[0013] In a possible implementation, the confidence extraction subunit is configured to extract an initial steel material crack confidence degree, an initial wood hole confidence degree, and an initial interface debonding confidence degree based on the composite defect atlas; the weighted analysis subunit is configured to perform weighted analysis according to the initial steel material crack confidence degree, the initial wood hole confidence degree, and the initial interface debonding confidence degree, and obtain a spatio-temporal weighted comprehensive confidence degree; the pseudo-color layer construction subunit is configured to perform color space mapping on the composite defect atlas according to the spatio-temporal weighted comprehensive confidence degree, and construct a pseudo-color layer; and the adaptive projection subunit is configured to perform curved surface adaptive projection based on the pseudo-color layer, and draw the component surface defect distribution map.

[0014] In a possible implementation, the grid coordinate system establishment unit is configured to establish a grid coordinate system based on the component surface defect distribution map, and traverse the grid coordinate system to extract a plurality of grid units; the influence factor determination unit is configured to traverse the plurality of grid units to perform local influence calculation, determine a plurality of local influence factors, and set a plurality of defect levels according to the plurality of local influence factors; the structure safety index acquisition unit is configured to map the plurality of defect levels to the component surface defect distribution map to perform structure safety analysis, and obtain a structure safety index; and the warning box determination unit is configured to project the component surface defect distribution map to a steel-wood combined component entity according to the structure safety index, determine a warning box, and lock the surface defect of the steel-wood combined component through the warning box.

[0015] In a possible implementation manner, the influence factor analysis subunit is configured to analyze and determine a plurality of steel local influence factors, a plurality of wood local influence factors and a plurality of combination local influence factors based on the plurality of local influence factors; the first defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of steel local influence factors, and set a first defect influence level; the second defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of wood local influence factors, and set a second defect influence level; the third defect influence level setting subunit is configured to perform defect influence judgment on the component surface defect distribution map according to the plurality of combination local influence factors, and set a third defect influence level; and the influence level integration subunit is configured to integrate the first defect influence level, the second defect influence level and the third defect influence level, and construct the plurality of defect levels.

[0016] In a second aspect of the present application, a steel-wood combination component surface defect detection method based on image sensing is provided, and the method comprises the following steps:

[0017] A multi-modal image dataset of a steel-wood combination component surface is collected, a region identification is performed based on the multi-modal image dataset, a steel region subset and a wood region subset are determined; a double-layer mapping is performed by traversing the steel region subset and the wood region subset, an overlap analysis is performed according to the mapping result, and steel-wood combination interface information is determined; surface anomaly analysis is respectively performed on the steel region subset and the wood region subset, steel defect features and wood defect features are extracted; combination surface anomaly analysis is performed based on the steel-wood combination interface information, and steel-wood defect features are extracted; the steel-wood defect features are verified based on the steel defect features and the wood defect features, a defect detection result of the steel-wood combination component is constructed according to the verification result, and a component surface defect distribution map is drawn; defect influence analysis is performed according to the component surface defect distribution map, a plurality of defect levels are set to be mapped to the component surface defect distribution map, and surface defects of the steel-wood combination component are locked.

[0018] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0019] A region identification module is configured to collect a multi-modal image dataset of a surface of a steel-wood combined component, determine a steel region subset and a wood region subset; an overlap analysis module is configured to traverse the steel region subset and the wood region subset to perform double-layer mapping, and determine steel-wood combined interface information; a surface anomaly analysis module is configured to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract steel defect features and wood defect features; a defect feature extraction module is configured to perform combined surface anomaly analysis based on the steel-wood combined interface information, and extract steel-wood defect features; a defect distribution map drawing module is configured to verify the steel-wood defect features, and draw a component surface defect distribution map; and a defect influence analysis module is configured to perform defect influence analysis according to the component surface defect distribution map, set a plurality of defect levels to be mapped to the component surface defect distribution map, and lock the surface defects of the steel-wood combined component. The steel-wood combined component surface defect can be accurately detected, and the comprehensiveness of the component structure safety evaluation is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 A structural schematic diagram of a steel-wood combined component surface defect detection system based on image sensing provided by the embodiments of the present application is shown in the figure.

[0022] Figure 2 A flowchart of a steel-wood combined component surface defect detection method based on image sensing provided by the embodiments of the present application is shown in the figure.

[0023] Legend of the drawings: region identification module 10, overlap analysis module 20, surface anomaly analysis module 30, defect feature extraction module 40, defect distribution map drawing module 50, and defect influence analysis module 60. DETAILED DESCRIPTION

[0024] The present application provides a steel-wood combined component surface defect detection system and method based on image sensing, which is used to solve the technical problem that the steel-wood combined component surface defect detection is not accurate enough in the prior art, resulting in that the structure safety evaluation is not comprehensive enough.

[0025] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0026] Embodiment one, as shown in the present application provides an image sensor-based steel-wood combined component surface defect detection system, which comprises: Figure 1

[0027] A region identification module 10 is configured to collect a multi-modal image dataset of the surface of a steel-wood combined component, perform region identification based on the multi-modal image dataset, and determine a steel region subset and a wood region subset.

[0028] Specifically, a multi-modal image dataset of the surface of a steel-wood combined component is collected from different angles and under different lighting conditions by using various image collection devices such as industrial cameras, 3D cameras, and infrared sensors, covering visible light, near-infrared, thermal imaging, and other image information. Subsequently, a semantic segmentation algorithm based on deep learning, such as Mask R-CNN, is used. In the training stage, a large number of labeled steel-wood combined component image samples are used to enable the model to learn the differences between steel and wood in terms of texture, color, and spectral features. In the inference process, the collected multi-modal image is input into the trained model, and the model classifies each pixel in the image. Based on the classification results, the steel region subset and the wood region subset are accurately determined, and effective division of different material regions on the surface of the steel-wood combined component is achieved.

[0029] An overlap analysis module 20 is configured to traverse the steel region subset and the wood region subset for double-layer mapping, perform overlap analysis based on the mapping results, and determine steel-wood combined interface information.

[0030] Specifically, when traversing the steel region subset and the wood region subset for double-layer mapping and determining the steel-wood combined interface information, the steel region subset is first mapped to a first layer to generate a steel spatial grid layer, and the wood region subset is simultaneously mapped to a second layer to generate a wood spatial grid layer. Then, double-layer projection is performed on the two layers to generate a mapping result and extract a projection overlap region therefrom. Next, interface normal projection is performed on the projection overlap region to determine a projection plane, a plurality of grid points are determined in the region by traversing the region in a spiral path, multi-modal feature analysis is performed based on the grid points to form a feature set, and an effective overlap label is generated through third-order effectiveness verification. Finally, the effective overlap label is matched with the grid points, thereby determining the steel-wood combined interface information.

[0031] ​The surface anomaly analysis module 30 is configured to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract steel defect features and wood defect features.

[0032] Specifically, the image edge flow analysis is performed on the steel region subset, the gradient vector field is constructed, and the gradient change direction and amplitude are monitored. The average gradient amplitude is calculated as a constraint, and the target pixel chain is tracked along the gradient change direction. The target pixel chain is mapped to the steel region subset to obtain a surface anomaly data set containing coordinates, length, and gradient data. The depth step verification generates a step height parameter, and the thermal field verification generates a temperature gradient direction parameter, thereby determining the physical crack defect features. At the same time, the non-pixel chain region is verified by near-infrared reflection to generate a rust area parameter and determine the chemical corrosion defect features. For the wood region subset, the wavelet texture decomposition is performed to extract high-frequency components, the morphological opening operation is performed to filter out wood grain noise, and the connected domain with an area greater than 5 mm² is retained as a candidate area of wormhole. Then, the curvature mutation of the depth point cloud is calculated, and when the local curvature is greater than 0.3 mm⁻¹ and presents a linear distribution, it is marked as dry cracking, thereby extracting the wood defect features.

[0033] The defect feature extraction module 40 is configured to perform combined surface anomaly analysis based on the steel-wood joint interface information, and extract steel-wood defect features.

[0034] Specifically, when performing combined surface anomaly analysis based on the steel-wood joint interface information, the multi-modal image data of the steel-wood joint interface region is analyzed in a targeted manner according to the determined steel-wood joint interface information. By identifying the texture mutation, color difference, and three-dimensional topographic features at the interface, combined with the grid point distribution of the interface normal projection plane, the connection tightness of the interface region is analyzed. When it is detected that there is a gap with a width greater than 0.2 mm or a near-infrared reflectivity difference greater than 15% or a point cloud curvature mutation region presents a continuous distribution at the interface, it is judged that there is an abnormal situation such as debonding or cracking at the steel-wood joint interface, and then the steel-wood defect features are extracted, such as the position, size, and severity of the interface debonding.

[0035] The defect distribution map drawing module 50 is configured to verify the steel-wood defect features based on the steel defect features, the wood defect features, and construct a defect detection result of the steel-wood joint component according to the verification result, and draw a component surface defect distribution map.

[0036] Specifically, based on the steel defect features (covering steel corrosion, welding cracks, porosity, coating peeling, etc.) and wood defect features (including wood cracking, decay, insect damage, coating peeling, etc.), the steel-wood combination defect features (such as glue failure, loose connectors, and joint surface cracking, etc.) are cross-verified. First, the steel defect features are used to verify whether the joint interface anomaly is caused by the expansion of steel corrosion or the extension of welding cracks. For example, when the distance between the steel crack endpoint and the debonding area of the joint interface is less than 0.5 mm, it is determined that there is an extension correlation. At the same time, the wood defect features are used to verify whether the joint anomaly is caused by wood dry cracking penetrating to the interface or insect damage to the interface structure. If the coincidence degree of the wood insect hole cluster area and the joint interface gap is more than 30%, it is determined that there is a causal relationship. After verification, the steel defect layer (annotating parameters such as corrosion area and crack length), the wood defect layer (annotating information such as insect hole position and dry cracking depth), and the steel-wood defect layer (recording data such as interface debonding range and connector loosening torque value) are constructed. The three layers of defect data are mapped to a three-dimensional space grid. The D-S evidence theory is used to fuse the confidence of multiple sources (steel crack confidence, wood insect hole confidence, and interface debonding confidence). The HSV color space is used to map the comprehensive confidence to a pseudo-color code (for example, set confidence 0.0~0.2 to blue, 0.2~0.5 to yellow, 0.5~0.8 to orange, and 0.8~1.0 to red). Finally, based on the curvature of the component surface, an adaptive projection is performed to generate a three-dimensional defect distribution map containing defect type, size, and risk level. Each defect point in the map is associated with multiple modal feature parameters (such as steel corrosion spectrum reflectivity, wood dry cracking point cloud curvature, and interface debonding infrared temperature difference value).

[0037] The defect impact analysis module 60 is used to perform defect impact analysis based on the component surface defect distribution map. A plurality of defect levels are mapped to the component surface defect distribution map, and the surface defects of the steel-wood combination component are locked.

[0038] Specifically, when performing defect impact analysis based on the component surface defect distribution map, a grid coordinate system is first established on the distribution map. A plurality of grid elements are extracted, and local impact factors are calculated for each grid element, including steel local impact factors (such as corrosion area ratio and crack depth), wood local impact factors (such as insect hole density and dry cracking length), and combination local impact factors (such as interface debonding width and connector loosening degree). Based on these factors, the first, second, and third defect impact levels are set and integrated into a plurality of defect levels. The defect levels are mapped to the distribution map for structural safety analysis, and a structural safety index is calculated. Then, the distribution map is projected onto the steel-wood combination component entity according to the index, and warning boxes of different colors (such as red for high risk and yellow for medium risk) are determined. The surface defects of the component are precisely locked through the warning boxes, and the defect location and damage degree are intuitively displayed.

[0039] In a possible implementation manner, the overlap analysis module 20 further includes:

[0040] A steel material spatial mesh layer generation unit is configured to map the steel material region subset to a first layer to generate a steel material spatial mesh layer.

[0041] A wood material spatial mesh layer generation unit is configured to map the wood material region subset to a second layer to generate a wood material spatial mesh layer.

[0042] A mapping result generation unit is configured to perform double-layer projection on the steel material spatial mesh layer and the wood material spatial mesh layer to generate a mapping result, and extract a projection overlap region according to the mapping result.

[0043] An overlap validity analysis unit is configured to traverse the projection overlap region to determine a plurality of grid points, perform overlap validity analysis based on the plurality of grid points, and generate a plurality of valid overlap labels.

[0044] A steel-wood combination interface information determination unit is configured to match the plurality of valid overlap labels with the plurality of grid points to determine the steel-wood combination interface information.

[0045] Specifically, when the steel material region subset is mapped to the first layer, based on pixel coordinates and spatial position information of the steel material region subset, a regular spatial mesh is constructed in the first layer with a preset mesh precision (for example, 0.5 mm*0.5 mm), so that each mesh unit corresponds to an actual physical region of a steel material surface one by one, each pixel point in the steel material region subset is accurately positioned into a corresponding mesh unit of the mesh layer by a coordinate mapping algorithm, and thus a steel material spatial mesh layer containing spatial distribution information of the steel material surface is generated, providing a structured spatial data basis for subsequent double-layer projection and combination interface analysis.

[0046] When the wood material region subset is mapped to the second layer, a regular spatial mesh system is established in the second layer according to a preset mesh density (for example, 0.5 mm*0.5 mm) based on pixel coordinates and three-dimensional spatial coordinate information of the wood material region subset, each pixel point in the wood material region subset is one by one corresponded to a specific mesh unit of the mesh layer by a coordinate transformation algorithm, so that each mesh unit accurately maps an actual physical region of a wood material surface, and thus a wood material spatial mesh layer representing spatial distribution characteristics of the wood material surface is generated, providing a structured data support for subsequent double-layer projection analysis and accurate positioning of a steel-wood combination interface.

[0047] According to the double-layer projection of the steel spatial grid layer and the wood spatial grid layer, first, the two layers are unified to the same three-dimensional coordinate system to ensure the consistency of the spatial position. Then, the steel spatial grid layer and the wood spatial grid layer are superimposed and projected on the specified projection plane in the way of orthogonal projection or perspective projection, so that the grid units of the two layers form overlapping or non-overlapping area distribution on the projection plane, thereby generating a mapping result. Then, the mapping result is analyzed by an image recognition algorithm to identify the overlapping part of the steel grid and the wood grid on the projection plane, and the overlapping part is extracted as the projection overlapping area, which is the area where the steel and wood materials may be combined on the surface of the component, providing a key spatial position reference for subsequent determination of the steel-wood combination interface information.

[0048] When traversing the projection overlapping area to determine a plurality of grid points and performing overlapping effectiveness analysis, first, the projection overlapping area is projected on the interface normal to determine the interface normal projection plane, and then the projection overlapping area is traversed in a spiral path according to the projection plane to determine a plurality of grid points. Then, based on these grid points, multi-modal feature analysis is performed to extract the features of each grid point in the multi-modal data such as visible light images and near-infrared images to form a multi-modal feature set, and then the three-order effectiveness verification is performed according to the feature set, that is, the first-order verification calculates the reflectivity gradient of adjacent grid points to judge the material continuity, the second-order verification detects the thermal conductivity mutation to verify the thermal conduction consistency, and the third-order verification determines the geometric fit degree by calculating the local curvature difference of the depth point cloud. According to the third-order verification result, a plurality of effective overlapping labels are generated to determine the overlapping effectiveness of each grid point.

[0049] When matching the plurality of effective overlapping labels with the plurality of grid points, each effective overlapping label is associated with its corresponding grid point spatial coordinates by a coordinate mapping algorithm, and a set of grid points marked as “effective” is selected. Based on the spatial distribution characteristics of the effective grid point set, the Delaunay triangulation algorithm or the least squares method is used to fit the steel-wood material interface curve / surface, thereby determining the specific position, trend and geometric form of the steel-wood combination interface. In this process, the effective overlapping label is used to verify whether the grid point belongs to the true combination interface, and the continuous area formed by the matched effective grid points is the steel-wood combination interface, and the coordinate data and geometric features thereof can be directly used for subsequent combination surface anomaly analysis.

[0050] In one possible implementation manner, the overlapping effectiveness analysis unit further includes:

[0051] The interface normal projection sub-unit is configured to project the projection overlapping area on the interface normal to determine the interface normal projection plane.

[0052] The grid point determination subunit is configured to traverse the projection overlap region in a spiral path according to the interface normal projection plane to determine a plurality of grid points.

[0053] The multi-modal feature set determination subunit is configured to perform multi-modal feature analysis based on the plurality of grid points to determine a multi-modal feature set, perform third-order effectiveness verification according to the multi-modal feature set, and generate the plurality of effective overlap labels.

[0054] Specifically, when performing interface normal projection on the projection overlap region to determine the interface normal projection plane, principal component analysis (PCA) is first performed on the point cloud data in the projection overlap region. By calculating the variance distribution of the point cloud data in each dimension, the direction with the largest variance is determined as the normal direction of the interface, which is perpendicular to the steel-wood combination interface. Then, taking the normal direction as the reference, a plane perpendicular to the normal direction is constructed, which is the interface normal projection plane. The projection plane is used for subsequent feature analysis and grid point traversal of the projection overlap region, and provides a projection reference for accurately determining the steel-wood combination interface information.

[0055] When traversing the projection overlap region in a spiral path according to the interface normal projection plane, the interface normal projection plane is taken as the reference plane, and the traversal starts from the geometric center of the projection overlap region and expands outward along the Archimedean spiral trajectory. During the traversal process, position points are uniformly selected on the spiral path at a predetermined spatial sampling interval (such as 0.1 mm), and each position point corresponds to a spatial coordinate in the projection overlap region to determine a plurality of grid points. This spiral traversal method can ensure that each part of the projection overlap region is covered in an orderly and non-missing manner, thereby obtaining uniformly distributed grid points, which provides a basis for subsequent multi-modal feature analysis and overlap effectiveness verification based on the grid points.

[0056] When performing multi-modal feature analysis and completing third-order effectiveness verification based on multiple grid points, visible light images, near-infrared spectra and thermal imaging data are synchronously collected for each grid point, steel features (such as reflectivity at a spectral reflection peak of iron element of 680 nm, a thermal conductivity threshold of 15 W / (m·K)) and wood features (such as absorbance at a spectral absorption valley of cellulose of 1450 nm, a thermal conductivity threshold of 0.15 W / (m·K)) are extracted, and a multi-modal feature set including spectral reflectivity, thermal conductivity coefficient and three-dimensional coordinates of point cloud is constructed. First-order verification is performed by calculating a near-infrared reflectivity gradient (a threshold is set to 0.2 / μm) of adjacent grid points, and if the gradient suddenly changes, it is determined that the material is discontinuous. Second-order verification is performed by comparing a measured value of thermal conductivity of the grid point with a standard thermal conductivity of steel and wood materials, and when the difference is greater than 20%, thermal conduction abnormality is marked. Third-order verification is performed by using deep point cloud to calculate a local curvature difference (a threshold is 0.15 mm⁻¹), and a curvature mutation area is determined as geometric mismatch. Finally, according to the third-order verification result, an “effective overlap” or “invalid overlap” label is generated for each grid point, and only when the third-order verification passes, it is marked as effective.

[0057] The third-order effectiveness verification evaluation criteria of the multi-modal feature set are shown in Table 1:

[0058] Table 1: Third-order effectiveness verification evaluation criteria table

[0059]

[0060] In one possible implementation manner, the surface anomaly analysis module further includes:

[0061] A gradient vector field construction unit is configured to traverse the steel area subset to perform image edge flow analysis and construct a gradient vector field.

[0062] A change monitoring unit is configured to perform change monitoring based on the gradient vector field to determine a gradient change direction and a gradient change amplitude.

[0063] An average gradient amplitude acquisition unit is configured to perform average value calculation according to the gradient change amplitude to obtain an average gradient amplitude.

[0064] A target pixel chain determination unit is configured to take the average gradient amplitude as a limiting constraint, track according to the gradient change direction, and determine a target pixel chain.

[0065] A record result analysis unit is configured to record the target pixel chain as surface anomaly data of the steel area subset, analyze according to a record result, and obtain the steel defect feature.

[0066] Specifically, when performing image edge flow analysis on the steel material region subset, an edge detection algorithm such as a Canny operator or a Sobel operator is used to calculate the gradient amplitude and direction of each pixel point in the region, thereby constructing a gradient vector field. In this process, by calculating the gradient components of the pixel point in the x and y directions, the gradient value and gradient direction of each pixel point are obtained, and these gradient vectors collectively constitute a gradient vector field that describes the edge features of the steel material region subset, providing a basis for subsequent change monitoring and defect feature extraction based on the gradient vector field.

[0067] When monitoring changes based on the gradient vector field, the gradient vectors of adjacent pixel points are compared point by point to analyze the spatial change trend of the gradient, thereby determining the gradient change direction, i.e., the direction in which the pixel intensity changes most quickly; at the same time, the difference in gradient amplitude between adjacent pixel points is calculated to obtain the gradient change amplitude, quantifying the degree of change in the gradient. This process can effectively identify the edge and defect features on the surface of the steel material, providing key data support for subsequent extraction of steel defect features.

[0068] When calculating the average value based on the gradient change amplitude, the data of all gradient change amplitudes in the gradient vector field are first collected, then these data are added together and divided by the number of data to obtain the average gradient amplitude. This average gradient amplitude can reflect the overall level of gradient change on the surface of the steel material region subset, providing an important reference for subsequent determination of the target pixel chain.

[0069] When tracking according to the gradient change direction with the average gradient amplitude as a limiting constraint, the pixel points with a gradient amplitude greater than or equal to the average gradient amplitude in the gradient change direction are selected from the gradient vector field as starting points, and adjacent pixel points are connected in sequence along the gradient change direction to form a continuous pixel chain. In the tracking process, it is continuously judged whether the gradient amplitudes of adjacent pixel points satisfy the constraint condition of being greater than or equal to the average gradient amplitude, and if not, the tracking is terminated. The final continuous pixel chain is the target pixel chain, which can effectively represent the abnormal feature region on the surface of the steel material.

[0070] When the target pixel chain is recorded as the surface anomaly data of the steel material region subset, the endpoint coordinates of the target pixel chain are automatically extracted by a computer vision algorithm, the precise positions of the two endpoints are determined by using a contour detection and edge fitting algorithm, and the maximum pixel spacing of the pixel chain in the direction perpendicular to the strike direction is calculated and converted into the actual physical size to obtain the maximum width data. When analyzing the recording results, if the target pixel chain presents a continuous and steep step feature and the temperature gradient direction is consistent with the crack propagation direction, it is determined as a physical crack defect, and the depth and strike of the crack are determined in combination with the step height parameter and the temperature gradient direction parameter; if the near-infrared reflectivity of a non-pixel chain region is lower than the standard reflectivity threshold (such as 0.6) of the steel material, a chemical corrosion defect is determined according to the corrosion area parameter, so as to obtain the steel defect features including the defect type, position, size and morphology.

[0071] In a possible implementation manner, the recording result analysis unit further includes:

[0072] A surface anomaly data set acquisition subunit is configured to map the target pixel chain to the steel material region subset for analysis, and obtain a surface anomaly data set, wherein the surface anomaly data set includes target pixel chain coordinate data, target pixel chain length data and target pixel chain gradient data.

[0073] A first verification result generation subunit is configured to perform depth step verification based on the target pixel chain coordinate data and the target pixel chain gradient data, generate a first verification result, and mark according to the first verification result to obtain a step height parameter.

[0074] A second verification result generation subunit is configured to perform a thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data, generate a second verification result, and mark according to the second verification result to obtain a temperature gradient direction parameter.

[0075] A feature analysis subunit is configured to perform feature analysis according to the step height parameter and the temperature gradient direction parameter, and determine a physical crack defect feature.

[0076] A third verification result generation subunit is configured to perform near-infrared reflectance verification on a non-pixel chain region extracted according to the target pixel chain, generate a third verification result, mark according to the third verification result to obtain a corrosion area parameter, perform feature analysis according to the corrosion area parameter, and determine a chemical corrosion defect feature.

[0077] A steel defect feature adding subunit is configured to add the physical crack defect feature and the chemical corrosion defect feature to a steel defect feature.

[0078] Specifically, when mapping the target pixel chain to a subset of the steel region for analysis, the pixel coordinates of the target pixel chain are converted into coordinate data in the actual physical coordinate system through a conversion matrix of image coordinates and physical coordinates. Meanwhile, the number of consecutive pixel points of the target pixel chain is calculated using a contour tracking algorithm and converted into an actual length. The gradient amplitude and direction data of each pixel point are extracted to obtain a surface anomaly data set containing the coordinate data, length data, and gradient data of the target pixel chain, which provides basic data support for subsequent depth step verification, thermal field verification, etc.

[0079] When performing depth step verification based on the coordinate data and gradient data of the target pixel chain, the coordinate data is associated with the height information in the three-dimensional point cloud model to obtain the three-dimensional coordinate values of each point on the target pixel chain. Then, the gradient data is used to determine the positions where steps may exist. By calculating the height difference between adjacent point clouds, when the height difference exceeds a preset threshold (such as 0.1 mm), it is determined that there is a depth step, and a first verification result is generated. Then, according to the verification result, the depth step region is marked, and the maximum height difference is extracted as a step height parameter to represent the depth feature of the physical crack on the steel surface.

[0080] When performing thermal field verification based on the coordinate data and length data of the target pixel chain, the coordinate data is associated with the infrared thermal imaging data to obtain the temperature values at each position of the target pixel chain. Then, the temperature sampling path is determined according to the length data. Along the direction of the target pixel chain, temperature data is collected at fixed intervals (such as every 0.5 mm), the temperature difference between adjacent sampling points is calculated, and the direction with the maximum temperature change rate is determined as the temperature gradient direction. When the temperature gradient direction is consistent with the extension direction of the target pixel chain and the temperature change rate exceeds a preset threshold (such as 5°C / mm), a second verification result is generated, the temperature gradient direction is marked, and the direction is extracted as a temperature gradient direction parameter to represent the thermal conduction abnormal feature of the physical crack on the steel surface.

[0081] When performing feature analysis based on the step height parameter and the temperature gradient direction parameter, if the step height parameter exceeds a preset threshold (such as 0.1 mm), it indicates that there is a significant height mutation on the surface of the steel, and the temperature gradient direction parameter shows that the direction with the maximum temperature change rate is consistent with the extension direction of the target pixel chain and the change rate exceeds the threshold (such as 5°C / mm), then it can be determined that the target pixel chain is a physical crack defect. The step height parameter reflects the depth feature of the crack, and the temperature gradient direction parameter reflects the influence direction of the crack on heat conduction. The combination of the two can accurately represent the position, depth, and expansion trend of the physical crack and other defect features.

[0082] When the non-pixel chain area is extracted according to the target pixel chain for near-infrared reflection verification, first, the non-pixel chain area is demarcated by taking the target pixel chain as a reference and expanding a certain range (for example, 5 pixels on both sides of the pixel chain) outward. The reflection spectrum data of the area is collected by using a near-infrared camera, and compared with the standard near-infrared reflection spectrum of the steel material (for example, the reflectivity threshold at a wavelength of 680 nm is 0.6). When the reflectivity is lower than the threshold, it is determined as a rust area, and a third verification result is generated. Then, the rust area boundary is marked according to the verification result, the proportion of the rust area in the non-pixel chain area is calculated, and the actual rust area is converted to obtain the rust area parameter. If the rust area parameter exceeds a preset threshold (for example, the proportion is 10%), the chemical rust defect features are determined in combination with the shape and distribution characteristics of the rust area, such as the rust type (overall rust or local rust) and severity.

[0083] When the physical crack defect features and the chemical rust defect features are added to the steel material defect features, first, the characteristic parameters of the physical crack, such as the step height and the temperature gradient direction, and the characteristic parameters of the chemical rust, such as the rust area and the reflectivity, are integrated, and then these characteristic parameters are classified and recorded in the steel material defect feature database. Among them, the physical crack defect features are stored according to the position, depth and expansion trend, and the chemical rust defect features are classified according to the type, area and severity. Finally, a complete steel material defect feature set containing two types of defects, physical cracks and chemical rust, is formed, which provides comprehensive feature data support for subsequent defect detection and evaluation of steel-wood combined components.

[0084] In one possible implementation manner, the defect distribution mapping module 50 further includes:

[0085] The steel material crack verification unit is configured to perform steel material crack verification on the steel-wood defect features according to the steel material defect features, and obtain a steel material defect verification result.

[0086] The wood dryness verification unit is configured to perform wood dryness verification on the steel-wood defect features according to the wood defect features, and obtain a wood defect verification result.

[0087] The wood defect layer construction unit is configured to construct a steel material defect layer based on the steel material defect features, construct a wood defect layer based on the wood defect features, and construct a steel-wood defect layer based on the steel material defect verification result and the wood defect verification result.

[0088] The composite defect atlas construction unit is configured to map the steel material defect layer, the wood defect layer and the steel-wood defect layer to a three-dimensional space grid for fusion, and construct a composite defect atlas.

[0089] The composite defect atlas projection unit is configured to perform confidence analysis based on the composite defect atlas, project the composite defect atlas using pseudo-color encoding according to a plurality of confidence levels, and draw the component surface defect distribution map.

[0090] Specifically, when verifying the steel crack defect feature of the steel-wood defect feature according to the steel defect feature, the depth point cloud data of the steel part in the steel-wood defect feature is extracted first, and is compared with the step height parameter recorded in the steel defect feature to calculate the step height difference of the depth point cloud at the crack. If the step height difference of the depth point cloud is consistent with the step height parameter in the steel defect feature and exceeds a preset threshold (such as 0.1 mm), the verification is passed, the steel crack defect feature in the steel-wood defect feature is determined to be valid, and a steel defect verification result is generated, indicating that the steel part of the steel-wood combined component has a crack defect conforming to the feature; if not, the verification is failed, indicating that the defect feature does not conform to the steel crack feature.

[0091] When verifying the wood crack defect feature of the steel-wood defect feature according to the wood defect feature, the texture data and humidity distribution data of the wood part in the steel-wood defect feature are extracted first, and are compared with the crack feature parameters (such as texture fracture degree and humidity change threshold) recorded in the wood defect feature. By analyzing the continuity and integrity of the wood surface texture, if the texture is found to have obvious fracture and gap, and the humidity of the corresponding area is lower than the humidity threshold of the wood crack (such as lower than 12%), the verification is passed, the wood crack defect feature in the steel-wood defect feature is determined to be valid, and a wood defect verification result is generated, indicating that the wood part of the steel-wood combined component has a crack defect conforming to the feature; if these conditions are not met, the verification is failed, indicating that the defect feature does not conform to the wood crack feature.

[0092] When constructing the steel defect layer based on the steel defect feature, the physical crack defect feature (such as the step height parameter and the temperature gradient direction parameter) and the chemical corrosion defect feature (such as the corrosion area parameter) of the steel are mapped to a two-dimensional plane according to the spatial position and feature type to form a steel defect layer reflecting the steel surface defect distribution; when constructing the wood defect layer based on the wood defect feature, the crack and other defect features of the wood are mapped according to the corresponding rules to generate a wood defect layer; then, based on the steel defect verification result and the wood defect verification result, the defect features at the steel-wood combined interface are fused to construct a steel-wood defect layer to represent the defect condition of the steel-wood combined area.

[0093] When the steel defect layer, the wood defect layer and the steel-wood defect layer are mapped to a three-dimensional space grid for fusion, first, the defect feature data of each layer (including the physical crack, chemical corrosion feature of steel, dry crack feature of wood and defect feature of the steel-wood joint interface) is converted into three-dimensional space coordinates, and then mapped according to a unified grid coordinate system (such as taking the geometric center of the component as the origin, and the XYZ axes corresponding to the length, width and height directions), so that different types of defect features are accurately positioned in the three-dimensional space. Then, a weighted fusion algorithm is used, different weights are given according to the type and severity of the defect feature, the defect features in the overlapping area are fused and processed, the conflict data is eliminated, the typical defect features are retained, and finally a composite defect map containing the defect information of steel, wood and joint interface is constructed, realizing the three-dimensional visualization integration of multi-dimensional defect information.

[0094] When the composite defect map is used for confidence analysis, first, multiple confidence parameters such as initial steel crack confidence, initial wood hole confidence and initial interface debonding confidence are extracted from the composite defect map, then different weights are given according to the influence degree of each confidence on the safety of the component structure, and the spatio-temporal weighted comprehensive confidence is obtained by weighted calculation. Then, the composite defect map is mapped in color space according to the comprehensive confidence, different confidence intervals are corresponded to different color channels, a pseudo-color layer is constructed, and finally the surface adaptive projection is performed based on the pseudo-color layer, so that the defect map is matched with the surface of the component, and a defect distribution map directly reflecting the defect distribution and severity of the component surface is drawn.

[0095] In a possible implementation manner, the composite defect map projection unit further includes:

[0096] A confidence extraction subunit is configured to extract an initial steel crack confidence, an initial wood hole confidence and an initial interface debonding confidence based on the composite defect map.

[0097] A weighted analysis subunit is configured to perform weighted analysis according to the initial steel crack confidence, the initial wood hole confidence and the initial interface debonding confidence, and obtain a spatio-temporal weighted comprehensive confidence.

[0098] A pseudo-color layer construction subunit is configured to map the composite defect map in color space according to the spatio-temporal weighted comprehensive confidence, and construct a pseudo-color layer.

[0099] An adaptive projection subunit is configured to perform surface adaptive projection based on the pseudo-color layer, and draw the component surface defect distribution map.

[0100] Specifically, when extracting the initial steel crack confidence based on the composite defect map, the initial steel crack confidence is calculated by analyzing the step height parameter distribution density of the steel area in the map and the proportion exceeding the threshold of 0.1 mm, combining the consistency degree of the temperature gradient direction and the crack extension direction, and using a fuzzy logic algorithm; when extracting the initial wood wormhole confidence, the humidity distribution of the wood area texture fracture (the area proportion below the humidity threshold of 12%) and the wormhole morphological characteristics (such as hole diameter and depth) are counted, and a neural network model is used to output the initial wood wormhole confidence; when extracting the initial interface debonding confidence, the debonding area proportion of the steel-wood interface area and the number of overlapping invalid grid points in the interface normal projection are evaluated, and the initial interface debonding confidence is determined by an analytic hierarchy process. The three initial confidences respectively quantify the possibility and severity of different types of defects.

[0101] When performing weighted analysis on the initial steel crack confidence, the initial wood wormhole confidence, and the initial interface debonding confidence, first determine the weight coefficients of each confidence in the time and space dimensions, for example, set the time weight of the steel crack confidence to 0.2 and the space weight to 0.1, the time weight of the wood wormhole confidence to 0.2 and the space weight to 0.1, and the time weight of the interface debonding confidence to 0.2 and the space weight to 0.2, ensuring that the sum of all weight coefficients is 1. Then add each initial confidence to the corresponding time weight and space weight to obtain the time-space comprehensive weight of each confidence, and then multiply the initial confidence by the corresponding time-space comprehensive weight. Finally, add the products, that is, calculate the time-space weighted comprehensive confidence by the formula: time-space weighted comprehensive confidence = initial steel crack confidence × (0.2 + 0.1) + initial wood wormhole confidence × (0.2 + 0.1) + initial interface debonding confidence × (0.2 + 0.2). The time-space weighted comprehensive confidence takes into account the influence of different types of defects on the component in the time and space dimensions.

[0102] When performing color space mapping on the composite defect map according to the time-space weighted comprehensive confidence, first establish the mapping relationship between the confidence and the color space, for example, set confidence 0.0~0.2 to correspond to blue, 0.2~0.5 to correspond to yellow, 0.5~0.8 to correspond to orange, and 0.8~1.0 to correspond to red. Then substitute the time-space weighted comprehensive confidence of each grid point in the composite defect map into the mapping relationship to assign a corresponding color value to each grid point, forming a pseudo-color layer, realizing the visual expression of the defect confidence, and intuitively presenting the defect areas of different confidences in different colors.

[0103] When the pseudo-color layer is projected on the curved surface, the surface curvature data of the steel-wood combined component is obtained through three-dimensional modeling, each pixel point of the pseudo-color layer is mapped and matched with the triangular mesh model of the component surface, the projection angle of the pixel point is adjusted according to the surface normal vector, the color transition of the concave-convex area of the curved surface is processed by using the bicubic spline interpolation, the pseudo-color layer is deformed and fitted with the component surface curvature, and finally the visualization distribution map containing the defect position, type and severity information is generated in the component entity surface coordinate system, so that the precise superposition of the defect information and the component geometry is realized.

[0104] In a possible implementation manner, the defect influence analysis module 60 further includes:

[0105] A grid coordinate system establishing unit is configured to establish a grid coordinate system based on the component surface defect distribution map, and extract a plurality of grid units by traversing the grid coordinate system.

[0106] An influence factor determining unit is configured to perform local influence calculation by traversing the plurality of grid units, determine a plurality of local influence factors, and set a plurality of defect levels according to the plurality of local influence factors.

[0107] A structure safety index obtaining unit is configured to map the plurality of defect levels to the component surface defect distribution map for structure safety analysis, and obtain a structure safety index.

[0108] A warning box determining unit is configured to project the component surface defect distribution map to the entity of the steel-wood combined component according to the structure safety index, determine a warning box, and lock the surface defect of the steel-wood combined component through the warning box.

[0109] Specifically, when the grid coordinate system is established based on the component surface defect distribution map, the grid lines are equally divided along the length and width directions with the geometric center of the component as the origin, to form a regular grid coordinate system. The grid line spacing can be set according to the component size and detection accuracy requirement, for example, 10 mm x 10 mm. After the coordinate system is established, the grid coordinate system is traversed in the order from top to bottom and from left to right, each region formed by the intersection of each grid line is extracted as a grid unit, to ensure that all areas of the component surface defect distribution map are covered, thereby obtaining a plurality of grid units, which provide basic analysis units for subsequent local influence calculation.

[0110] When traversing multiple grid cells for local influence calculation, first, for each grid cell, parameters such as defect type, size, number, and confidence level are established to build a local influence calculation model. For example, for a grid cell in the steel area, the weakening ratio of the steel strength caused by the crack length and depth is calculated, for a grid cell in the wood area, the influence degree of the wood carrying capacity caused by the wormhole or dry crack is evaluated, and the influence of the steel-wood interface area debonding area on the overall structure connection strength is considered, so as to determine multiple local influence factors. Then, multiple defect levels are set according to the numerical range of the local influence factor, for example, the local influence factor ≤ 0.3 is set as a low level, 0.3 < local influence factor ≤ 0.6 is set as a medium level, and local influence factor > 0.6 is set as a high level, so as to realize the grading and definition of the defect severity of different grid cells.

[0111] When mapping multiple defect levels to the component surface defect distribution map for structure safety analysis, first, each defect level is assigned a corresponding safety influence coefficient, such as a low-level defect corresponding to a safety influence coefficient of 0.2, a medium level corresponding to 0.5, and a high level corresponding to 0.8. Then, the defect level and its safety influence coefficient of each grid cell are superimposed on the component surface defect distribution map, and according to the location, type and severity of the defect, the influence on the overall structure strength and stiffness of the component is analyzed, and the structure safety index is calculated by weighted summation, for example, structure safety index = 1 - Σ (defect level safety influence coefficient x defect distribution area proportion), so as to quantitatively evaluate the structure safety condition of the component.

[0112] According to the structure safety index, the component surface defect distribution map is projected to the steel-wood combined component entity, first, the structure safety index threshold is set, such as 0.6, when the structure safety index of a certain area is lower than the threshold, it is determined that the defect area needs to be warned. Then, based on the three-dimensional model of the component, the low safety index area on the defect distribution map is mapped to the corresponding position on the entity surface, and a warning box is generated around the area, and the size and color of the warning box are adjusted according to the defect level, such as high-level defect with red large box, medium-level defect with yellow medium box, and low-level defect with blue small box. In this way, the two-dimensional defect distribution map is accurately projected onto the three-dimensional entity, realizing the visualization locking of the surface defects of the steel-wood combined component, and facilitating the rapid positioning and processing of the defect area.

[0113] In one possible implementation, the influence factor determination unit further includes:

[0114] The influence factor analysis subunit is configured to analyze the multiple local influence factors to determine multiple steel local influence factors, multiple wood local influence factors, and multiple combination local influence factors.

[0115] The first defect influence level setting subunit is configured to determine defect influence of the component surface defect distribution map according to the plurality of steel local influence factors, and set a first defect influence level.

[0116] The second defect influence level setting subunit is configured to determine defect influence of the component surface defect distribution map according to the plurality of wood local influence factors, and set a second defect influence level.

[0117] The third defect influence level setting subunit is configured to determine defect influence of the component surface defect distribution map according to the plurality of combination local influence factors, and set a third defect influence level.

[0118] The influence level integrating subunit is configured to integrate the first defect influence level, the second defect influence level and the third defect influence level, and construct the plurality of defect levels.

[0119] Specifically, when analyzing based on the plurality of local influence factors, the local influence factors are divided into steel local influence factors, wood local influence factors and combination local influence factors according to the material type and position of the defects. The steel local influence factors include the influence degree of physical crack characteristics (such as step height parameter, temperature gradient direction parameter) and chemical corrosion characteristics (such as corrosion area parameter) on the performance of steel; the wood local influence factors include the influence parameters of the width of wood dryness, the size and density of worm holes on the strength of wood; and the combination local influence factors involve the influence indexes of the steel-wood interface debonding area ratio and the interface normal projection characteristics on the steel-wood combination strength. Through the analysis of these factors, the classification and quantification of the influence of defects on different materials and interfaces are realized.

[0120] When determining defect influence of the component surface defect distribution map according to the plurality of steel local influence factors, the step height parameter, the temperature gradient direction parameter and the corrosion area parameter in the steel local influence factors are analyzed first. When the step height is greater than 0.3 mm and the corrosion area ratio is greater than 5%, it is determined as high influence; when the step height is 0.1-0.3 mm and the corrosion area ratio is 2%-5%, it is determined as medium influence; and when the step height is less than 0.1 mm and the corrosion area ratio is less than 2%, it is determined as low influence. Accordingly, the first defect influence level is set, and the classification of the influence degree of the steel area defect is realized.

[0121] When the component surface defect distribution map is judged for defect influence according to multiple wood local influence factors, first, parameters such as the width of dry cracking, the size and density of worm holes in the wood local influence factors are analyzed. When the width of dry cracking is more than 1.5 mm, the diameter of worm holes is greater than 3 mm, and the density is more than 3 per 10 cm2, it is judged as high influence; when the width of dry cracking is between 0.5 and 1.5 mm, the diameter of worm holes is between 1 and 3 mm, and the density is between 1 and 3 per 10 cm2, it is judged as medium influence; when the width of dry cracking is less than 0.5 mm, the diameter of worm holes is less than 1 mm, and the density is less than 1 per 10 cm2, it is judged as low influence, thereby setting the second defect influence level and realizing the grading of the influence degree of the wood area defect.

[0122] When the component surface defect distribution map is judged for defect influence according to multiple combined local influence factors, mainly parameters such as the area ratio of steel-wood interface debonding and the normal projection characteristics of the interface are analyzed. When the area ratio of debonding is more than 10% and the number of overlapping invalid grid points in the normal projection of the interface is relatively large, it is judged as high influence; when the area ratio of debonding is between 5% and 10% and the number of overlapping invalid grid points is moderate, it is judged as medium influence; when the area ratio of debonding is less than 5% and the number of overlapping invalid grid points is relatively small, it is judged as low influence, thereby setting the third defect influence level and realizing the grading of the influence degree of the steel-wood combined area defect.

[0123] When the first defect influence level, the second defect influence level and the third defect influence level are integrated, first, a three-dimensional defect influence matrix is established, taking the defect influence levels of the steel, wood and steel-wood combined area as three dimensions, and an integration rule is set. When any one of the three areas is of a high influence level, the integrated defect level is judged as a high level; if two areas are of a medium influence level and the other is of a low influence level, it is judged as a medium-high level; if all the three areas are of a low influence level or only one area is of a medium influence level, it is judged as a low level. Through this multi-dimensional weighted integration method, multiple defect levels covering the comprehensive influence of defects of different materials and interfaces are constructed, and the severity of the component surface defect is comprehensively reflected.

[0124] In the second embodiment, based on the same inventive concept as the steel-wood combined component surface defect detection system based on image sensing in the foregoing embodiments, as shown in Figure 2 The present application provides a steel-wood combined component surface defect detection method based on image sensing, and the method and system embodiments in the present application are based on the same inventive concept. The method comprises the following steps:

[0125] Step S100: Collecting a multi-modal image data set of the steel-wood combined component surface, identifying regions based on the multi-modal image data set, and determining a steel region subset and a wood region subset.

[0126] Step S200: traversing the steel material region subset and the wood material region subset to perform double-layer mapping, performing overlap analysis according to the mapping result, and determining steel-wood combination interface information.

[0127] Step S300: performing surface anomaly analysis on the steel material region subset and the wood material region subset respectively, and extracting steel defect features and wood defect features.

[0128] Step S400: performing combination surface anomaly analysis based on the steel-wood combination interface information, and extracting steel-wood defect features.

[0129] Step S500: verifying the steel-wood defect features based on the steel defect features and the wood defect features, constructing a defect detection result of a steel-wood combination component according to the verification result, and drawing a component surface defect distribution map.

[0130] Step S600: performing defect influence analysis according to the component surface defect distribution map, mapping a plurality of defect levels to the component surface defect distribution map, and locking the surface defects of the steel-wood combination component.

[0131] Further, the method further comprises:

[0132] mapping the steel material region subset to a first layer to generate a steel space grid layer, mapping the wood material region subset to a second layer to generate a wood space grid layer, performing double-layer projection on the steel space grid layer and the wood space grid layer to generate a mapping result, extracting a projection overlap region according to the mapping result, traversing the projection overlap region to determine a plurality of grid points, performing overlap effectiveness analysis based on the plurality of grid points to generate a plurality of effective overlap labels, and matching the plurality of effective overlap labels with the plurality of grid points to determine the steel-wood combination interface information.

[0133] Further, the method further comprises:

[0134] performing interface normal projection on the projection overlap region to determine an interface normal projection plane, traversing the projection overlap region in a spiral path according to the interface normal projection plane to determine a plurality of grid points, performing multi-modal feature analysis based on the plurality of grid points to determine a multi-modal feature set, and performing third-order effectiveness verification according to the multi-modal feature set to generate the plurality of effective overlap labels.

[0135] Further, the method further comprises:

[0136] Traverse the steel material region subset to perform image edge flow analysis and construct a gradient vector field; perform change monitoring based on the gradient vector field to determine the gradient change direction and the gradient change amplitude; calculate the average value based on the gradient change amplitude to obtain the average gradient amplitude; use the average gradient amplitude as a limiting constraint to track according to the gradient change direction to determine the target pixel chain; record the target pixel chain as the surface anomaly data of the steel material region subset, analyze the recording results to obtain the steel defect characteristics.

[0137] Further, the method further comprises:

[0138] Map the target pixel chain to the steel material region subset for analysis to obtain a surface anomaly data set, which includes target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data; perform depth step verification based on the target pixel chain coordinate data and the target pixel chain gradient data to generate a first verification result, and mark according to the first verification result to obtain a step height parameter; perform thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data to generate a second verification result, and mark according to the second verification result to obtain a temperature gradient direction parameter; perform feature analysis according to the step height parameter and the temperature gradient direction parameter to determine the physical crack defect feature; perform near-infrared reflection verification on the non-pixel chain region extracted according to the target pixel chain to generate a third verification result, and mark according to the third verification result to obtain a rust area parameter, and perform feature analysis according to the rust area parameter to determine the chemical rust defect feature; add the physical crack defect feature and the chemical rust defect feature to the steel defect feature.

[0139] Further, the method further comprises:

[0140] Perform steel crack verification on the steel-wood defect feature according to the steel defect feature to obtain a steel defect verification result; perform wood dryness crack verification on the steel-wood defect feature according to the wood defect feature to obtain a wood defect verification result; construct a steel defect layer based on the steel defect feature, construct a wood defect layer based on the wood defect feature, and construct a steel-wood defect layer based on the steel defect verification result and the wood defect verification result; map the steel defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional space grid for fusion to construct a composite defect map; perform confidence analysis based on the composite defect map, project the composite defect map using pseudo-color coding according to multiple confidences, and draw the component surface defect distribution map.

[0141] Further, the method further comprises:

[0142] extract an initial steel crack confidence, an initial wood wormhole confidence and an initial interface debonding confidence based on the composite defect map; perform weighted analysis according to the initial steel crack confidence, the initial wood wormhole confidence and the initial interface debonding confidence to obtain a spatio-temporal weighted comprehensive confidence; perform color space mapping on the composite defect map according to the spatio-temporal weighted comprehensive confidence to construct a pseudo-color layer; and perform curved surface adaptive projection based on the pseudo-color layer to draw the component surface defect distribution map.

[0143] Further, the method further comprises:

[0144] establishing a grid coordinate system based on the component surface defect distribution map, extracting a plurality of grid units by traversing the grid coordinate system; performing local influence calculation by traversing the plurality of grid units to determine a plurality of local influence factors, setting a plurality of defect levels according to the plurality of local influence factors; mapping the plurality of defect levels to the component surface defect distribution map for structure safety analysis to obtain a structure safety index; and projecting the component surface defect distribution map to the entity of the steel-wood combined component according to the structure safety index to determine a warning box, and locking the surface defects of the steel-wood combined component through the warning box.

[0145] Further, the method further comprises:

[0146] performing analysis based on the plurality of local influence factors to determine a plurality of steel local influence factors, a plurality of wood local influence factors and a plurality of combined local influence factors; performing defect influence determination on the component surface defect distribution map according to the plurality of steel local influence factors to set a first defect influence level; performing defect influence determination on the component surface defect distribution map according to the plurality of wood local influence factors to set a second defect influence level; performing defect influence determination on the component surface defect distribution map according to the plurality of combined local influence factors to set a third defect influence level; and integrating the first defect influence level, the second defect influence level and the third defect influence level to construct the plurality of defect levels.

[0147] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. Moreover, the above-mentioned describes specific embodiments of the present application. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0148] The above only describes the preferred embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0149] The specification and drawings are, of course, to be regarded in an illustrative rather than a restrictive sense. It is to be understood that any such modifications, variations, changes or equivalents which fall within the spirit and scope of the application are intended to be embraced by the application. Thus, it is intended that the application covers all modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A surface defect detection system for steel-wood composite components based on image sensing, characterized in that, The system includes: The region recognition module is used to collect a multimodal image dataset of the surface of the steel-wood composite component, and perform region recognition based on the multimodal image dataset to determine the steel region subset and the wood region subset; The overlap analysis module is used to traverse the steel region subset and the wood region subset to perform dual-layer mapping, perform overlap analysis based on the mapping results, and determine the steel-wood interface information. The surface anomaly analysis module is used to perform surface anomaly analysis on the steel region subset and the wood region subset respectively, and extract the defect features of the steel and the defect features of the wood. The defect feature extraction module is used to perform surface anomaly analysis based on the steel-wood bonding interface information and extract steel-wood defect features. The defect distribution map drawing module is used to verify the steel-wood defect features based on the steel defect features and the wood defect features, construct the defect detection results of the steel-wood composite component based on the verification results, and draw the defect distribution map of the component surface. The defect impact analysis module is used to perform defect impact analysis based on the defect distribution map of the component surface, set multiple defect levels and map them to the defect distribution map of the component surface, and lock the surface defects of the steel-wood composite component. The overlap analysis module also includes: A steel spatial mesh layer generation unit is used to map the subset of the steel region to the first layer to generate a steel spatial mesh layer; The timber spatial grid layer generation unit is used to map the subset of the timber region to the second layer to generate a timber spatial grid layer. The mapping result generation unit is used to perform dual-layer projection on the steel space grid layer and the wood space grid layer to generate a mapping result, and to extract the projection overlap area based on the mapping result; The overlap validity analysis unit is used to traverse the projected overlap area to determine multiple grid points, perform overlap validity analysis based on the multiple grid points, and generate multiple valid overlap labels. The steel-wood interface information determination unit is used to match the plurality of valid overlapping labels with the plurality of grid points to determine the steel-wood interface information.

2. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 1, characterized in that, The overlap validity analysis unit further includes: An interface normal projection subunit is used to perform interface normal projection on the overlapping projection area to determine the interface normal projection surface. The grid point determination sub-unit is used to traverse the overlapping area of ​​the projection along a spiral path according to the interface normal projection surface to determine multiple grid points; A multimodal feature set determination subunit is used to perform multimodal feature analysis based on the multiple grid points, determine the multimodal feature set, perform third-order validity verification according to the multimodal feature set, and generate the multiple valid overlapping labels.

3. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 1, characterized in that, The surface anomaly analysis module also includes: The gradient vector field construction unit is used to traverse the subset of the steel region to perform image edge flow analysis and construct a gradient vector field. A change monitoring unit is used to monitor changes based on the gradient vector field and determine the gradient change direction and gradient change magnitude. The average gradient magnitude acquisition unit is used to calculate the average value based on the gradient change magnitude to obtain the average gradient magnitude. The target pixel chain determination unit is used to determine the target pixel chain by using the average gradient magnitude as a limiting constraint and tracking according to the gradient change direction. The recording result analysis unit is used to record the target pixel chain as a subset of the steel region surface anomaly data, analyze the recording results, and obtain the steel defect characteristics.

4. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 3, characterized in that, The record result analysis unit also includes: The surface anomaly dataset acquisition subunit is used to map the target pixel chain to the steel region subset for analysis to obtain the surface anomaly dataset, which includes target pixel chain coordinate data, target pixel chain length data, and target pixel chain gradient data. The first verification result generation subunit is used to perform depth step verification based on the target pixel chain coordinate data and the target pixel chain gradient data, generate a first verification result, mark it according to the first verification result, and obtain the step height parameter. The second verification result generation subunit is used to perform thermal field verification based on the target pixel chain coordinate data and the target pixel chain length data, generate a second verification result, mark it according to the second verification result, and obtain the temperature gradient direction parameter. The feature analysis subunit is used to perform feature analysis based on the step height parameter and the temperature gradient direction parameter to determine the physical crack defect characteristics. The third verification result generation subunit is used to extract non-pixel chain regions based on the target pixel chain for near-infrared reflection verification, generate the third verification result, mark the third verification result, obtain the corrosion area parameter, perform feature analysis based on the corrosion area parameter, and determine the chemical corrosion defect characteristics. A steel defect feature addition subunit is used to add the physical crack defect feature and the chemical corrosion defect feature to the steel defect feature.

5. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 1, characterized in that, The defect distribution map drawing module also includes: A steel crack verification unit is used to verify the steel cracks according to the steel defect characteristics and obtain the steel defect verification results. The wood crack verification unit is used to verify the wood cracking based on the wood defect characteristics and the steel-wood defect characteristics to obtain the wood defect verification results. A wood defect layer construction unit is used to construct a steel defect layer based on the steel defect characteristics, a wood defect layer based on the wood defect characteristics, and a steel-wood defect layer based on the steel defect verification results and the wood defect verification results. The composite defect map construction unit is used to map the steel defect layer, the wood defect layer, and the steel-wood defect layer to a three-dimensional spatial mesh for fusion to construct a composite defect map. The composite defect map projection unit is used to perform confidence analysis based on the composite defect map, and to project the composite defect map using pseudo-color encoding according to multiple confidence levels to draw the surface defect distribution map of the component.

6. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 5, characterized in that, The composite defect map projection unit further includes: The confidence extraction subunit is used to extract the initial confidence of steel cracks, the initial confidence of wood wormholes, and the initial confidence of interface debonding based on the composite defect map. The weighted analysis subunit is used to perform weighted analysis based on the initial confidence level of steel cracks, the initial confidence level of wood wormholes, and the initial confidence level of interface debonding to obtain a spatiotemporal weighted comprehensive confidence level. The pseudo-color layer construction subunit is used to perform color space mapping on the composite defect spectrum according to the spatiotemporal weighted comprehensive confidence level to construct a pseudo-color layer. An adaptive projection subunit is used to perform adaptive surface projection based on the pseudo-color layer to draw a surface defect distribution map of the component.

7. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 1, characterized in that, The defect impact analysis module also includes: A grid coordinate system establishment unit is used to establish a grid coordinate system based on the surface defect distribution map of the component, and to extract multiple grid units by traversing the grid coordinate system. An impact factor determination unit is used to traverse the multiple grid cells to perform local impact calculations, determine multiple local impact factors, and set multiple defect levels according to the multiple local impact factors. The structural safety index acquisition unit is used to map the multiple defect levels to the defect distribution map on the surface of the component for structural safety analysis and to obtain the structural safety index. The warning frame determination unit is used to project the surface defect distribution map of the component onto the solid of the steel-wood composite component according to the structural safety index, determine the warning frame, and lock the surface defects of the steel-wood composite component through the warning frame.

8. The surface defect detection system for steel-wood composite components based on image sensing as described in claim 7, characterized in that, The impact factor determination unit further includes: The influence factor analysis subunit is used to analyze the multiple local influence factors to determine multiple local influence factors for steel, multiple local influence factors for timber, and multiple combined local influence factors. The first defect impact level setting subunit is used to determine the defect impact of the component surface defect distribution map according to the multiple local influence factors of steel, and set the first defect impact level. The second defect impact level setting subunit is used to determine the defect impact of the component surface defect distribution map according to the multiple wood local impact factors and set the second defect impact level. The third defect impact level setting subunit is used to determine the defect impact of the component surface defect distribution map according to the multiple combined local impact factors, and set the third defect impact level. The impact level integration subunit is used to integrate the first defect impact level, the second defect impact level, and the third defect impact level to construct the multiple defect levels.

9. A method for detecting surface defects in steel-wood composite components based on image sensing, characterized in that, The method is implemented using the image sensing-based surface defect detection system for steel-wood composite components as described in any one of claims 1-8, and the method includes: A multimodal image dataset of the surface of the steel-wood composite component is collected, and region identification is performed based on the multimodal image dataset to determine the steel region subset and the wood region subset; The steel region subset and the wood region subset are traversed to perform dual-layer mapping. Based on the mapping results, overlap analysis is performed to determine the steel-wood interface information. Surface anomaly analysis was performed on the steel region subset and the wood region subset respectively to extract steel defect features and wood defect features; Based on the information of the steel-wood interface, anomaly analysis of the bonding surface is performed to extract the defect features of the steel and wood. Based on the defect characteristics of the steel and the defect characteristics of the wood, the defect characteristics of the steel and wood are verified. Based on the verification results, the defect detection results of the steel-wood composite component are constructed, and the surface defect distribution map of the component is drawn. Based on the surface defect distribution map of the component, a defect impact analysis is performed, and multiple defect levels are set and mapped to the surface defect distribution map of the component to lock the surface defects of the steel-wood composite component.

Images