Aluminum bar surface quality detection method and system based on visual detection

Abstract

The invention discloses an aluminum bar surface quality detection method and system based on visual inspection, and relates to the technical field of image processing, the system is composed of a plurality of functional modules, and the system comprises: a multi-modal data acquisition module for performing phase solution and triangulation by acquiring an aluminum bar structured light image to generate a three-dimensional point cloud; capturing an aluminum bar surface temperature field by using an infrared camera to generate a temperature image; the defect feature extraction module is used for performing normal vector estimation on the three-dimensional point cloud, identifying a surface discontinuous region through a normal vector change rate to generate a defect region, constructing a morphological feature vector, performing hierarchical feature extraction on the morphological feature vector by adopting a PointNet + + architecture, and outputting a morphological feature map; calculating temperature features based on the temperature image to construct a thermal field feature vector, inputting the thermal field feature vector into a preset thermal resistance defect correlation model, and outputting a thermal field feature map; and the defect identification and classification module is used for acquiring a morphological feature map and a thermal field feature map.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an aluminum bar surface quality detection method and system based on visual detection. BACKGROUND

[0002] Aluminum bars are widely used in aerospace, automotive manufacturing, building structures, electronic devices, etc. These industries have strict requirements for the mechanical properties and appearance quality of materials. Surface defects such as cracks, scratches, pits, and oxidation spots can become stress concentration points, reducing the fatigue strength, tensile strength, and other key properties of aluminum bars, thereby affecting the safety and service life of the final product. Aluminum bars are often used as raw materials for further processing such as extrusion, forging, rolling, and machining. Surface defects often reflect problems in the production process, such as cold shut, porosity during casting, poor lubrication or mold wear during extrusion, and oxidation or corrosion during heat treatment or cooling. If there are defects on the surface, it may cause tool wear or damage, lead to excessive surface roughness of the processed parts, cause defective or even scrap products, and increase production costs.

[0003] Traditional 3D structured light detection technology can achieve three-dimensional shape measurement, but it faces multiple challenges in the aluminum bar production scene. On a high-speed production line, traditional fixed-frequency stripe projection is prone to reconstruction blur due to aluminum bar movement. The existing system has a reconstruction frame rate of less than 30 Hz, which is difficult to match the production rhythm. The shielding problem caused by the cylindrical surface of the aluminum bar and the environmental light interference cause the stripe contrast to decrease and the point cloud noise rate to increase, directly affecting the defect feature extraction accuracy. At the same time, existing defect recognition relies on general deep learning models, lacks specific feature design for aluminum bar-specific defects, and the large number of model parameters causes long inference delay. There is an urgent need to develop an aluminum bar surface quality detection technology that adapts to high-speed production lines and has strong anti-interference ability. SUMMARY

[0004] To achieve the above purpose, the present application realizes the following technical solutions: An aluminum bar surface quality detection system based on visual detection, the system comprising: A multi-modal data acquisition module acquires aluminum bar structured light images for phase unwrapping and triangulation to generate three-dimensional point clouds; an infrared camera is used to capture the temperature field of the aluminum bar surface to generate a temperature image; A defect feature extraction module estimates the normal vector of the three-dimensional point cloud, identifies the surface discontinuous area through the normal vector change rate, and constructs a shape feature vector. PointNet++ architecture is used to extract hierarchical features from the shape feature vector, and output a shape feature map. Temperature features are calculated based on the temperature image to construct a thermal field feature vector, which is input into a pre-set thermal resistance defect correlation model to output a thermal field feature map; The defect recognition classification module obtains the morphology feature map and the thermal field feature map, uses a dual-modal attention fusion network to dynamically allocate weights of the morphology feature map and the thermal field feature map, inputs the full connection classifier, and outputs a defect type.

[0005] Further, the process of generating the three-dimensional point cloud is: A specially coded sinusoidal fringe structured light pattern is projected onto the surface of the aluminum bar, a plurality of cameras are used to synchronously capture images of the structured light pattern on the surface of the aluminum bar, phase unwrapping processing is performed on the captured plurality of structured light patterns, a four-step phase shift method is used to analyze the phase change of the fringes in the structured light pattern, and the phase distribution corresponding to each pixel point on the surface of the aluminum bar is calculated; based on the principle of triangulation, the phase distribution information is converted into three-dimensional space coordinates of each point on the surface of the aluminum bar, and through integration of the three-dimensional space coordinates, point cloud data of the three-dimensional topography of the surface of the aluminum bar is generated.

[0006] Further, the process of generating the temperature image is: The infrared camera captures the infrared radiation energy of the surface of the aluminum bar at a set frame rate, converts the infrared radiation energy into an electrical signal, converts the electrical signal into a gray value corresponding to the temperature through signal processing, and forms an infrared temperature image.

[0007] Further, the process of estimating the normal vector of the three-dimensional point cloud is: For each point in the three-dimensional point cloud, a plurality of points closest to the point are searched using k-neighborhood search to construct a neighborhood point set, and the centroid of the neighborhood point set is calculated; the coordinates of all points in the neighborhood are added and divided by the number of points to obtain a centralized point set; the coordinates of each point in the centralized point set are multiplied and summed, and then divided by the number of points to obtain a covariance matrix; eigenvalue decomposition is performed on the covariance matrix to obtain three eigenvalues arranged in descending order and corresponding three eigenvectors; the eigenvector corresponding to the smallest eigenvalue in the covariance matrix is taken as the normal vector of the point cloud surface, and the eigenvector is taken as the initial estimate of the normal vector of the point.

[0008] Further, the process of constructing the morphology feature vector is: The direction difference of the normal vectors of adjacent points is measured according to the point cloud normal vector estimation, the surface smoothness of the region where each target point in the three-dimensional point cloud is located is judged, a threshold of the normal vector change rate is set, abnormal points are selected from the point cloud, and a preliminary discontinuous region is formed by region aggregation, the preliminary discontinuous region contains the normal structure of the aluminum bar, and the non-defect region is excluded by combining the geometric characteristics and prior knowledge of the aluminum bar, the real defect region is segmented, the key parameters representing the geometric properties of each independent defect region are extracted, and the morphology feature vector is formed by combining the key parameters in a fixed order.

[0009] Further, the process of outputting the morphology feature map is: The three-dimensional point cloud and the morphological feature vector are integrated into the PointNet++ architecture, key areas in the three-dimensional point cloud are focused through grouping, sampling points are screened, and local neighborhoods are divided, the local neighborhoods are merged into local point cloud groups, local coordinate system conversion, feature mapping and global pooling are performed on the local point cloud groups, and the morphological features of the discrete points in the local point cloud groups are converted into aggregated features of local correlation. On the basis of the first aggregation layer, the multi-level features are extracted by repeating the sampling, grouping and aggregation processes.

[0010] Further, the process of constructing the thermal field feature vector is: The temperature image is preprocessed, the temperature reference is set and the region is analyzed, the temperature abnormal area related to the defect is located, the normal temperature fluctuation is excluded, the key features of the thermal physical properties of the aluminum rod defect are extracted from each temperature abnormal area, and the features are integrated in a fixed order. The normalized thermal field feature vector is formed by eliminating the order of magnitude difference.

[0011] Further, the process of outputting the thermal field feature map is: First, the importance of the five dimensions in the thermal field feature vector is evaluated, each thermal field feature vector corresponds to a temperature abnormal area, and the thermal field feature vector is corresponded to the spatial position of the area in the temperature image. The supervised learning model trained by a large number of defect samples converts the thermal field feature into a thermal resistance correlation feature that can represent the thermal resistance properties of the defect. The thermal resistance correlation feature is distributed to the corresponding pixel area of the temperature image through spatial position mapping. The eight-dimensional thermal resistance correlation feature of each grid unit is converted into pixel information of a two-dimensional image through feature dimension distribution, pixel value mapping and interpolation supplement, and a complete thermal field feature map is formed.

[0012] Further, the process of outputting the defect type is: The morphological feature map and the thermal field feature map are obtained, and a dual-modal attention fusion network is input. The feature is aligned, independently encoded, attention calculated, and dynamically weighted to assign weights to the two modal feature maps. The modal features with higher contribution degree are automatically strengthened in different areas of the defect. The feature map fused with the advantages of the two modalities is output, and is converted into a one-dimensional feature vector that can be received by a full connection classifier. A pre-trained full connection classifier is loaded, including an input layer, a double hidden layer, and an output layer. The input vector is calculated to output the probability of each class, and the class with the highest probability is selected as the defect type.

[0013] A method for detecting the surface quality of an aluminum rod based on visual detection, comprising the following steps: Step one, generate a three-dimensional point cloud by phase unwrapping and triangulation of the structured light image of the aluminum rod; and capture the temperature field of the aluminum rod surface to generate a temperature image using an infrared camera; Step two, normal vector estimation is performed on the three-dimensional point cloud, surface discontinuity areas are identified through normal vector change rate, and a morphological feature vector is constructed, a PointNet++ architecture is used to perform hierarchical feature extraction on the morphological feature vector, and a morphological feature map is output; a thermal field feature vector is constructed based on temperature image calculation temperature feature, and the thermal field feature vector is input into a preset thermal resistance defect correlation model to output a thermal field feature map; Step three, the morphological feature map and the thermal field feature map are obtained, the weight of the morphological feature map and the thermal field feature map is dynamically allocated by using a dual-modal attention fusion network, and the full connection classifier is input, and the defect type is output.

[0014] The aluminum bar surface quality detection method and system based on visual detection provided by the application have the following beneficial effects: (1) The three-dimensional point cloud generated by phase solving and triangulation can accurately capture the three-dimensional geometric morphology of defects such as cold separation and cracks, such as step height and crack depth, and make up for the deficiency of 2D technology that cannot quantify three-dimensional parameters; the infrared temperature image can reflect the thermal field distribution of the aluminum bar surface, and the thermal field features extracted by the thermal resistance defect correlation model can identify hidden defects such as subcutaneous bubbles and internal inclusions formed due to differences in heat conduction. The three-dimensional morphological features of such defects are not obvious, but the thermal field is significantly abnormal, and through the complementation of dual-modal data, the missing detection rate of hidden defects in the prior art is effectively reduced, and defect recognition in both geometric and thermal physical dimensions is realized.

[0015] (2) The normal vector estimation is performed by k-neighborhood search and covariance matrix feature decomposition, the surface discontinuity area is identified by using the normal vector change rate, the defects in the fuzzy area caused by the movement of the aluminum bar in high-speed production are located, such as the small cracks under high-speed transmission, and the reconstruction noise interference is avoided; at the same time, the PointNet++ architecture is used to perform hierarchical extraction on the morphological feature vector, the feature vector is constructed in combination with the exclusive geometric properties of the aluminum bar defects such as cold separation linear step and pore circular depression, the matching degree of the feature and the defect type is improved, and the thermal resistance defect correlation model is used to further filter the noise caused by environmental light and aluminum bar reflection through temperature abnormal amplitude, gradient and other exclusive thermal field features. In the scene where the production line speed is greater than or equal to 1 m / s, high feature extraction accuracy can still be maintained, and the adaptability problem of the existing 3D system in high-speed misalignment is solved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is a system flowchart of the application; Figure 2 It is a whole method schematic diagram of the application. DETAILED DESCRIPTION

[0017] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0018] Embodiment 1 Please refer to Figure 1 Embodiment 1 of the present application provides an aluminum bar surface quality detection system based on visual detection, which comprises: A multi-modal data acquisition module, which generates a three-dimensional point cloud by phase unwrapping and triangulation through acquisition of aluminum bar structured light images; and an infrared camera is used to capture the temperature field of the aluminum bar surface to generate a temperature image; Acquisition of aluminum bar structured light images: A structured light projection device is used to project a specially coded sinusoidal fringe pattern onto the surface of the aluminum bar. The specially coded sinusoidal fringe pattern includes background light intensity, contrast, spatial frequency and phase information, and can uniformly cover the circumferential surface of the aluminum bar. Three groups of high-resolution linear array structured light cameras are arranged at an angle of 120° along the circumference of the aluminum bar. When the aluminum bar is transported at high speed on the production line, the three groups of cameras synchronously acquire images of the structured light pattern on the surface of the aluminum bar, ensuring that the optical information on the surface of the aluminum bar is completely captured from different angles. The acquired multiple groups of structured light images are processed by phase unwrapping. A four-step phase shift method is used to accurately calculate the phase distribution of each pixel point on the surface of the aluminum bar by analyzing the phase change of the fringes in the images.

[0019] Process of generating three-dimensional point cloud: The acquired structured light images are processed by the four-step phase shift method. Four sinusoidal fringe patterns with fixed phase differences are sequentially projected by the structured light projection device, and the expression formulas are as follows:

[0020]

[0021]

[0022]

[0023] Wherein, A is the background light intensity, B is the fringe contrast, f is the spatial frequency, is the initial phase; , , and are sinusoidal fringe patterns with fixed phase differences, x and y are the horizontal and vertical coordinates of the pattern respectively.

[0024] After the line array structured light camera synchronously collects the four images, the phase value of each pixel point is calculated by the formula wherein, are four different fixed phase difference sinusoidal fringe patterns, respectively. The phase information related to the height of the aluminum rod surface is extracted by using the gray level change difference of the phase shift fringe to eliminate the interference of the background light intensity and the fringe contrast.

[0025] Phase unwrapping and continuous phase field construction: The reliable pixel points with continuous phase change and low noise are identified by calculating the gradient information of the phase; based on the reliable pixel points, the wrapped phase is gradually unwrapped into a continuous absolute phase field by the region growing method , the interference such as reflection and shadow on the surface of the aluminum rod is processed to ensure the continuity and accuracy of the phase unwrapping, and the continuous phase distribution corresponding to the height of the aluminum rod surface is obtained; Based on the principle of triangulation, the mapping relationship between the phase and the height of the aluminum rod surface is established; in the structured light system, the spatial positions of the camera and the projector are fixed, forming a triangulation system: the projector projects the fringe onto the surface of the aluminum rod, and the camera captures the modulated fringe image from another angle, the height change of the aluminum rod surface will cause the phase shift of the fringe, and the shift amount has a geometric correspondence with the height; let the baseline distance of the camera and the projector be d, the working distance, i.e. the distance from the camera to the reference plane of the aluminum rod be L, and the spatial frequency of the fringe be f, then the height h(x, y) of a point on the surface of the aluminum rod; based on the phase change and the height change, the optical phase information is directly related through the geometric relationship of the triangle, and the height information in the three-dimensional space is converted.

[0026] Based on the height calculation result, combined with the intrinsic parameters of the camera such as focal length, principal point coordinates and extrinsic parameters such as the position and attitude of the camera in the world coordinate system, the two-dimensional image coordinates (u, v) of each pixel point are converted into three-dimensional space coordinates (x, y, z), wherein the z component is the height h(x, y); the three-dimensional coordinates of all pixel points are integrated to form a three-dimensional point cloud data set wherein, P is the cloud data set, is each point in the three-dimensional point cloud ; is the three-dimensional space coordinate of the i-th point, the h(x, y) component corresponds to the height, i is the index of the point, used to identify different points in the point cloud, the value range is , and N is the total number of three-dimensional points in the point cloud.

[0027] The three-dimensional point cloud data set is preprocessed, statistical filtering or radius filtering algorithm is adopted, and abnormal points caused by aluminum rod surface reflection, camera noise are removed; three sets of structured light cameras are used to collect data from different angles, and point cloud registration algorithm is used to fuse multi-view point cloud into complete aluminum rod surface point cloud; and interpolation is performed on the sparse area and down sampling is performed on the over-dense area, so as to ensure the uniformity of point cloud density and facilitate the subsequent defect detection algorithm processing.

[0028] Temperature image acquisition process: By installing the infrared thermal imaging camera on the detection station of the aluminum rod production line, it is ensured that the field of view can completely cover the surface area of the aluminum rod. When the aluminum rod is transported on the production line, the infrared camera captures the infrared radiation energy of the aluminum rod surface at a set frame rate, which is usually adjusted according to the aluminum rod transmission speed to ensure the capture of the entire area of the aluminum rod surface. The temperature difference of different areas of the aluminum rod surface will cause different infrared radiation intensities, and the detector of the infrared camera will convert these infrared radiation energies into electrical signals. Through the signal processing circuit inside the camera, the electrical signals are converted into gray value or color value corresponding to the temperature, forming an infrared temperature image. At the same time, the lens of the infrared camera focuses the infrared radiation, and through the built-in temperature calibration module, the collected radiation energy is accurately converted into temperature value according to the physical characteristics of infrared radiation, such as Planck's law, so as to generate a temperature field image that can directly reflect the temperature distribution of the aluminum rod surface.

[0029] The defect feature extraction module estimates the normal vector of the three-dimensional point cloud, identifies the surface discontinuous area through the normal vector change rate to generate the defect area, constructs the morphological feature vector, adopts the PointNet++ architecture, performs hierarchical feature extraction on the morphological feature vector, and outputs the morphological feature map; the temperature feature is calculated based on the temperature image to construct the thermal field feature vector, which is input into the preset thermal resistance defect correlation model to output the thermal field feature map. Normal vector estimation of three-dimensional point cloud: For each point in the three-dimensional point cloud , determine its neighborhood point set, adopt k-neighborhood search or radius neighborhood search to find the k nearest points or all points within a specified radius, to form the neighborhood point set . The neighborhood point set is centered, that is, the centroid of the neighborhood point set is calculated , wherein is the centroid, and is each neighborhood point, then each neighborhood point is subtracted from the centroid to obtain the centered point set ; construct the covariance matrix, the formula is:

[0030] For covariance matrix Eigenvalue decomposition yields three eigenvalues. and the corresponding three feature vectors Since the normal vector of the point cloud surface corresponds to the smallest eigenvalue in the covariance matrix. The corresponding eigenvector Therefore, As a point The initial estimation of the normal vector is performed; to ensure the consistency of the normal vectors, i.e., the normal vectors of adjacent points point in the same approximate direction, the normal vector direction needs to be adjusted; by checking the orientation relationship between the normal vector and the local surface of the point cloud, or by using the normal vectors of neighboring points for voting adjustment, the normal vector direction of the entire point cloud is kept consistent, thus obtaining the accurate normal vector of each point. .

[0031] Constructing morphological feature vectors: The geometric information of 3D point cloud is transformed into features required for defect identification. Surface anomalies are located by quantifying the spatial variation differences of normal vectors. Key morphological parameters are then extracted from the anomaly regions to finally form feature vectors that can be used for defect classification and assessment.

[0032] Based on the estimated point cloud normal vectors, each point corresponds to a normal vector with a clear direction. By measuring the difference in the direction of the normal vectors of adjacent points, the surface smoothness of the area where the target point is located is determined. For each target point in the 3D point cloud, the neighborhood point set determined when its normal vector was estimated is used, that is, the several points closest to the target point. The number of neighborhood points needs to be set in advance according to the point cloud density to ensure that it can reflect the local surface features and ensure the consistency of the calculation benchmark. The angle between the normal vector of the target point and the normal vector of each point in the neighborhood is calculated. The cosine value of the angle between the two normal vectors is calculated by the dot product of the two normal vectors, and then the specific angle is obtained by the inverse cosine operation. Since the normal vector direction of the defect-free area of ​​the normal surface of the aluminum rod changes gently, the angle between the normal vectors of adjacent points is small. However, the abrupt change in surface morphology of defective areas such as cracks and dents will cause the angle between the normal vectors of adjacent points to increase significantly.

[0033] By setting a threshold for the rate of change of the normal vector, outlier points, i.e., points whose rate of change of the normal vector exceeds the standard, are screened out from the point cloud. Then, preliminary discontinuous regions are formed through region aggregation, while eliminating isolated noise interference. Determining the threshold for the rate of change of the normal vector: This threshold needs to be pre-calibrated based on the point cloud data of defect-free aluminum rods. First, a large number of three-dimensional point clouds of defect-free aluminum rods are collected, and the distribution of the rate of change of the normal vector of all points on their surface is statistically analyzed. The 95th percentile of the distribution is taken as the threshold. For example, the average rate of change of the normal vector of a normal aluminum rod surface is 0.1 radians, and the threshold can be set to 0.3 radians. When the rate of change of the normal vector of a certain point exceeds the threshold, the point is determined to be a surface discontinuity point, i.e., a candidate point of defect.

[0034] The connected component analysis is performed on the screened discontinuous points, and a region growing algorithm is used to take any one discontinuous point as a seed point. All points within a fixed spatial radius around the seed point that also satisfy the threshold of the normal vector change rate are grouped into the same region. The points in the region are taken as new seeds, and the expansion continues until no new points can be added. A minimum region point threshold, for example, 50 points, is set to filter out isolated small regions formed by point cloud noise, surface micro scratches, and non-quality-affecting defects. The number of points in a small region is usually less than the threshold, and the small region is directly removed. Multiple surface discontinuous regions are obtained, which are the preliminary candidate regions of defects.

[0035] The preliminary discontinuous regions may contain normal structures of the aluminum rod, such as end cutting marks and processing joints. The non-defect regions are excluded by combining the geometric features of the aluminum rod with the common sense of the production process, and the true defect regions are finally accurately segmented; the effective detection region of the aluminum rod is determined: according to the production parameters of the aluminum rod, such as diameter and length, the end region of 50 mm at both ends is excluded. The end region will form normal surface discontinuity due to the cutting process, and the normal vector change rate is high, which does not belong to defects. Only the main body region of the aluminum rod in the middle is retained as the detection object. If the preliminary discontinuous region is located at the end, it is directly determined as a non-defect region and removed.

[0036] The true defect region is verified in combination with the regular curved surface characteristics of the aluminum rod. The normal vector direction of the normal aluminum rod surface should be along the radial direction, outward or inward, and should match the direction adjusted in the previous normal vector consistency. If the overall direction of the normal vector of a preliminary discontinuous region deviates from the radial direction by more than 30°, and the region shape conforms to the common defect shape, such as linear cracks, block-shaped depressions, and circular-shaped bubble protrusions, the region is determined as a true defect region. If the region shape is a regular ring, it may be a positioning mark left by processing or the normal vector direction does not deviate significantly from the radial direction, and the region is determined as a non-defect region and removed.

[0037] The true defect region is independently segmented: if the distance between multiple true defect regions is less than 1 mm, it may be a segmented part of the same defect, such as a long crack being segmented into multiple small regions by local noise. The shortest distance between the regions is calculated to determine whether it is an extension of the same defect. If it is, the regions are merged into a single defect region. Finally, each independent defect region corresponds to a specific quality defect, such as a single crack, a single depression, or a single bubble protrusion. Key parameters that can represent the geometric properties of each independent defect region are extracted to distinguish different defect types, such as cracks, depressions, and bubbles, and to reflect the severity of the defects. The shape feature vectors are formed by combining the parameters in a fixed order. The defect area is first counted for the number of point clouds in the defect area, and then combined with the resolution during point cloud collection, i.e., the transverse and longitudinal dimensions covered by a single point on the surface of the aluminum bar, for example, 0.1 mm / point in the transverse direction and 0.1 mm / point in the longitudinal direction, and the coverage area of a single point is 0.01 mm2. The actual planar coverage area of the defect is obtained by multiplying the number of point clouds in the defect area by the coverage area of a single point, reflecting the size range of the defect; The maximum depth of the defect is taken as the reference plane of the normal surface points within a range of 5 mm around the periphery of the defect area. This area is free of defects and the surface is flat, which can represent the reference surface of the aluminum bar. The vertical distance between each point in the defect area and the reference plane is calculated, and the maximum value along the radial direction of the aluminum bar is taken. If it is a depression, the distance value is positive; if it is a protrusion, such as a bubble, the distance value is negative, thereby distinguishing the defect type and reflecting the longitudinal severity of the defect.

[0038] The defect perimeter is obtained by fitting the boundary points of the defect area through an algorithm, finding the outermost points of the area edge, connecting the boundary points in a clockwise or counterclockwise order to form a closed polygon, and calculating the total length of all sides of the polygon to obtain the perimeter of the defect, reflecting the complexity of the defect boundary. For example, the crack has a longer perimeter and a straight edge, while the depression has a shorter perimeter and an irregular edge.

[0039] The defect roundness is calculated by multiplying 4 times the circumference by the defect area and then dividing by the square of the perimeter. The closer the roundness value is to 1, the closer the defect shape is to a circle, such as a bubble protrusion. The smaller the value, the more irregular the defect shape, such as a crack, which usually has a roundness less than 0.2, used to quickly distinguish the geometric shape type of the defect.

[0040] The defect aspect ratio is fitted to the smallest circumscribed rectangle of the defect area, i.e., the smallest rectangle that can completely wrap the defect area. The rectangle edges are parallel or perpendicular to the aluminum bar axis. The length of the long side and the length of the short side of the rectangle are measured, and the aspect ratio is obtained by dividing the length of the long side by the length of the short side. The aspect ratio of a crack is usually greater than 10, which is elongated, while the aspect ratio of a depression is close to 1, which is blocky, and is a key parameter for distinguishing cracks from depressions.

[0041] The standard deviation of the normal vector of the defect area is calculated by calculating the standard deviation of the included angle of the normal vector of all points in the defect area. First, the average value of all included angles is calculated, then the square sum of the deviation of each included angle from the average value is calculated, and then the square root of the square sum divided by the number of points is obtained, reflecting the roughness of the surface of the defect area. For example, the surface of a corrosion defect is rough, and the standard deviation of the normal vector is large. The surface of a smooth depression is flat, and the standard deviation of the normal vector is small.

[0042] Output morphology feature map: Taking the three-dimensional point cloud of the defect area of the aluminum bar and the corresponding morphology feature vector as input, the discrete point-level morphology features are gradually converted into global feature maps with spatial correlation according to the hierarchical sampling, local aggregation, and multi-scale fusion of PointNet++. The three-dimensional point cloud and the morphological feature vector are integrated into a joint input data that can be processed by PointNet++, so as to eliminate the interference of data noise and scale difference on feature extraction; the input data includes two parts, including the three-dimensional point cloud of the aluminum rod defect area, that is, the spatial coordinates of each point, that is, the position information of x, y and z axes, and the morphological feature vector corresponding to each point, such as the local area contribution of the point in the defect area, the relative depth, the normal vector direction and the like, which are all from the defect morphological feature vector constructed in the early stage.

[0043] During preprocessing, isolated points generated by equipment noise in the point cloud are removed by statistical filtering, such as points with a distance from the neighborhood points more than 3 times the normal point cloud density, to ensure the integrity of the input point cloud; by feature normalization, each parameter in the morphological feature vector, such as depth and area contribution value, is mapped to the [0, 1] interval, which is obtained by linear transformation, to avoid the model paying excessive attention to the features with large magnitude and ignoring the key small magnitude features due to the magnitude difference of the parameters, such as depth in millimeters and area in square millimeters. Finally, a joint input matrix of point cloud coordinates and normalized morphological features is formed, each point corresponds to a high-dimensional feature vector, 3 coordinate dimensions and 6 morphological feature dimensions, a total of 9 dimensions.

[0044] By sampling and grouping the key areas in the point cloud, invalid calculation of redundant points is avoided, and local feature correlation is captured; by hierarchical operation, sampling points are selected from the initial joint input and local neighborhoods are divided; the farthest point sampling uniformly selects key sampling points from the preprocessed point cloud, the sampling number is usually 1 / 4-1 / 2 of the total number of input point clouds, which is adjusted according to the size of the aluminum rod defect, if the defect is small, the sampling ratio can be increased to 1 / 2 to retain more details; and a random initial point is selected as the first sampling point, then the point with the farthest distance from the selected sampling point is selected as the new sampling point each time, to ensure that the sampling points can uniformly cover the entire defect area, neither missing the defect edge details nor repeatedly collecting redundant points in the normal area; the spherical neighborhood grouping takes each sampling point as the center, sets a fixed radius of the spherical neighborhood, the radius size is set according to the average size of the defect, such as setting the neighborhood radius to 2 mm when the average diameter of the defect is 5 mm, to ensure that each neighborhood can cover the local area of the defect, and all original points in the neighborhood, including the morphological feature vector, are grouped into a group, forming multiple local point cloud groups; each local point cloud group corresponds to a sampling point, and the group contains the local spatial information and morphological feature information around the sampling point, laying a foundation for subsequent local feature extraction.

[0045] By local feature aggregation, the first aggregation layer extracts local morphological feature correlation, and through local coordinate system conversion, feature mapping and global pooling, the morphological features of the discrete points in the group are converted into aggregated features with local correlation. The local coordinate system conversion eliminates the influence of point cloud translation on the features. The coordinates of all points in each local point cloud group are centered on the sampling point of the group, i.e., the coordinates of each point are subtracted by the coordinates of the sampling point, so that the position information of the points in the group reflects their local spatial relationship relative to the sampling point, such as the offset direction and distance of the defect edge point relative to the sampling point, and the normalized morphological feature vector of each point in the group is retained to ensure that the local morphological information is not lost.

[0046] The multi-layer perceptron feature mapping concatenates the centered local point cloud coordinates and the morphological feature vector and inputs them into a 3-layer MLP, such as a first layer with an output dimension of 64, a second layer with an output dimension of 128, and a third layer with an output dimension of 256, for feature mapping. The MLP converts the low-dimensional coordinates and basic morphological features into high-dimensional local feature vectors through a nonlinear activation function, which can capture the spatial position correlation and morphological feature correlation of the points in the group, such as the corresponding relationship between the defect depth variation and the normal vector direction.

[0047] Max-pooling aggregation: for each local point cloud group after MLP mapping, a global max-pooling operation is performed, i.e., the maximum value of the feature vector of all points in the group is taken in each feature dimension to obtain a fixed-dimensional local aggregation feature. The advantage of max-pooling is that it can capture the most significant features in the local area, such as the feature corresponding to the maximum depth of the defect and the most obvious normal vector change, and it also unifies the features of each local point cloud group into a fixed dimension for subsequent processing. At this time, each sampling point corresponds to a 256-dimensional local aggregation feature, completing the first level of feature extraction.

[0048] Multi-level feature extraction extracts the subtle changes and global correlations of the crack edge by iteratively aggregating and fusing multi-scale morphological features, such as the overall relationship between the defect and the normal surface of the aluminum rod. On the basis of the first aggregation layer, the sampling, grouping, and aggregation process is repeated to realize multi-level feature extraction. The second level of sampling and grouping: taking the sampling points and corresponding 256-dimensional local aggregation features output by the first aggregation layer as input, the FPS is used again for sampling, with the number of samples being 1 / 2 of the first level of sampling points, to obtain fewer but more widely covered second level of sampling points. Then, taking the second level of sampling points as the center, a larger radius spherical neighborhood is set, such as 4mm, which is twice the radius of the first level of neighborhood, and the first level of sampling points in the neighborhood, including their local aggregation features, are grouped into new local point cloud groups.

[0049] Second level feature aggregation: for each second level local point cloud group, repeat the coordinate system conversion, take the second level sampling point as the origin, MLP feature mapping, such as 3-layer MLP, output dimension 512, maximum pooling process, get the 512-dimensional global aggregation feature corresponding to each second level sampling point; this feature combines the local detail features of the first level and the spatial correlation features of a wider range, such as the morphological differences between different regions of the defect.

[0050] Feature interpolation and fusion: since the number of second level sampling points is less than the first level, the 512-dimensional global aggregation features of the second level need to be mapped to all sampling points of the first level through interpolation and inverse distance weighted interpolation, so that each sampling point of the first level has both 256-dimensional local features and 512-dimensional global features. Then concatenate the two types of features and input them into a 1-layer MLP with an output dimension of 512 for fusion to obtain a 512-dimensional multi-scale aggregation feature for each first level sampling point. This feature contains both local detail information of the defect and global spatial correlation information, fully representing the morphological features of the defect.

[0051] Since multi-modal fusion needs to match the two-dimensional structured data of the infrared image, the discrete point-level features extracted by multi-level extraction are converted into two-dimensional structured morphological feature maps.

[0052] According to the geometric parameters of the aluminum rod, such as diameter and detection area length, the spatial coordinates of the three-dimensional point cloud are mapped to a two-dimensional plane. The axial direction of the aluminum rod is taken as the y-axis, and the circumferential direction is taken as the x-axis along the length direction of the aluminum rod. The 360° circumference is expanded into a linear coordinate. The three-dimensional coordinates (x, y, z) of each point are converted into two-dimensional coordinates (u, v) through circumferential expansion and axial projection, where u corresponds to the linear position after circumferential expansion, and v corresponds to the axial position. The z-coordinate height has been integrated into the multi-scale aggregation feature.

[0053] Feature map construction: according to the two-dimensional coordinates (u, v), the 512-dimensional multi-scale aggregation features of all first level sampling points are filled into a two-dimensional grid in the order of u-axis sorting and v-axis sorting. If there is no sampling point at a certain grid position, the feature value is supplemented through neighborhood interpolation, such as bilinear interpolation, to form a fixed-size two-dimensional morphological feature map, such as a resolution of 256x256. Each pixel corresponds to a fixed region on the surface of the aluminum rod, and the pixel value is the 512-dimensional multi-scale aggregation feature of the region.

[0054] The optimization of the feature map is performed by normalizing the generated morphological feature map in the channel dimension, the mean value of each feature channel is 0 and the variance is 1, so as to eliminate the magnitude difference of different channel features; meanwhile, the smoothing processing is performed through a 3*3 convolution kernel, the feature noise caused by interpolation is reduced, and a clear structure and complete feature morphological feature map is output, which not only retains the spatial distribution relationship of the aluminum bar surface morphological features, but also has the feature expression ability of multi-scale fusion, and can be directly used for multi-modal fusion with the infrared thermal field feature map.

[0055] Constructing a thermal field feature vector: Based on the aluminum bar surface temperature image collected by the infrared camera, the key features representing the defect thermal characteristics are extracted by pre-processing to eliminate interference, locate the temperature abnormal area and integrate into a structured thermal field feature vector; through temperature image preprocessing, interference is eliminated and the real temperature is calibrated.

[0056] The image is processed by 3*3 mean filtering or Gaussian filtering. The mean filtering replaces the original pixel gray value by calculating the average gray value of 9 pixels around each pixel, which can smooth the local high-frequency noise caused by the reflection of the aluminum bar surface, such as isolated bright spots or dark spots; the Gaussian filtering assigns weights according to the pixel distance, the center pixel has high weight and the edge pixel has low weight, which can remove noise while retaining the edge details of the temperature abnormal area, such as the temperature boundary between the defect and the normal area, to avoid feature loss caused by excessive filtering; The non-detection area in the temperature image is removed by image segmentation. The infrared image of the aluminum bar is a continuous long strip high-light area, and the background is a low-light area. Threshold segmentation is adopted, the gray threshold is set, the aluminum bar area is higher than the threshold, and the background is lower than the threshold. The background is removed. According to the production parameters of the aluminum bar, such as length and diameter, the end area of 50mm at both ends is cut off. The end area has normal heat residue due to cutting process, and the temperature distribution is abnormal, which is not caused by the thermal field change of the defect. Only the main body detection area in the middle is reserved to ensure that the subsequent feature calculation focuses on the effective area.

[0057] The gray value of the infrared image is converted into the real temperature value. The original image collected by the infrared camera represents the radiation intensity in gray value. The temperature calibration curve provided by the camera is used to map the gray value of each pixel to the actual temperature. For example, the gray value 200 corresponds to the temperature 35℃, and the gray value 220 corresponds to the temperature 40℃. Through linear interpolation or table lookup method, the value of each pixel in the image directly reflects the real temperature of the aluminum bar surface, avoiding the error caused by the subsequent feature calculation based on the gray value.

[0058] The surface defects of the aluminum bar, such as subcutaneous bubbles and internal inclusions, can cause local temperature abnormalities due to differences in heat conduction coefficients. By setting a temperature reference and conducting area analysis, these temperature abnormal areas related to defects can be located, and the interference of normal temperature fluctuations can be excluded.

[0059] The normal temperature reference of the effective detection area of the aluminum bar is calculated. Ten non-overlapping normal sub-areas are randomly selected in the effective area, each with an area of 5mm x 5mm. The average temperature of each sub-area is calculated, and the average of the ten average temperatures is taken as the global normal temperature The standard deviation of the temperatures of these sub-areas is also calculated , reflecting the temperature fluctuation range of the normal area. For example, if the average temperature of the normal area is 38℃ and the standard deviation is 0.8℃, the normal temperature fluctuation range is 38℃ ± 0.8℃.

[0060] By connected component analysis, abnormal pixels are aggregated to form continuous temperature abnormal areas. With any one abnormal pixel as a seed point, all pixels within a 3x3 range that also meet the abnormal condition are grouped into the same area, and the growth continues until no new pixel is added. At the same time, a minimum abnormal area threshold is set, such as 10 pixels, corresponding to an actual area of 0.2mm x 0.2mm, matching the thermal influence range of small defects in the aluminum bar, filtering out isolated small areas formed by local noise, and finally obtaining multiple temperature abnormal areas, i.e. the thermal influence areas of defects.

[0061] From each temperature abnormal area, key features reflecting the thermal physical properties of defects are extracted to distinguish different types of defects, such as bubbles and inclusions, and to reflect the severity of defects. The maximum deviation of the abnormal area from the normal temperature is calculated to reflect the influence strength of the defect on the thermal field. First, the temperature values of all pixels in the abnormal area are counted, and the highest or lowest temperature is found. Different defects have different temperature performances: bubbles will cause local temperature to be higher due to the large internal gas thermal resistance; metal inclusions will cause local temperature to be lower due to fast heat conduction. The difference between the highest or lowest temperature and the global normal temperature, i.e. the temperature abnormal amplitude, is calculated. For example, the highest temperature of a bubble area is 42℃, and the normal temperature is 38℃, so the abnormal amplitude is 4℃. The larger the amplitude, the more significant the difference in heat conduction of the defect, and the more severe the defect may be.

[0062] The temperature change rate of the abnormal region edge is calculated, reflecting the thermal transition characteristics of the defect and the normal region. First, the edge pixels of the abnormal region are found, i.e. the abnormal pixels adjacent to the normal temperature pixels, the temperature difference between each edge pixel and the adjacent normal pixel is calculated, and then the temperature difference is divided by the actual distance between the two pixels, which is determined according to the resolution of the infrared camera, for example, the pixel pitch is 0.1 mm, to obtain the temperature gradient of a single edge pixel; take the average of all edge pixel gradients as the average temperature gradient of the abnormal region. For example, the temperature abnormal region of the crack is linear, the temperature change of the edge is very steep, and the gradient is about 5 ℃ / mm, while the abnormal region of the bubble is circular, the temperature change of the edge is relatively gentle, and the gradient is about 2 ℃ / mm, the difference in gradient can be used to distinguish the morphology of the defect.

[0063] The area and shape factor of the abnormal region are included, reflecting the size and shape of the thermal influence range of the defect. The area is calculated by counting the number of pixels in the abnormal region and multiplying the actual area corresponding to a single pixel, for example, the camera resolution is 0.1 mm / pixel, and the area of a single pixel is 0.01 mm2, to obtain the actual thermal influence area of the abnormal region, for example, 100 pixels correspond to an area of 1 mm2, the larger the area, the larger the actual size of the defect usually means. The shape factor is calculated by the relationship between the perimeter and the area of the abnormal region to judge the shape, specifically by dividing the perimeter by the square root of the area, the ratio of a circular region is smaller, about 3.5; the ratio of a linear region is larger, about 10, for example, the abnormal region of the bubble is close to circular, the shape factor is smaller, the abnormal region of the crack is linear, the shape factor is larger, which can be used to intuitively distinguish the morphology of the defect.

[0064] The difference degree of temperature in the abnormal region is calculated, reflecting the consistency of the thermal distribution inside the defect. First, the average temperature of all pixels in the abnormal region is calculated, then the deviation of each pixel temperature from the average temperature is calculated, the sum of the squares of all deviations is added and divided by the number of pixels, and finally the square root is taken to obtain the standard deviation of the temperature distribution, the smaller the standard deviation, the more uniform the temperature in the abnormal region; the larger the standard deviation, the more obvious the temperature difference; for example, the standard deviation of the temperature distribution of the subcutaneous bubble is about 0.5 ℃ due to the uniform distribution of the internal gas, while the standard deviation of the temperature distribution of the internal inclusion is about 1.5 ℃ due to the uneven composition, which can assist in judging the internal properties of the defect.

[0065] If the aluminum bar is dynamically transmitted on the production line, for example, the transmission speed is 1 m / s, then 3 frames of temperature images of the same defect region are continuously collected, the frame interval is 0.1 s, the average temperature difference of the abnormal region between adjacent two frames is calculated, which is the thermal hysteresis characteristic. The temperature of the normal aluminum bar surface will change stably with the transmission, while the temperature change rate of the defect region is different from that of the normal region, for example, the normal region cools by 0.2 ℃ per frame, and the metal inclusion region cools by 0.5 ℃ per frame due to fast heat conduction, the thermal hysteresis characteristic can be used to further distinguish the defect types in the dynamic transmission scene.

[0066] The core temperature feature is normalized to eliminate the magnitude difference of different features, and finally a standardized thermal field feature vector is formed to ensure that it can be effectively fused with the morphological feature vector for defect recognition. The most critical 5 features for defect recognition are selected in a fixed order according to the influence intensity, transition characteristics, geometric shape, uniformity, and dynamic characteristics. If it is static detection, the aluminum rod transmission sequence is temperature abnormal amplitude, temperature gradient, abnormal area, shape factor, and temperature distribution uniformity. If it is dynamic detection, i.e. aluminum rod transmission, the temperature distribution uniformity is replaced by thermal hysteresis feature, so that the feature order conforms to the logical priority of defect recognition. The numerical value of each feature is mapped to the interval of 0 to 1 to avoid the model paying excessive attention to features with large magnitudes due to different magnitudes of features, such as area in square millimeters and temperature amplitude in degrees Celsius.

[0067] Specifically, the linear normalization method is used: first, the minimum and maximum values of the feature in a large number of defect samples are counted, then the feature value of a certain defect is subtracted from the minimum value, and finally divided by the difference between the maximum and minimum values, so that all features are in the same order of magnitude. For example, the sample minimum value of the temperature abnormal amplitude is 1℃, the maximum value is 10℃, and the amplitude of a certain defect is 4℃, then the normalized value is (4-1) ÷ (10-1) = 0.33; the normalized 5 features are arranged in a fixed order to form a thermal field feature vector with a dimension of 5. For example, the normalized features of a subcutaneous bubble are 0.4 temperature abnormal amplitude, 0.2 temperature gradient, 0.3 abnormal area, 0.18 shape factor, and 0.1 temperature distribution uniformity, respectively. The final thermal field feature vector is a set of these 5 values in order; this vector completely represents the thermal physical properties of the defect, and can be directly fused with the morphological feature vector through the attention mechanism to provide key basis for subsequent defect classification, such as distinguishing bubbles, inclusions, and cracks, in the thermal field dimension.

[0068] Output of the thermal field feature map: The thermal field feature vector constructed is taken as the input, the mapping relationship between the thermal field feature and the defect thermal resistance property is established according to the thermal resistance defect correlation model, and then the discrete vector is converted into a structured thermal field feature map through spatial position matching and feature reconstruction, which provides a spatial distribution bound thermal physical feature expression for subsequent multi-modal fusion.

[0069] The thermal field feature vector is discrete high-dimensional data. Through preprocessing, data interference is eliminated, and spatial position information is bound, so that the specific area of the aluminum bar surface can be associated after inputting the model, and the features and positions are avoided to be dislocated. The importance of the five dimensions in the thermal field feature vector, such as temperature anomaly amplitude, temperature gradient, and abnormal area, is evaluated. Based on the feature contribution degree during model training, such as the feature importance score calculated by the random forest algorithm, the core features with the top four contribution degrees, such as temperature anomaly amplitude, temperature gradient, shape factor, and temperature distribution uniformity, are retained, and the redundant features with low contribution degrees, such as the thermal lag feature in the dynamic scene, are removed if the model verifies that the effect of the defect thermal resistance correlation is small. Then, the numerical range of the core features is adjusted to the input interval of the model, such as 0-1, through linear scaling. Each thermal field feature vector corresponds to a temperature anomaly area. The vector is bound with the spatial position of the area in the original temperature image, the center pixel coordinates of the temperature anomaly area are recorded, such as the pixel center point coordinates of the area in the original temperature image, and the boundary pixel range of the area is labeled, such as the pixel box from the upper left corner to the lower right corner. The spatial coordinate information is used as an additional parameter, and the thermal field feature vector after preprocessing is integrated into a feature vector with a spatial label, so that the features output by the model can accurately correspond to the specific position of the aluminum bar surface.

[0070] The supervised learning model trained by a large number of defect samples converts the thermal field features into thermal resistance correlation features that can represent the thermal resistance properties of defects. The preset model file contains the trained weight parameters and network structure, which is trained by 5000 samples of known defect types, thermal field feature vectors, and actual thermal resistance measurement values. The samples cover common defects such as bubbles, inclusions, and cracks. The thermal resistance of each defect is measured by a laboratory thermal conductivity instrument. For example, the thermal resistance of a bubble is about 2.5 W / (m·K), and the thermal resistance of an aluminum bar substrate is about 237 W / (m·K). The input layer, hidden layer, and output layer parameters of the model are initialized, so that the input layer dimension is consistent with the dimension of the thermal field feature vector after preprocessing, such as 4-dimensional input, and the output layer dimension is the preset thermal resistance correlation feature dimension of the model, such as 8-dimensional, which covers the defect thermal resistance size, thermal resistance distribution uniformity, and difference between thermal resistance and substrate; The core features in the feature vector with a space label are input into the model input layer, and the correlation between the features and thermal resistance is established through the nonlinear transformation of the hidden layer, such as the ReLU activation function. The model hidden layer first calculates the correlation between the thermal field features and the defect thermal resistance, such as the greater the temperature anomaly amplitude, the more significant the difference between the thermal resistance of the corresponding defect and the matrix. Then, through multi-layer iterative optimization, the 4-dimensional thermal field features are converted into 8-dimensional thermal resistance correlation features. For example, the thermal field features of a subcutaneous bubble, with an amplitude of 0.4, a gradient of 0.2, a shape factor of 0.18, and a distribution uniformity of 0.1, after inputting into the model, the 8-dimensional thermal resistance correlation features output by the model have a thermal resistance difference degree dimension value of 0.8, close to 1, indicating a large difference in thermal resistance from the matrix, and a thermal resistance distribution uniformity dimension value of 0.9, indicating uniform bubble thermal resistance distribution, directly reflecting the thermal physical nature of the defect.

[0071] After the model outputs the 8-dimensional thermal resistance correlation features, the preset effectiveness threshold is automatically called, such as the feature confidence threshold of 0.7 determined by the validation set. The confidence of each dimension feature is calculated, i.e., the prediction reliability of the model for the feature value, which is determined based on the error distribution during training. If the confidence of all dimensions is higher than the threshold, it is determined that the thermal resistance correlation features are effective, and the subsequent steps are entered. If there is a dimension with a confidence lower than the threshold, such as a thermal resistance size dimension with a confidence of 0.5, the previous step is returned to, and the thermal field feature vector of the region is re-extracted and input into the model again until effective features are output, preventing distortion of the subsequent feature map due to feature errors.

[0072] The thermal resistance correlation features are high-dimensional data bound with a space label, which are accurately allocated to the corresponding pixel region of the original temperature image through spatial position mapping, ensuring that the spatial distribution of the thermal field feature map is consistent with the actual thermal field distribution of the aluminum bar surface. The temperature image of the effective area of the aluminum rod after pretreatment is divided into a fixed-size feature mapping grid, such as 8x8 pixels, with each grid cell corresponding to a small area on the surface of the aluminum rod, such as a grid cell size of 0.8mmx0.8mm, matching the spatial coverage range of the thermal resistance correlation feature. Record the center pixel coordinates of each grid cell, such as the center coordinates of the first row and first column grid (4, 4), as the positioning reference for subsequent feature mapping. Extract the center coordinates of the temperature abnormal area in the feature vector with a spatial label, calculate the Euclidean distance between the coordinates and the center coordinates of all grid cells, and find the nearest grid cell as the target grid cell. If the boundary range of the temperature abnormal area covers multiple grid cells, such as the abnormal area being large and covering three adjacent grid cells, the thermal resistance correlation feature is assigned to the three grid cells at the same time, labeled as the relevant grid group, and the model ensures that all grid cells in the defect heat affected zone can obtain the corresponding thermal resistance feature. For normal area grid cells in the original temperature image that do not correspond to the thermal field feature vector, i.e. areas without temperature abnormalities, the model will automatically generate basic thermal resistance correlation features based on the standard thermal resistance properties of the aluminum rod matrix, such as thermal resistance 237W / (m・K), generate 8-dimensional basic features, such as thermal resistance difference degree dimension value 0 and thermal resistance distribution uniformity dimension value 1, and assign them to all normal area grid cells. The generated thermal field feature map covers the entire effective detection area of the aluminum rod without spatial blanking.

[0073] By assigning feature dimensions, mapping pixel values, and interpolating, the 8-dimensional thermal resistance correlation features of each grid cell are transformed into pixel information of a 2D image, forming a complete thermal field feature map. The 8-dimensional thermal resistance correlation features are divided into two groups according to their thermophysical meaning, corresponding to the channel dimension and pixel grayscale dimension of the thermal field feature map, respectively. The first four core dimensions, thermal resistance difference, thermal resistance distribution uniformity, thermal resistance gradient, and thermal resistance stability, are selected as the four channels of the feature map, similar to the three channels of an RGB image. The last four auxiliary dimensions (such as the correlation coefficient between thermal resistance and temperature) are converted into pixel grayscale values ​​by weighted summation, ranging from 0 to 255. For example, if the four core dimension values ​​of a certain grid cell are [0.8, 0.9, 0.6, 0.7], then the four channels of the feature map are assigned values ​​of 0.8, 0.9, 0.6, and 0.7, respectively. At the same time, the weighted summation of the last four auxiliary dimension values ​​is 0.65, which is then scaled to 166. 0.65 × 255 ≈ 166 is used as the pixel grayscale value of this grid cell. The channel feature value and grayscale value of each grid cell are filled into all the corresponding pixels of that grid. For example, the 64 pixels of an 8×8 grid cell are all assigned the same 4-channel feature value and grayscale value. For associated grid groups, such as a defect area covering 3 grid cells, feature smoothing is performed at the boundary of the grid cells. The channel feature value of the pixels at the boundary is calculated by bilinear interpolation, and the average value of the channel values ​​of the adjacent grid cells is taken to avoid feature abrupt changes caused by grid boundaries. For edge pixels in normal regions that lack assigned basic features, such as incomplete grid cells at image edges, neighborhood interpolation is used to supplement feature values ​​by taking the average of the features from the three surrounding normal grid cells. This ensures that each pixel in the thermal field feature map has complete channel features and grayscale values. The size of the initially constructed thermal field feature map is adjusted to match the size of the previous morphological feature map. Figure 1 To achieve the desired resolution, such as 256×256 pixels, if the original temperature image resolution is 512×512, then downsampling, such as average pooling, is used to reduce the feature map size to 256×256 while maintaining the channel dimension of 4 channels. If the original resolution is lower than 256×256, then upsampling, such as bilinear interpolation, is used to amplify it to the target resolution, so that the spatial scale of the thermal field feature map and the morphological feature map are matched, avoiding feature misalignment caused by size differences during subsequent multimodal fusion.

[0074] After the preliminary thermal field feature map, it is necessary to pass through noise suppression, feature enhancement and dimension adjustment to optimize the clarity and availability of the feature map, so that it can be directly fused with the morphological feature map; 5*5 Gaussian filter is used to smooth each channel of the thermal field feature map, the average value of the channel features of the 25 pixels around each pixel is calculated to replace the original pixel value, and the local noise caused by interpolation is eliminated, such as the feature fluctuation of the normal area grid unit; then the Sobel edge detection algorithm is used to enhance the edge features of the defect area and the grid group, calculate the gradient value of the edge pixel of the defect area, and increase the gray value of the edge pixel by 20%, such as from 166 to 199, so that the thermal field feature boundary of the defect is clearer, and the edge of the morphological feature is aligned with the edge of the morphological feature.

[0075] The feature values of the four channels of the thermal field feature map are uniformly mapped to the interval of 0-1, the maximum and minimum values of the feature values of all pixels in each channel are calculated, and the channel value of each pixel is adjusted to 0-1 through linear transformation, such as (current value-minimum value) / (maximum value-minimum value), to avoid the difference in the order of magnitude of the features of different channels, such as the thermal resistance difference range 0-1, the thermal resistance gradient range 0-0.8, which causes excessive attention to a certain channel feature during subsequent fusion; the optimized thermal field feature map is converted into the output format preset by the model, if the subsequent multi-modal fusion model requires input in the form of channel priority tensor, such as 4*256*256, channel number*height*width, the dimension order of the feature map is adjusted; at the same time, the spatial coordinate mapping table of the feature map is saved, which records the correspondence between the feature map pixels and the actual position of the aluminum bar surface, such as the feature map (100, 100) pixel corresponds to the position of 100mm axial and 30° circumferential of the aluminum bar, which is convenient for subsequent defect positioning and result tracing; finally, the structured, high-definition and spatially bound thermal field feature map is output, completing the conversion from discrete vector to image.

[0076] The defect recognition and classification module acquires the morphological feature map and the thermal field feature map, uses the dual-modal attention fusion network to dynamically allocate the weights of the morphological feature map and the thermal field feature map, and inputs the full connection classifier to output the defect type; The process of dynamically allocating the weights of the morphological feature map and the thermal field feature map is as follows: The morphological feature map representing the geometric properties of the defect and the thermal field feature map representing the thermal physical properties of the defect are taken as inputs, the intelligent weight allocation of the two modal features is realized through the feature alignment, independent coding, attention calculation and dynamic weighting of the dual-modal attention fusion network, the modal features with higher contribution degree in different areas of the defect, such as the edge of the crack and the center of the bubble, are automatically enhanced, such as the morphological feature in the crack area and the thermal field feature in the bubble area, and a feature map fused with the advantages of dual modal is output. The dual-modal feature map preprocessing and spatial alignment eliminate the modal difference and make the position match. The resolution of the two kinds of feature maps is calibrated. If the resolution of the morphological feature map is 256x256 pixels, 4 channels, corresponding to different dimensions of geometric features, and the resolution of the thermal field feature map is 512x512 pixels, 4 channels, corresponding to different dimensions of thermal resistance correlation features, then the thermal field feature map is down-sampled to 256x256 pixels by bilinear interpolation to make the size of the two consistent. If the number of channels is different, such as 4 channels for the morphological map and 8 channels for the thermal field map, then the number of channels of the thermal field feature map is compressed to 4 channels by a 1x1 convolution kernel without activation function to make the heightxwidthxchannel number of the dual-modal feature map completely match, which is 256x256x4, avoiding the bias of attention calculation caused by dimension difference.

[0077] The numerical distribution of the two modal features may be different, such as the channel numerical range of the morphological feature map is 0-0.8 and the thermal field feature map is 0-1.2. Batch normalization is needed to eliminate the distribution difference. For each channel of each feature map, the mean and standard deviation of all pixels in the channel are calculated, and then the numerical value is converted to a normal distribution with mean 0 and standard deviation 1, so that the dual-modal features are in the same order of magnitude during subsequent attention calculation.

[0078] Since the two kinds of feature maps are from the same detection area of the aluminum bar, the position is matched by corresponding to the physical space coordinates of the aluminum bar. Taking the axial direction, length direction and circumferential direction of the original aluminum bar as the reference, the spatial coordinate anchor points are marked in the morphological feature map and the thermal field feature map, such as the position of the aluminum bar axial 100mm, circumferential 0°, corresponding to the pixel coordinates (50, 50) of the feature map. By comparing the pixel coordinates of more than 3 anchor points, if there is a deviation, such as the thermal field map anchor point shifts by 2 pixels, then the position of the thermal field feature map is adjusted by pixel translation to make the same pixel point in the dual-modal feature map accurately correspond to the same physical area on the surface of the aluminum bar, such as the feature map (50, 50) both correspond to the position of the aluminum bar axial 100mm, circumferential 0°, avoiding the misalignment of features and positions during subsequent weight distribution.

[0079] Independent encoding of dual-modal feature maps extracts deep layer modal exclusive features. Lightweight convolutional encoders with the same structure are configured for the two modalities, 3 layers of convolutional layers + 2 layers of pooling layers, with the structure parameters unified as follows: the first layer of convolution kernel size 3x3, step 1, output channel number 8, and the activation function is ReLU to enhance the non-linear feature extraction capability; the first layer of pooling layer is 2x2 max pooling, which reduces the feature map size to 128x128 while retaining key features; the second layer of convolution kernel size 3x3, step 1, output channel number 16, and the activation function is ReLU; the second layer of pooling layer is 2x2 max pooling, which reduces the size to 64x64; the third layer of convolution kernel size 3x3, step 1, output channel number 32, without pooling, retaining the 64x64 size for subsequent spatial attention calculation.

[0080] The pre-processed morphological feature map is input into a morphological encoder, and through step-by-step filtering of 3 layers of convolution, the first layer of convolution captures the basic geometric features of the defect, such as edges and corners, the second layer of convolution strengthens the combined features of the defect, such as the linear profile of the crack and the blocky morphology of the depression, and the third layer of convolution extracts the deep geometric features of the defect, such as the depth variation rule and the edge roughness, and outputs a morphological deep feature map of 64x64x32; the pre-processed thermal field feature map is input into a thermal field encoder, and through the same convolution operation, the deep thermal physical features of the defect are extracted, such as the thermal resistance distribution gradient and the thermal field abnormal boundary, and a thermal field deep feature map of 64x64x32 is output. The two kinds of deep feature maps respectively reserve the core information of their own modalities, providing high-quality input for subsequent attention weight calculation.

[0081] The dual-modality attention fusion network calculates the importance of different positions in each modality by combining the intra-modality spatial attention and the inter-modality correlation attention, such as the crack edge being more important than the normal area in the morphological map, and calculates the overall correlation of the two modalities, such as the thermal field modality being more important than the morphological modality in the bubble area, and finally fuses to obtain dynamic weights; Intra-modality spatial attention weight calculation: for each deep feature map, the spatial importance weight of each pixel inside it is calculated, highlighting the defect area features and suppressing background interference; taking the morphological deep feature map 64x64x32 as an example: the feature map is globally averaged pooled, and the average value of all 64x64 pixels in each channel is calculated, a total of 32, to obtain a 1x1x32 channel global feature vector.

[0082] The vector is input into a two-layer fully connected network, the first layer outputs 16 dimensions, and the activation function is ReLU; the second layer outputs 32 dimensions without activation, to obtain a 1x1x32 channel weight vector; the channel weight vector is multiplied with the original morphological deep feature map channel by channel, and all pixels in each channel are multiplied by the weight value of that channel, to obtain a 64x64x32 channel weighted feature map; finally, the channel dimension of the channel weighted feature map is summed to add the pixel values of the 32 channels, to obtain a 64x64x1 spatial weight map, and the weight value is converted to a probability value of 0-1 through the Softmax activation function, the sum of all pixel weights is 1, and the higher the value, the more important the position in the morphological modality, such as the crack edge pixel weight close to 0.8 and the normal area close to 0.1. The same method is used to process the thermal field deep feature map to obtain a 64x64x1 thermal field spatial weight map, highlighting the importance of the temperature abnormal area in the thermal field modality, such as the bubble center pixel weight close to 0.9 and the normal area close to 0.05.

[0083] Inter-modality correlation attention weight calculation: calculate the overall contribution weight of the two modalities to determine which modality has more reference value in the current defect scene: First, the morphological deep feature map and the thermal field deep feature map are spatially dimensioned, converting the 64x64x32 feature map into a 1x64x64x32 vector, i.e., a 1x131072 vector, respectively denoted as morphological vector V m and thermal field vector V h ; The correlation matrix of the two vectors is calculated by multiplying V m and the transpose of V h , obtaining a 1x1 correlation coefficient, which ranges from -1 to 1. The closer the coefficient is to 1, the more similar the two modal features are, and the more balanced the contribution is. The closer the coefficient is to 0, the higher the contribution of a certain modality is. The correlation coefficient is input into the Sigmoid activation function to convert it into a 0-1 modality weight value. If the coefficient is 0.3, it means that the thermal field modality is more unique. If the coefficient is 0.8, it means that the contributions of the two modalities are balanced, and the output is 0.5, ω h =0.5, realizing dynamic allocation of the overall weight according to the modality correlation.

[0084] The intra-modality spatial weight and the inter-modality modality weight are combined to dynamically weight the morphological and thermal field deep feature maps, and then the final bimodal fusion feature map is obtained through fusion operation. The morphological deep feature map is weighted: the morphological spatial weight map 64x64x1 is multiplied by the morphological modality weight ω m to obtain a 64x64x1 morphological most weight map; then the weight map is multiplied by the morphological deep feature map pixel by pixel, and each pixel's 32 channel values are multiplied by the corresponding weight value to obtain a 64x64x32 weighted morphological feature map. The same method is used to process the thermal field deep feature map: the thermal field spatial weight map is multiplied by ω h to obtain a thermal field most weight map, which is then multiplied by the thermal field deep feature map pixel by pixel to obtain a 64x64x32 weighted thermal field feature map.

[0085] The core of this step is double reinforcement, which highlights the defect area within the modality, such as the crack edge of the morphological map, and strengthens the modality with higher contribution between modalities, such as the thermal field modality in the bubble scene, avoiding feature deviation caused by single weight.

[0086] The two weighted feature maps are merged by channel-by-channel splicing and convolution fusion: First, the weighted morphological feature map (64x64x32) and the weighted thermal field feature map (64x64x32) are spliced in the channel dimension to obtain a 64x64x64 spliced feature map, retaining all the deep features of the two modalities. The convolutional fusion is performed on the spliced feature map, 1 layer of 3x3 convolution kernel is used, the output channel number is 32, the activation function is ReLU, 64-channel features are compressed into 32-channel features through sliding convolution, and the correlation information of the two modalities is fused, the crack edge features of the morphology map are combined with the crack area thermal resistance features of the thermal field map; Finally, the feature map size is enlarged from 64x64 to 256x256 through 2x2 deconvolution, which is consistent with the original feature map size, and the dual-modality fusion feature map of 64x64x32 to 256x256x32 is obtained, each pixel of which fuses the advantages of morphology and thermal field, such as crack area mainly with morphology features and bubble area mainly with thermal field features; In order to eliminate noise interference in the fusion process and verify the rationality of weight distribution, post-processing is needed for the fusion feature map; 3x3 Gaussian filter is used to process the fusion feature map, the weight is assigned according to the pixel distance, the center pixel weight is 0.2, and the weight of the surrounding 8 pixels is 0.1; the weighted average value of each pixel and the surrounding 9 pixels is calculated to replace the original pixel value, and the local feature fluctuation caused by weight distribution is smoothed, such as the boundary area of the weight mutation of the two modalities; the defect features are strengthened through contrast stretching, the minimum and maximum values of each channel of the fusion feature map are calculated, and the pixel value is linearly mapped to the 0-1 interval, so that the feature difference between the defect area and the normal area is more obvious, such as the crack area feature value is increased from 0.4 to 0.7, and the normal area is decreased from 0.1 to 0.05; the defect area feature contribution degree analysis is used to verify the rationality of the weight: in the fusion feature map, typical defect areas such as 1 crack area and 1 bubble area are selected, the feature contribution proportion of the morphology modality and the thermal field modality in the area is calculated respectively, and the weight proportion of the two modalities is calculated by backtracking the weighting process; if the crack area morphology contribution proportion is ≥60% and the bubble area thermal field contribution proportion is ≥60%, it is determined that the weight distribution is effective, which is consistent with the physical characteristics of the defects: the crack geometry feature is more significant, and the bubble thermal physical feature is more significant; if the contribution proportion does not meet the standard, such as the bubble area thermal field contribution proportion is only 40%, return to the inter-modality correlation attention calculation step, adjust the calculation parameters of the correlation matrix, such as increasing the weight coefficient of the thermal field vector, recalculating the modality weight, until the contribution proportion meets the standard, so that the accuracy of subsequent defect recognition is improved.

[0087] Output defect type: The dual-modality fusion feature map is preprocessed to convert it into a one-dimensional feature vector that can be received by the full-connection classifier. The fusion feature map output previously is a three-dimensional structure of 256x256x32, which needs to be flattened in the order of channel, height, and width: first traverse the 32 feature channels, read the feature values of 256x256 pixels in each channel in row priority order, and finally concatenate them into a one-dimensional feature vector of 1x256x256x32 or 1x2097152, so that the vector dimension is completely matched with the number of input layer neurons of the classifier.

[0088] A pre-trained full connection classifier is loaded, which is structured as an input layer, a hidden layer 1, a hidden layer 2 and an output layer, the input layer is provided with 2097152 neurons, which is consistent with the dimension of the flattened feature vector; the hidden layer 1 is provided with 1024 neurons, which performs nonlinear mapping on the input features through a ReLU activation function, extracts more abstract defect-related features, and avoids gradient disappearance at the same time; the hidden layer 2 is provided with 256 neurons, which continues to compress the feature dimension through a ReLU activation function, and strengthens the key classification information; the number of neurons of the output layer is matched with the number of aluminum bar defect categories, for example, four categories corresponding to cracks, bubbles, inclusions and normal, and a Softmax activation function is used to convert the output value into a probability distribution of 0-1, and the sum of the probabilities of all categories is 1.

[0089] After the one-dimensional feature vector is input into the classifier, it is first transmitted to the hidden layer 1 through the input layer, and a 1024-dimensional feature vector is obtained through weight matrix multiplication, bias addition and ReLU activation; then it is transmitted to the hidden layer 2, and a 256-dimensional key feature vector is obtained by repeating the same operation; finally, it is input into the output layer, and the Softmax operation is performed to output the probability of each category, such as crack 0.05, bubble 0.92, inclusion 0.02 and normal 0.01. Determine the defect type, select the category with the maximum output probability as the final result, such as bubble corresponding to the probability of 0.92, and set a probability threshold, such as 0.7, if the maximum probability is lower than the threshold, it is determined as an unknown defect and triggers manual review, so as to improve the classification accuracy. The whole process relies on the classifier weight pre-trained by 5000 defect samples in the early stage.

[0090] Embodiment 2 Please refer to Figure 2 Based on embodiment 1, the application embodiment 2 further provides an aluminum bar surface quality detection method based on visual detection, which comprises the following specific steps: Step one, phase unwrapping and triangulation are performed on the structured light image of the aluminum bar to generate a three-dimensional point cloud; an infrared camera is used to capture the temperature field of the aluminum bar surface to generate a temperature image; Step two, normal vector estimation is performed on the three-dimensional point cloud, the surface discontinuous area is generated by identifying the normal vector change rate, and a morphological feature vector is constructed; a PointNet++ architecture is used to extract hierarchical features from the morphological feature vector, and output a morphological feature map; a thermal field feature vector is constructed based on the temperature features calculated from the temperature image, and the thermal field feature vector is input into a pre-set thermal resistance defect correlation model to output a thermal field feature map; Step three, the morphological feature map and the thermal field feature map are obtained, a dual-modal attention fusion network is used to dynamically allocate the weights of the morphological feature map and the thermal field feature map, and input into a full connection classifier to output the defect type.

[0091] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art can be aware that units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on the specific application and design constraints of the technical solutions.

[0092] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, and can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiments according to actual needs.

[0093] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A visual inspection-based surface quality inspection system for aluminum rods, characterized in that, The system includes: The multimodal data acquisition module generates a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculations and triangulation; and uses an infrared camera to capture the surface temperature field of the aluminum rod to generate a temperature image. The defect feature extraction module estimates the normal vector of the 3D point cloud, identifies discontinuous areas on the surface by the rate of change of the normal vector, and constructs a morphological feature vector. Using the PointNet++ architecture, it performs hierarchical feature extraction on the morphological feature vector and outputs a morphological feature map. Based on the temperature image, it calculates the temperature features to construct a thermal field feature vector, inputs the thermal field feature vector into a preset thermal resistance defect association model, and outputs a thermal field feature map. The defect identification and classification module acquires morphological feature maps and thermal feature maps, uses a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, and inputs them into a fully connected classifier to output the defect type.

2. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of generating a 3D point cloud is as follows: A specially coded sinusoidal striped structured light pattern is projected onto the surface of an aluminum rod. Multiple cameras simultaneously acquire images of the structured light pattern on the aluminum rod surface. The acquired structured light patterns are processed by phase calculation, and a four-step phase shift method is used to analyze the phase change of the stripes in the structured light pattern to calculate the phase distribution corresponding to each pixel on the aluminum rod surface. Based on the principle of triangulation, the phase distribution information is converted into three-dimensional spatial coordinates of each point on the aluminum rod surface. By integrating the three-dimensional spatial coordinates, point cloud data of the three-dimensional morphology of the aluminum rod surface is generated.

3. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of generating the temperature image is as follows: An infrared camera collects infrared radiation energy from the surface of an aluminum rod at a set frame rate, converts the infrared radiation energy into an electrical signal, and then processes the signal to convert the electrical signal into a grayscale value corresponding to the temperature, thus forming an infrared temperature image.

4. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of estimating the normal vector of the 3D point cloud is as follows: For each point in the 3D point cloud, k-neighborhood search is used to find the nearest points to the current point, a neighborhood point set is constructed, and the centroid of the neighborhood point set is calculated. The coordinates of all points in the neighborhood are added together and divided by the number of points. The centroid is then subtracted from each neighborhood point to obtain the centered point set. Multiply and sum the coordinates of each point in the centered point set, then divide by the number of points to obtain the covariance matrix. Perform eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​arranged in descending order and three corresponding eigenvectors. Based on the eigenvector of the normal vector of the point cloud surface corresponding to the smallest eigenvalue in the covariance matrix, use this eigenvector as the initial estimate of the normal vector of that point.

5. The surface quality inspection system for aluminum rods based on vision inspection according to claim 4, characterized in that, The process of constructing the morphological feature vector is as follows: Based on the point cloud normal vector estimation, the directional difference of the normal vectors of adjacent points is measured to determine the surface smoothness of the region where each target point is located in the 3D point cloud. A threshold for the rate of change of the normal vector is set to filter out abnormal points from the point cloud. Then, a preliminary discontinuous region is formed by region aggregation. The preliminary extracted discontinuous region contains the normal structure of the aluminum rod. Non-defect regions are excluded by combining the geometric features of the aluminum rod with prior knowledge, and the real defect regions are segmented. Key parameters characterizing the geometric properties of each independent defect region are extracted and combined in a fixed order to form a morphological feature vector.

6. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of outputting the morphological feature map is as follows: The PointNet++ architecture integrates 3D point cloud and morphological feature vectors. It focuses on key regions in the 3D point cloud by grouping, and simultaneously selects sampling points and divides them into local neighborhoods. The local neighborhoods are merged into local point cloud groups. Local coordinate system transformation, feature mapping and global pooling are performed on the local point cloud groups. The morphological features of discrete points in the local point cloud groups are transformed into aggregated features of local correlation. Based on the first aggregation layer, the aggregated features are repeatedly sampled, grouped and aggregated to extract multi-level features.

7. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of constructing the thermal field feature vector is as follows: Temperature images are acquired and preprocessed. Temperature benchmarks are set and regions are analyzed to locate temperature anomaly regions related to defects. Interference from normal temperature fluctuations is eliminated. Key features of the thermophysical properties of aluminum rod defects are extracted from each temperature anomaly region and integrated in a fixed order. Normalization is used to eliminate magnitude differences and form a standardized thermal field feature vector.

8. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process of outputting the thermal field feature map is as follows: First, the importance of the five dimensions in the thermal field feature vector is evaluated. Each thermal field feature vector corresponds to a temperature anomaly region, and the thermal field feature vector is mapped to the spatial location of the region in the temperature image. Through a supervised learning model trained with a large number of defect samples, the thermal field features are transformed into thermal resistance correlation features that can characterize the thermal resistance properties of defects. Through spatial location mapping, the thermal resistance correlation features are assigned to the corresponding pixel regions of the temperature image. Through feature dimension allocation, pixel value mapping and interpolation, the 8-dimensional thermal resistance correlation features of each grid cell are transformed into pixel information of a two-dimensional image, forming a complete thermal field feature map.

9. The surface quality inspection system for aluminum rods based on vision inspection according to claim 1, characterized in that, The process for specifying the output defect type is as follows: The system acquires morphological and thermal feature maps, inputs them into a bimodal attention fusion network, and assigns weights to the two modal feature maps through feature alignment, independent encoding, attention calculation, and dynamic weighting. It automatically strengthens the modal features with higher contribution in different defect regions, outputs a feature map that integrates the advantages of both modalities, and transforms it into a one-dimensional feature vector that can be received by a fully connected classifier. After loading the pre-trained fully connected classifier, which includes an input layer, two hidden layers, and an output layer, the system calculates the probability of each class and selects the one with the highest probability as the defect type.

10. A method for inspecting the surface quality of aluminum rods based on visual inspection, wherein the method is applied to the surface quality inspection system for aluminum rods based on visual inspection as described in any one of claims 1-9, characterized in that, Includes the following steps: Step 1: Generate a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculation and triangulation; and generate a temperature image by capturing the surface temperature field of the aluminum rod using an infrared camera. Step 2: Estimate the normal vector of the 3D point cloud, identify discontinuous areas on the surface by the rate of change of the normal vector, and construct morphological feature vectors. Using the PointNet++ architecture, perform hierarchical feature extraction on the morphological feature vectors and output the morphological feature map. Calculate the temperature features based on the temperature image to construct the thermal field feature vector, input the thermal field feature vector into the preset thermal resistance defect association model, and output the thermal field feature map. Step 3: Obtain morphological feature maps and thermal feature maps. Use a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, and input them into a fully connected classifier to output the defect type.

Images