Aluminum bar surface quality detection method and system based on visual detection

Abstract

The application discloses an aluminum bar surface quality detection method and system based on visual detection, and relates to the technical field of image processing.The system is composed of a plurality of function modules, and comprises a multi-modal data acquisition module, which generates a three-dimensional point cloud through phase solving and triangulation measurement by acquiring a structured light image of an aluminum bar; an infrared camera is used to capture the temperature field of the surface of the aluminum bar to generate a temperature image; a defect feature extraction module is used to estimate the normal vector of the three-dimensional point cloud, identify the surface discontinuous area by the normal vector change rate to generate a defect area, and construct a morphological feature vector; a PointNet++ architecture is used to perform hierarchical feature extraction on the morphological feature vector and output a morphological feature map; a temperature feature is calculated based on the temperature image to construct a thermal field feature vector; the thermal field feature vector is input into a preset thermal resistance defect correlation model to output a thermal field feature map; and a defect recognition and classification module is used to acquire the morphological feature map and the thermal field feature map.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for detecting the surface quality of aluminum rods based on visual inspection. Background Technology

[0002] Aluminum bars are widely used in aerospace, automotive manufacturing, building structures, and electronic components, industries that have strict requirements for the mechanical properties and appearance quality of materials. Surface defects, such as cracks, scratches, pits, and oxide spots, can become stress concentration points, reducing the fatigue strength, tensile strength, and other key properties of aluminum bars, thus affecting the safety and service life of the final product. Aluminum bars are usually used as raw materials for further processing, such as extrusion, forging, rolling, and machining. Surface defects often reflect problems in the production process, such as cold shuts and porosity in casting, poor lubrication or mold wear in extrusion, and oxidation or corrosion in heat treatment or cooling. If surface defects exist, they may cause tool wear or damage, lead to excessive surface roughness of processed parts, and result in defective or even scrap products, increasing production costs.

[0003] While traditional 3D structured light inspection technology can achieve three-dimensional morphology measurement, it faces multiple challenges in aluminum rod production scenarios: Under high-speed production lines, traditional fixed-frequency stripe projection is prone to reconstruction blurring due to the movement of aluminum rods, and the reconstruction frame rate of existing systems is mostly below 30Hz, making it difficult to match the production rhythm; in addition, the occlusion problem caused by the cylindrical surface of aluminum rods and the interference of ambient light reduce stripe contrast and increase point cloud noise rate, directly affecting the accuracy of defect feature extraction; at the same time, existing defect recognition mostly relies on general deep learning models, lacking specific feature design for defects unique to aluminum rods, and the large number of model parameters leads to excessive inference latency. Therefore, there is an urgent need to develop an aluminum rod surface quality inspection technology that is adaptable to high-speed production lines and has strong anti-interference capabilities. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A vision-based surface quality inspection system for aluminum rods, comprising:

[0006] The multimodal data acquisition module generates a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculations and triangulation; and uses an infrared camera to capture the surface temperature field of the aluminum rod to generate a temperature image.

[0007] The defect feature extraction module estimates the normal vector of the 3D point cloud, identifies discontinuous areas on the surface by the rate of change of the normal vector, and constructs a morphological feature vector. Using the PointNet++ architecture, it performs hierarchical feature extraction on the morphological feature vector and outputs a morphological feature map. Based on the temperature image, it calculates the temperature features to construct a thermal field feature vector, inputs the thermal field feature vector into a preset thermal resistance defect association model, and outputs a thermal field feature map.

[0008] The defect identification and classification module acquires morphological feature maps and thermal feature maps, uses a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, and inputs them into a fully connected classifier to output the defect type.

[0009] Furthermore, the process of generating the 3D point cloud is as follows:

[0010] A specially coded sinusoidal striped structured light pattern is projected onto the surface of an aluminum rod. Multiple cameras simultaneously acquire images of the structured light pattern on the aluminum rod surface. The acquired structured light patterns are processed by phase calculation, and a four-step phase shift method is used to analyze the phase change of the stripes in the structured light pattern to calculate the phase distribution corresponding to each pixel on the aluminum rod surface. Based on the principle of triangulation, the phase distribution information is converted into three-dimensional spatial coordinates of each point on the aluminum rod surface. By integrating the three-dimensional spatial coordinates, point cloud data of the three-dimensional morphology of the aluminum rod surface is generated.

[0011] Furthermore, the process of generating the temperature image is as follows:

[0012] An infrared camera collects infrared radiation energy from the surface of an aluminum rod at a set frame rate, converts the infrared radiation energy into an electrical signal, and then processes the signal to convert the electrical signal into a grayscale value corresponding to the temperature, thus forming an infrared temperature image.

[0013] Furthermore, the process of estimating the normal vector of the 3D point cloud is as follows:

[0014] For each point in the 3D point cloud, a k-neighborhood search is used to find the nearest points to that point, constructing a neighborhood point set. The centroid of the neighborhood point set is calculated, and the coordinates of all points in the neighborhood are summed and divided by the number of points. The centroid is then subtracted from each neighborhood point to obtain the centered point set. The coordinates of each point in the centered point set are multiplied and summed, then divided by the number of points to obtain the covariance matrix. The covariance matrix is ​​then decomposed into eigenvalues ​​to obtain three eigenvalues ​​arranged in descending order and their corresponding three eigenvectors. The eigenvector corresponding to the smallest eigenvalue in the covariance matrix based on the normal vector of the point cloud surface is used as the initial estimate of the normal vector of that point.

[0015] Furthermore, the process of constructing the morphological feature vector is as follows:

[0016] Based on the point cloud normal vector estimation, the directional difference of the normal vectors of adjacent points is measured to determine the surface smoothness of the region where each target point is located in the 3D point cloud. A threshold for the rate of change of the normal vector is set to filter out abnormal points from the point cloud. Then, a preliminary discontinuous region is formed by region aggregation. The preliminary extracted discontinuous region contains the normal structure of the aluminum rod. Non-defect regions are excluded by combining the geometric features of the aluminum rod with prior knowledge, and the real defect regions are segmented. Key parameters characterizing the geometric properties of each independent defect region are extracted and combined in a fixed order to form a morphological feature vector.

[0017] Furthermore, the process of outputting the morphological feature map is as follows:

[0018] The PointNet++ architecture integrates 3D point cloud and morphological feature vectors. It focuses on key regions in the 3D point cloud by grouping, and simultaneously selects sampling points and divides them into local neighborhoods. The local neighborhoods are merged into local point cloud groups. Local coordinate system transformation, feature mapping and global pooling are performed on the local point cloud groups. The morphological features of discrete points in the local point cloud groups are transformed into aggregated features of local correlation. Based on the first aggregation layer, the aggregated features are repeatedly sampled, grouped and aggregated to extract multi-level features.

[0019] Furthermore, the process of constructing the thermal field feature vector is as follows:

[0020] Temperature images are acquired and preprocessed. Temperature benchmarks are set and regions are analyzed to locate temperature anomaly regions related to defects. Interference from normal temperature fluctuations is eliminated. Key features of the thermophysical properties of aluminum rod defects are extracted from each temperature anomaly region and integrated in a fixed order. Normalization is used to eliminate magnitude differences and form a standardized thermal field feature vector.

[0021] Furthermore, the process of outputting the thermal field feature map is as follows:

[0022] First, the importance of the five dimensions in the thermal field feature vector is evaluated. Each thermal field feature vector corresponds to a temperature anomaly region, and the thermal field feature vector is mapped to the spatial location of the region in the temperature image. Through a supervised learning model trained with a large number of defect samples, the thermal field features are transformed into thermal resistance correlation features that can characterize the thermal resistance properties of defects. Through spatial location mapping, the thermal resistance correlation features are assigned to the corresponding pixel regions of the temperature image. Through feature dimension allocation, pixel value mapping and interpolation, the 8-dimensional thermal resistance correlation features of each grid cell are transformed into pixel information of a two-dimensional image, forming a complete thermal field feature map.

[0023] Furthermore, the process for outputting the defect type is as follows:

[0024] The system acquires morphological and thermal feature maps, inputs them into a bimodal attention fusion network, and assigns weights to the two modal feature maps through feature alignment, independent encoding, attention calculation, and dynamic weighting. It automatically strengthens the modal features with higher contribution in different defect regions, outputs a feature map that integrates the advantages of both modalities, and transforms it into a one-dimensional feature vector that can be received by a fully connected classifier. After loading the pre-trained fully connected classifier, which includes an input layer, two hidden layers, and an output layer, the system calculates the probability of each class and selects the one with the highest probability as the defect type.

[0025] A method for inspecting the surface quality of aluminum rods based on visual inspection includes the following steps:

[0026] Step 1: Generate a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculation and triangulation; and generate a temperature image by capturing the surface temperature field of the aluminum rod using an infrared camera.

[0027] Step 2: Estimate the normal vector of the 3D point cloud, identify discontinuous areas on the surface by the rate of change of the normal vector, and construct morphological feature vectors. Using the PointNet++ architecture, perform hierarchical feature extraction on the morphological feature vectors and output the morphological feature map. Calculate the temperature features based on the temperature image to construct the thermal field feature vector, input the thermal field feature vector into the preset thermal resistance defect association model, and output the thermal field feature map.

[0028] Step 3: Obtain morphological feature maps and thermal feature maps. Use a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, and input them into a fully connected classifier to output the defect type.

[0029] The present invention provides a method and system for detecting the surface quality of aluminum rods based on visual inspection, which has the following beneficial effects:

[0030] (1) This invention uses a multimodal data acquisition module to simultaneously acquire three-dimensional point clouds of structured light and infrared temperature images of aluminum rods. The three-dimensional point cloud generated by phase calculation and triangulation can accurately capture the three-dimensional geometric morphology of defects such as cold shuts and cracks, such as step height and crack depth, making up for the inability of 2D technology to quantify three-dimensional parameters. The infrared temperature image can reflect the thermal field distribution on the surface of the aluminum rod. Combined with the thermal field features extracted by the thermal resistance defect association model, it can identify hidden defects such as subcutaneous bubbles and internal inclusions caused by differences in thermal conduction. The three-dimensional morphological features of such defects are not obvious, but the thermal field is abnormally significant. Through the complementary use of dual-modal data, the missed detection rate of hidden defects in the existing technology can be effectively reduced, and defect identification in both geometric and thermophysical dimensions can be achieved.

[0031] (2) This invention estimates the normal vector by using k-neighborhood search and covariance matrix eigenvalue decomposition, and uses the normal vector change rate to identify surface discontinuities and locate defects in the fuzzy area caused by the movement of aluminum rods during high-speed production, such as micro-cracks under high-speed transmission, thus avoiding reconstruction noise interference. At the same time, the PointNet++ architecture is used to extract the morphological feature vector in layers, and the feature vector is constructed by combining the exclusive geometric attributes of aluminum rod defects, such as cold shut linear steps and circular pore depressions, to improve the matching degree between features and defect types. Furthermore, the thermal resistance defect association model is used to further filter noise caused by ambient light and aluminum rod reflection through exclusive thermal field features such as temperature anomaly amplitude and gradient. Even in scenarios where the production line speed is ≥1m / s, it can still maintain high feature extraction accuracy and solve the adaptation problem of high-speed inaccuracy of existing 3D systems. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the system flow of the present invention;

[0033] Figure 2 This is a schematic diagram of the overall method of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0035] Example 1

[0036] Please see Figure 1 Embodiment 1 of this application provides a visual inspection-based aluminum rod surface quality inspection system, which includes:

[0037] The multimodal data acquisition module generates a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculations and triangulation; and uses an infrared camera to capture the surface temperature field of the aluminum rod to generate a temperature image.

[0038] Acquiring structured light images of an aluminum rod:

[0039] A specially coded sinusoidal stripe pattern is projected onto the surface of an aluminum rod using a structured light projection device. This pattern, which includes background light intensity, contrast, spatial frequency, and phase information, uniformly covers the circumference of the aluminum rod. Three sets of high-resolution linear structured light cameras are deployed at a 120° angle along the circumference of the aluminum rod. As the aluminum rod is transported at high speed on the production line, these three sets of cameras simultaneously acquire images of the structured light pattern on the surface of the aluminum rod, ensuring complete capture of the optical information of the aluminum rod surface from different angles. The acquired structured light images are then processed for phase resolution. A four-step phase-shifting method is used to accurately calculate the phase distribution of each pixel on the surface of the aluminum rod by analyzing the phase changes of the stripes in the image.

[0040] The process of generating a 3D point cloud:

[0041] The acquired structured light images are processed using a four-step phase-shifting method. Four sinusoidal fringe patterns with fixed phase differences are then sequentially projected through a structured light projection device. The formulas are as follows:

[0042]

[0043]

[0044]

[0045]

[0046] Where A is the background light intensity, B is the stripe contrast, and f is the spatial frequency. This is the initial phase; , , and These are sinusoidal fringe patterns with a fixed phase difference, where x and y are the abscissa and ordinate of the pattern, respectively.

[0047] After the linear array structured light camera simultaneously acquires these four images, the formula is used to... ,in, For each of the four sinusoidal fringe patterns with different fixed phase differences, calculate the phase value of each pixel. By utilizing the difference in grayscale variation of phase-shifted fringes, interference from background light intensity and fringe contrast is eliminated, and phase information highly correlated with the surface height of the aluminum rod is extracted.

[0048] Phase unfolding and construction of continuous phase fields:

[0049] By calculating the phase gradient information, reliable pixels with continuous phase changes and low noise are identified. Based on these reliable pixels, the encapsulated phase is gradually unfolded into a continuous absolute phase field through region growing. To address interference such as reflections and shadows on the aluminum rod surface, ensure the continuity and accuracy of phase unfolding, and obtain a continuous phase distribution corresponding to the height of the aluminum rod surface.

[0050] Based on the principle of triangulation, a mapping relationship between phase and height on the aluminum rod surface is established. In the structured light system, the spatial positions of the camera and projector are fixed, forming a triangulation system: the projector projects stripes onto the aluminum rod surface, and the camera captures the modulated stripe image from another angle. Changes in the height of the aluminum rod surface cause a shift in the stripe phase, and this shift has a geometric correspondence with the height. Let the baseline distance between the camera and the projector be d, the working distance (i.e., the distance from the camera to the reference plane of the aluminum rod) be L, and the spatial frequency of the stripes be f. Then, the height h(x,y) of a point on the aluminum rod surface is calculated. Based on the direct correlation between the phase change and the height change through trigonometric geometry, the optical phase information is converted into three-dimensional spatial height information.

[0051] Based on the height calculation results, and combined with the camera's intrinsic parameters, such as focal length and principal point coordinates, and extrinsic parameters, such as the camera's position and attitude in the world coordinate system, the two-dimensional image coordinates (u,v) of each pixel are converted into three-dimensional spatial coordinates (x,y,z), where the z component is the height h(x,y). The three-dimensional coordinates of all pixels are then integrated to form a three-dimensional point cloud dataset. Where P is the cloud dataset, For each point in a 3D point cloud ; Let i be the three-dimensional spatial coordinates of the i-th point. The component corresponds to the height h(x,y), where i is the index of the point used to identify different points in the point cloud, and its value range is... N is the total number of 3D points in the point cloud.

[0052] The 3D point cloud dataset is preprocessed by using statistical filtering or radius filtering algorithms to remove abnormal points caused by surface reflection and camera noise of the aluminum rod. Data is collected from different angles using three sets of structured light cameras, and the multi-view point clouds are fused into a complete point cloud of the aluminum rod surface through point cloud registration algorithm. Interpolation is performed on sparse areas and downsampling is performed on overly dense areas to ensure uniform point cloud density, which is convenient for subsequent defect detection algorithm processing.

[0053] The process of acquiring temperature images:

[0054] By installing an infrared thermal imaging camera at the inspection station of the aluminum rod production line, it is ensured that its field of view can completely cover the surface area of ​​the aluminum rod. When the aluminum rod is transported on the production line, the infrared camera, at a set frame rate (usually adjusted according to the transport speed of the aluminum rod), ensures full-area capture of the aluminum rod surface, continuously collecting infrared radiation energy. Temperature differences in different areas of the aluminum rod surface result in different infrared radiation intensities. The infrared camera's detector converts this infrared radiation energy into electrical signals. Through the signal processing circuit inside the camera, the electrical signals are converted into grayscale or color values ​​corresponding to the temperature, forming an infrared temperature image. At the same time, the infrared camera lens focuses the infrared radiation, and the camera's built-in temperature calibration module accurately converts the collected radiation energy into temperature values ​​based on the physical properties of infrared radiation, such as Planck's law, thereby generating a temperature field image that can intuitively reflect the temperature distribution on the aluminum rod surface.

[0055] The defect feature extraction module estimates the normal vector of the 3D point cloud, identifies surface discontinuities by the rate of change of the normal vector, generates defect regions, and constructs morphological feature vectors. Using the PointNet++ architecture, it performs hierarchical feature extraction on the morphological feature vectors and outputs a morphological feature map. Based on the temperature image, it calculates temperature features to construct a thermal field feature vector, inputs a preset thermal resistance defect association model, and outputs a thermal field feature map.

[0056] 3D point cloud normal vector estimation:

[0057] For each point in the 3D point cloud Determine its neighborhood set and find it using k-neighborhood search or radius neighborhood search. The k nearest points or all points within a specified radius constitute the neighborhood point set. The neighborhood point set is centered, that is, the centroid of the neighborhood point set is calculated. ,in, Center of mass, For each neighboring point, then for each neighboring point Subtract the center of mass The centralized point set is obtained. Construct the covariance matrix using the following formula:

[0058]

[0059] For covariance matrix Eigenvalue decomposition yields three eigenvalues. and the corresponding three feature vectors Since the normal vector of the point cloud surface corresponds to the smallest eigenvalue in the covariance matrix. The corresponding eigenvector Therefore, As a point The initial estimation of the normal vector is performed; to ensure the consistency of the normal vectors, i.e., the normal vectors of adjacent points point in the same approximate direction, the normal vector direction needs to be adjusted; by checking the orientation relationship between the normal vector and the local surface of the point cloud, or by using the normal vectors of neighboring points for voting adjustment, the normal vector direction of the entire point cloud is kept consistent, thus obtaining the accurate normal vector of each point. .

[0060] Constructing morphological feature vectors:

[0061] The geometric information of 3D point cloud is transformed into features required for defect identification. Surface anomalies are located by quantifying the spatial variation differences of normal vectors. Key morphological parameters are then extracted from the anomaly regions to finally form feature vectors that can be used for defect classification and assessment.

[0062] Based on the estimated point cloud normal vectors, each point corresponds to a normal vector with a clear direction. By measuring the difference in the direction of the normal vectors of adjacent points, the surface smoothness of the area where the target point is located is determined. For each target point in the 3D point cloud, the neighborhood point set determined when its normal vector was estimated is used, that is, the several points closest to the target point. The number of neighborhood points needs to be set in advance according to the point cloud density to ensure that it can reflect the local surface features and ensure the consistency of the calculation benchmark. The angle between the normal vector of the target point and the normal vector of each point in the neighborhood is calculated. The cosine value of the angle between the two normal vectors is calculated by the dot product of the two normal vectors, and then the specific angle is obtained by the inverse cosine operation. Since the normal vector direction of the defect-free area of ​​the normal surface of the aluminum rod changes gently, the angle between the normal vectors of adjacent points is small. However, the abrupt change in surface morphology of defective areas such as cracks and dents will cause the angle between the normal vectors of adjacent points to increase significantly.

[0063] By setting a threshold for the rate of change of the normal vector, outlier points, i.e., points whose rate of change of the normal vector exceeds the standard, are screened out from the point cloud. Then, preliminary discontinuous regions are formed through region aggregation, while eliminating isolated noise interference. Determining the threshold for the rate of change of the normal vector: This threshold needs to be pre-calibrated based on the point cloud data of defect-free aluminum rods. First, a large number of three-dimensional point clouds of defect-free aluminum rods are collected, and the distribution of the rate of change of the normal vector of all points on their surface is statistically analyzed. The 95th percentile of the distribution is taken as the threshold. For example, the average rate of change of the normal vector of a normal aluminum rod surface is 0.1 radians, and the threshold can be set to 0.3 radians. When the rate of change of the normal vector of a certain point exceeds the threshold, the point is determined to be a surface discontinuity point, i.e., a candidate point of defect.

[0064] Connectivity analysis is performed on the selected discontinuous points. A region growing algorithm is used, with any discontinuous point as a seed point. All points within a fixed spatial radius (the radius must match the common size of defects on the aluminum rod surface, usually set to 0.5 mm) that also satisfy the condition that the rate of change of the normal vector exceeds a threshold are grouped into the same region. Points in this region are used as new seeds, and the process continues until no new points can be added. A minimum threshold for the number of points in a region is set, for example, 50 points. Isolated small regions formed by point cloud noise, minor surface scratches, and defects that do not affect quality are filtered out. The number of points in a small region is usually less than the threshold, and these are directly removed. After this process, multiple surface discontinuous regions are obtained, which are the initial candidate regions for defects.

[0065] The initially extracted discontinuous areas may contain normal structures of the aluminum rod, such as end cutting marks and processing seams. It is necessary to combine the geometric characteristics of the aluminum rod with common sense of production process to exclude non-defect areas and finally accurately segment the real defect areas. Delineate the effective inspection area of ​​the aluminum rod: According to the production parameters of the aluminum rod, such as diameter and length, exclude the end areas of 50 mm at both ends. The ends will form normal surface discontinuities due to the cutting process, and the normal vector change rate is high, which is not a defect. Only the main body area in the middle of the aluminum rod is retained as the inspection object. If the initial discontinuous area is located at the end, it is directly judged as a non-defect area and rejected.

[0066] Verification based on the regular curved surface characteristics of aluminum rods shows that the surface of a normal aluminum rod is a standard cylindrical surface, and its normal vector direction should be radial, outward or inward, and must match the direction of the previous normal vector consistency adjustment. If the overall direction of the normal vector of a certain initially discontinuous area deviates from the radial direction by more than 30°, and the shape of the area conforms to common defect morphologies, such as linear cracks, blocky depressions, and circular bubbles, then it is determined to be a real defect area. If the shape of the area is a regular ring, it may be a positioning mark left by processing or the normal vector direction does not deviate significantly from the radial direction, then it is determined to be a non-defect area and is removed.

[0067] Independent segmentation of real defect areas: If the distance between multiple real defect areas is less than 1 mm, they may be segments of the same defect, such as a long crack being divided into multiple small areas by local noise. By calculating the shortest distance between the areas, it is determined whether they are extensions of the same defect. If so, they are merged into a single defect area. Finally, each independent defect area is obtained, and each area corresponds to a specific quality defect, such as a single crack, a single depression, or a single bubble protrusion.

[0068] Key parameters that characterize the geometric properties of each independent defect region are extracted to distinguish different defect types, such as cracks, dents, and bubbles, while also reflecting the severity of the defects. These parameters are then combined in a fixed order to form a morphological feature vector.

[0069] The area of ​​the defect region is determined by first counting the number of point clouds within the defect region, and then combining this with the resolution during point cloud acquisition, i.e. the horizontal and vertical dimensions of a single point covering the surface of the aluminum rod, for example, 0.1 mm / point horizontally and 0.1 mm / point vertically, the coverage area of ​​a single point is 0.01 square millimeters. The actual planar coverage area of ​​the defect is obtained by multiplying the number of point clouds in the defect region by the coverage area of ​​a single point, which reflects the size range of the defect.

[0070] The maximum depth of the defect is determined by taking the normal surface points within a 5 mm radius of the defect area as the reference plane. This area is defect-free and has a flat surface, which can represent the reference surface of the aluminum rod. The vertical distance between each point in the defect area and the reference plane is calculated, and the maximum value is taken along the radial direction of the aluminum rod. If it is a depression, the distance value is positive; if it is a bulge, such as a bubble, the distance value is negative. This is used to distinguish the defect type and reflect the longitudinal severity of the defect.

[0071] The perimeter of a defect is obtained by fitting the boundary points of the defect region using an algorithm, finding the outermost point of the region's edge, connecting the boundary points in a clockwise or counterclockwise order to form a closed polygon, and calculating the sum of the lengths of all sides of the polygon. This yields the perimeter of the defect, reflecting the complexity of the defect boundary. For example, a crack has a longer perimeter and straight edges, while a depression has a shorter perimeter and irregular edges.

[0072] Defect roundness is calculated by multiplying the defect area by 4 times pi and then dividing by the square of the perimeter. The closer the roundness value is to 1, the closer the defect shape is to a circle, such as a bulging bubble; the smaller the value, the more irregular the defect shape is, such as the roundness of a crack, which is usually less than 0.2. It is used to quickly distinguish the geometric shape of defects.

[0073] The aspect ratio of the defect is fitted to the minimum bounding rectangle of the defect area, that is, the smallest rectangle that can completely enclose the defect area. The sides of the rectangle are parallel or perpendicular to the axis of the aluminum rod. The length of the long side and the length of the short side of the rectangle are measured, and the aspect ratio is obtained by dividing the length of the long side by the length of the short side. The aspect ratio of cracks is usually greater than 10 and they are slender, while the aspect ratio of depressions is close to 1 and they are blocky. It is a key parameter to distinguish between cracks and depressions.

[0074] The standard deviation of the normal vector of the defect area is calculated by taking the standard deviation of the angle between the normal vectors of all points in the defect area. First, calculate the average value of all angles, then calculate the sum of squares of the deviations of each angle from the average value, divide by the number of points, and take the square root. This reflects the roughness of the surface of the defect area. For example, the surface of a corrosion defect is rough and has a larger standard deviation of the normal vector; the surface of a smooth and concave defect is flat and has a smaller standard deviation of the normal vector.

[0075] Output morphological feature map:

[0076] Using the three-dimensional point cloud and corresponding morphological feature vector of the defect region of aluminum rod as input, and based on PointNet++'s hierarchical sampling, local aggregation, and multi-scale fusion, the discrete point-level morphological features are gradually transformed into a global feature map with spatial correlation.

[0077] The input data is based on the integration of 3D point cloud and morphological feature vector into PointNet++-processable joint input data, eliminating the interference of data noise and scale differences on feature extraction. The input data consists of two parts: the 3D point cloud of the aluminum rod defect area, i.e., the spatial coordinates of each point, i.e., the x, y, z axis position information, and the morphological feature vector corresponding to each point, such as the local area contribution, relative depth, and normal vector direction of the point in the defect area, all of which come from the previously constructed defect morphological feature vector.

[0078] During preprocessing, statistical filtering is used to remove isolated points in the point cloud caused by device noise, such as points whose distance from neighboring points exceeds three times the normal point cloud density, ensuring the integrity of the input point cloud. Feature normalization maps the parameters in the morphological feature vector, such as depth and area contribution values, to the [0,1] interval through linear transformation. This avoids the model overemphasizing large-scale features and ignoring crucial small-scale features due to differences in parameter magnitudes (e.g., depth in millimeters, area in square millimeters). Finally, a joint input matrix of point cloud coordinates and normalized morphological features is formed, with each point corresponding to a high-dimensional feature vector: 3 coordinate dimensions and 6 morphological feature dimensions, for a total of 9 dimensions.

[0079] By sampling and grouping key regions in the point cloud, invalid calculations of redundant points are avoided, while capturing local feature correlations. Through hierarchical operations, sampling points are filtered from the initial joint input and local neighborhoods are divided. For farthest-point sampling, key sampling points are uniformly selected from the preprocessed point cloud, with the sampling quantity typically being 1 / 4 to 1 / 2 of the total input point cloud. This is adjusted according to the size of the aluminum rod defect; if the defect is small, the sampling ratio can be increased to 1 / 2 to retain more details. An initial point is randomly selected as the first sampling point, and subsequently, the point farthest from the selected sampling point is chosen as the new sampling point each time, ensuring uniform sampling. The sampling method covers the entire defect area, ensuring that no detail of the defect edge is missed and that redundant points in the normal area are not repeatedly collected. A spherical neighborhood group is set with each sampling point as the center, and the radius is determined based on the average size of the defect. For example, if the average diameter of the defect is 5mm, the neighborhood radius is set to 2mm. This ensures that each neighborhood can cover the local area of ​​the defect. All original points within the neighborhood, including morphological feature vectors, are grouped together to form multiple local point cloud groups. Each local point cloud group corresponds to one sampling point and contains local spatial and morphological feature information around that sampling point, laying the foundation for subsequent local feature extraction.

[0080] By local feature aggregation, the first aggregation layer extracts local morphological feature associations. Through local coordinate system transformation, feature mapping and global pooling, the morphological features of discrete points within a group are transformed into aggregated features with local associations.

[0081] The local coordinate system is converted to eliminate the influence of point cloud translation on features. All points in each local point cloud group are centered with the sampling point of the group as the origin. That is, the coordinates of each point are subtracted from the coordinates of the sampling point, so that the position information of the points in the group reflects their local spatial relationship with respect to the sampling point, such as the offset direction and distance of the defect edge point relative to the sampling point. The normalized morphological feature vector of each point in the group is retained to ensure that local morphological information is not lost.

[0082] Multilayer perceptron feature mapping concatenates the centered local point cloud coordinates with morphological feature vectors and inputs them into a 3-layer MLP, such as the first layer outputting 64 dimensions, the second layer 128, and the third layer 256 for feature mapping. The MLP uses a non-linear activation function to transform low-dimensional coordinates and basic morphological features into high-dimensional local feature vectors. These vectors can capture the spatial positional correlation and morphological feature correlation of points within a group, such as the correspondence between defect depth changes and normal vector directions.

[0083] Max pooling aggregation: For the high-dimensional feature vectors of each local point cloud group after MLP mapping, a global max pooling operation is performed. That is, for the feature vectors of all points in the group, the maximum value is taken in each feature dimension to obtain a fixed-dimensional local aggregated feature. The advantage of max pooling is that it can capture the most significant features in the local region, such as the feature corresponding to the maximum depth of the defect or the feature corresponding to the most obvious change in the normal vector. At the same time, it unifies the features of each local point cloud group to a fixed dimension, which is convenient for subsequent processing. At this time, each sampling point corresponds to a 256-dimensional local aggregated feature, completing the first level of feature extraction.

[0084] Multi-level feature extraction is achieved by iteratively aggregating and fusing multi-scale morphological features to extract subtle changes and global correlations at the crack edge, such as the overall relationship between defects and the normal surface of the aluminum rod. Based on the first aggregation layer, the sampling, grouping, and aggregation processes are repeated to achieve multi-level feature extraction. Second-level sampling and grouping: Using the sampling points output from the first aggregation layer and the corresponding 256-dimensional local aggregation features as input, FPS is used again for sampling. The number of samples is half that of the first-level sampling points, resulting in fewer but wider-coverage second-level sampling points. Then, a spherical neighborhood with a larger radius, such as 4mm, is set as the center of the second-level sampling points. This is twice the radius of the first-level neighborhood. The first-level sampling points within the neighborhood, including their local aggregation features, are grouped into a new local point cloud group.

[0085] Second-level feature aggregation: For each local point cloud group of the second level, repeat the coordinate system transformation, take the sampling point of the second level as the origin, MLP feature mapping, such as 3-layer MLP, output dimension 512, max pooling process, to obtain the 512-dimensional global aggregated feature corresponding to each sampling point of the second level; this feature integrates the local detail features of the first level with the spatial correlation features of a wider range, such as the morphological differences between different regions of the defect.

[0086] Feature Interpolation and Fusion: Since the number of sampling points in the second level is less than that in the first level, interpolation is required. Inverse distance weighted interpolation maps the 512-dimensional global aggregated features of the second level onto all sampling points in the first level, so that each sampling point in the first level simultaneously possesses 256-dimensional local features and 512-dimensional global features. Then, the two types of features are concatenated and input into a single-layer MLP. The output dimension of 512 is fused to obtain a 512-dimensional multi-scale aggregated feature for each sampling point in the first level. This feature contains both local detail information and global spatial correlation information of the defect, fully representing the morphological features of the defect.

[0087] Since multimodal fusion needs to be matched with the two-dimensional structured data of infrared images, the discrete point-level features extracted from multiple levels are transformed into two-dimensional structured morphological feature maps.

[0088] Based on the geometric parameters of the aluminum rod, such as diameter and detection area length, the spatial coordinates of the 3D point cloud are mapped to a 2D plane. The axial direction of the aluminum rod is taken as the y-axis, and the circumferential direction is taken as the x-axis along the length of the aluminum rod. The 360° circumference is unfolded into linear coordinates. The 3D coordinates (x, y, z) of each point are transformed into 2D coordinates (u, v) through circumferential unfolding and axial projection. Here, u corresponds to the linear position after circumferential unfolding, v corresponds to the axial position, and the z coordinate height has been integrated into the multi-scale aggregation feature.

[0089] The feature map is constructed by filling a two-dimensional grid with the 512-dimensional multi-scale aggregated features of all sampling points in the first level, based on the two-dimensional coordinates (u,v), sorted by the u-axis and v-axis. If there is no sampling point at a certain grid position, the feature value is supplemented through neighborhood interpolation, such as bilinear interpolation, to form a two-dimensional morphological feature map of fixed size, such as a resolution of 256×256. Each pixel corresponds to a fixed area on the surface of the aluminum rod, and the pixel value is the 512-dimensional multi-scale aggregated feature of that area.

[0090] The feature map is optimized by normalizing the channel dimension of the generated morphological feature map, with the mean of each feature channel being 0 and the variance being 1, thus eliminating the magnitude difference of features in different channels. At the same time, a 3×3 convolution kernel is used for smoothing to reduce feature noise caused by interpolation, resulting in a morphological feature map with a clear structure and complete features. This feature map not only preserves the spatial distribution relationship of the surface morphological features of the aluminum rod, but also has the feature expression capability of multi-scale fusion, and can be directly used for multimodal fusion with infrared thermal field feature maps.

[0091] Constructing thermal field feature vectors:

[0092] Based on the surface temperature images of aluminum rods acquired by infrared cameras, interference is eliminated through preprocessing, abnormal temperature areas are located, and key features characterizing the thermal properties of defects are extracted and integrated into a structured thermal field feature vector. Temperature image preprocessing is then used to eliminate interference and calibrate the true temperature.

[0093] The image is processed using 3×3 mean filtering or Gaussian filtering. Mean filtering calculates the average gray value of the nine pixels surrounding each pixel and replaces the original pixel's gray value, which can smooth out local high-frequency noise caused by reflection on the aluminum rod surface, such as isolated bright or dark spots. Gaussian filtering, on the other hand, assigns weights according to pixel distance, with high weights for center pixels and low weights for edge pixels. This preserves edge details in temperature abnormal areas, such as the temperature boundary between defects and normal areas, while removing noise, thus avoiding over-filtering that could lead to feature loss.

[0094] Non-detection areas in the temperature image are removed by image segmentation. Utilizing the geometric contour of the aluminum rod, the aluminum rod appears as a continuous, elongated, bright area in the infrared image, while the background is a low-brightness area. Threshold segmentation is used, setting a grayscale threshold. Areas above the threshold are considered aluminum rod areas, and areas below the threshold are considered background, which is then removed. Based on the aluminum rod's production parameters, such as length and diameter, 50mm sections are cut off at both ends. The ends have normal residual heat from the cutting process, resulting in abnormal temperature distribution, which is not due to thermal field changes caused by defects. Only the main detection area in the middle is retained to ensure that subsequent feature calculations focus on the effective area.

[0095] The grayscale values ​​of the infrared image are converted into actual temperature values. The raw image captured by the infrared camera represents the radiation intensity in grayscale values. These values ​​need to be calibrated at the factory using the camera's built-in temperature calibration curve. The grayscale values ​​correspond to the temperature, mapping the grayscale value of each pixel to the actual temperature. For example, a grayscale value of 200 corresponds to a temperature of 35℃, and a grayscale value of 220 corresponds to a temperature of 40℃. Through linear interpolation or a lookup table method, the value of each pixel in the image directly reflects the true temperature of the aluminum rod surface, avoiding errors caused by subsequent feature calculations based on grayscale values.

[0096] Surface defects in aluminum rods, such as subcutaneous bubbles and internal inclusions, can cause localized temperature anomalies due to differences in thermal conductivity. By setting a temperature baseline and performing regional analysis, these temperature anomaly areas related to defects can be located, eliminating interference from normal temperature fluctuations.

[0097] Calculate the normal temperature baseline for the effective detection area of ​​the aluminum rod. Randomly select 10 non-overlapping normal sub-regions within the effective area. Each sub-region has an area of ​​5mm × 5mm, avoiding visible surface textures or scratches. Calculate the average temperature of each sub-region, and then take the average of these 10 average temperatures as the global normal temperature. Simultaneously calculate the standard deviation of the temperature in these sub-regions. This reflects the temperature fluctuation range within the normal range. For example, if the average temperature of the normal range is 38℃ and the standard deviation is 0.8℃, then the normal temperature fluctuation range is 38℃ ± 0.8℃.

[0098] By aggregating anomalous pixels through connected component analysis, continuous temperature anomaly regions are formed. Taking any anomalous pixel as a seed point, all pixels within a 3×3 radius that also meet the anomalous conditions are grouped into the same region, and this process is gradually expanded until no new pixels are added. At the same time, a minimum anomalous region area threshold is set, such as 10 pixels, corresponding to an actual area of ​​0.2mm×0.2mm, to match the thermal impact range of the small defects in the aluminum rod. Isolated small regions formed by local noise are filtered out, and finally multiple temperature anomaly regions are obtained, which are the thermal impact regions of the corresponding defects.

[0099] From each temperature anomaly region, key features reflecting the thermophysical properties of the defect are extracted to distinguish different types of defects, such as bubbles and inclusions, and to indicate the severity of the defect. The maximum deviation between the anomaly region and the normal temperature is calculated to reflect the intensity of the defect's influence on the thermal field. First, the temperature values ​​of all pixels within the anomaly region are statistically analyzed to find the highest or lowest temperature. Different defects exhibit different temperature characteristics: bubbles, due to their high internal gas thermal resistance, result in locally higher temperatures; metallic inclusions, due to their rapid thermal conductivity, result in locally lower temperatures. The difference between the highest or lowest temperature and the global normal temperature is calculated, i.e., the temperature anomaly amplitude. For example, if the highest temperature in the bubble region is 42℃ and the normal temperature is 38℃, then the anomaly amplitude is 4℃. The larger the amplitude, the more significant the difference in thermal conduction of the defect, and the more severe the defect may be.

[0100] The temperature change rate at the edge of the abnormal region is calculated to reflect the thermal transition characteristics between the defect and the normal region. First, the edge pixels of the abnormal region are identified, i.e., the abnormal pixels adjacent to the normal temperature pixels. The temperature difference between each edge pixel and its adjacent normal pixel is calculated, and then this temperature difference is divided by the actual distance between the two pixels, determined based on the resolution of the infrared camera (e.g., a pixel spacing of 0.1 mm), to obtain the temperature gradient of a single edge pixel. The average of all edge pixel gradients is taken as the average temperature gradient of the abnormal region. For example, the temperature anomaly region of a crack is linear, with a steep temperature change at the edge, a gradient of approximately 5℃ / mm, while the anomaly region of a bubble is circular, with a gentler temperature change at the edge, a gradient of approximately 2℃ / mm. This difference in gradient can be used to distinguish the morphology of the defect.

[0101] This includes the area and shape factor of the anomalous region, reflecting the size and shape of the defect's thermally affected area. Area calculation involves counting the number of pixels within the anomalous region and multiplying it by the actual area corresponding to a single pixel. For example, with a camera resolution of 0.1 mm / pixel, the area of ​​a single pixel is 0.01 square millimeters, yielding the actual thermally affected area of ​​the anomalous region. For instance, 100 pixels correspond to an area of ​​1 square millimeter. A larger area generally indicates a larger actual defect size. Shape factor calculation determines the shape by the relationship between the perimeter and area of ​​the anomalous region. Specifically, it's calculated by dividing the perimeter by the square root of the area. This ratio is smaller for circular regions (approximately 3.5) and larger for linear regions (approximately 10). For example, the anomalous region of a bubble is nearly circular with a small shape factor, while the anomalous region of a crack is linear with a large shape factor. This feature allows for a direct differentiation of defect morphology.

[0102] The degree of temperature difference within the abnormal region reflects the consistency of thermal distribution inside the defect. First, the average temperature of all pixels within the abnormal region is calculated. Then, the deviation of each pixel's temperature from this average temperature is calculated. The squares of all deviations are summed, divided by the number of pixels, and the square root is taken to obtain the standard deviation of the temperature distribution. The smaller the standard deviation, the more uniform the temperature within the abnormal region; the larger the standard deviation, the more significant the temperature difference. For example, subcutaneous bubbles have a standard deviation of approximately 0.5℃ due to uniform internal gas distribution, while internal inclusions have a standard deviation of approximately 1.5℃ due to uneven composition. This characteristic can help determine the internal properties of the defect.

[0103] If aluminum bars are dynamically transported on a production line, for example at a speed of 1 meter per second, then three consecutive frames of temperature images of the same defective area are acquired, with a frame interval of 0.1 seconds. The average temperature difference between adjacent frames of the abnormal area is calculated; this difference is the thermal hysteresis characteristic. The surface temperature of a normal aluminum bar changes steadily with transmission, while the temperature change rate of the defective area differs from that of the normal area. For example, the normal area cools down by 0.2°C per frame, while the area with metal inclusions cools down by 0.5°C per frame due to its faster thermal conductivity. The thermal hysteresis characteristic can be used to further distinguish defect types in dynamic transmission scenarios.

[0104] The core temperature features are normalized to eliminate the magnitude differences between different features, ultimately forming a standardized thermal field feature vector. This ensures that the vector can be effectively integrated with the morphological feature vector for defect identification. Five features most critical for defect identification are selected and logically ordered according to their influence intensity, transition characteristics, geometric shape, uniformity, and dynamic characteristics. For static detection, the aluminum rod transmission order is: temperature anomaly amplitude, temperature gradient, anomaly area, shape factor, and temperature distribution uniformity. For dynamic detection (aluminum rod transmission), temperature distribution uniformity is replaced with thermal hysteresis features, ensuring the feature order aligns with the logical priority of defect identification. The value of each feature is mapped to the range of 0 to 1 to avoid the model overemphasizing features with large magnitudes due to differences in feature magnitude (e.g., area measured in square millimeters, temperature amplitude in degrees Celsius).

[0105] Specifically, linear normalization is used: first, the minimum and maximum values ​​of the feature are statistically analyzed in a large number of defect samples; then, the minimum value is subtracted from the feature value of a certain defect; finally, the result is divided by the difference between the maximum and minimum values ​​to ensure that all features are on the same order of magnitude. For example, if the minimum value of the temperature anomaly amplitude sample is 1℃ and the maximum value is 10℃, and the amplitude of a certain defect is 4℃, then the normalized value is (4-1)÷(10-1)=0.33. The five normalized features are arranged in a fixed order to form a thermal field feature vector with a dimension of 5. For example, the normalized features of a subcutaneous bubble are 0.4 temperature anomaly amplitude, 0.2 temperature gradient, 0.3 anomaly area, 0.18 shape factor, and 0.1 temperature distribution uniformity. The final thermal field feature vector is the set of these five values ​​arranged in order. This vector fully represents the thermophysical properties of the defect and can be directly fused with the morphological feature vector through an attention mechanism to provide key evidence of the thermal field dimension for subsequent defect classification, such as distinguishing between bubbles, inclusions, and cracks.

[0106] Output of the thermal field feature map:

[0107] The constructed thermal field feature vector is obtained as input. Based on the thermal resistance defect correlation model, the mapping relationship between thermal field features and defect thermal resistance properties is established. Then, through spatial location matching and feature reconstruction, the discrete vector is transformed into a structured thermal field feature map, providing a thermophysical feature expression bound to spatial distribution for subsequent multimodal fusion.

[0108] The thermal field feature vector is discrete high-dimensional data. Preprocessing eliminates data interference and binds spatial location information, ensuring that the input model can associate it with specific regions on the aluminum rod surface, avoiding feature-location misalignment. The importance of the five dimensions in the thermal field feature vector—temperature anomaly amplitude, temperature gradient, and anomaly area—is evaluated based on the feature contribution during model training, such as feature importance scores calculated using a random forest algorithm. The top four core features with the highest contribution, such as temperature anomaly amplitude, temperature gradient, shape factor, and temperature distribution uniformity, are retained, while redundant features with low contribution, such as thermal hysteresis features in dynamic scenarios, are removed. If the model validates its correlation with defect thermal resistance, this will be considered. The value range of the core features is then adjusted to the model's preset input range, such as 0-1, through linear scaling. Each thermal field feature vector corresponds to a temperature anomaly region. For the previously located defect thermal influence region, the vector is bound to the spatial position of the region in the original temperature image. The center pixel coordinates of the temperature anomaly region are recorded, such as the pixel center point coordinates of the region in the original temperature image. The boundary pixel range of the region is then marked, such as the pixel box from the upper left corner to the lower right corner. The spatial coordinate information is used as an additional parameter and integrated with the preprocessed thermal field feature vector into a feature vector with spatial labels, so that the features output by the model can accurately correspond to the specific position on the surface of the aluminum rod.

[0109] A supervised learning model trained with a large number of defect samples transforms thermal field features into thermal resistance correlation features that characterize the thermal resistance properties of defects. A pre-set model file, containing trained weight parameters and network structure, is loaded and trained using 5000 sets of samples with known defect types, thermal field feature vectors, and actual thermal resistance measurements. The samples cover common defects such as bubbles, inclusions, and cracks. The thermal resistance of each defect is measured using a laboratory thermal conductivity meter; for example, the thermal resistance of a bubble is approximately 2.5 W / (m·K), and the thermal resistance of the aluminum rod matrix is ​​approximately 237 W / (m·K). The model's input, hidden, and output layer parameters are initialized so that the input layer dimension matches the pre-processed thermal field feature vector dimension (e.g., 4-dimensional input), and the output layer dimension is the model's pre-set thermal resistance correlation feature dimension (e.g., 8-dimensional), covering defect thermal resistance magnitude, thermal resistance distribution uniformity, and the difference between thermal resistance and the matrix.

[0110] The core features from the spatially labeled feature vectors are input into the model's input layer. Nonlinear transformations in the hidden layers, such as the ReLU activation function, establish the correlation between features and thermal resistance. The model's hidden layer first calculates the correlation between thermal field features and defect thermal resistance. For example, the larger the temperature anomaly amplitude, the more significant the difference between the defect's thermal resistance and the matrix. Then, through multi-layer iterative optimization, the 4-dimensional thermal field features are transformed into 8-dimensional thermal resistance correlation features. For instance, the thermal field features of a subcutaneous bubble have an amplitude of 0.4, a gradient of 0.2, a shape factor of 0.18, and a distribution uniformity of 0.1. After inputting these features into the model, the output 8-dimensional thermal resistance correlation features show a thermal resistance difference dimension of 0.8, close to 1, indicating a large difference in thermal resistance from the matrix. The thermal resistance distribution uniformity dimension is 0.9, indicating a uniform distribution of thermal resistance in the bubble, directly reflecting the thermophysical nature of the defect.

[0111] After the model outputs 8-dimensional thermal resistance correlation features, it automatically calls the preset validity threshold, such as a feature confidence threshold of 0.7 determined by the validation set. It calculates the confidence of each dimension feature, that is, the model's prediction credibility for that feature value, which is determined based on the error distribution during training. If the confidence of all dimensions is higher than the threshold, the thermal resistance correlation feature is determined to be valid and proceeds to the next step. If there is a dimension with a confidence of lower than the threshold, such as a confidence of 0.5 for the thermal resistance size dimension, it returns to the previous step, re-extracts the thermal field feature vector of that region and inputs it into the model again until a valid feature is output, to prevent the subsequent feature map from being distorted due to feature errors.

[0112] Thermal resistance correlation features are high-dimensional data bound with spatial labels. Through spatial location mapping, they are accurately assigned to the corresponding pixel regions of the original temperature image, ensuring that the spatial distribution of the thermal field feature map is consistent with the actual thermal field distribution on the surface of the aluminum rod.

[0113] The pre-processed temperature image of the effective area of ​​the aluminum rod, with a resolution of 512×512 pixels, is divided into a fixed-size feature mapping grid, such as 8×8 pixels per grid cell. Each grid cell corresponds to a small area on the surface of the aluminum rod, such as a grid cell size of 0.8mm×0.8mm, matching the spatial coverage of the thermal resistance correlation feature. The center pixel coordinates of each grid cell are recorded, such as the center coordinates of the first row and first column grid (4,4), which serve as the positioning reference for subsequent feature mapping. The center coordinates of the temperature anomaly region are extracted from the spatially labeled feature vector, and the Euclidean distance between these coordinates and the center coordinates of all grid cells is calculated. The grid cell with the closest distance is selected as the target grid cell. If the boundary of the temperature anomaly region covers multiple grid cells, such as a large anomaly region covering three adjacent grid cells, the thermal resistance correlation feature is assigned to these three grid cells simultaneously and labeled as an associated grid group, ensuring that all grids in the defect thermally affected area can obtain the corresponding thermal resistance feature. For normal region grid cells in the original temperature image that do not correspond to the thermal field feature vector, i.e. regions without temperature anomalies, the model will automatically generate basic thermal resistance correlation features. Based on the standard thermal resistance properties of the aluminum rod matrix, such as thermal resistance 237W / (m・K), 8-dimensional basic features are generated, such as thermal resistance difference dimension value of 0 and thermal resistance distribution uniformity dimension value of 1, and are assigned to all normal region grid cells, so that the subsequently generated thermal field feature map covers the entire effective detection area of ​​the aluminum rod without any spatial gaps.

[0114] By assigning feature dimensions, mapping pixel values, and interpolating, the 8-dimensional thermal resistance correlation features of each grid cell are transformed into pixel information of a 2D image, forming a complete thermal field feature map. The 8-dimensional thermal resistance correlation features are divided into two groups according to their thermophysical meaning, corresponding to the channel dimension and pixel grayscale dimension of the thermal field feature map, respectively. The first four core dimensions, thermal resistance difference, thermal resistance distribution uniformity, thermal resistance gradient, and thermal resistance stability, are selected as the four channels of the feature map, similar to the three channels of an RGB image. The last four auxiliary dimensions (such as the correlation coefficient between thermal resistance and temperature) are converted into pixel grayscale values ​​by weighted summation, ranging from 0 to 255. For example, if the four core dimension values ​​of a certain grid cell are [0.8, 0.9, 0.6, 0.7], then the four channels of the feature map are assigned values ​​of 0.8, 0.9, 0.6, and 0.7, respectively. At the same time, the weighted summation of the last four auxiliary dimension values ​​is 0.65, which is then scaled to 166. 0.65 × 255 ≈ 166 is used as the pixel grayscale value of this grid cell. The channel feature value and grayscale value of each grid cell are filled into all the corresponding pixels of that grid. For example, the 64 pixels of an 8×8 grid cell are all assigned the same 4-channel feature value and grayscale value. For associated grid groups, such as a defect area covering 3 grid cells, feature smoothing is performed at the boundary of the grid cells. The channel feature value of the pixels at the boundary is calculated by bilinear interpolation, and the average value of the channel values ​​of the adjacent grid cells is taken to avoid feature abrupt changes caused by grid boundaries.

[0115] For edge pixels in normal regions that lack assigned basic features, such as incomplete grid cells at image edges, neighborhood interpolation is used to supplement feature values ​​by taking the average of the features from the three surrounding normal grid cells. This ensures that each pixel in the thermal field feature map has complete channel features and grayscale values. The size of the initially constructed thermal field feature map is adjusted to match the size of the previous morphological feature map. Figure 1 To achieve the desired resolution, such as 256×256 pixels, if the original temperature image resolution is 512×512, then downsampling, such as average pooling, is used to reduce the feature map size to 256×256 while maintaining the channel dimension of 4 channels. If the original resolution is lower than 256×256, then upsampling, such as bilinear interpolation, is used to amplify it to the target resolution, so that the spatial scale of the thermal field feature map and the morphological feature map are matched, avoiding feature misalignment caused by size differences during subsequent multimodal fusion.

[0116] After the initial thermal field feature map is generated, noise suppression, feature enhancement, and dimensionality adjustment are needed to optimize the clarity and usability of the feature map, enabling it to be directly fused with the morphological feature map for multimodal processing. A 5×5 Gaussian filter is used to smooth each channel of the thermal field feature map. The average channel feature value of the 25 pixels surrounding each pixel is calculated and replaced with the original pixel value to eliminate local noise caused by interpolation, such as feature fluctuations of grid cells in normal areas. Then, the Sobel edge detection algorithm is used to enhance the defect area, associate the edge features of the grid group, calculate the gradient value of the edge pixels of the defect area, and increase the gray value of the edge pixels by 20%, such as from 166 to 199, so that the thermal field feature boundary of the defect is clearer and it is easier to align with the edge of the morphological feature during subsequent fusion.

[0117] The feature values ​​of the four channels of the thermal field feature map are uniformly mapped to the 0-1 range. The maximum and minimum feature values ​​of all pixels in each channel are calculated. Through linear transformation, such as (current value - minimum value) / (maximum value - minimum value), the channel value of each pixel is adjusted to 0-1. This avoids the overemphasis on a certain channel feature during subsequent fusion due to differences in feature magnitude between different channels, such as thermal resistance difference range of 0-1 and thermal resistance gradient range of 0-0.8. The optimized thermal field feature map is converted into the model's preset output format. If the subsequent multimodal fusion model requires the input to be a channel-priority tensor format, such as 4×256×256, channel number × height × width, the dimensional order of the feature map is adjusted. At the same time, the spatial coordinate mapping table of the feature map is saved to record the correspondence between the feature map pixels and the actual position on the aluminum rod surface. For example, the feature map pixel (100,100) corresponds to the position of 100mm axially and 30° circumferentially on the aluminum rod, which is convenient for subsequent defect localization and result traceability. Finally, a structured, high-definition thermal field feature map bound to spatial position is output, completing the transformation from discrete vector to image.

[0118] The defect identification and classification module acquires morphological feature maps and thermal feature maps, uses a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, inputs them into a fully connected classifier, and outputs the defect type.

[0119] The process of dynamically allocating the weights of the morphological feature map and the thermal feature map is as follows:

[0120] The system acquires morphological feature maps to represent the geometric properties of defects and thermal feature maps to represent the thermophysical properties of defects. Using these as inputs, the system performs feature alignment, independent encoding, attention calculation, and dynamic weighting through a dual-modal attention fusion network. This intelligently allocates weights to the two modal features, automatically strengthening modal features with higher contribution in different regions of the defect, such as crack edges and bubble centers. For example, it strengthens morphological features in crack regions and strengthens thermal field features in bubble regions. The output is a feature map that integrates the advantages of both modalities.

[0121] Bimodal feature map preprocessing and spatial alignment eliminate modal differences, ensuring positional matching. The resolution of the two feature maps is calibrated. If the morphological feature map has a resolution of 256×256 pixels and 4 channels, corresponding to different dimensions of geometric features, and the thermal field feature map has a resolution of 512×512 pixels and 4 channels, corresponding to different dimensions of thermal resistance correlation features, then bilinear interpolation downsampling is used to reduce the thermal field feature map to 256×256 pixels, ensuring that the two are completely identical in size. If the number of channels differs, such as 4 channels for the morphological map and 8 channels for the thermal field map, then a 1×1 convolution kernel without activation functions is used to compress the number of channels in the thermal field feature map to 4 channels, ensuring that the height × width × number of channels of the bimodal feature maps are perfectly matched, both being 256×256×4, avoiding attention calculation deviations due to dimensional differences.

[0122] The numerical distributions of the two modal features may differ. For example, the channel values ​​of the morphological feature map range from 0 to 0.8, while those of the thermal feature map range from 0 to 1.2. Batch normalization is needed to eliminate the distribution differences. For each channel of each feature map, the mean and standard deviation of all pixels in that channel are calculated separately. Then, the values ​​are converted into a normal distribution with a mean of 0 and a standard deviation of 1, so that the two modal features are on the same order of magnitude when attention is calculated later.

[0123] Since both feature maps originate from the same detection area of ​​the aluminum rod, their corresponding physical spatial coordinates ensure position matching. Based on the axial, length, and circumferential directions of the original aluminum rod, spatial coordinate anchor points are marked in both the morphological feature map and the thermal field feature map. For example, the position of 100mm axially and 0° circumferentially corresponds to the pixel coordinates (50, 50) in the feature map. By comparing the pixel coordinates of more than three anchor points, if there is a deviation, such as the anchor point of the thermal field map being offset by 2 pixels, the position of the thermal field feature map is adjusted by pixel translation. This ensures that the same pixel in the dual-modal feature map accurately corresponds to the same physical area on the surface of the aluminum rod. For example, the feature map (50, 50) corresponds to the position of 100mm axially and 0° circumferentially on the aluminum rod, avoiding misalignment between features and positions during subsequent weight allocation.

[0124] By independently encoding the dual-modal feature maps, deep modality-specific features are extracted and configured into lightweight convolutional encoders with identical structures for both modalities. These encoders consist of 3 convolutional layers and 2 pooling layers, with unified structural parameters: the first convolutional layer has a 3×3 kernel size, a stride of 1, and 8 output channels, using the ReLU activation function to enhance non-linear feature extraction; the first pooling layer uses 2×2 max pooling to retain key features while reducing the feature map size to 128×128; the second convolutional layer has a 3×3 kernel size, a stride of 1, and 16 output channels, using the ReLU activation function; the second pooling layer uses 2×2 max pooling, reducing the size to 64×64; the third convolutional layer has a 3×3 kernel size, a stride of 1, and 32 output channels, with no pooling, maintaining the 64×64 size for subsequent spatial attention calculation.

[0125] The preprocessed morphological feature map is input into the morphological encoder. Through a three-layer convolutional filtering process, the first convolutional layer captures the basic geometric features of the defect, such as edges and corners. The second convolutional layer enhances the combined features of the defect, such as the linear contour of cracks and the blocky morphology of depressions. The third convolutional layer extracts the deep geometric features of the defect, such as depth variation patterns and edge roughness, outputting a 64×64×32 morphological deep feature map. The preprocessed thermal field feature map is input into the thermal field encoder, and the same convolutional operation extracts the deep thermophysical features of the defect, such as thermal resistance distribution gradients and thermal field anomaly boundaries, outputting a 64×64×32 thermal field deep feature map. Both deep feature maps retain the core information of their respective modalities, providing high-quality input for subsequent attention weight calculations.

[0126] The bimodal attention fusion network calculates the importance of different locations within each modality by combining intramodal spatial attention and intermodal correlation attention. For example, in the morphology map, the crack edge is more important than the normal area. It also calculates the overall correlation between the two modalities, such as the thermal field mode being more important than the morphology mode in the bubble region. Finally, it fuses the two modalities to obtain dynamic weights.

[0127] Intramodal spatial attention weight calculation: For each deep feature map, the spatial importance weight of each pixel within it is calculated to highlight the features of the defect area and suppress background interference. Taking a morphological deep feature map of 64×64×32 as an example: global average pooling is performed on the feature map, and the average value of all 64×64 pixels in each channel is calculated to obtain a 1×1×32 channel global feature vector.

[0128] The vector is input into a two-layer fully connected network. The first layer outputs a dimension of 16 and uses the ReLU activation function; the second layer outputs a dimension of 32 and has no activation, resulting in a 1×1×32 channel weight vector. The channel weight vector is then multiplied channel-by-channel by the original morphological deep feature map, with all pixels in each channel multiplied by the weight value of that channel, resulting in a 64×64×32 channel-weighted feature map. Finally, the channel-weighted feature map is summed by channel dimension, adding the pixel values ​​of the 32 channels to obtain a 64×64×1 spatial weight map. The weight values ​​are then converted to 0-1 probability values ​​using the Softmax activation function, with the sum of all pixel weights being 1. Pixels with higher weight values ​​are more important in the morphological mode; for example, the pixel weight at the crack edge is close to 0.8, while that in the normal area is close to 0.1. The same method is used to process the thermal field deep feature map, resulting in a 64×64×1 thermal field spatial weight map, highlighting the importance of temperature anomaly regions in the thermal field mode; for example, the pixel weight at the bubble center is close to 0.9, while that in the normal area is close to 0.05.

[0129] Intermodal correlation attention weight calculation: Calculate the overall contribution weight of the two modalities to determine which modality is more valuable in the current defect scenario.

[0130] First, the morphological deep feature map and the thermal deep feature map are flattened in terms of spatial dimensions, transforming the 64×64×32 feature map into a 1×64×64×32 vector, which is a 1×131072 vector, denoted as the morphological vector V. m and thermal field vector V h ;

[0131] Calculate the correlation matrix between the two vectors, using V m With V h Multiplying the transposes of the two modalities yields a 1×1 correlation coefficient, ranging from -1 to 1. The closer the coefficient is to 1, the more similar the characteristics of the two modalities are, and the more balanced their contributions are; the closer the coefficient is to 0, the higher the contribution of one modality.

[0132] The correlation coefficient is input into the Sigmoid activation function, which converts it into a 0-1 modal weight value. If the coefficient is 0.3, it indicates that the thermal field mode is more unique; if the coefficient is 0.8, it indicates that the contributions of the two modes are balanced, and the output is 0.5, ω. h =0.5, which enables dynamic allocation of overall weights based on modal correlation.

[0133] By combining intramodal spatial weights and intermodal modal weights, the deep feature maps of morphology and thermal field are dynamically weighted. A fusion operation is then performed to obtain the final bimodal fused feature map, which weights the deep feature map of morphology by combining the morphological spatial weight map (64×64×1) with the morphological modal weights ω. m Multiply to obtain a final morphological weight map of 64×64×1; then multiply this weight map with the deep morphological feature map pixel by pixel, multiplying the 32 channel values ​​of each pixel by the weight value of the corresponding position to obtain a weighted morphological feature map of 64×64×32.

[0134] The same method is used to process the deep feature map of the thermal field: the spatial weight map of the thermal field and ω. h The final weighted thermal field map is obtained by multiplying the two maps, and then multiplied pixel by pixel with the deep feature map of the thermal field to obtain a 64×64×32 weighted thermal field feature map.

[0135] The core of this step is dual reinforcement: highlighting defective areas within a modality, such as the crack edges in a morphological graph, and reinforcing modalities with higher contributions between modalities, such as the thermal field modality in a bubble scene, to avoid feature bias caused by a single weight.

[0136] The two weighted feature maps are merged using a channel-by-channel stitching and convolutional fusion method:

[0137] First, the weighted morphological feature map (64×64×32) and the weighted thermal field feature map (64×64×32) are concatenated along the channel dimension to obtain a 64×64×64 concatenated feature map, which retains all deep features of the two modalities.

[0138] Convolutional fusion is performed on the spliced ​​feature maps using a 1-layer 3×3 convolution kernel with 32 output channels and ReLU activation function. The 64-channel features are compressed into 32 channels through sliding convolution, and the correlation information of the two modalities is fused at the same time, combining the crack edge features of the morphology map with the thermal resistance features of the crack region of the thermal field map.

[0139] Finally, the feature map size was enlarged from 64×64 to 256×256 by 2×2 deconvolution, which is consistent with the original feature map size, resulting in a dual-modal fusion feature map of 64×64×32 to 256×256×32. Each pixel of this feature map integrates the advantages of morphology and thermal field, such as the crack area being dominated by morphological features and the bubble area being dominated by thermal field features.

[0140] To eliminate noise interference during the fusion process and verify the rationality of weight allocation, post-processing of the fused feature map is necessary. A 3×3 Gaussian filter is used to process the fused feature map, with weights allocated based on pixel distance: the center pixel has a weight of 0.2, and the surrounding 8 pixels have a weight of 0.1. The weighted average of the 9 surrounding pixels for each pixel is calculated and used to replace the original pixel value, smoothing out local feature fluctuations caused by weight allocation, such as the boundary regions where the weights of the two modes abruptly change. Defect features are enhanced through contrast stretching. The minimum and maximum pixel values ​​of each channel in the fused feature map are calculated, and the pixel values ​​are linearly mapped to the 0-1 range, making the feature differences between defective and normal areas more obvious. For example, the feature value of the crack area increases from 0.4 to 0.7, while that of the normal area decreases from 0.1 to 0.05. This is achieved by using the defect area feature contribution... Contribution analysis verifies the rationality of the weights: Typical defect regions, such as a crack region and a bubble region, are selected in the fused feature map. The feature contribution ratios of the morphological mode and the thermal field mode within each region are calculated. By backtracking the weighting process, the total weight ratio of the two modes is statistically analyzed. If the morphological contribution ratio of the crack region is ≥60% and the thermal field contribution ratio of the bubble region is ≥60%, the weight allocation is deemed effective and consistent with the physical characteristics of the defects: cracks have more significant geometric features, while bubbles have more significant thermophysical features. If the contribution ratio is insufficient (e.g., the thermal field contribution ratio of the bubble region is only 40%), the process returns to the intermodal correlation attention calculation step. The calculation parameters of the correlation matrix are adjusted, such as increasing the weight coefficient of the thermal field vector, and the modal weights are recalculated until the contribution ratio meets the standard, thus improving the accuracy of subsequent defect identification.

[0141] Output defect type:

[0142] The dual-modal fusion feature map is preprocessed to transform it into a one-dimensional feature vector that can be accepted by a fully connected classifier. The previously output fusion feature map is a 256×256×32 three-dimensional structure, which needs to be flattened in the order of channel, height, and width: first, traverse the 32 feature channels, and read the feature values ​​of 256×256 pixels in each channel in row-major order. Finally, it is concatenated into a 1×256×256×32, or 1×2097152, one-dimensional feature vector, so that the vector dimension perfectly matches the number of neurons in the input layer of the classifier.

[0143] A pre-trained fully connected classifier is loaded, with an input layer, hidden layer 1, hidden layer 2, and output layer. The input layer has 2,097,152 neurons, consistent with the dimension of the flattened feature vector. Hidden layer 1 has 1,024 neurons, which use the ReLU activation function to perform non-linear mapping on the input features, extracting more abstract defect-related features while avoiding gradient vanishing. Hidden layer 2 has 256 neurons, which continue to use the ReLU activation function to compress the feature dimension and strengthen key classification information. The number of neurons in the output layer matches the number of defect categories of the aluminum rod. For example, if there are 4 categories, corresponding to cracks, bubbles, inclusions, and normal, the Softmax activation function is used to transform the output value into a 0-1 probability distribution, and the sum of the probabilities of all categories is 1.

[0144] For classification calculations, a one-dimensional feature vector is input into the classifier and first passed through the input layer to hidden layer 1. There, a 1024-dimensional feature vector is obtained through weight matrix multiplication, bias term addition, and ReLU activation. This vector is then passed to hidden layer 2, where the same operation is repeated to obtain a 256-dimensional key feature vector. Finally, this vector is input to the output layer, where a Softmax operation outputs the probabilities of each category, such as cracks (0.05), bubbles (0.92), inclusions (0.02), and normal (0.01). The defect type is determined by selecting the category with the highest output probability, such as bubbles (probability 0.92). A probability threshold, such as 0.7, is set. If the maximum probability is lower than the threshold, the defect is classified as unknown and manual review is triggered to ensure classification accuracy. The entire process relies on the classifier weights pre-trained from 5000 defect samples.

[0145] Example 2

[0146] Please see Figure 2 Based on Example 1, Example 2 of this application also provides a method for detecting the surface quality of aluminum rods based on visual inspection, including the following specific steps:

[0147] Step 1: Generate a 3D point cloud by acquiring structured light images of an aluminum rod, performing phase calculation and triangulation; and generate a temperature image by capturing the surface temperature field of the aluminum rod using an infrared camera.

[0148] Step 2: Estimate the normal vector of the 3D point cloud, identify the surface discontinuity region by the normal vector change rate to generate the defect region, and construct the morphological feature vector. Using the PointNet++ architecture, perform hierarchical feature extraction on the morphological feature vector and output the morphological feature map. Calculate the temperature features based on the temperature image to construct the thermal field feature vector, input the thermal field feature vector into the preset thermal resistance defect association model, and output the thermal field feature map.

[0149] Step 3: Obtain morphological feature maps and thermal feature maps. Use a bimodal attention fusion network to dynamically allocate weights between the morphological feature maps and thermal feature maps, and input them into a fully connected classifier to output the defect type.

[0150] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A visual inspection-based aluminum bar surface quality detection system, characterized by, The system comprises: A multi-modal data acquisition module, which performs phase unwrapping and triangulation to generate a three-dimensional point cloud by acquiring a structured light image of an aluminum rod; and an infrared camera is used to capture a temperature field of the surface of the aluminum rod to generate a temperature image; A defect feature extraction module, which estimates a normal vector of the three-dimensional point cloud, identifies a surface discontinuous region by a normal vector change rate, constructs a morphological feature vector, extracts hierarchical features of the morphological feature vector by using a PointNet++ architecture, and outputs a morphological feature map; calculates a temperature feature based on the temperature image to construct a thermal field feature vector, inputs the thermal field feature vector into a preset thermal resistance defect correlation model, and outputs a thermal field feature map; The process of outputting the morphological feature map is: Integrates the three-dimensional point cloud and the morphological feature vector to input the PointNet++ architecture, focuses on key regions in the three-dimensional point cloud by grouping, filters to obtain sampling points and divides local neighborhoods, merges the local neighborhoods into local point cloud groups, performs local coordinate system conversion, feature mapping and global pooling on the local point cloud groups, converts the morphological features of discrete points in the local point cloud groups into aggregated features of local correlation, and extracts multi-level features based on the aggregated features in the first aggregation layer by repeating the sampling, grouping and aggregation processes; The process of estimating the normal vector of the three-dimensional point cloud is: For each point in the three-dimensional point cloud, a plurality of points closest to the point are searched by using k-neighborhood search to construct a neighborhood point set, and a centroid of the neighborhood point set is calculated; the coordinates of all points in the neighborhood are added and then divided by the number of points, and each neighborhood point is subtracted from the centroid to obtain a centralized point set; the coordinates of each point in the centralized point set are multiplied and summed, and then divided by the number of points to obtain a covariance matrix; eigenvalue decomposition is performed on the covariance matrix to obtain three eigenvalues arranged in descending order and corresponding three eigenvectors; the eigenvector corresponding to the smallest eigenvalue in the covariance matrix is taken as the initial estimation of the normal vector of the point based on the normal vector of the point cloud surface; The process of constructing the morphological feature vector is: According to the point cloud normal vector estimation, the directional difference of adjacent point normal vectors is measured to determine the surface smoothness of the region where each target point in the three-dimensional point cloud is located, a threshold of the normal vector change rate is set, abnormal points are filtered out from the point cloud, a preliminary discontinuous region is formed by region aggregation, the preliminary discontinuous region contains the normal structure of the aluminum rod, and the non-defect region is excluded by combining the geometric features and prior knowledge of the aluminum rod to segment the real defect region; key parameters representing the geometric properties of each independent defect region are extracted, and the morphological feature vector is formed by combining the key parameters in a fixed order; A defect recognition and classification module, which acquires the morphological feature map and the thermal field feature map, dynamically allocates the weights of the morphological feature map and the thermal field feature map by using a dual-modal attention fusion network, inputs the full connection classifier, and outputs the defect type.

2. The aluminum bar surface quality detection system based on visual detection according to claim 1, characterized in that, The process of generating the three-dimensional point cloud is: A specially coded sinusoidal fringe structured light pattern is projected onto the surface of the aluminum rod. A plurality of cameras synchronously capture the structured light pattern on the surface of the aluminum rod. The plurality of captured structured light patterns are subjected to phase unwrapping processing. A four-step phase shift method is used to analyze the phase change of the fringes in the structured light pattern, and the phase distribution corresponding to each pixel point on the surface of the aluminum rod is calculated. Based on the principle of triangulation, the phase distribution information is converted into the three-dimensional spatial coordinates of each point on the surface of the aluminum rod. Through the integration of the three-dimensional spatial coordinates, the point cloud data of the three-dimensional topography of the surface of the aluminum rod is generated. 3.The aluminum bar surface quality detection system based on visual detection according to claim 1, characterized in that, The process of generating the temperature image is: The infrared camera acquires the infrared radiation energy of the surface of the aluminum rod at a set frame rate, converts the infrared radiation energy into an electrical signal, and converts the electrical signal into a gray value corresponding to the temperature through signal processing to form an infrared temperature image.

4. The aluminum bar surface quality detection system based on visual detection according to claim 1, characterized in that, The process of constructing the thermal field feature vector is: The temperature image is preprocessed, and the temperature abnormal area related to the defect is located through temperature benchmark setting and region analysis. The interference of normal temperature fluctuation is excluded. The key features of the thermal physical properties of the aluminum rod defect are extracted from each temperature abnormal area, and are integrated in a fixed order. Through normalization, the order of magnitude difference is eliminated to form a standardized thermal field feature vector.

5. The aluminum bar surface quality detection system based on visual detection according to claim 1, characterized in that, The process of outputting the thermal field feature map is: First, the importance of the five dimensions in the thermal field feature vector is evaluated. Each thermal field feature vector corresponds to a temperature abnormal area, and the thermal field feature vector is corresponded to the spatial position of the area in the temperature image. Through the supervised learning model trained by a large number of defect samples, the thermal field feature is converted into a thermal resistance correlation feature that can represent the thermal resistance properties of the defect. Through spatial position mapping, the thermal resistance correlation feature is assigned to the corresponding pixel area of the temperature image. Through feature dimension allocation, pixel value mapping and interpolation supplement, the eight-dimensional thermal resistance correlation feature of each grid element is converted into the pixel information of the two-dimensional image to form a complete thermal field feature map.

6. The aluminum bar surface quality detection system based on visual detection according to claim 1, characterized in that, The process of outputting the defect type is: The morphology feature map and the thermal field feature map are obtained, input into the dual-modal attention fusion network, and the weights of the two modal feature maps are allocated through feature alignment, independent coding, attention calculation and dynamic weighting. The modal features with higher contribution degree are automatically strengthened in different areas of the defect. The feature map fused with the advantages of the two modalities is output, and is converted into a one-dimensional feature vector that can be received by a full connection classifier. A pre-trained full connection classifier is loaded, including an input layer, a double hidden layer and an output layer. The input vector is calculated to output the probability of each class. The class with the maximum probability is selected as the defect type.

7. A method for detecting surface quality of an aluminum bar based on visual detection, the method being applied to the system for detecting surface quality of an aluminum bar based on visual detection according to any one of claims 1-6, characterized in that, The process includes the following steps: Step 1: generate three-dimensional point cloud by phase unwrapping and triangulation through acquisition of aluminum rod structured light image; generate temperature image by capturing temperature field of aluminum rod surface with infrared camera; Step 2: estimate normal vector of three-dimensional point cloud, identify surface discontinuous area through normal vector change rate, and construct morphology feature vector; output morphology feature map by hierarchical feature extraction of morphology feature vector using PointNet++ architecture; calculate temperature feature based on temperature image to construct thermal field feature vector, input thermal field feature vector into pre-set thermal resistance defect correlation model, and output thermal field feature map; Step three, obtain the morphological feature map and the thermal field feature map, utilize the dual-modal attention fusion network to dynamically allocate the weight of the morphological feature map and the thermal field feature map, and input the full connection classifier to output the defect type.

Images