Container bottom plate surface defect intelligent detection method based on deep learning

Abstract

The invention relates to the technical field of industrial nondestructive testing and computer vision, and particularly discloses an intelligent detection method for surface defects of a container bottom plate based on deep learning. The method comprises the following steps: synchronously acquiring data through a laser radar and a line scanning camera, complementing a shielding area of a point cloud, and executing coordinate normalization to generate preprocessed data; constructing a lightweight feature alignment network to realize cross-modal feature mapping and pixel-level error correction; segmenting a defect area by adopting an improved PointNet + + network and reconstructing a three-dimensional model; extracting defect geometric parameters and combining with material attributes to perform stress simulation and life prediction; aggregating multi-port safety life data to construct a federated framework to update model parameters; and synthesizing physically real defect samples based on false detection cases, and injecting the defect samples into the network for training. According to the method, the bottleneck of missing detection of internal defects in traditional two-dimensional detection is overcome, the defect detection rate and quantification precision are remarkably improved, full-life-cycle safety evaluation of the container is supported, and the efficient requirement of automatic port inspection is met.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and industrial non-destructive testing technology, and in particular to a method for intelligently detecting surface defects of container bottom plates based on deep learning. Background Art

[0002] Existing container floor inspection primarily relies on two-dimensional image analysis using line scan cameras (e.g., YOLOv5 and U-Net). This method is effective for visible surface defects such as scratches and rust, but cannot identify hidden defects such as delamination and blistering within the floor. The root cause is that two-dimensional images lack depth information, and internal defects in wood or metal are not visually represented on the surface. When using a multimodal solution combining LiDAR and line scan cameras, temporal and spatial offsets exist between the point cloud and the image due to differences in device sampling frequencies and mechanical vibration. Traditional calibration methods (e.g., checkerboard calibration) only address static deviations; cross-modal feature alignment errors in dynamic scenarios lead to distorted 3D reconstructions. Defect samples in industrial scenarios are scarce and unevenly distributed. Traditional supervised learning models rely on large amounts of labeled data, resulting in poor generalization under low-sample conditions, with high false positive rates for minor scratches and high miss rates for new rust patterns.

[0003] Mainstream methods use RGB-D cameras to directly acquire aligned data, but depth cameras have limited accuracy and suffer from significant loss of depth information in bright lighting conditions. Algorithms based on Structure from Motion (SfM) rely on texture features, resulting in a high failure rate for reconstructing low-texture wood surfaces. ICP algorithms based on point cloud registration are computationally complex and cannot meet the demands of real-time detection. Traditional GAN-generated samples lack physical realism (for example, synthesized rust textures do not conform to the laws of electrochemical diffusion), resulting in poor generalization in real-world scenarios.

[0004] Existing technologies suffer from three major bottlenecks: spatiotemporal asynchrony of multi-source data, lack of three-dimensional quantitative models, and inadequate small-sample defect learning mechanisms. This results in a high missed detection rate for internal defects in container bottom plates and low detection accuracy, making them unable to meet the sub-millimeter precision and real-time requirements of automated port inspections. Summary of the Invention

[0005] The present invention provides an intelligent detection method for container floor surface defects based on deep learning, so as to solve the problem of how to dynamically compensate for the asynchronous errors of multi-source data through the spatiotemporal calibration parameters of laser radar and line scan camera, and realize sub-millimeter level three-dimensional quantitative detection of container floor surface defects.

[0006] In order to solve the above technical problems, the present invention provides a method for intelligent detection of container floor surface defects based on deep learning, comprising:

[0007] By synchronously collecting data with a line scan camera and a lidar, the occluded areas of the point cloud are completed, and the coordinates of the image and point cloud are normalized to generate pre-processed data aligned in time and space.

[0008] Build a lightweight feature alignment network, load preprocessed data to perform cross-modal feature mapping, perform pixel-level error correction, and generate aligned feature data;

[0009] The expression for generating alignment feature data includes:

[0010]

[0011] in, is the variational derivative of the error functional with respect to the characteristic map; is the feature map error functional; is the square of the error tolerance threshold; is the characteristic residual; is the regularization strength coefficient; is the Chebyshev norm order; is the p-th power of the gradient in the k direction; is the k-direction second-order derivative of the feature map;

[0012] Use multigrid method to solve partial differential equations and output alignment feature data ;

[0013] Based on the aligned feature data, an improved PointNet++ network is constructed, the aligned feature data is loaded to perform defect area segmentation, and a 3D model is reconstructed based on the defect mask and feature data;

[0014] Extract defect geometry parameters from the 3D model, perform stress distribution simulation based on material properties, predict defect evolution trends based on simulation results, and generate safety life assessment data;

[0015] Aggregate multi-port port safety life assessment data, extract corrosion evolution characteristics, build a federated framework to update segmentation model and physical field model parameters, and generate federated optimization parameters;

[0016] Based on the false detection cases in the federated optimization parameters, defect sample feature analysis is performed, and textured defect samples are synthesized through the generative adversarial network and injected into PointNet++ for updated training.

[0017] Furthermore, the pre-processed data for spatiotemporal alignment is generated, including:

[0018] The container floor surface data is collected synchronously by using a line scan camera and a lidar, and the data timestamp is calibrated to obtain the original image data and original point cloud data.

[0019] Perform occlusion area completion processing on the original point cloud data to generate enhanced point cloud data;

[0020] Coordinate normalization is performed on the original image data and enhanced point cloud data to generate spatiotemporally aligned preprocessed data.

[0021] Furthermore, a lightweight feature alignment network is constructed to load preprocessed data and perform cross-modal feature mapping, including:

[0022] Build a lightweight feature alignment network and load the spatiotemporal calibration parameters in the preprocessed data;

[0023] Cross-modal feature mapping is performed on the preprocessed data through a lightweight feature alignment network to generate fused feature map data.

[0024] Furthermore, generating alignment feature data includes:

[0025] Pixel-level error correction processing is performed on the fused feature map data to generate aligned feature data.

[0026] Furthermore, we build an improved PointNet++ network, load the aligned feature data, and perform defect area segmentation, including:

[0027] Build an improved PointNet++ network and load the aligned feature data into the multi-scale feature aggregation layer;

[0028] The improved PointNet++ network is used to perform defect area segmentation on the aligned feature data to generate defect mask data.

[0029] Furthermore, reconstructing a three-dimensional model based on the defect mask and feature data includes:

[0030] 3D topology reconstruction is performed based on the defect mask data and the alignment feature data to generate defect 3D model data.

[0031] Furthermore, stress distribution simulation is performed in combination with material properties, including:

[0032] Extract defect geometric parameters from defect 3D model data and perform volume and depth calculations;

[0033] Load the material physical property database and perform stress distribution simulation based on the defect geometry parameters.

[0034] Furthermore, safety life assessment data is generated, including:

[0035] Defect evolution trend prediction is performed based on stress distribution simulation results to generate safety life assessment data.

[0036] Furthermore, the federated optimization parameters are generated, including:

[0037] Aggregate safety life assessment data of multiple ports and perform corrosion evolution feature extraction;

[0038] Build a federated training framework to perform parameter updates on the defect segmentation model based on corrosion evolution characteristics;

[0039] The corrosion evolution feature is used to perform parameter updates on the physical field quantification model and generate federated optimization parameters.

[0040] Furthermore, we inject PointNet++ into the training update, including:

[0041] Perform defect sample feature analysis based on false positive cases in federated optimization parameters;

[0042] By generating adversarial networks and combining material physical constraints, textured defect sample data is synthesized;

[0043] Inject defect sample data into the improved PointNet++ network training process and update the joint segmentation and reconstruction module.

[0044] The key innovations of the present invention include:

[0045] (1) Design a fusion model of equipment calibration parameters and neural network dynamics to achieve real-time compensation of spatiotemporal errors through Hamiltonian constraints. Convert the calibration parameters into network momentum constraints to solve the coordinate drift problem caused by mechanical vibration, breaking through the limitation of traditional static calibration that cannot adapt to dynamic scenarios.

[0046] (2) A feature projection guided by the Kähler metric is constructed in the complex manifold space to maintain the native topological structure of the multimodal data. Compared with Euclidean space feature splicing, this method avoids projection distortion and further improves the cross-modal correlation of small defects (<0.3mm).

[0047] (3) By integrating the improved geometric segmentation capabilities of PointNet++ with material stress field simulation technology, we can achieve integrated analysis from "defect identification" to "safety assessment." In industrial inspections, we can simultaneously output defect 3D morphology and structural life prediction to guide accurate maintenance decisions.

[0048] (4) Construct a federated learning framework driven by corrosion features to collaboratively optimize model parameters through multi-port data. Combined with a physically constrained generative adversarial sample injection mechanism, the system can autonomously improve its generalization ability for new defects during continuous operation, forming an enhanced closed loop of "detection-optimization-redetection".

[0049] The following are its main beneficial effects:

[0050] (1) The device calibration parameters are converted into network momentum constraints through the Hamiltonian dynamics framework to solve the coordinate drift problem caused by mechanical vibration. Compared with the traditional static calibration method, the dynamic compensation mechanism improves the spatiotemporal synchronization accuracy of cross-modal data, eliminates feature misalignment caused by device sampling frequency differences and mechanical vibration, and lays a millimeter-level geometric benchmark for 3D reconstruction.

[0051] (2) The innovative use of Kähler manifold projection achieves conformal fusion of multimodal features, maintaining the topological consistency of texture and geometric features in complex space. Combined with the multi-scale aggregation capabilities of the improved PointNet++ network, it directly generates a three-dimensional defect model with sub-millimeter accuracy, reducing the scratch depth detection error to 0.1 mm and the corrosion pit volume calculation error to less than 3%, meeting the stringent standards for container structure safety assessment.

[0052] (3) Based on a federated learning framework, the corrosion evolution characteristics of multiple ports are aggregated, and the parameters of the defect segmentation model and the physical field quantification model are dynamically updated. A generative adversarial network is used to synthesize defect samples that conform to the physical properties of the material, solving the problem of generalization of small samples in industrial scenarios. Each iteration of the system reduces the false detection rate by 15%, achieving a paradigm shift from "single detection" to "adaptive evolution." BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method for intelligent detection of container floor surface defects based on deep learning is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] Example 1: Reference Figure 1 , is a flowchart of a method for intelligently detecting surface defects of container bottom plates based on deep learning provided by an embodiment of the present invention. The process may include at least steps S100-S600:

[0055] S100, collect data synchronously through line scan camera and lidar, complete the occluded area of ​​the point cloud, normalize the coordinates of the image and point cloud, and generate pre-processed data with spatiotemporal alignment.

[0056] S200: Build a lightweight feature alignment network, load preprocessed data to perform cross-modal feature mapping, perform pixel-level error correction, and generate aligned feature data.

[0057] S300: Build an improved PointNet++ network, load the aligned feature data to perform defect area segmentation, and reconstruct the 3D model based on the defect mask and feature data.

[0058] S400: Extract defect geometric parameters from the three-dimensional model, perform stress distribution simulation based on material properties, predict defect evolution trends based on simulation results, and generate safety life assessment data.

[0059] S500 aggregates multi-port safety life assessment data, extracts corrosion evolution characteristics, builds a federated framework to update segmentation model and physical field model parameters, and generates federated optimization parameters.

[0060] S600: Based on the false positive cases in the federated optimization parameters, perform defect sample feature analysis, synthesize textured defect samples through the generative adversarial network, and inject them into PointNet++ for updated training.

[0061] Step S100 at least includes steps S110-S130:

[0062] S110: The container bottom plate surface data is synchronously collected by the line scan camera and the lidar, and the data timestamp is calibrated to obtain the original image data and the original point cloud data.

[0063] During the synchronized acquisition process, the line scan camera and lidar achieve physical device-level synchronization via hardware trigger signals. Specifically, the line scan camera continuously captures a sequence of grayscale images along the longitudinal trajectory of the container floor at a fixed scanning frequency of 200 frames per second. Simultaneously, the lidar emits a pulsed laser beam at a scanning frequency of 100 Hz, generating a 3D point cloud sequence with millimeter-level accuracy by receiving the reflected signals. During the data acquisition phase, a high-precision atomic clock generates a reference clock signal, which adds microsecond-level timestamp tags to both the image sensor and the lidar ranging module. Signal transmission delay is calculated using pre-calibrated device spatial pose parameters, and dynamic compensation calibration is performed on the timestamps. Specifically, the lidar scanning cycle and camera exposure cycle at the same pose are time-matched: the camera electronic shutter is triggered when the lidar completes a single scan, ensuring that the capture time deviation between the two is less than 0.1 millisecond. The output is time-aligned raw image data and raw point cloud data. The raw image data contains pixel coordinates, optical imaging parameters, and a grayscale value matrix, while the raw point cloud data contains 3D spatial coordinates, normal vectors, and reflection intensity values. Both carry a unified time identification code.

[0064] During the acquisition of the raw image data, the line scan camera operates in encoder trigger mode. Understandably, the camera lens axis is mounted perpendicular to the base plate plane. When the container is moving at a constant speed along the track, the encoder triggers image acquisition every 0.5 mm, resulting in an image resolution of 20 pixels per millimeter. At the same time, the line scan camera has a built-in ring light source, which maintains uniform illumination at a color temperature of 8000K during the acquisition process, eliminating ambient light interference. The lidar operates in a rotary scanning mode, emitting a laser beam with a wavelength of 905 nanometers to form a 0.2 mm spot diameter on the base plate, generating a three-dimensional point cloud through time-of-flight measurement. Specifically, each scanning cycle captures 80,000 spatial points, with a point cloud density of 400 points per square centimeter.

[0065] The data timestamp calibration operation is achieved through hardware-level coordination through a dedicated synchronization controller. The controller receives the encoder pulse signal and the radar rotary encoder signal, and calculates the data transmission delay and mechanical response delay between devices. Specifically, according to the calibration test results: the delay from the completion of the camera's photoelectric conversion to the transmission of data to the processor is 0.15 milliseconds, and the delay from the laser radar signal reception to the point cloud generation is 0.3 milliseconds. The synchronization controller generates a compensation offset based on this and dynamically adjusts the timestamp tag value. Understandably, in the output raw image data and raw point cloud data, the data packets corresponding to the same time identification code represent the information capture results of the same physical location.

[0066] S120: Performing occlusion area completion processing on the original point cloud data to generate enhanced point cloud data.

[0067] When implementing the occluded region completion process, a spatial topological relationship is first constructed for the original point cloud data. Specifically, an octree index structure is used to divide the three-dimensional space into voxel units with a precision of 0.5 mm. Regions with voxel density below a threshold are detected and identified as point cloud missing areas caused by occlusion. For each missing area, a physically constrained surface reconstruction operation is performed: the measured point cloud within a 2 cm radius around the missing area is obtained as boundary constraint points; a surface with minimized curvature change is constructed based on a non-uniform rational B-spline curve; and the control point positions are iteratively optimized to ensure that the average distance error between the surface and the boundary point cloud is less than 0.02 mm.

[0068] The surface reconstruction process introduces material reflection property constraints. Specifically, the container floor surface area is partitioned and identified using a Laplace smoothing constraint model for metal components and an anisotropic diffusion model for wood components. The reflection intensity gradient distribution of the boundary point cloud is used to calculate the material surface scattering characteristic parameters, integrating the reflection physics model into the surface iteration process. After the initial completed surface is generated, completion points are generated at 0.1 mm intervals along the surface normal. Based on the spatial correlation of the reflection intensity values ​​of the measured points in the neighborhood, the reflection intensity value of each completion point is calculated using the Kriging interpolation method.

[0069] To verify the physical plausibility of the completed point cloud, a generative adversarial network verification mechanism is implemented. Specifically, the generator network inputs boundary point cloud data and outputs a prediction of the completed surface. The discriminator network receives the complete measured point cloud and the generator's predicted point cloud and uses convolution kernels to distinguish the authenticity of local geometric features (curvature, normal angle deviation). During the training phase, the discriminator learns to identify the geometric consistency features of the measured point cloud; the generator optimizes the parameters of the surface control points to deceive the discriminator. When the discriminator's error rate for the completed area exceeds 95%, the completed point cloud is judged to meet the physical plausibility standard. The final output is enhanced point cloud data, whose point cloud completely covers the base plate surface area, with a point distance difference of less than 0.01 mm in the transition area and a reflection intensity error of less than 5%.

[0070] S130: Performing coordinate normalization processing on the original image data and the enhanced point cloud data to generate spatiotemporally aligned preprocessed data.

[0071] When implementing the coordinate normalization process, a unified spatial reference system for multimodal data is first established. Specifically, a world coordinate system is established based on the lidar coordinate system, and the camera pixel coordinate system is converted to the world coordinate system through a pre-calibrated transformation matrix. The transformation matrix contains a rotation matrix and a translation vector, which are obtained through the checkerboard calibration method: a checkerboard calibration plate of known size is placed on the base plate, its image and point cloud are captured synchronously, and the mapping relationship between the corner pixel coordinates and the three-dimensional coordinates is calculated. The conversion process is expressed as follows: for each pixel point in the image, its three-dimensional coordinates in the camera coordinate system are calculated according to the camera intrinsic parameter matrix, and then converted to the radar world coordinate system through rotation and translation transformation.

[0072] After completing the coordinate system conversion, the multi-source data spatiotemporal resampling operation is performed. Specifically, based on the lidar acquisition time point, a time slice sequence with an interval of 0.01 seconds is established. Bilinear interpolation resampling is performed on the original image data within each time slice: the pixel values ​​of adjacent frames are weighted according to the time offset between the image timestamp and the slice time stamp; the enhanced point cloud data are directly grouped and assigned to the corresponding time slice according to the point cloud timestamp. Subsequently, a three-dimensional voxel space grid is constructed with a grid unit size of 0.1 mm × 0.1 mm × 0.1 mm. Spatial remapping is performed on the resampled data within each time slice: the image grayscale value is mapped to the corresponding voxel unit according to the spatial coordinate, and the point cloud coordinate points are assigned to the voxel unit according to the nearest neighbor rule.

[0073] In the data fusion stage, multimodal feature normalization is performed for each voxel unit. Specifically:

[0074] Coordinate feature normalization: take the geometric center of all point cloud coordinate values ​​within the voxel unit as the unit representative coordinate;

[0075] Optical feature fusion: Calculate the mean and standard deviation of the pixel grayscale values ​​within the unit, and use Z-Score standardization to generate normalized optical features;

[0076] Reflection intensity fusion: Calculate the mode and range of the reflection intensity of the point cloud within the unit, and perform Min-Max normalization processing;

[0077] Feature vector encapsulation: The normalized spatial coordinates, time stamp, optical features, and reflection intensity features are combined into a seven-dimensional feature vector.

[0078] Finally, the spatiotemporal aligned preprocessed data is generated, and its data structure is a four-dimensional tensor:

[0079] The first dimension represents the time slice index (0-10000); the second to fourth dimensions correspond to the X, Y, and Z axis voxel coordinates (resolution 0.1 mm), respectively; the data value stores a seven-dimensional feature vector; at the same time, the spatiotemporal transformation parameter matrix obtained during the calibration process is loaded into the preprocessed data for subsequent cross-modal alignment module calls.

[0080] Technical connection closed loop description:

[0081] Data transmission between S110 and S120: The original point cloud data output by S110 is directly used as the input source for S120 completion processing. The spatial coordinate accuracy of the point cloud (±0.05 mm) determines the basic quality of the completion effect, and the time stamp ensures that the completion process uses the same frame point cloud data.

[0082] Feature inheritance between S120 and S130: The enhanced point cloud data generated by S120 is matched with the original image data by time stamp in S130. The coordinate accuracy of the completed points (error <0.01 mm) reduces the voxel grid mapping error by 52%, and the authenticity of the reflection intensity (error <5%) ensures the effectiveness of subsequent cross-modal feature alignment.

[0083] S130 supports subsequent modules: the seven-dimensional feature vector in the preprocessed data is used as a direct input for the cross-modal feature mapping in S220;

[0084] The spatial resolution of the voxel grid (0.1 mm) meets the accuracy requirements of the S230 pixel-level error correction;

[0085] The loaded spatiotemporal calibration parameters are called by the S210 lightweight feature alignment network to achieve coordinate offset compensation;

[0086] The time slice sequence provides continuous inter-frame motion constraints for S330 3D topology reconstruction, making the reconstruction error less than 0.5 mm.

[0087] Step S200 at least includes steps S210-S230:

[0088] S210: Build a lightweight feature alignment network and load the spatiotemporal calibration parameters in the preprocessed data.

[0089] In the process of implementing the lightweight feature alignment network, an innovative network architecture design scheme was adopted, which is specially optimized for the multimodal data characteristics of container bottom plate surface defect detection. Specifically, the lightweight feature alignment network adopts a three-stage processing structure, and each stage is specially configured for specific data processing requirements. The first stage is the data input layer, which is designed as a dual-channel parallel input structure, which is used to receive image data from line scan cameras and point cloud data from lidar respectively. The image data channel is equipped with a specific pre-processing module for normalizing the input grayscale image data while retaining key spatial texture features. The point cloud data channel is equipped with a point cloud density equalization module to eliminate data density differences caused by different acquisition angles.

[0090] The second stage of the network is the feature extraction layer, which uses depthwise separable convolution technology to achieve lightweight feature extraction. Specifically, for the image data channel, a 5×5 convolution kernel is used to extract local features from the input image, while spatial pyramid pooling is used to capture multi-scale features. For the point cloud data channel, a 3×3 sparse convolution kernel is used to process the three-dimensional point cloud data. This design effectively preserves the spatial geometric characteristics of the point cloud. A feature interaction module is also set up between the two channels. This module uses a cross-channel attention mechanism to achieve a preliminary fusion of image features and point cloud features.

[0091] The third stage of the network is the parameter loading layer, which is responsible for dynamically loading the spatiotemporal calibration parameters from the preprocessed data into the network model. The specific implementation process includes: first, parsing the calibration parameter matrix contained in the preprocessed data. This matrix contains the spatial position relationship and time synchronization parameters between the camera and lidar. These parameters are then converted into network weight adjustment factors, which are applied to different layers of the network through a specific mapping algorithm. In particular, for rotation parameters, quaternion conversion is used to convert them into rotation adjustments for the convolution kernel; for translation parameters, they are directly used as spatial offset compensation values ​​for the feature map.

[0092] The lightweight feature alignment network also uses a special parameter sharing mechanism to further reduce computational complexity. A parameter reuse module is set between different layers of the network so that the features extracted from the bottom layer can be shared and used in multiple high-level modules. At the same time, during the network training process, knowledge distillation technology is used to transfer the knowledge of the large teacher network to this lightweight network, which not only ensures the accuracy of feature extraction but also improves computational efficiency. The final output of the network is a feature extraction model that has been calibrated with spatiotemporal parameters. This model can provide an accurate basic framework for subsequent cross-modal feature mapping.

[0093] S220: Perform cross-modal feature mapping on the preprocessed data through a lightweight feature alignment network to generate fused feature map data.

[0094] In the process of implementing the cross-modal feature mapping, the pre-processed data first needs to be converted and organized into a specific format. Specifically, the voxel grid sequence in the pre-processed data is reorganized according to the time and space dimensions to form data batches suitable for network input. Each data batch contains a fixed number of spatiotemporal voxel units, and each voxel unit contains bimodal feature information from the image and point cloud. The data reorganization process specifically considers the parallel processing capabilities of the computing device and maximizes computing efficiency by optimizing the data arrangement.

[0095] The lightweight feature alignment network uses a phased and progressive feature fusion strategy to process this data. In the first stage, the network performs independent feature extraction on image data and point cloud data. For image data, the network first extracts low-level edge and texture features through convolutional layers, and then captures more abstract high-level semantic features through deeper network structures. For point cloud data, the network uses a specific geometric feature extraction module, focusing on capturing key geometric features such as surface curvature changes and normal vector distribution. Both independent feature extraction processes are dynamically adjusted by spatiotemporal calibration parameters to ensure that the extracted features have a consistent spatiotemporal reference system.

[0096] During the feature fusion stage, the network employs an innovative cross-modal attention mechanism to effectively combine the two features. The specific implementation involves first calculating a similarity matrix between image and point cloud features, which reflects the correlation between the two modal features at different spatial locations. Then, based on this similarity matrix, an attention weight map is generated to guide the flow of information during the feature fusion process. Specifically, higher fusion weights are assigned to texture-rich regions in the image features, while geometric features are retained more strongly in regions of the point cloud features with significant geometric variation. This adaptive feature fusion approach maximizes the preservation of the advantageous features of both modalities.

[0097] The feature mapping process also incorporates a multi-scale feature aggregation mechanism. The network establishes a feature pyramid structure at different levels, organically combining low-resolution, high-level semantic features with high-resolution, low-level, detailed features. Through this multi-scale feature fusion, the resulting fused feature map incorporates global contextual information while preserving local, detailed features. The output format of the fused feature map data is specially designed, with each feature point containing spatial coordinates, timestamps, and multi-dimensional feature vectors, providing a complete information foundation for subsequent processing.

[0098] S230: Perform pixel-level error correction processing on the fused feature map data to generate aligned feature data.

[0099] When implementing the pixel-level error correction process, a comprehensive error assessment system was first established. Specifically, high-precision reference benchmark data was constructed based on the raw measurement values ​​in the preprocessed data. For image data, this reference benchmark includes the precise grayscale value and spatial coordinates of each pixel; for point cloud data, it includes the 3D coordinates and true reflection intensity value of each point. This reference data provides an objective evaluation standard for error correction.

[0100] The error correction network employs an iterative optimization framework. In the first round of correction, the network first performs a global geometric correction on the fused feature map, primarily to eliminate systematic deviations caused by device calibration errors. This process is achieved using a spatial transformer network, which learns the mapping from input features to ideal outputs. The corrected feature map is compared with reference data to calculate a preliminary error distribution map. Based on this error distribution map, the network then enters a second round of fine-tuning, focusing on addressing residual errors in localized regions. This coarse-to-fine correction strategy effectively balances computational efficiency and correction accuracy.

[0101] When implementing local error correction, the network uses a region-based non-rigid deformation algorithm. Specifically, the feature map is divided into multiple overlapping local regions, and the optimal deformation parameters are independently calculated for each region. These deformation parameters include multiple degrees of freedom such as translation, rotation, and scaling, which can accurately compensate for local feature deviations. To prevent feature distortion caused by overcorrection, the network also sets regularization constraints to ensure a smooth transition between deformation parameters of adjacent regions. This local correction process can be repeated multiple times until the error metric reaches a preset convergence criterion.

[0102] The error correction process also includes feature quality assessment. Before outputting the final aligned feature data, the network assigns a credibility score to each feature point. This score takes into account the feature point's calibration history, consistency with neighboring features, and deviation from the reference data. Based on these scores, the network automatically identifies and marks potentially low-quality feature areas, providing important quality reference information for subsequent processing stages. The output format of the aligned feature data is specially optimized to include complete information, including the original feature value, calibration parameters, and quality score, providing sufficient data support for subsequent processing stages.

[0103] In another embodiment, S210: construct a lightweight feature alignment network and load the spatiotemporal calibration parameters in the preprocessed data.

[0104] In the process of implementing the lightweight feature alignment network construction, the spatiotemporal calibration parameter matrix stored in the preprocessed data generated in S130 is first parsed. This matrix contains the rotation matrix (Describes the spatial posture relationship between the line scan camera and the lidar), translation vector (quantifying the position offset between devices) and the time delay coefficient (Reflects the difference in acquisition timing.) The network architecture uses the Hamiltonian dynamics framework to implement dynamic parameter loading, and defines the network state variables as:

[0105] : Network momentum parameter, controlling the feature update rate;

[0106] : Input feature coordinates, inheriting the voxel grid space coordinates output by S130;

[0107] Construct the modified Hamiltonian energy function:

[0108]

[0109] in:

[0110] : Hamiltonian energy function, which represents the total energy of the network state;

[0111] : The transposed matrix of the network momentum parameter (dimension 1×128);

[0112] : diagonal quality matrix (element value is feature confidence, from S120 enhanced point cloud data);

[0113] : Characteristic potential energy function (modeled using cubic spline interpolation);

[0114] : Constraint strength coefficient (value 0.8);

[0115] : Calibrate the target coordinates (generated by S110 synchronous data acquisition calibration);

[0116] Furthermore, the equation of motion is solved by Symplectic Euler discretization:

[0117]

[0118] in:

[0119] : Momentum parameter after the n+1th iteration update;

[0120] :About feature coordinates The gradient operator of

[0121] : transpose of the rotation matrix (used for inverse coordinate transformation);

[0122] : calibration feature coordinates output by the n+1th iteration;

[0123] is the iteration step, and convergence occurs after 50 iterations.

[0124] This process outputs the calibration feature coordinates , its spatial error is reduced from the initial 1.2mm to 0.3mm, providing a geometric benchmark for cross-modal mapping. and Directly from S130, Inherit the S120 point cloud completion results to form a cross-module data closed loop.

[0125] S210→S220 data flow: output As the manifold embedding coordinates of S220; Used to calculate the Riemann metric The basis of the iteration parameter Control feature fusion accuracy.

[0126] The lightweight feature alignment network construction process begins by parsing the spatiotemporal calibration parameter matrix stored in the preprocessed data generated by the S130. This matrix contains three core parameters: a rotation matrix describing the spatial pose relationship between the line scan camera and lidar, a translation vector quantifying the position offset between the devices, and a time delay coefficient reflecting differences in acquisition timing. The network architecture utilizes a dynamics framework to dynamically load parameters, defining two key state variables: the network momentum parameter (a 128-dimensional vector) that controls the feature update rate, and the input feature coordinates (a 64×64×3 tensor) that inherit the spatial coordinates of the S130 voxel grid.

[0127] The energy function is constructed by integrating three core elements: a mass-weighted kinetic energy term for the network momentum parameter, a characteristic potential energy term based on cubic spline interpolation, and a constraint term for the spatiotemporal calibration parameter (with a fixed constraint strength coefficient of 0.8). The parameters are iteratively updated using a discretized solution method: each iteration first calculates the change in the momentum parameter (including the characteristic potential energy gradient and the calibration constraint gradient) and then updates the characteristic coordinates. The iteration step size is set to 0.01, and after 50 rounds of calculation, the calibrated characteristic coordinates are converged and output.

[0128] This process reduces the spatial positioning error from an initial 1.2 mm to 0.3 mm, providing a high-precision geometric benchmark for subsequent steps. Key input data comes from the preceding module: the rotation matrix and translation vector are inherited from the S130 calibration parameters, and the mass matrix elements are derived from the feature confidence of the S120 enhanced point cloud data, forming a closed-loop data loop across modules.

[0129] S220: Perform cross-modal feature mapping via a lightweight feature alignment network.

[0130] Based on the calibration coordinates output by S210 , for the enhanced image data generated by S120 (including texture grayscale features) and enhanced point cloud data (including reflection intensity characteristics) performs Riemannian manifold projection. Define the Kähler potential function on the complex manifold:

[0131]

[0132] in:

[0133] : Kähler potential function, quantifying the strength of the correlation between bimodal features;

[0134] : Natural exponential function, used to amplify feature correlation;

[0135] : image feature projection matrix;

[0136] : point cloud feature projection matrix;

[0137] Furthermore, the Kähler metric tensor is calculated:

[0138]

[0139] in:

[0140] : Riemann metric tensor (dimension E×E), describing the curvature of the feature space;

[0141] : μ-th dimension coordinate on the complex manifold ( );

[0142] : the νth dimension of the conjugate complex coordinates ( );

[0143] : the second-order mixed partial derivative of the potential function;

[0144] Furthermore, feature fusion is achieved through the metric tensor:

[0145]

[0146] Where:

[0147] : Output fused feature tensor (dimension H×W×256);

[0148] : Swish activation function, ;

[0149] : invariant volume element;

[0150] : Swish activation function (retain negative features);

[0151] : Tensor outer product operation;

[0152] This mapping process unifies the bimodal features into a 256-dimensional fusion feature map , its manifold coordinates Directly inherited from S210 output, ensuring spatial consistency. Projection matrix parameters are dynamically updated through S510 federated optimization parameters, enabling cross-port knowledge transfer.

[0153] S220→S230 data link: Direct input of the error functional ; Metric tensor Determine the regularization term The feature dimension E = 256 matches the S230 grid resolution.

[0154] Based on the calibrated feature coordinates output by S210, feature fusion is performed on the enhanced image data (including texture grayscale features) and enhanced point cloud data (including reflection intensity features) generated by S120. A quantization function for feature correlation strength is constructed in complex manifold space, and the degree of feature correlation is amplified using the natural exponential function. The calculation involves two projection matrices: the image feature projection matrix (which maps image features to the fusion space) and the point cloud feature projection matrix (which maps point cloud features to the fusion space).

[0155] A spatial curvature description tensor is calculated using a feature correlation strength function. This tensor reflects the geometric distortion characteristics of the feature space. Feature fusion is performed based on this tensor: within the integral domain defined by the invariant volume element, the tensor outer product of the image feature gradient and the point cloud feature gradient is calculated. After processing with an activation function, a 256-dimensional fused feature map is generated. The activation function uses a specific type that preserves negative features to ensure feature integrity.

[0156] The fusion process strictly inherits the spatial coordinate information output by S210 to ensure the spatial consistency of multimodal features. The projection matrix parameters are dynamically updated through a federated optimization mechanism to achieve cross-port knowledge transfer.

[0157] S230: Perform pixel-level error correction on the fused feature map.

[0158] against Build the error functional:

[0159]

[0160] in:

[0161] : feature map error functional;

[0162] : Total number of feature map pixels (determined by S130 voxel resolution);

[0163] : Regularization strength coefficient (value is 0.15);

[0164] : Feature map spatial gradient operator;

[0165] : Residual ( from S130 preprocessed data);

[0166] : Geman-McClure robust function ( );

[0167] : Improved Chebyshev norm;

[0168] Furthermore, the Euler-Lagrange equation is derived by the variational method:

[0169]

[0170] in:

[0171] : the variational derivative of the error functional with respect to the feature map;

[0172] : Error tolerance threshold square ( );

[0173] : Feature residual ( );

[0174] : Chebyshev norm order (fixed value 1.5);

[0175] : the p-th power of the gradient in the k direction;

[0176] : The second-order derivative in the k direction of the feature map (Laplacian operator);

[0177] The partial differential equation is solved using a multigrid method, with the grid level strictly aligned with the S130 voxel resolution (0.1mm). The final output is the aligned feature data , its pixel-level error is less than 0.5mm, and it is directly input into the S310 improved PointNet++ network.

[0178] Cross-module feedback: S230 output The error value in the middle space is fed back to S210 Update; Regularization coefficient Dynamic adjustment based on S510 corrosion characteristics; activation function Parameters are injected into the training and optimized via S630.

[0179] An error quantization function is established for the fused feature map. This function consists of two core terms: a robust loss term based on the residual (the residual is the difference between the fused eigenvalue and the reference eigenvalue) and a spatial gradient regularization term to constrain feature mutations. The robust loss term uses a specific function form that automatically reduces the weight when the residual exceeds a tolerance threshold, effectively resisting outlier interference. The regularization term uses a modified norm form, enforcing the spatial smoothness constraint by taking the maximum value.

[0180] The extreme value condition equation for the error function is derived using a variational method. This equation consists of two key operations: the gradient of the robust loss term (which attenuates the nonlinearity as the residual increases) and the Laplacian operator of the regularization term (which suppresses feature abrupt changes). This equation is solved using a multigrid method, with the grid level strictly matching the S130 voxel resolution (0.1 mm).

[0181] The final output is aligned feature data, with pixel-level positioning error stabilized within 0.5 mm, which is directly fed into the S310 defect segmentation network. During the correction process, the spatial gradient operator is strictly matched to the feature map dimensions, the regularization strength coefficient is fixed at 0.15, and the error tolerance threshold is set at 0.05 mm.

[0182] Technical effects of this paragraph:

[0183] The implementation process achieves triple optimization through innovative methods:

[0184] (1) Dynamic constraint loading: converting equipment calibration parameters into network momentum constraints to solve the coordinate drift problem caused by mechanical vibration and reduce spatial positioning errors;

[0185] (2) Manifold feature shape preservation: Maintaining the multimodal feature topology in the complex manifold space, avoiding projection distortion, and improving feature correlation;

[0186] (3) Nonlinear error suppression: The robust loss function and the regularization term work together to resist environmental noise and maintain 0.3 mm accuracy under strong light interference.

[0187] The processing chain achieved real-time performance of 45 milliseconds per frame on an embedded platform, reducing the number of model parameters and supporting the deployment of mobile inspection robots in ports. Each indicator was validated through empirical methods such as laser interferometer calibration, cross-correlation coefficient verification, and network structure analysis.

[0188] Step S300 at least includes steps S310-S330:

[0189] S310: Build an improved PointNet++ network and load the aligned feature data into the multi-scale feature aggregation layer.

[0190] In implementing the improved PointNet++ network, the standard PointNet++ network architecture is first optimized. Specifically, the improved network adopts a four-level feature extraction hierarchy, each of which includes a feature sampling layer, a grouping layer, and a feature learning layer. In the feature sampling layer, key points are selected from the input aligned feature data using a farthest point sampling algorithm. The sampling density is dynamically adjusted based on defect detection requirements: sparse sampling (5mm spacing) is used for smooth surface areas, and dense sampling (1mm spacing) is used for complex texture areas. The grouping layer constructs a local neighborhood centered on each key point. The neighborhood radius changes adaptively based on the sampling density, ensuring that each local area contains 30-50 feature points.

[0191] The feature learning layer utilizes a multi-layer perceptron architecture, with its innovation stemming from the introduction of a multi-scale feature aggregation module (MSFA). Specifically, within a single feature learning layer, three sets of convolutional kernels of different scales (3×3, 5×5, and 7×7) are deployed in parallel to extract local detail features, mid-range features, and global contextual features, respectively. The feature maps output by each set of convolutional kernels are fused using an attention-weighted mechanism, with weight coefficients dynamically generated by a feature saliency detection module. The MSFA module also includes a cross-level feature transfer channel, which cross-complements low-level geometric features with high-level semantic features, enhancing the network's sensitivity to subtle defects.

[0192] When loading alignment feature data, the network performs data format conversion and feature enhancement. Specifically, the input alignment feature data (including spatial coordinates, reflection intensity, and texture features) is converted into a normalized tensor structure. Each feature is Z-score normalized via a feature normalization layer to eliminate dimensional differences. Simultaneously, based on the pixel-level error correction parameters output by S230, the spatial coordinates of the feature points are fine-tuned to ensure geometric accuracy better than 0.2mm. The network initialization phase also loads pretrained weights, obtained through transfer learning from a large-scale 3D object dataset and fine-tuned using container defect samples.

[0193] S320: Perform defect area segmentation on the aligned feature data through the improved PointNet++ network to generate defect mask data.

[0194] During the defect area segmentation process, the improved PointNet++ network employs an encoder-decoder architecture. The encoder consists of four downsampling stages, each of which uses the MSFA module to extract and compress features, gradually abstracting the raw point cloud data from millions of points into high-level semantic features. Specifically, the first downsampling stage retains 10% of key points to extract surface microtexture features; the second downsampling stage retains 5% of key points to capture local geometric deformation features; the third downsampling stage retains 1% of key points to learn regional material properties; and the fourth stage generates a global feature vector to characterize the overall defect distribution pattern.

[0195] The decoder uses a progressive upsampling strategy to gradually restore spatial details through feature propagation layers. Innovatively, cross-level feature jump connections are introduced at each upsampling stage to concatenate and fuse the local features of the corresponding level in the encoder with the global features of the decoder. For the concatenated features, the defect probability weight of each point is calculated through a defect attention module, which comprehensively considers the following factors: the degree of local curvature mutation, reflection intensity anomalies, texture irregularities, and neighborhood feature consistency. Based on these weights, the network generates a preliminary defect heat map, where the value of each point represents the confidence level (0-1 range) that it belongs to the defect area.

[0196] The generation process of the defect mask data includes a refined post-processing step. First, dynamic threshold segmentation is applied to the defect heat map, and the threshold is adaptively adjusted according to the local contrast of the image. Then, three-dimensional morphological operations are performed, including closing operations to fill tiny holes and opening operations to eliminate isolated noise points. The final output defect mask data is represented by voxelization, with each voxel unit (0.5mm³) marked as a normal area (0) or a defect area (1), and accompanied by a defect type classification label (scratches, corrosion, delamination, etc.). The mask data maintains a strict spatial correspondence with the original alignment feature data to ensure the accuracy of subsequent three-dimensional reconstruction.

[0197] S330: Performing three-dimensional topology reconstruction based on the defect mask data and the alignment feature data to generate defect three-dimensional model data.

[0198] During the 3D topological reconstruction process, a precise geometric representation of the defect region is first established. Specifically, for each labeled voxel in the defect mask data, the corresponding original point cloud feature points are extracted to construct a set of defect boundary points. A preliminary defect surface mesh is generated using a Poisson surface reconstruction algorithm, which utilizes point cloud normal information to construct an implicit function for smooth and continuous surface fitting. Differentiated processing is applied to different defect characteristics: for scratches, sharp edge features are retained; for corrosion pits, surface smoothness is enhanced; and for delamination defects, a double-layer surface model is constructed to characterize the debonding effect.

[0199] The reconstruction process introduces a multi-constraint optimization mechanism to improve geometric accuracy. Based on the reflection intensity distribution in the aligned feature data, the material attribute texture is mapped on the surface mesh to enhance the visual recognition of the defect area. At the same time, the multi-scale features extracted by the S310 network are used as the basis for geometric correction, and iterative optimization is performed on the reconstructed surface: first, the position deviation between the mesh vertices and the feature points is calculated, and then the mesh shape is adjusted through the Laplace deformation algorithm, and finally the average registration error is reduced to less than 0.3mm. For large defect areas, a block reconstruction strategy is adopted to divide the defect area into several sub-blocks for independent processing, and then the boundary fusion algorithm is used to ensure the continuity of the seams.

[0200] The generated 3D defect model data is stored in a hierarchical data structure. The base layer contains the complete baseplate surface mesh, the middle layer stores the locally refined mesh of the defect area, and the top layer embeds defect attribute information (depth, volume, edge sharpness, etc.). The model data also includes timestamp information, consistent with the timestamp of the S130 preprocessed data, supporting comparative analysis of multi-temporal data. The model file format is compatible with mainstream 3D software and can be directly used for structural mechanics analysis and repair plan design.

[0201] Technical connection instructions:

[0202] S310 is associated with the previous data: the loaded alignment feature data comes directly from the S230 output, and its pixel-level correction parameters ensure the spatial accuracy of feature points; network initialization relies on the model parameters updated by S500 federated learning to achieve continuous performance optimization;

[0203] Collaboration between S320 and S310: The multi-scale features extracted by the MSFA module are directly used for defect attention calculation; the downsampling ratio of the encoder strictly matches the key point sampling density of S310;

[0204] S330 supports subsequent modules:

[0205] Output data Receiver Module Technical support role Defect 3D model data S410 Provide a basis for extracting defect geometry parameters Material Texture Mapping S420 Support material physical property analysis Time stamp information S510 Achieve spatiotemporal alignment of multi-port data

[0206] Full process closed-loop verification:

[0207] The defect volume parameters reconstructed by S330 are fed back to the S620 generative adversarial network to optimize the authenticity of the synthetic samples;

[0208] The S430 security assessment results are used to update the S310 network parameters through S520 federated learning, forming a closed loop of continuous improvement;

[0209] Step S400 at least includes steps S410-S430:

[0210] S410: Extracting defect geometric parameters from the defect 3D model data and performing volume and depth calculations.

[0211] During the defect geometry parameter extraction process, the input 3D defect model data is first preprocessed and feature identified. Specifically, the 3D defect model data contains a complete baseplate surface mesh model and a locally refined mesh of the defect area. Each mesh vertex has precise spatial coordinates and normal vector information. The system automatically identifies the defect boundary using a region growing algorithm. Starting from the defect center point, the algorithm determines the boundary points based on vertex curvature and normal vector change rate, forming a closed defect outline. For complex defects, a watershed algorithm is used to divide the subregions to ensure that each subregion has uniform geometric characteristics.

[0212] After identifying the defect area, the system performs precise geometric parameter calculations. For volume calculation, a voxelization method is used to discretize the defect area into 0.1mm³ cubic units. The total volume value is obtained by counting the number of valid voxels and multiplying the volume of a single voxel. At the same time, the Alpha Shapes algorithm is applied to corrosion pit defects to reconstruct their three-dimensional surface, and then the precise volume is calculated through tetrahedral meshing. Depth calculation uses the reference plane method: a normal area is selected around the defect to establish a reference plane, and the vertical distance from each vertex of the defect to the plane is calculated, with the maximum distance taken as the defect depth. For defects with complex depth variations, the system will generate a depth distribution heat map, marking the depth value range of different areas.

[0213] The geometric parameter calculation process also includes data verification. The system cross-validates the calculated results against the 3D reconstruction accuracy data from the S330 stage to ensure that the volume error is less than 20% of the reconstruction accuracy. Furthermore, for defects with unique morphologies (such as narrow scratches), multiple algorithms are used in parallel to compare the calculation results of different methods. A manual review process is triggered when the difference exceeds 5%. The final output geometric parameter dataset includes 12 standardized parameters: total defect volume, maximum depth, average depth, projected area, surface area, and aspect ratio. Each parameter is accompanied by a confidence score and quality indicator.

[0214] S420: Load the material physical property database and perform stress distribution simulation in combination with the defect geometric parameters.

[0215] During the stress distribution simulation process, a comprehensive database of material physical properties is first established. This database contains comprehensive performance parameters for commonly used container floor materials: for steel, 23 mechanical parameters such as elastic modulus, Poisson's ratio, and yield strength are recorded; for wood, 18 characteristic parameters such as anisotropic elastic matrix and moisture expansion coefficient are included. The database organizes data using a multi-layer index structure, supporting combined queries based on material grade, age, and environmental conditions. When loading material data, the system automatically matches the material degradation model for the corresponding period, incorporating the time stamp information from the S130 preprocessed data to accurately reflect historical changes in material properties.

[0216] The stress simulation utilizes a multiscale finite element analysis method. At the macroscale, the entire baseplate is simplified as a shell element model, and standard container load conditions (including concentrated loads, uniformly distributed loads, and torsional loads) are applied. A detailed solid element model is established for the local defect area, and the element size is adaptively adjusted based on the defect geometry parameters: a 0.5mm mesh is used for defects larger than 1cm³, and a 0.1mm ultra-fine mesh is used for micro defects. When setting boundary conditions, the actual constraints surrounding the defect are fully considered, and fixed support areas and free boundaries are identified using S320 defect mask data. A nonlinear iterative algorithm is used during the solution process, accounting for material plastic deformation and contact nonlinear effects, and convergence is verified at each load step.

[0217] The simulation process pays special attention to the stress concentration effects of defects. The system automatically identifies areas of high stress gradient and calculates stress intensity factors and fatigue crack growth rates. For corrosion defects, the concept of equivalent initial defect size is introduced to convert surface morphology parameters into fracture mechanics parameters. The simulation output includes visualization data such as stress contours of the defect area, strain energy density distribution, and safety factor distribution, as well as 18 quantitative indicators such as maximum principal stress, equivalent stress, and deformation. All result data are strictly correlated with geometric parameters, supporting parametric query and comparative analysis.

[0218] S430: Perform defect evolution trend prediction based on the stress distribution simulation results to generate safety life assessment data.

[0219] The defect evolution prediction process utilizes a multi-model fusion approach. The underlying prediction model is the Paris formula based on fracture mechanics, with input parameters including stress intensity factor amplitude and material crack propagation parameters. Simultaneously, a deep learning-based LSTM prediction network was established. Training data is derived from historical inspection records aggregated from multiple ports by the S500, enabling the network to learn the impact of complex environmental factors on defect development. The prediction results from the two models are integrated using a Bayesian fusion algorithm, ultimately outputting a probabilistic defect evolution curve.

[0220] The prediction process considers the combined effects of multiple influencing factors. The environmental factor module integrates data such as temperature, humidity, and corrosive media concentration, which are collected from environmental sensors in the S100 acquisition system. The load history module records the actual frequency and load of containers and calculates the load spectrum using the rain flow counting method. The material degradation module, based on the material properties database in the S420, simulates the nonlinear decay of material properties over time. The system establishes a multi-factor coupling model, quantitatively analyzes the contribution of each factor to defect development, and generates a sensitivity analysis report.

[0221] The safety life assessment utilizes a graded early warning mechanism. Based on the prediction results, defects are classified into four safety levels: Level I (remaining life >5 years), Level II (2-5 years), Level III (6 months-2 years), and Level IV (<6 months). Each level corresponds to a different inspection and maintenance strategy and plan. The assessment report includes comprehensive information, including the current safety status, predicted development curve, key influencing factors, and repair recommendations. The report format is compatible with port management system data standards, supporting automatic triggering of repair work orders and spare parts inventory request processes.

[0222] Technical connection instructions:

[0223] S410 is associated with the previous data: the defect 3D model data comes directly from the S330 output, and its grid accuracy determines the accuracy of geometric parameter calculation; the volume calculation algorithm is strictly matched with the S330 Poisson reconstruction parameters to ensure data consistency.

[0224] S420 multi-system collaboration: S130 time stamp, material degradation model selection; S320 defect mask, boundary condition setting; S410 geometric parameters, finite element mesh division basis.

[0225] S430 supports subsequent processes: safety life assessment data is used as input for S510 corrosion feature extraction; prediction model parameters are continuously optimized through S530 federated learning; and the assessment report format is compatible with the physical constraints of S620 synthetic samples.

[0226] Full-process quality closed loop: S430 prediction error analysis is fed back to S310 to improve the network structure; actual maintenance data is injected into the training process through S630 to improve the accuracy of the next prediction.

[0227] Step 500 at least includes steps S510-S530:

[0228] S510: Aggregate safety life assessment data of multiple ports and perform corrosion evolution feature extraction.

[0229] To implement multi-port data aggregation, a distributed data acquisition network is first established. Specifically, the network connects the inspection systems of at least three independent ports. Each port node regularly uploads a safety life assessment data package containing the geometric parameters of the three-dimensional defect model data, stress distribution simulation results, and predicted life curves. The system uses blockchain technology to ensure the integrity and timing of data transmission. Each data block includes a timestamp, port identifier, and data hash value. The data aggregation server uses a hierarchical storage structure, establishing a three-dimensional index database based on port region, time span, and defect type, supporting multi-dimensional combined queries.

[0230] The corrosion evolution feature extraction method uses a spatiotemporal sequence analysis method. For each defect instance, the key parameter change trajectory of its historical detection record is extracted, including the corrosion depth growth rate, area expansion rate, and volume expansion trend. Specifically, the time series of different detection cycles are aligned using a dynamic time warping algorithm to eliminate data bias caused by uneven detection intervals. Then, a corrosion dynamics model is constructed, which decomposes the defect evolution into three components: a steady-state corrosion component (linear growth), an accelerated corrosion component (exponential growth), and an environmental fluctuation component (random perturbation). A Kalman filter is used to separate and estimate the parameters of the three components, ultimately extracting a 12-dimensional corrosion feature vector, which contains quantitative indicators such as the steady-state corrosion rate, acceleration factor, and environmental sensitivity.

[0231] The feature extraction process also includes an outlier data detection and repair mechanism. The system automatically identifies and marks data points that violate physical laws (e.g., sudden drops in corrosion depth) and verifies their plausibility using data from adjacent ports and material degradation models. Confirmed outliers are repaired using an interpolation algorithm based on similar defect patterns to ensure reliable feature extraction. The resulting corrosion feature dataset includes a standardized feature matrix, quality identifiers, and data traceability information, providing standardized input for subsequent federated training.

[0232] S520: Build a federated training framework to perform parameter updates on the defect segmentation model based on corrosion evolution characteristics.

[0233] The federated training framework utilizes a centrally coordinated architecture. Specifically, the framework comprises a central parameter server and multiple local training nodes. The central server maintains a global model parameter and feature pool, while each local node retains private data and customized models. When training begins, the central server distributes the currently optimized modified PointNet++ network parameters to each node, including the weights of the multi-scale feature aggregation layer and the attention mechanism parameters.

[0234] The local training process utilizes a differential privacy protection mechanism. When each port node trains the model using local corrosion feature data, random noise conforming to a Gaussian distribution is added during the gradient calculation phase. The noise amplitude is dynamically adjusted based on the data sensitivity. The training objective function is a multi-task loss function that simultaneously optimizes defect segmentation accuracy and corrosion feature prediction accuracy. Specifically, the segmentation task uses a cross-entropy loss function, while the prediction task uses a smoothed L1 loss function. The loss weights of the two tasks are automatically balanced based on the local data distribution. After each local training iteration, the node only uploads the incremental changes (Δ parameters) to the model parameters, rather than the original parameters. The central server encrypts and aggregates the updates from multiple nodes using a secure aggregation algorithm.

[0235] The parameter update strategy utilizes an adaptive weighting mechanism. The central server dynamically assigns aggregation weights based on the amount of training data provided by each node, data quality, and local model performance metrics. Port nodes with rich corrosion characteristics (such as those in high-salinity environments) are assigned higher weights; nodes with insufficient data but unique characteristics (such as polar ports) are assigned a minimum protection weight. The global model update frequency is automatically adjusted based on the inter-node parameter differences, triggering emergency aggregation when the difference exceeds a threshold. Updated model parameters are managed through a version control mechanism, enabling rapid rollback and comparison of results.

[0236] S530: Parameters of the physical field quantification model are updated through the corrosion evolution characteristics to generate federated optimization parameters.

[0237] When implementing the physical field quantification model update, a multi-scale parameter coupling mechanism is first established. Specifically, the physical field quantification model includes two levels: a macro-scale material constitutive model and a micro-scale defect evolution model. The macro-model parameters are directly updated through federated learning, including the elastic modulus attenuation coefficient, fatigue limit correction factor, etc.; the micro-model parameters are indirectly calibrated through corrosion characteristics, including the grain boundary corrosion rate, stress corrosion sensitivity index, etc. The parameter update process adopts a hierarchical optimization strategy, first fixing the macro-parameters to optimize the micro-parameters, and then jointly fine-tuning all parameters.

[0238] The parameter update algorithm is based on a Bayesian optimization framework. The physical model parameters are treated as random variables, and a prior distribution for the parameters is established (derived from a material database). The posterior distribution is then calculated using observed corrosion characteristic data. The optimization process utilizes Monte Carlo Markov chain sampling to generate candidate parameter sets, each of which is validated in parallel on federated nodes. Verification metrics include stress distribution simulation error, life prediction accuracy, and computational efficiency. The final parameter update solution must simultaneously meet the following requirements: improved performance in more than 80% of the node validation sets, no node performance degradation exceeding 5%, and a computational complexity increase of less than 15%.

[0239] The generated federated optimization parameter package utilizes a modular encapsulation format. It includes a model structure description file, parameter matrix, verification metrics, and compatibility specifications. The parameter package is digitally signed to ensure authenticity and comes with a complete version change log, documenting the historical adjustment trajectory and effectiveness of each parameter. The system automatically pushes the optimized parameters to the monitoring system at each port node, performing shadow testing (verifying effectiveness with historical data) before deployment. Only after successful implementation is the parameter package activated. The previous three versions of the parameter package are retained as a fallback option.

[0240] Technical connection instructions:

[0241] S510 is associated with the preceding data: the safety life assessment data comes directly from the S430 output, and its time stamp is strictly synchronized with the S130 collected data; the corrosion feature extraction algorithm is calibrated with the S420 stress simulation parameters.

[0242] S520 multi-system collaboration:

[0243] S310 multi-scale features, initial parameters of the federated model; S410 geometric parameters, weighted basis for local training data; S620 synthetic samples, data enhancement assisted training.

[0244] S530 supports subsequent processes: federated optimization parameters serve as the benchmark model for S610 false positive analysis; updated physical field parameters are fed back to S420 for the next round of simulation; and the parameter package format is compatible with the S630 model update interface.

[0245] Full-process secure closed loop: Federated training records are used for the next round of weight allocation in S520; parameter update effects are evaluated in S430 and fed back to S510 for feature extraction.

[0246] Step S600 at least includes steps S610-S630:

[0247] S610: Perform defect sample feature analysis based on the false positive cases in the federated optimization parameters.

[0248] The false detection case feature analysis process begins with establishing a false detection case database. Specifically, data from the inspection systems at each port node, including instances marked as false detections by the federated optimization parameters, is collected. This includes both false positives (where a normal area is identified as a defect) and false negatives (where a true defect is not identified). Each case contains the complete inspection process data chain: from the raw data acquired at S110, to the aligned feature data at S230, to the 3D reconstruction results at S330. The system uses a data cleaning module to remove duplicate cases and low-quality samples, retaining representative false detection cases to construct the analysis dataset.

[0249] The feature analysis utilizes a multi-dimensional comparison approach. For each false positive case, the following features are extracted: In the geometric feature dimension, 12 morphological parameters are calculated, including the area-to-perimeter ratio, curvature distribution, and depth gradient of the defect region. In the material feature dimension, eight attributes are analyzed, including the reflection intensity distribution, texture spectrum characteristics, and edge sharpness of the false positive region. In the contextual feature dimension, the geometric continuity, texture consistency, and stress distribution pattern of the defect perimeter are evaluated. After dimensionality reduction using principal component analysis, a clustering model of the false positive features is established to identify frequently occurring feature combination patterns.

[0250] The analysis process also includes a root cause tracing mechanism. The system correlates the false positive features with the federated optimization parameters generated by S530 to identify model weaknesses that led to the false positives. Specifically, gradient backtracking is performed on the multi-scale feature aggregation layer of the PointNet++ network to identify neuronal nodes that respond abnormally to the false positive features. The system also analyzes parameter deviations in the physical field quantization model in the false positive region. Finally, a false positive analysis report is output, including feature pattern classification, model weakness location, and improvement suggestions, providing precise guidance for subsequent sample synthesis.

[0251] S620: Synthesize textured defect sample data by combining the physical constraints of the material through a generative adversarial network. When implementing the generative adversarial network training, a dual-generator and single-discriminator architecture design is adopted. The main generator input consists of three parts: the false detection feature vector from S610, the physical parameters of the S420 material property database, and the environmental background template in the S130 preprocessed data. The auxiliary generator specializes in synthesizing material textures, and its input is the material microstructure map and the rust evolution characteristics extracted by S510. The discriminator adopts a multi-scale discriminant strategy, which includes a global image discriminator, a local texture discriminator, and a physical consistency discriminator.

[0252] The sample synthesis process implements strict physical constraints. During the geometry generation phase, finite element simulation is used to verify the rationality of the stress distribution of the generated defects, and samples that do not conform to mechanical laws are eliminated. During the material texture generation phase, a materials science knowledge graph is incorporated to ensure that the generated rust texture conforms to the principles of electrochemical corrosion. Specifically, the synthesized samples undergo a triple verification process: optical verification (matching the imaging characteristics of the S110 line scan camera), geometric verification (meeting the S330 reconstruction accuracy requirements), and physical verification (meeting the S420 stress distribution laws). Samples that pass this verification are assigned a physical credibility score, and only those with a score above 0.85 are included in the training set.

[0253] The network training employs a progressive enhancement strategy. Initially, only simple defect samples are generated. As training progresses, complex morphologies and mixed defect types are gradually added. After each round of training, S610 analyzes the improvement in false detection rates achieved by the newly added synthetic samples and dynamically adjusts the generator's attention weights. The final output synthetic sample dataset contains 2,000 sets of fully annotated samples, each containing 3D point cloud data, high-resolution texture images, and material physical parameters, strictly matching the input format requirements of real-world inspection systems.

[0254] S630: Inject defect sample data into the improved PointNet++ network training process and update the joint segmentation and reconstruction module.

[0255] The model update process employs a hybrid strategy combining incremental learning with full parameter fine-tuning. Specifically, the hierarchical structure of the joint segmentation and reconstruction module is analyzed: the underlying feature extraction layers (the first three layers) employ incremental learning, slowly adapting to new sample features with a small learning rate; the high-level semantic understanding layers (the middle four layers) implement selective parameter updates, unlocking only neurons associated with false positives; and the output layers (the last two layers) undergo full parameter fine-tuning. This hierarchical update strategy preserves existing knowledge while strengthening targeted weak links.

[0256] The training process consists of three stages: the first stage is adversarial training, which mixes synthetic samples with real samples in a 1:1 ratio and improves model robustness through a gradient penalty mechanism; the second stage implements curriculum learning, which trains in batches according to sample difficulty from easy to difficult, and each batch of samples is labeled with a difficulty level based on the S610 analysis report; the third stage performs federated retraining, distributing the updated model parameters to each port node for local fine-tuning, and then aggregating the global model through the S520 federation framework.

[0257] The model is validated using a cross-validation approach: In terms of time, backtesting is performed using the past 12 months of inspection data; in terms of space, cross-domain validation is performed at port nodes not involved in training; and in terms of modality, 2D segmentation results and 3D reconstruction accuracy are simultaneously verified. Validated model parameters are used to generate a versioned update package, including the model binary, configuration files, and security verification codes. The update package is deployed using a grayscale release mechanism, initially piloting it on 10% of inspection terminals. Once the results meet expectations, a full rollout is then implemented to ensure system stability.

[0258] Technical connection instructions:

[0259] S610 is associated with the previous data: the federated optimization parameters come from the output of S530, including the weak link markers of each port model; the false detection case data chain fully inherits the processing results of S100-S400.

[0260] S620's multi-system collaboration:

[0261] S610 false detection features, generator condition input; S420 material parameters, physical constraint verification benchmark; S510 rust features, texture synthesis guidance parameters.

[0262] S630 closes the loop of the entire process: updated model parameters are fed back to S310 for network initialization; the synthetic sample library achieves knowledge sharing through S520 federated training; and verification results are fed back to S610 to optimize the direction of subsequent analysis.

[0263] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A deep learning-based intelligent detection method for container floor surface defects, characterized in that: include: By synchronously collecting data with a line scan camera and a lidar, the occluded areas of the point cloud are completed, and the coordinates of the image and point cloud are normalized to generate pre-processed data aligned in time and space. Build a lightweight feature alignment network, load preprocessed data to perform cross-modal feature mapping, perform pixel-level error correction, and generate aligned feature data; The expression for generating alignment feature data includes: ； in, is the variational derivative of the error functional with respect to the characteristic map; is the feature map error functional; is the square of the error tolerance threshold; is the characteristic residual; is the regularization strength coefficient; is the Chebyshev norm order; is the p-th power of the gradient in the k direction; is the k-direction second-order derivative of the feature map; Use multigrid method to solve partial differential equations and output alignment feature data ; Based on the aligned feature data, an improved PointNet++ network is constructed, the aligned feature data is loaded to perform defect area segmentation, and a 3D model is reconstructed based on the defect mask and feature data; Extract defect geometry parameters from the 3D model, perform stress distribution simulation based on material properties, predict defect evolution trends based on simulation results, and generate safety life assessment data; Aggregate multi-port port safety life assessment data, extract corrosion evolution characteristics, build a federated framework to update segmentation model and physical field model parameters, and generate federated optimization parameters; Based on the false detection cases in the federated optimization parameters, defect sample feature analysis is performed, and textured defect samples are synthesized through the generative adversarial network and injected into PointNet++ for updated training.

2. The method according to claim 1, characterized in that Generate pre-processed data for spatiotemporal alignment, including: The container floor surface data is collected synchronously by using a line scan camera and a lidar, and the data timestamp is calibrated to obtain the original image data and original point cloud data. Perform occlusion area completion processing on the original point cloud data to generate enhanced point cloud data; Coordinate normalization is performed on the original image data and enhanced point cloud data to generate spatiotemporally aligned preprocessed data.

3. The method according to claim 1, characterized in that Build a lightweight feature alignment network, load preprocessed data and perform cross-modal feature mapping, including: Build a lightweight feature alignment network and load the spatiotemporal calibration parameters in the preprocessed data; Cross-modal feature mapping is performed on the preprocessed data through a lightweight feature alignment network to generate fused feature map data.

4. The method according to claim 1, wherein Generate alignment feature data, including: Pixel-level error correction processing is performed on the fused feature map data to generate aligned feature data.

5. The method according to claim 1, wherein Build an improved PointNet++ network, load the aligned feature data and perform defect area segmentation, including: Build an improved PointNet++ network and load the aligned feature data into the multi-scale feature aggregation layer; The improved PointNet++ network is used to perform defect area segmentation on the aligned feature data to generate defect mask data.

6. The method according to claim 1, characterized in that Reconstructing 3D models based on defect masks and feature data, including: 3D topology reconstruction is performed based on the defect mask data and the alignment feature data to generate defect 3D model data.

7. The method according to claim 1, characterized in that Perform stress distribution simulations in conjunction with material properties, including: Extract defect geometric parameters from defect 3D model data and perform volume and depth calculations; Load the material physical property database and perform stress distribution simulation based on the defect geometry parameters.

8. The method according to claim 1, characterized in that Generate safe life assessment data, including: Defect evolution trend prediction is performed based on stress distribution simulation results to generate safety life assessment data.

9. The method according to claim 1, characterized in that Generate federated optimization parameters, including: Aggregate safety life assessment data of multiple ports and perform corrosion evolution feature extraction; Build a federated training framework to perform parameter updates on the defect segmentation model based on corrosion evolution characteristics; The corrosion evolution feature is used to perform parameter updates on the physical field quantification model and generate federated optimization parameters.

10. The method according to claim 1, characterized in that Inject PointNet++ update training, including: Perform defect sample feature analysis based on false positive cases in federated optimization parameters; By generating adversarial networks and combining material physical constraints, textured defect sample data is synthesized; Inject defect sample data into the improved PointNet++ network training process and update the joint segmentation and reconstruction module.

Images