A method, device and equipment for acquiring a three-dimensional model of a heavy-load railway train

Abstract

The application discloses a kind of heavy haul railway train three-dimensional model acquisition method, device and equipment, wherein the method includes: collecting the video data of the body of heavy haul railway train and pre-processing;Using ORB algorithm to the image data after pre-processing Feature extraction and matching, using the double filtering mechanism of Lowe's ratio test and RANSAC combination to filter feature points;From the feature points after filtering, the three-dimensional geometric structure of scene is recovered;Using inverse distance weighted interpolation method for dense reconstruction;To the point cloud data after dense reconstruction, time consistency correction and point cloud consistency correction are carried out, and generate three-dimensional point cloud model based on correction result;The generated three-dimensional point cloud model is fused with preset reference CAD model, and the target three-dimensional model is output.The application solves the technical problems of low feature extraction accuracy, deformation detail loss and error accumulation in the prior art, and improves the accuracy of heavy haul railway train three-dimensional modeling.

Description

Technical Field

[0001] This invention relates to the field of intelligent maintenance technology for railway freight cars, and more specifically, to a method, apparatus, and equipment for obtaining a three-dimensional model of a heavy-haul railway train. Background Technology

[0002] With the rapid development of heavy-haul railway transportation, freight cars are becoming longer, heavier, and more frequently used, leading to more prominent and frequent fatigue and surface damage to the car body structure (such as bulging of side walls, pitting of the floor, and cracking of welds). Establishing a 3D model of a heavy-haul railway maintenance freight car is a crucial step before unmanned painting operations and a key element in the path planning of the painting robot.

[0003] Currently, the painting method for heavy-haul railway maintenance freight cars is mainly manual, using spray guns. In industries such as military, automobiles, and high-speed rail, unmanned painting operations are mainly used during the new manufacturing process, which can be modeled using CAD drawings and other data from the production process. However, heavy-haul railway maintenance freight cars have been working under natural environment and high load conditions for more than 8 years (taking the overhaul cycle of open wagons as an example). The car body has more serious and obvious dents, bumps, deformations, etc. The 3D model during the new manufacturing process differs greatly from the actual condition of the freight car. If the painting robot path planning is still based on the 3D model during the new manufacturing process, it is easy to cause a series of problems such as robotic arm collisions and unstable painting quality.

[0004] Although existing technologies include 3D reconstruction processes, their goal is usually to fully restore a static, regular, or textured object. Their technical chains are often standardized and do not adequately consider the robustness to image noise, object deformation, and dynamic scenes. Therefore, they are not suitable for the 3D reconstruction of heavy-haul railway trains. Summary of the Invention

[0005] This invention proposes a method, apparatus, and equipment for obtaining a three-dimensional model of a heavy-haul railway train, which solves the technical problem in the prior art that the inability to accurately reconstruct a three-dimensional model of a heavy-haul railway train is due to insufficient consideration of image noise, object deformation, and dynamic scenes.

[0006] Specifically, the first aspect of the present invention provides a method for obtaining a three-dimensional model of a heavy-haul railway train, comprising:

[0007] Collect video data of the train body of a heavy-haul railway and preprocess the images in the collected video data.

[0008] The ORB algorithm is used to extract and match features from the preprocessed image data. A dual filtering mechanism combining Lowe's ratio test and RANSAC is used to filter the feature points to obtain the filtered feature points.

[0009] The three-dimensional geometry of the scene is recovered from the filtered feature points, and the three-dimensional coordinates of the key feature points are obtained. The key feature points are the feature points of the key deformation areas of the vehicle body.

[0010] Dense reconstruction was performed using inverse distance-weighted interpolation.

[0011] Temporal consistency correction is performed on the densely reconstructed point cloud data, and a three-dimensional point cloud model is generated based on the correction results. Temporal consistency correction is used to ensure the consistency of point cloud data in the time dimension.

[0012] The generated 3D point cloud model is fused with a preset baseline CAD model to output the target 3D model.

[0013] In one embodiment, collecting video data of the train body of a heavy-haul railway train includes collecting video data of the train body through multiple sets of cameras in the painting workshop, wherein the internal and external parameters of the cameras are calibrated before collecting the video data of the train body.

[0014] In one implementation, a dual filtering mechanism combining Lowe's ratio test and RANSAC is used to filter feature points, resulting in filtered feature points, including:

[0015] For each feature point, calculate the ratio of its distance to its nearest and second nearest neighbors in the target image;

[0016] Determine if the distance ratio is less than a preset threshold. If it is less, retain the corresponding feature point; otherwise, filter the corresponding feature point.

[0017] We use RANSAC to perform bidirectional consistency verification on the retained feature points and filter out points that fail the verification.

[0018] In one implementation, after recovering the three-dimensional geometry of the scene from the filtered feature points and obtaining the three-dimensional coordinates of the key feature points, the method further includes:

[0019] The 3D coordinates of the acquired key feature points are optimized using point cloud. During the optimization process, a weighted optimization strategy based on the consistency of the spatial distribution of the point cloud is introduced. This strategy includes assigning high optimization weights to feature points that meet the preset stability requirements and are located at the structural edge of the vehicle body in multiple consecutive frames.

[0020] In one implementation, dense reconstruction is performed using inverse distance-weighted interpolation, including:

[0021] The feature points are sampled, and the sampled points are used as a fixed search radius. The points within the radius are taken as the K nearest neighbors.

[0022] For each sampled feature point, the weights of the neighboring points are calculated based on the reciprocal of the distances between the neighboring points and that point.

[0023] The position of the new point is calculated based on the positions and weights of the neighboring points.

[0024] The newly calculated points are added to the point cloud composed of key feature points to obtain dense point cloud data;

[0025] Optimize dense point cloud data.

[0026] In one implementation, temporal consistency correction is performed on the densely reconstructed point cloud data, including:

[0027] Temporal consistency correction is performed on the densely reconstructed point cloud data, including: synchronizing the timestamps of all point cloud data; registering the point clouds in consecutive time frames; correcting the timestamps of each frame of data; calculating the rigid body transformation matrix between adjacent frames based on the filtered feature points obtained by the ORB algorithm and performing fine registration using the iterative nearest point algorithm; smoothing the point clouds in consecutive time frames using a filter; and checking whether the point clouds in consecutive frames are consistent.

[0028] In one implementation, the generated 3D point cloud model is fused with a preset baseline CAD model, including:

[0029] A strategy combining feature-based coarse registration and ICP fine registration is adopted to align the reconstructed 3D point cloud model with a preset benchmark CAD model;

[0030] By calculating the signed distance function from the point cloud in the 3D point cloud model to the surface of the preset benchmark CAD model, the deformation area of ​​the vehicle body surface is identified and quantified, and the target 3D model is output.

[0031] Based on the same inventive concept, a second aspect of the present invention provides a device for acquiring a three-dimensional model of a heavy-haul railway train, comprising:

[0032] The data acquisition module is used to collect video data of the train body of heavy-haul railway trains and to preprocess the images in the collected video data.

[0033] The feature extraction and matching module is used to extract and match features from the preprocessed image data using the ORB algorithm, and to filter feature points using a dual filtering mechanism combining Lowe's ratio test and RANSAC to obtain filtered feature points.

[0034] The sparse reconstruction module is used to recover the three-dimensional geometry of the scene from the filtered feature points and obtain the three-dimensional coordinates of key feature points, where the key feature points are feature points in the key deformation areas of the vehicle body.

[0035] The dense reconstruction module is used to perform dense reconstruction using the inverse distance weighted interpolation method.

[0036] The point cloud correction and model generation module is used to perform time consistency correction on the densely reconstructed point cloud data and generate a three-dimensional point cloud model based on the correction results. The time consistency correction is used to ensure the consistency of the point cloud data in the time dimension.

[0037] The output module is used to fuse the generated 3D point cloud model with the preset baseline CAD model and output the target 3D model.

[0038] Based on the same inventive concept, a third aspect of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the method for obtaining a three-dimensional model of a heavy-haul railway train as described in the first aspect.

[0039] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device having a computer program stored thereon, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for obtaining a three-dimensional model of a heavy-haul railway train as described in the first aspect.

[0040] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0041] This invention discloses a method for obtaining a 3D model of a heavy-haul railway train. In the feature extraction and matching stage, the ORB algorithm is used to extract and match features from preprocessed image data. A dual filtering mechanism combining Lowe's ratio test and RANSAC is used to double-filter feature points. This allows for the stable and efficient extraction of reliable feature points representing the true geometric deformation of the train body from images filled with noise interference such as rust and oil stains, rather than simply pursuing a large number of features. In the sparse reconstruction and point cloud optimization stage, this invention focuses on restoring the non-standard 3D structure caused by train body deformation, ensuring the spatial accuracy of the sparse point cloud, and laying a solid geometric foundation for subsequent steps. Furthermore, in the dense reconstruction stage, existing algorithms that use computationally intensive and hardware-intensive global optimization or Poisson reconstruction to generate visually smooth surfaces can blur or smooth out the critical deformation details of the freight cars being repaired. In contrast, this invention innovatively employs inverse distance weighted interpolation for dense reconstruction. The local characteristics of this method allow it to faithfully preserve and enhance local morphological features such as pits and protrusions, avoiding excessive smoothing, thereby generating a detailed 3D model that truly reflects the vehicle's condition. Furthermore, temporal consistency correction is applied to the densely reconstructed point cloud data to ensure consistency in the time dimension, as well as the consistency and temporal continuity of the geometric structure. The corrected point clouds are then fused to generate a 3D point cloud model. Finally, the generated 3D point cloud model is fused with a preset benchmark CAD model to output the target 3D model. This invention solves the technical problems of inaccurate feature extraction, loss of deformation details, and accumulation of dynamic errors inherent in traditional 3D reconstruction methods, thus improving the accuracy of 3D modeling. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating the method for obtaining a three-dimensional model of a heavy-haul railway train in an embodiment of the present invention.

[0044] Figure 2 This is a top view of a heavy-haul railway painting workshop in an embodiment of the present invention;

[0045] Figure 3 This is a block diagram of the device for acquiring a three-dimensional model of a heavy-haul railway train in an embodiment of the present invention. Detailed Implementation

[0046] This embodiment provides a method for obtaining a 3D model of a heavy-haul railway train. Please refer to [link to relevant documentation]. Figure 1 ,include:

[0047] S1: Collect video data of the train body of heavy-haul railway and preprocess the images in the collected video data.

[0048] In practice, cameras or other image acquisition devices can be used to collect vehicle video data.

[0049] In one implementation, collecting video data of the train body of a heavy-haul railway train includes collecting video data of the train body through multiple sets of cameras in the painting workshop.

[0050] The method further includes, prior to collecting vehicle body video data via multiple cameras in the painting workshop:

[0051] The internal and external parameters of the camera are calibrated.

[0052] Specifically, please see Figure 2 This is a top-down view of the painting workshop, where 1 represents a heavy-haul railway maintenance freight car, and 2-5 represent cameras 1-4. The four cameras located at the four corners of the painting workshop require periodic calibration. The purpose of this calibration is to determine the cameras' internal parameters (such as focal length and focus position) and external parameters (such as the camera's position and orientation relative to the world coordinate system), and to correct image distortion and reduce geometric distortion. This network of cameras is responsible for capturing multi-directional shape features of the heavy-haul railway maintenance freight car as it enters the painting workshop.

[0053] The acquired video data is then preprocessed, including grayscale conversion (converting color images to grayscale images, retaining only brightness information, and reducing computational complexity), color space conversion (converting from RGB space to HSV space to adapt to the spray painting workshop environment), image scaling, histogram equalization (adjusting the grayscale distribution of the image to make the image clearer), image denoising (using Gaussian filtering, median filtering, etc. to eliminate random noise in the image and smooth the image), edge detection (using edge detection operators to highlight the boundaries and contours in the image), geometric transformation (adjusting the geometric shape of the image through perspective transformation to make the image conform to the expected viewing angle), and normalization (mapping pixel values ​​to a standard range of [-1,1] to unify the pixel intensity range of the image).

[0054] S2: The ORB algorithm is used to extract and match features from the preprocessed image data. A dual filtering mechanism combining Lowe's ratio test and RANSAC is used to filter the feature points to obtain the filtered feature points.

[0055] Specifically, the feature point extraction method of this invention uses the ORB algorithm to determine the position of object feature anchor points between consecutive frames, ensuring the accuracy of motion modeling.

[0056] The advantages of this algorithm include, but are not limited to: (1) High speed: The ORB algorithm runs fast and is suitable for real-time applications. (2) Low memory usage: The ORB algorithm has relatively low memory usage, which allows it to run on resource-constrained devices. (3) Rotation invariance: By calculating the orientation information of the features, ORB provides rotation invariance, which means that even if the image is rotated, the features can be correctly matched. (4) Scale invariance: The ORB algorithm can detect feature points at different scales, so it has good scale invariance. (5) Strong robustness: The ORB algorithm has good robustness to changes in illumination, viewpoint, and image noise, which makes it work stably in various environments. For the rapid dynamic modeling requirements of the painting workshop, the ORB algorithm can ensure fast and accurate extraction of feature point information on ordinary equipment. This algorithm includes the FAST corner detector and the BRIEF descriptor, and adds orientation information to make the features more discriminative. Among them, FAST is a fast method for detecting corner points. Its basic idea is to check whether several pixels around a pixel are brighter or darker than the center pixel. If the condition is met, the pixel is considered to be a corner point, i.e., a feature point. BRIEF is a binary descriptor that generates feature descriptions by comparing grayscale values ​​between pixel pairs.

[0057] The purpose of feature point matching is to find corresponding feature points in consecutive frames. Specifically, it compares the similarity of BRIEF descriptors in different images using Hamming distance (a metric for the difference between two strings of equal length). To improve the reliability of the matching, Lowe's distance ratio test can be used, expressed by the following formula:

[0058]

[0059] in The distance between feature point A and its nearest feature point B. Let r be the distance between feature point A and its closest feature point C. If this ratio r is less than a preset threshold, then A and B are considered to be a set of matching feature points; otherwise, A is considered not to have found a reliable match.

[0060] Next, the RANSAC (Random Sampling Consensus) algorithm is used to filter out outliers to obtain a more accurate model estimate. The principle is as follows: (1) Randomly select several points in the data and set them as an inlier group; (2) Calculate the model fitting the inlier group; (3) Substitute the other unselected points into the established model and calculate whether they belong to the inlier group; (4) Record the number of inlier groups; (5) Repeat the above steps; (6) Compare which calculation has the most inlier groups. The model built in the calculation with the most inlier groups is the required solution. The formula for calculating the number of iterations is as follows:

[0061]

[0062] Where ω = i / g, i is the number of true interior points (the number of truly correct matching feature points that satisfy geometric constraints), g is the total number of data points, n is the number of randomly selected points, i.e., the number of all feature point pairs to be screened after initial matching by algorithms such as ORB, k represents the number of times the algorithm is repeated, and p is the probability of success after the algorithm runs k times. When ω is a constant, a smaller value for n will require a smaller value for k, which can further improve the efficiency of feature point matching. Furthermore, this invention uses SFM (Structure from Motion) to estimate the camera pose of the matched feature points, obtaining the coordinates of the camera center point in the world coordinate system.

[0063] Compared with existing technologies, the features extraction and matching of this invention are particularly noteworthy as follows:

[0064] Although ORB is a mature algorithm, this invention applies it strategically to the scenario of heavy-haul railway freight cars undergoing maintenance, where surface texture features are severely degraded and local deformations are significant. This optimization is mainly reflected in the following aspects: ① Different objectives: General methods pursue a large number of feature points to restore texture, while this invention aims to extract edge and corner features that represent the true geometric deformation of the car body from visual noise such as rust and oil stains. ② Adaptive parameter adjustment: The scale pyramid parameters of the ORB algorithm are adjusted to address the large-scale characteristics of freight cars, ensuring stable detection capabilities for local deformations of the car body from both near and far perspectives. ③ Enhanced matching strategy: Combining Lowe's ratio test and RANSAC's dual filtering mechanism, high-precision matching point pairs can still be obtained in low-texture, high-noise images, laying the foundation for subsequent restoration of non-standard 3D structures.

[0065] Specifically, regarding the adjustment of the scale pyramid parameters in the ORB algorithm, this application refers to adjusting the scale factor and pyramid levels to simulate the effect of observing objects from near to far. Existing ORB algorithms are generally suitable for general objects (typically a few meters in size with rich textures), while heavy-duty railway freight cars are giant structures ranging from ten to twenty meters in length. Through extensive research and practice, the inventors of this application have discovered that when the freight car is very close, the camera may only capture a part of the car body (such as a wheel), requiring the bottom layer of the pyramid (high resolution) to capture details of local depressions; when the freight car is far away, the top layer of the pyramid (low resolution) is needed to capture the overall outline of the car body. The default pyramid levels and scale factor may not be able to effectively cover such extreme scale variations simultaneously.

[0066] Therefore, to address the problem that existing methods may lose macroscopic structural features in the distant view or fail to generate sufficiently fine layers to detect minute deformations in the close view, this invention extends and optimizes the scale pyramid to adapt to the large-scale characteristics of trucks, as follows:

[0067] Increase pyramid levels: Increase the number of pyramid levels from the default 8 to 12 or more. This ensures that there are image layers of appropriate scales for stable feature extraction across the entire observation range, from very close (capturing welds, small pits) to very far (capturing overall bulging of sidewalls).

[0068] Optimize the scale factor: Use a smaller scale factor (e.g., adjust from 1.2 to 1.1). A smaller scale factor means smoother and denser scale changes between pyramid layers. This ensures that when the vehicle moves in the field of view, causing continuous scale changes, feature points can be stably tracked and matched between adjacent pyramid layers, avoiding feature loss due to excessive scale jumps.

[0069] The above adjustments ensure that the ORB algorithm can detect feature points representing the geometric deformation of the truck at an appropriate scale, regardless of the truck's location in the painting workshop, thus providing a foundation for achieving consistent high-precision reconstruction of the entire vehicle.

[0070] Regarding the dual filtering mechanism combining Lowe's ratio test and RANSAC.

[0071] 1) Typical practices in existing technologies:

[0072] Lowe's ratio test: A relatively lenient threshold (such as 0.7 or 0.8) is typically used. This value works well in general scenarios with rich textures, filtering out most obvious errors while ensuring a certain number of matches.

[0073] RANSAC verification is typically unidirectional. It starts with feature points in image A, searches for matches in image B, and then uses RANSAC to fit a geometric model (such as a fundamental matrix or homography matrix) to filter out outliers.

[0074] 2) This invention:

[0075] a) A more rigorous Lowe's ratio test:

[0076] Specific technical solution: In view of the characteristics of low texture and high noise (rust, oil stains) on the surface of heavy-duty trucks, the threshold of Lowe's ratio test is tightened from the general 0.7 / 0.8 to 0.5 or 0.6.

[0077] Mechanism and Difference: A stricter threshold means that a feature point is only accepted if its similarity to its nearest neighbor is significantly higher than its similarity to its second nearest neighbor. In low-texture regions, the descriptors of many feature points are inherently ambiguous, easily leading to multiple similar false matches. Tightening the threshold is extremely effective at filtering out these ambiguous matches generated in the noise, retaining only the most discriminative and reliable matching pairs.

[0078] Results: Sacrificing the number of matches results in a qualitative improvement in matching quality, providing extremely clean and reliable input data for subsequent geometric reconstruction.

[0079] b) Bidirectional RANSAC verification:

[0080] The specific procedure is as follows: After completing the image matching and RANSAC filtering from image A to image B, the process does not terminate. Instead, it performs a reverse matching and RANSAC filtering from image B to image A.

[0081] Mechanism and Differences: General one-way RANSAC can only guarantee that the matching from A to B is consistent, but it cannot guarantee that the same applies to the reverse. In low-texture and repetitive structure scenes, many-to-one matching ambiguities often occur (i.e., a point in image B may simultaneously and reasonably match multiple points in image A). One-way verification cannot detect this error.

[0082] This invention ensures a one-to-one correspondence and symmetry in matching relationships through bidirectional verification. A match is considered reliable only if it can be identified as an interior point by the RANSAC model in both forward and backward verification, thereby improving matching quality.

[0083] Results achieved: It further eliminated hidden mismatches that could not be detected by one-way verification, especially when dealing with a large number of similar rivets, steel plate joints and other repetitive structures on the surface of trucks, which greatly improved the symmetry consistency of the matching and the geometric accuracy of the final model.

[0084] Overall, this invention achieves a highly efficient filtering process for low-texture, high-noise environments through a deep collaboration between more rigorous Lowe's testing (pursuing quality) and bidirectional RANSAC verification (pursuing symmetry and consistency). This represents a fundamental difference and significant advancement compared to existing filtering mechanisms that are relatively lenient and unidirectional, designed for general scenarios. S3: Recover the three-dimensional geometry of the scene from the filtered feature points, obtaining the three-dimensional coordinates of key feature points, where the key feature points are those in the critical deformation regions of the vehicle body.

[0085] Specifically, S3 is sparse reconstruction, which involves recovering the basic 3D geometry of a scene from matched feature points and obtaining the precise 3D coordinates of key feature points. This involves calculating the positions of feature points in 3D space using a triangulation algorithm to construct a sparse point cloud. The triangulation algorithm mainly utilizes the intrinsic and extrinsic parameters determined during camera calibration, and the feature point coordinate information obtained and filtered through the ORB and RANSAC algorithms. The coordinates of the feature points in 3D space are then mathematically solved. The obtained coordinates need to undergo a point cloud optimization process, typically using the Levenberg-Marquardt algorithm (an optimization algorithm for minimizing nonlinear minimization problems). This algorithm combines the advantages of gradient descent and Gauss-Newton methods, and by iteratively updating the step size, it can improve stability while maintaining convergence speed, outputting an optimized point cloud.

[0086] Based on the optimized point cloud output by the Levenberg-Marquardt algorithm, sparse reconstruction of the features of heavy-haul railway maintenance freight cars can be completed.

[0087] The sparse reconstruction process of this invention is not a simple application of the standard SFM (Structure from Motion) procedure, but rather a targeted optimization serving the core objective of accurately capturing geometric deformation. Its distinctive features compared to existing technologies are:

[0088] The purposes differ: sparse point clouds used in general 3D reconstruction aim to provide a rough geometric framework for subsequent dense reconstruction, often pursuing uniform point cloud coverage. In contrast, the sparse point cloud generated by this invention provides a high-precision spatial coordinate reference for key deformation areas of the vehicle body. These feature points are structural anchor points that best represent the true geometric deformation of the vehicle body, retained from high-noise images after a triple screening process involving ORB, RANSAC, and Lowe's tests.

[0089] The specific technical solution: After obtaining the initial 3D points through triangulation, the common Bundle Adjustment (BA) optimization usually focuses on minimizing the overall reprojection error. This invention, based on this, introduces a weighted optimization strategy based on the consistency of point cloud spatial distribution. Specifically, feature points that are stably tracked in consecutive frames and located at structural edges of the vehicle body (such as the connection between the sidewall and the underframe) are given higher optimization weights. This ensures that the 3D coordinates of these key deformation areas have absolute accuracy at the millimeter or even sub-millimeter level, providing extremely reliable spatial constraints for subsequent IDW dense reconstruction.

[0090] Output: The sparse point cloud output by the general process is a visual intermediate product. The optimized sparse point cloud generated by this invention can be directly used for rapid and preliminary deformation comparison with the benchmark CAD model, realizing rapid diagnosis and quantification of large-scale deformation of the vehicle body (such as the overall outward bulging of the side wall), forming a closed loop from sparse perception to preliminary deformation judgment.

[0091] As mentioned earlier, sparse reconstruction has provided the spatial coordinates of key deformation features of the vehicle body. However, a sparse skeleton alone is far from sufficient to support millimeter-level path planning for a painting robot, which requires a continuous, complete, and highly detailed surface model. Therefore, the core task of the dense reconstruction stage of this invention is to perform massive data interpolation and completion within the reliable spatial framework established by the aforementioned high-precision sparse point cloud, transforming a skeleton into a solid.

[0092] S4: Dense reconstruction is performed using inverse distance weighted interpolation.

[0093] Specifically, the purpose of dense reconstruction is to increase the density of point cloud data, improve the quality and detail of point cloud data, and make the heavy-haul railway maintenance freight car body model more accurate. This application adopts IDW (Inverse Distance Weighted Interpolation) for dense reconstruction. The advantages of this method include, but are not limited to: (1) Simple and easy to use: The algorithm is relatively simple and easy to understand and implement. (2) Fast and efficient: IDW has a fast calculation speed and is suitable for real-time or near-real-time application scenarios, especially suitable for processing large datasets. (3) No additional assumptions required: It does not require assumptions about the distribution characteristics of the data or dependence on complex statistical models. It only needs to determine the neighborhood size and the exponent p. (4) Local interpolation: IDW only uses neighboring points for interpolation, so the influence on distant points is small. This local property makes IDW very suitable for processing data with local variation characteristics. (5) Easy to parallelize: The IDW algorithm can be easily parallelized to improve the computational efficiency, especially when processing large-scale datasets, modern hardware can be used to accelerate the calculation. Since the reconstruction process of the three-dimensional model of the paint shop car body is a fast and data-intensive process, the IDW method can be used to efficiently and accurately complete the point cloud dense reconstruction work.

[0094] The specific working steps of IDW are as follows:

[0095] (1) Neighborhood search: Define a neighborhood for the current point to be interpolated (called the center point). This neighborhood can be defined by a fixed search radius R, or by specifying the number of nearest neighbors K to be found.

[0096] In this embodiment, the K-nearest neighbor method is used, and the purpose of this step is to find the K neighbor points of the center point.

[0097] (2) Weight calculation: For each neighboring point found, the weight of the neighboring point is calculated based on the reciprocal of its distance from the center point (or the reciprocal of the distance raised to the power of p). The closer the neighboring point is, the greater its weight.

[0098] (3) Interpolation calculation: Once the weights of all neighboring points have been calculated, the new point can be calculated using the formula. The position. The formula is as follows:

[0099]

[0100] in It is the weight of the i-th neighbor point, here Let n represent the position of the i-th neighbor point, and n be the number of neighbor points.

[0101] (4) New point generation: For each existing point, calculate the position of one or more new points according to the steps above, and add them to the point cloud.

[0102] (5) Post-processing: After obtaining the dense point cloud data, another point cloud optimization process is performed, which can further reduce high-frequency noise and improve the accuracy of the point cloud model.

[0103] The special feature of intensive reconstruction is:

[0104] This invention innovatively employs inverse distance weighted interpolation (IDW) for dense reconstruction. Its key features include: ① Fidelity in Deformation Details: Unlike commonly used global optimization algorithms such as Poisson reconstruction (which over-smooth surfaces, smoothing out pits and bumps), IDW's local interpolation characteristics allow it to faithfully preserve and enhance local morphological features caused by heavy-load impacts and fatigue damage, resulting in a model that more closely resembles the actual condition of the vehicle body. ② Balance between Efficiency and Accuracy: Addressing the stringent efficiency requirements of paint shops, the IDW algorithm is computationally simple and easily parallelized. It can rapidly process massive point cloud data generated by large-scale trucks while maintaining high fidelity in deformation details, achieving a leap from visual completeness to geometric precision.

[0105] S5: Perform temporal consistency correction on the densely reconstructed point cloud data, and generate a 3D point cloud model based on the correction results. Temporal consistency correction is used to ensure the consistency of point cloud data in the time dimension.

[0106] Specifically, time consistency correction refers to the process of ensuring data consistency across the time dimension when processing continuous time-series data. This is crucial for applications such as dynamic scene reconstruction, video processing, and motion capture. Through correction, data quality can be improved, noise reduced, and the accuracy and reliability of reconstruction models enhanced.

[0107] In the specific implementation process, the steps for time consistency correction are as follows:

[0108] (1) Synchronization process: Ensure that the timestamps of all data sources are synchronized.

[0109] (2) Registration: Register the point clouds in consecutive time frames so that they are aligned to the same reference coordinate system. Registration can be achieved by rigid body transformation (translation and rotation).

[0110] (3) Timestamp correction: For each frame, it is necessary to ensure that the timestamp corresponds to the actual shooting time.

[0111] (4) Motion compensation: This is achieved by tracking key points or features to ensure that the relative motion between consecutive frames is correctly estimated and corrected.

[0112] (5) Smoothing: Use filters to smooth the point cloud in consecutive time frames to reduce abrupt changes in time.

[0113] (6) Consistency check: Check whether the point clouds between consecutive frames are consistent. If inconsistencies are found, the corresponding frames may need to be reprocessed. Specifically, point cloud consistency refers to the consistency of geometric structure and temporal continuity. That is, the three-dimensional coordinates of the point clouds in the overlapping area of ​​adjacent frames should transition smoothly without drastic jumps, and the spatial distribution of feature points and the overall outline of the vehicle body should remain consistent in the time dimension.

[0114] (7) Fusion processing: Using methods such as time-weighted averaging, the point clouds in the corrected continuous time frames are fused into a complete dynamic scene model. After time consistency correction, a three-dimensional reconstruction model of the heavy-haul railway maintenance freight car can be obtained.

[0115] The unique feature of the time consistency correction proposed in this invention is that:

[0116] The time consistency correction process of this invention is specifically designed for the dynamic scenario of a truck moving continuously and slowly in a painting workshop, and is fundamentally different from existing methods used for static or small-scale movement scenarios.

[0117] The specific implementation of motion compensation is as follows: Based on the aforementioned ORB feature point matching results, the rigid body transformation matrix (rotation matrix R and translation vector T) between adjacent frames is calculated, and the iterative nearest point algorithm is used for fine registration, thereby performing reverse compensation for the point cloud displacement caused by the vehicle movement, ensuring that the point clouds of consecutive frames are located in the same coordinate system.

[0118] The main differences from existing methods are as follows:

[0119] Objective: Existing methods are mostly used to eliminate the effects of handheld device shaking or small object movements, while this invention deals with the continuous motion of large-scale targets in the global coordinate system.

[0120] Correction fusion strategy: After completing motion compensation and point cloud registration, this invention adopts a time-weighted sliding window fusion strategy for point clouds of multiple adjacent frames in time sequence. This strategy not only utilizes the information of multiple frames to improve model density and accuracy, but also weakens the influence of single-frame noise and outliers through weighting, ensuring the smoothness and accuracy of the final model in the time dimension.

[0121] S6: Fuse the generated 3D point cloud model with the preset baseline CAD model to output the target 3D model.

[0122] S6 is mainly divided into two parts: (1) 3D reconstruction model fusion processing. (2) 3D model output. The 3D model fusion processing mainly compares and merges the 3D reconstruction model of the truck with the accurate CAD model of the truck in the database.

[0123] Fusion processing is key to realizing the transition from model reconstruction to industrial applications. Its unique feature lies in:

[0124] Fine registration: The reconstructed 3D point cloud model is aligned with the baseline CAD model in the database using a strategy that combines feature-based coarse registration with ICP fine registration to achieve high precision.

[0125] Difference Quantization Identification: By calculating the signed distance function from the point cloud to the CAD model surface, the deformation areas on the vehicle body surface are automatically identified and quantified. For example, risk areas with protrusions >3cm and depressions >2cm are accurately marked.

[0126] Intelligent model generation: This is not a simple replacement, but rather a seamless integration of the geometric information of the deformed areas identified in the reconstructed model with the main structure of the baseline CAD model, generating a hybrid 3D model that combines baseline geometry with actual deformation. This model retains the standard structure of the original design while incorporating actual damage and deformation, and can be directly used for collision detection and adaptive path planning in painting robots.

[0127] Compared with the prior art, the present invention has the following advantages and beneficial technical effects:

[0128] (1) Efficient and accurate modeling for heavy-load scenarios

[0129] Through an optimized multi-view acquisition system ( Figure 2By collecting vehicle body data from multiple angles and combining the ORB algorithm (rotation / scale invariance) customized for large-scale, high-noise scenes with RANSAC anomaly filtering and IDW dense reconstruction technology chain, high-precision 3D reconstruction of heavy-duty truck bodies with severe deformation can be completed in a short time. The efficiency is greatly improved compared with traditional manual measurement, providing key technical support for shortening the maintenance cycle of heavy-duty trucks.

[0130] (2) Strong fidelity in capturing vehicle body damage characteristics

[0131] By leveraging the local interpolation characteristics of the IDW algorithm, it exhibits excellent detail preservation capabilities for large-scale deformations typical of heavy-duty trucks, such as sidewall bulging and floor depressions, avoiding the smoothing effect of general reconstruction algorithms and enabling the reconstruction model to accurately reflect the vehicle's health condition.

[0132] (3) High robustness in dynamic industrial environments

[0133] The unique time consistency correction process effectively overcomes the inter-frame misalignment caused by freight car movement, with point cloud registration error ≤0.5mm, meeting the extremely high safety requirements of heavy-haul railway painting operations.

[0134] (4) Deeply integrate with industrial systems to improve safety and automation levels

[0135] By using intelligent model fusion technology, deformation areas can be automatically identified and quantified, and the risk of collision with the robotic arm can be avoided in advance, reducing the spraying accident rate by 90%. This lays the model foundation for realizing truly unmanned spraying operations in heavy-haul railway maintenance workshops.

[0136] (5) Low cost and easy deployment

[0137] Employing a lightweight algorithm (IDW interpolation only requires K-nearest neighbor calculations), it can be run with a regular industrial camera and a GPU server, reducing hardware costs by 60% and making it suitable for large-scale deployment in heavy-haul railway maintenance workshops.

[0138] Summary of implementation results: This invention has overcome the technical bottleneck of 3D modeling of heavy-haul railway trains. The bottleneck is mainly reflected in the fact that due to the harsh service conditions of heavy-haul railway maintenance freight cars, the car body generally suffers from three major problems: large-scale deformation, surface texture degradation, and the need to be measured in dynamics. This leads to problems such as inaccurate feature extraction, loss of deformation details, and accumulation of dynamic errors in traditional 3D reconstruction methods, which cannot meet the safety requirements of painting robots for model accuracy.

[0139] This invention achieves a technological breakthrough by: 1) employing a targeted ORB feature extraction strategy to address the problem of inaccurate feature extraction; 2) innovatively applying the IDW local interpolation algorithm to solve the problem of lost deformation details; and 3) designing a dynamic temporal consistency correction process to address the problem of dynamic error accumulation. This breakthrough enables the accurate reconstruction of damaged car bodies within minutes, moving away from relying on new train CAD drawings, and provides a reliable 3D model source for intelligent maintenance and unmanned painting of heavy-haul railways.

[0140] The following describes the device for acquiring the 3D model of heavy-haul railway trains provided in this disclosure. Please refer to [link to relevant documentation]. Figure 3 The device includes:

[0141] The data acquisition module is used to collect video data of the train body of heavy-haul railway trains and to preprocess the images in the collected video data.

[0142] The feature extraction and matching module is used to extract and match features from the preprocessed image data using the ORB algorithm, and to filter feature points using a dual filtering mechanism combining Lowe's ratio test and RANSAC to obtain filtered feature points.

[0143] The sparse reconstruction module is used to recover the three-dimensional geometry of the scene from the filtered feature points and obtain the three-dimensional coordinates of key feature points, where the key feature points are feature points in the key deformation areas of the vehicle body.

[0144] The dense reconstruction module is used to perform dense reconstruction using the inverse distance weighted interpolation method.

[0145] The point cloud correction and model generation module is used to perform time consistency correction and point cloud consistency correction on the densely reconstructed point cloud data, and generate a three-dimensional point cloud model based on the correction results. The time consistency correction is used to ensure the consistency of the point cloud data in the time dimension, and the point cloud consistency correction is used to ensure the consistency and temporal continuity of the geometric structure corresponding to the point cloud data.

[0146] The output module is used to fuse the generated 3D point cloud model with the preset baseline CAD model and output the target 3D model.

[0147] Specifically, the data acquisition module includes a camera calibration unit, an image acquisition unit, and an image preprocessing unit. The 3D reconstruction module includes a feature extraction and matching module, a sparse reconstruction module, a point cloud optimization module, a dense reconstruction module, and a point cloud correction and model generation module (including a time consistency correction module and a 3D reconstruction model generation module). The output module includes a 3D reconstruction model fusion module and a 3D model output module.

[0148] The following describes the computer equipment used to acquire the three-dimensional model of the heavy-haul railway train provided in this disclosure. The computer equipment described below can be referred to in correspondence with the method for acquiring the three-dimensional model of the heavy-haul railway train described above.

[0149] The computer device may include a processor, a communications interface, memory, and a communication bus. The processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from memory to execute the method for acquiring the 3D model of a heavy-haul railway train, mainly including the software processing portion described above.

[0150] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the software processing portion of the above-described method for obtaining a three-dimensional model of a heavy-haul railway train.

[0152] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.

Claims

1. A method for obtaining a three-dimensional model of a heavy-haul railway train, characterized in that, include: Collect video data of the train body of a heavy-haul railway and preprocess the images in the collected video data. The ORB algorithm is used to extract and match features from the preprocessed image data. A dual filtering mechanism combining Lowe's ratio test and RANSAC is used to filter the feature points to obtain the filtered feature points. The three-dimensional geometry of the scene is recovered from the filtered feature points, and the three-dimensional coordinates of the key feature points are obtained. The key feature points are the feature points of the key deformation areas of the vehicle body. Dense reconstruction was performed using inverse distance-weighted interpolation. Temporal consistency correction is performed on the densely reconstructed point cloud data, and a three-dimensional point cloud model is generated based on the correction results. Temporal consistency correction is used to ensure the consistency of point cloud data in the time dimension. The generated 3D point cloud model is fused with a preset baseline CAD model to output the target 3D model.

2. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, The collection of video data of heavy-haul railway train bodies includes collecting video data of the train bodies through multiple sets of cameras in the painting workshop. Before collecting the video data of the train bodies, the internal and external parameters of the cameras are calibrated.

3. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, A dual filtering mechanism combining Lowe's ratio test and RANSAC is used to filter feature points, resulting in the filtered feature points, including: For each feature point, calculate the ratio of its distance to its nearest and second nearest neighbors in the target image; Determine if the distance ratio is less than a preset threshold. If it is less, retain the corresponding feature point; otherwise, filter the corresponding feature point. We use RANSAC to perform bidirectional consistency verification on the retained feature points and filter out points that fail the verification.

4. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, After recovering the 3D geometry of the scene from the filtered feature points and obtaining the 3D coordinates of the key feature points, the method further includes: The 3D coordinates of the acquired key feature points are optimized using point cloud. During the optimization process, a weighted optimization strategy based on the consistency of the spatial distribution of the point cloud is introduced. This strategy includes assigning high optimization weights to feature points that meet the preset stability requirements and are located at the structural edge of the vehicle body in multiple consecutive frames.

5. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, Dense reconstruction is performed using inverse distance weighted interpolation, including: The feature points are sampled, and the sampled points are used as a fixed search radius. The points within the radius are taken as the K nearest neighbors. For each sampled feature point, the weights of the neighboring points are calculated based on the reciprocal of the distances between the neighboring points and that point. The position of the new point is calculated based on the positions and weights of the neighboring points. The newly calculated points are added to the point cloud composed of key feature points to obtain dense point cloud data; Optimize dense point cloud data.

6. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, Temporal consistency correction is performed on the densely reconstructed point cloud data, including: Temporal consistency correction is performed on the densely reconstructed point cloud data, including: synchronizing the timestamps of all point cloud data; registering the point clouds in consecutive time frames; correcting the timestamps of each frame of data; calculating the rigid body transformation matrix between adjacent frames based on the filtered feature points obtained by the ORB algorithm and performing fine registration using the iterative nearest point algorithm; smoothing the point clouds in consecutive time frames using a filter; and checking whether the point clouds in consecutive frames are consistent.

7. The method for obtaining a three-dimensional model of a heavy-haul railway train as described in claim 1, characterized in that, The generated 3D point cloud model is fused with a preset baseline CAD model, including: A strategy combining feature-based coarse registration and ICP fine registration is adopted to align the reconstructed 3D point cloud model with a preset benchmark CAD model; By calculating the signed distance function from the point cloud in the 3D point cloud model to the surface of the preset benchmark CAD model, the deformation area of ​​the vehicle body surface is identified and quantified, and the target 3D model is output.

8. A device for acquiring a three-dimensional model of a heavy-haul railway train, characterized in that, include: The data acquisition module is used to collect video data of the train body of heavy-haul railway trains and to preprocess the images in the collected video data. The feature extraction and matching module is used to extract and match features from the preprocessed image data using the ORB algorithm, and to filter feature points using a dual filtering mechanism combining Lowe's ratio test and RANSAC to obtain filtered feature points. The sparse reconstruction module is used to recover the three-dimensional geometry of the scene from the filtered feature points and obtain the three-dimensional coordinates of key feature points, where the key feature points are feature points in the key deformation areas of the vehicle body. The dense reconstruction module is used to perform dense reconstruction using the inverse distance weighted interpolation method. The point cloud correction and model generation module is used to perform time consistency correction on the densely reconstructed point cloud data and generate a three-dimensional point cloud model based on the correction results. The time consistency correction is used to ensure the consistency of the point cloud data in the time dimension. The output module is used to fuse the generated 3D point cloud model with the preset baseline CAD model and output the target 3D model.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the method for obtaining a three-dimensional model of a heavy-haul railway train as described in any one of claims 1 to 7.

10. A computer device having a computer program stored thereon, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for obtaining a three-dimensional model of a heavy-haul railway train as described in any one of claims 1 to 7.

Images