A three-dimensional mapping train wheel out-of-roundness detection method

Abstract

The application discloses a three-dimensional mapping train wheel out-of-roundness detection method, relates to the technical field of railway wheel out-of-roundness detection, and comprises the following steps: when a train is running, feature points of a wheel are marked, and a local state two-dimensional image of the wheel when the train is running is acquired; the method further comprises the following steps: feature point extraction, space extreme point detection, accurate positioning of the feature points, feature point direction information matching, feature point description and feature point matching are sequentially performed on the local state two-dimensional image, and movement pose data of the wheel between adjacent images is obtained; the movement pose data of the wheel is optimized, and global consistent wheel movement track data and wheel two-dimensional image data are obtained; wheel space three-dimensional image data is established according to the wheel movement track data and the wheel two-dimensional image data; and the wheel space three-dimensional image data is compared with standard wheel data, and wheel out-of-roundness information is obtained. The application is easier to acquire essential features of an image, and improves matching stability and anti-noise capability.

Description

Technical Field

[0001] This invention relates to the field of railway wheel out-of-roundness detection technology, and more specifically, to a three-dimensional mapping method for detecting the out-of-roundness of train wheels. Background Technology

[0002] After a period of operation, railway vehicles will experience uneven wear on the wheel treads. When the wear reaches a certain value, the wheel out-of-roundness will cause additional vibration, impact, and noise, affecting the vehicle's stability and threatening the safety of train operation.

[0003] Currently, mainstream wheel out-of-roundness detection technologies include manual inspection using various detectors, stress-strain testing, ultrasonic testing, and electrical signal monitoring. These methods suffer from several drawbacks: they cannot achieve online detection, require significant manpower, and have low efficiency. Furthermore, wheels cannot be inspected promptly, and the out-of-roundness cannot be visualized. Therefore, a dynamic, non-contact measurement method for train wheel out-of-roundness is urgently needed.

[0004] The center-point chord measurement method is a relatively new vehicle-mounted, non-contact, dynamic wheel out-of-roundness measurement method. This method requires the prior installation of a detection device, the main body of which consists of three laser displacement sensors. At the start of measurement, the three sensors, with their centers equidistant from the wheel tread, simultaneously measure the wheel tread. A rotary encoder records the acquisition step length, which is transmitted to a host computer via a Wi-Fi module. The measurement data is then reconstructed using an inverse filter to obtain the true out-of-roundness data of the wheel. This vehicle-mounted, non-contact, dynamic wheel out-of-roundness measurement method has high accuracy, but it is complex to install, requiring the installation and adjustment of the measuring device on each wheel. Furthermore, this method is only applicable to parking lots and low-speed train operation scenarios, and cannot be used for measurements while the train is running on a railway line.

[0005] Chinese patent ZL202210372005.1 discloses a method for detecting train wheel out-of-roundness based on three-dimensional information. This invention employs a combination of multiple line-scan cameras and a 3D laser scanner to generate multiple two-dimensional image data and laser scan data with depth information. The laser-scanned tread area is extracted, and the wheel tread contour with depth information is reconstructed based on an elliptical model. This measuring equipment is installed beside the track at the train's entry throat section, with five measuring devices installed on each side of the track. This method can perform dynamic non-contact measurement of train wheel out-of-roundness, but the visual effect is poor, failing to visualize wheel out-of-roundness, and the laser scanner is costly. Summary of the Invention

[0006] The present invention provides a method for detecting the out-of-roundness of train wheels in three-dimensional mapping.

[0007] To alleviate the above problems, the technical solution adopted by the present invention is as follows:

[0008] This invention provides a three-dimensional mapping method for detecting out-of-roundness of train wheels. During train operation, feature points are marked on the wheels to obtain a two-dimensional image of the local state of the wheels. The method further includes the following steps:

[0009] S100. Generate a Gaussian difference pyramid based on the local state two-dimensional image. Based on the Gaussian difference pyramid, perform spatial extreme value feature point detection, precise localization of feature points, feature point orientation information matching, feature point description, and feature point matching in sequence to obtain the motion pose data of the wheel between adjacent images.

[0010] S200. The nonlinear least squares method is sequentially used to optimize the motion pose image data of the wheel and to detect loop closure, so as to obtain globally consistent wheel motion trajectory data and wheel two-dimensional image data.

[0011] S300: Perform monocular dense reconstruction based on wheel motion trajectory data and wheel two-dimensional image data to establish wheel spatial three-dimensional image data;

[0012] S400: Calculate the distance between the 3D point cloud of the wheel space image and the standard wheel point cloud to obtain the wheel out-of-roundness information.

[0013] In a preferred embodiment of the present invention

[0014] A wheel feature point marking device is used to mark feature points on the wheel;

[0015] The wheel feature point marking device includes two tread feature marking devices arranged on the outer side of the train track and two wheel flange feature marking devices installed on the inner side of the track;

[0016] The feature marking device includes a feature marking device box and several feature marking pens disposed in the feature marking device box, which marks feature points on the wheel when the wheel contacts the feature marking pens.

[0017] Two-dimensional images of the local state of the train wheels during operation are acquired using a monocular camera group;

[0018] The monocular camera assembly is mounted on the train track and located behind the wheel feature point marking device;

[0019] When the wheel passes the feature marker pen closest to the monocular camera group, the monocular camera group is activated, and the monocular camera group starts to capture a two-dimensional image of the local state of the wheel during train operation. When the train has completely passed the wheel feature point marking device, the wheel has been in contact with the feature marker pen for a longer than the set time, and the monocular camera group is turned off.

[0020] In a preferred embodiment of the present invention

[0021] The real-time running speed of the train and the wheelbase of the bogie are calculated based on the time difference between the first and last feature markers that the wheel contacts, and the distance between the two feature markers.

[0022] The shooting frequency of the monocular camera group is determined based on the real-time operating speed of the train. The shooting frequency of the monocular camera group must ensure that when a wheel completely passes over the wheel feature point marking device, 24 local state two-dimensional images can be captured by the monocular camera group. The number of captured local state two-dimensional images must ensure strong robustness of three-dimensional mapping while reducing the intensity of real-time image processing by the computer.

[0023] In a preferred embodiment of the present invention, before performing step S100, the local state two-dimensional images of different groups of wheels are classified, stored, and built into a library by taking local state two-dimensional images of different groups of wheels at different intervals based on the wheelbase of the train bogie.

[0024] In a preferred embodiment of the present invention, step S100, the specific method for generating the Gaussian difference pyramid includes: performing Gaussian blurring on the two-dimensional image of the local state of the wheel at different scales, calculating the blur template using the Gaussian function, and performing a convolution operation between the template and the original two-dimensional image of the local state of the wheel to blur the two-dimensional image of the local state of the wheel, and then performing downsampling multiple times on the blurred two-dimensional image of the local state, each downsampling to obtain one layer of the Gaussian pyramid image, and several images of each layer are collectively referred to as a group of Gaussian pyramids, and subtracting the adjacent upper and lower layers of the Gaussian pyramid in each group to obtain the Gaussian difference pyramid.

[0025] Feature points are extracted at different scales to ensure scale invariance, making them invariant to image angles and rotations. The orientation is assigned by calculating the gradient of each feature point.

[0026] In a preferred embodiment of the present invention, spatial extreme feature point detection is performed by identifying potential feature points whose scale and rotation angle remain unchanged through the difference of Gaussian pyramid, and then performing spatial extreme point detection to obtain a set of spatial extreme feature points. Precise feature point localization involves accurately determining the position and scale of the feature points by fitting a three-dimensional quadratic function to the set of spatial extreme feature points, while removing low-contrast feature points and unstable edge response points to enhance matching stability and improve noise resistance. Feature point orientation information matching uses image gradient methods to obtain stable orientations for the set of spatial extreme feature points. Feature point description involves creating a descriptor for each spatial extreme feature point; this descriptor is a set of feature vectors that describe the feature point in a way that does not change with various factors. Feature point matching estimates the motion pose data of the wheel based on the set of spatial extreme feature points.

[0027] The feature point description divides the pixel region around the feature point into blocks, calculates the gradient histogram within the block, and generates a vector to abstractly represent the image information of the region. This information includes not only the feature point, but also the neighboring points around the feature point that contribute to it.

[0028] Feature point matching is achieved by calculating the 128-dimensional Euclidean distance between two sets of feature points. The smaller the Euclidean distance, the higher the similarity. When the Euclidean distance is less than the set threshold, it is considered a successful match. By matching the feature points of each key frame, the motion and pose relationship of the wheel are estimated. Key frame matching features are inserted, the localization of the frame is calculated, the wheel landmark points are calculated using the triangulation method, and all key frames and landmark points are assigned to estimate the motion and pose of the wheel.

[0029] In a preferred embodiment of the present invention, step S200, the nonlinear least squares optimization includes: using Bayes' rule to estimate the conditional distribution of wheel state variables in batches, obtaining the maximum likelihood estimate, and obtaining better wheel motion and pose estimates; substituting the better wheel motion and pose estimates into the motion and observation equations of SLAM, using the PNP algorithm to provide iterative initial values ​​for the wheel motion and pose estimates, and finally using the Gauss-Newton method to iteratively fine-tune the wheel motion and pose estimates, solving for a minimum value, and obtaining locally consistent wheel motion trajectory data and wheel two-dimensional image data.

[0030] In a preferred embodiment of the present invention, loop closure detection includes: using the K-means algorithm to cluster locally consistent wheel motion trajectory data and wheel two-dimensional image data; then describing all clustered wheel motion and pose images with description vectors; calculating the L1 norm of each description vector; comparing the similarity of each description vector; defining the similarity between each image; and determining that a similarity greater than 90% indicates successful loop closure detection. Detected loop closures may be from multiple frames. The clustering algorithm is used to group similar loop closures into one class, so that the algorithm does not repeatedly detect loop closures of the same class, ultimately obtaining globally consistent wheel motion trajectory data and wheel two-dimensional image data.

[0031] In a preferred embodiment of the present invention, step S300 specifically includes: calculating the depth value of the wheel motion and pose image using triangulation based on the wheel motion trajectory data and the wheel two-dimensional image data; calculating the uncertainty of the depth information based on geometric relationships; then fusing the current observed depth into the previous estimate; traversing each pixel of the current wheel motion and pose image; using the first captured wheel pose image as a reference frame; converting the pixel coordinates of the reference frame from pixel coordinates to three-dimensional coordinates in the camera coordinate system; multiplying the three-dimensional coordinates by a rotation matrix and converting them to the camera coordinate system in the current frame; then projecting them onto the pixel coordinates of the current frame; and then projecting the converted three-dimensional coordinates in the reference frame twice under the condition of maximum and minimum depth to obtain two projected coordinates. The line connecting these two points is the epipolar line to be searched. Using NCC to search for the best matching block on the epipolar line, after a successful search, updating the depth map and completing the establishment of the wheel space three-dimensional image data.

[0032] In a preferred embodiment of the present invention, step S400 specifically includes: transforming the wheel space three-dimensional image point cloud and the standard wheel point cloud in the wheel space three-dimensional image data to the world coordinate system, and then calculating their Euclidean distance to obtain the wheel's out-of-roundness information.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] This is a dynamic detection method for wheel out-of-roundness that saves manpower, provides timely health checks, and offers intelligent visualization. Compared with conventional detection methods, the three-dimensional image information of wheel out-of-roundness obtained by this method can be well visualized by computer, thus better assisting human decision-making.

[0035] The Gaussian difference pyramid is used to represent different wheel images. The Gaussian difference pyramid contains a series of low-pass filters, which has the advantage of spanning a large frequency range. It can better describe wheel images at multiple scales and make it easier to obtain wheel image features. At the same time, it overcomes problems such as wheel image rotation, scale scaling, and brightness changes. It has good stability against wheel image viewpoint changes and noise. Moreover, wheel feature point matching is accurate and fast, and the motion pose data of wheels between adjacent images are obtained efficiently.

[0036] Compared with traditional single-scale image information processing techniques, scale-space methods are more likely to obtain the essential features of images. The blurring degree of images at each scale in scale space gradually increases, which can simulate the formation process of a target on the retina when a person moves from near to far from the target.

[0037] By using loop closure detection technology, the constraints on wheel pose were increased and the cumulative error was reduced, resulting in more accurate global wheel trajectory data. Through continuous optimization, the wheel point cloud distance was reduced to the initial point cloud distance, the wheel pose offset was corrected, edge ghosting was reduced, and the wheel geometry was clear.

[0038] By fitting a three-dimensional quadratic function, the location and scale of key points are accurately determined, while low-contrast key points and unstable edge response points are removed, thereby enhancing matching stability and improving noise resistance.

[0039] To address the problems of complex installation, low applicable vehicle speed, and inability to achieve three-dimensional visualization of out-of-roundness in existing vehicle-mounted non-contact dynamic wheel out-of-roundness measurement methods based on the center point chord measurement method, this technical solution eliminates the need to repeatedly install and adjust the detection equipment for each wheel. The system can be installed on the train's entry line and maintained periodically to detect the out-of-roundness information of multiple wheels. This system is applicable to vehicle speeds of 40-80 km / h, higher than the 20 km / h measurement speed of the aforementioned methods. This technical solution provides effective three-dimensional visualization of wheel out-of-roundness information, facilitating observation, identification, and report writing by staff.

[0040] In response to the problem that the visual effect of the train wheel out-of-roundness detection method based on three-dimensional information disclosed in Chinese patent ZL202210372005.1 is poor and the cost of laser scanner is high, this technical solution adopts a monocular camera group, which greatly reduces the hardware cost. The monocular camera has a better visual imaging effect than the laser scanner.

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, embodiments of the present invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart for a method to detect out-of-roundness wheels in 3D modeling;

[0044] Figure 2 Layout diagram of the wheel out-of-roundness detection device;

[0045] Figure 3 A schematic diagram of a wheel feature point marking device;

[0046] Figure 4A schematic diagram of a frontal shot taken by the camera assembly;

[0047] Figure 5 A schematic diagram of a dynamic wheel being photographed from the side by a camera group;

[0048] Figure 6 A flowchart for estimating wheel motion pose data. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0051] Please refer to Figure 1 This is a flowchart of the method for three-dimensional modeling of train wheels and online visualization detection of wheel out-of-roundness in this embodiment.

[0052] like Figure 2 As shown, the wheel feature point marking device 3 is arranged on the inner and outer sides of the track 1. The wheel feature point marking device 3 includes four feature marking devices, namely two tread feature marking devices arranged on the outer side of the track and two wheel flange feature marking devices installed on the inner side of the track.

[0053] Figure 3 This is a schematic diagram of a single feature marking device. Based on the structure of wheel 2, the feature marking device includes a feature marking device box 302 and 24 feature marking pens 301 disposed on the feature marking device box 302. The design height of the feature marking device box 302 for the rim feature marking device is slightly lower than that for the tread feature marking device, and the distance between the feature marking pens 301 on the feature marking device box 302 is equal.

[0054] During installation, along the length of track 1, the distance between the first (the foremost) feature marker pen 301 and the last feature marker pen 301 on the feature marking device box 302 is greater than the wheel circumference. The design aims to ensure that the tread and wheel flange feature points can be marked evenly, clearly and densely when the wheel passes, thereby reducing the difficulty of wheel image processing and making wheel feature matching more accurate.

[0055] like Figure 2 As shown, the wheel feature point marking device 3 is arranged in front of the monocular camera group 4 (the direction of train travel is forward, and vice versa). The distance between the wheel feature point marking device 3 and the camera group is greater than the wheel diameter. When the wheel passes the last feature marker pen 301 (the marker pen closest to the monocular camera group 4), the monocular camera group start switch is triggered, and the camera group starts shooting. When the train completely passes the wheel feature point marking device 3, and the wheel contacts the feature marker pen 301 for a longer time than the set time, the camera group turns off.

[0056] Based on the time difference between the wheel contacting the first feature marker 301 and the last feature marker 301, combined with the distance between the two markers, the real-time running speed of the train can be obtained.

[0057] The system determines the shooting frequency of the monocular camera group based on the real-time operating speed of the train. The shooting frequency of the monocular camera group is ensured to be appropriate. Each wheel requires 24 images. While ensuring the robustness of the 3D mapping, the number of images acquired also needs to reduce the intensity of real-time image processing by the computer to save resources.

[0058] Generally, a train has two sets of wheels on one bogie, and a train has several bogies. The interval between the two sets of wheels on one bogie is relatively short. The system records the time when each set of wheels contacts the last feature marker pen 301 (i.e. the time when the wheel is marked). Based on the contact time interval recorded by the marker pen, the system numbers, sorts, and stores the two-dimensional images of the wheel's local state captured by the camera group, ensuring that the images are ordered and belong to the same wheel when feature matching is performed.

[0059] There are 24 monocular cameras in the group 4. Six are arranged inside and outside the left rail of track 1, and six are arranged inside and outside the right rail. The distance between the monocular cameras is adjusted according to the structure of track 1 and the road conditions. The distance between the first monocular camera and the last monocular camera is greater than the circumference of the wheel, and the wheel angle is adjusted accordingly.

[0060] like Figure 4 As shown, this ensures that the left and right monocular cameras can clearly and extensively capture the wheel tread surface, while also capturing as many wheel feature points as possible marked on the wheel by the feature marker pen 301.

[0061] In practical applications, the angle between a monocular camera and the horizontal is in the range of 30 to 60 degrees. Too small an angle may result in a small wheel tread area and fewer wheel feature points in the image, reducing the efficiency of image use and increasing the difficulty of calculating feature points and feature matching. Too large an angle may reduce the clarity of the image and lead to a decrease in the accuracy of wheel out-of-roundness calculation.

[0062] like Figure 5As shown, this is a side view of the wheel taken by the camera during the train's movement. Wheel track position 201 is the position of wheel 2 before it moves. Two monocular cameras 401 and 402 on the inner and outer sides of the monorail are responsible for taking pictures of the local positions of the inner and outer sides of the wheel, respectively. After the wheel has traveled one revolution, the monocular camera group 4 can completely record the displacement and movement trajectory of the wheel.

[0063] After acquiring a local two-dimensional image of the train wheel using the aforementioned device, a three-dimensional model of the train wheel out-of-roundness is constructed based on the acquired local two-dimensional image, such as... Figure 1 and Figure 6 As shown, the specific steps are as follows:

[0064] Step 100: Generate a Gaussian difference pyramid based on the local state two-dimensional image. Based on the Gaussian difference pyramid, perform spatial extreme value feature point detection, precise feature point localization, feature point orientation information matching, feature point description, and feature point matching in sequence to obtain the motion pose data of the wheel between adjacent images.

[0065] Step 101: Generate the Difference of Gaussian Pyramid (DOG Pyramid).

[0066] Gaussian blurring at different scales is applied to the two-dimensional image of the local state of the wheel. The blur template is calculated using the Gaussian function and then convolved with the original two-dimensional image of the local state of the wheel to achieve the desired blurring. The blurred image is then downsampled multiple times, with each downsampling yielding one layer of the Gaussian pyramid. Several images from each layer are combined to form a group of Gaussian pyramids. The difference between adjacent layers in each group of Gaussian pyramids is then subtracted to obtain the difference between Gaussian pyramids.

[0067] Among them, the scale-space method is more likely to obtain the essential features of the image than the traditional single-scale image information processing technology. The blurring degree of each scale image in the scale space gradually increases, which can simulate the formation process of the target on the retina when a person is moving from near to far from the target.

[0068] Step 102: Spatial extreme value feature point detection, to obtain a set of spatial extreme value feature points.

[0069] To find the DOG function extrema (spatial extrema feature points) between two adjacent layers of the Difference of Gaussian pyramid, each pixel is compared with all its neighbors, comparing the size of its neighbors in the image domain and scale domain.

[0070] In the two-dimensional image space, the central pixel is compared with 8 points in its 3*3 neighborhood. In the scale space within the same group, the central pixel is compared with 2*9 points in the two adjacent layers of the image above and below. That is, the central pixel is compared with 26 points to ensure that extreme points are detected in both the scale space and the two-dimensional image space. Finally, the spatial extreme feature point set is obtained, and the spatial extreme point detection is completed.

[0071] Step 103: Precise localization of feature points to obtain a set of spatial extreme feature points with precise localization.

[0072] Subpixel interpolation is used to interpolate the detected discrete spatial extrema points (spatial extrema feature points) to obtain continuous spatial extrema points. A three-dimensional quadratic function is used to fit the interpolated continuous spatial extrema points, and the offset of the interpolation center is calculated. When the center offset in any dimension is greater than 0.5, the position of the current feature point is changed, and interpolation is repeated at the new position. This process is iterated until convergence, and low-contrast and unstable feature points are removed, thus obtaining the precise position of the spatial extrema feature points.

[0073] Among them, the location and scale of key points are accurately determined by fitting a three-dimensional quadratic function, while removing low-contrast key points and unstable edge response points, thereby enhancing matching stability and improving noise resistance.

[0074] Step 104: Feature point orientation information matching. The stable orientation of the spatial extreme feature point set is obtained using the image gradient method.

[0075] Stable extrema are extracted at different scales to ensure the scale invariance of feature points, making the feature points invariant to image angles and rotations. The orientation assignment is achieved by calculating the gradient of each extrema.

[0076] Specifically, the set of spatial extreme feature points is traversed, and the gradient and orientation distribution features of pixels within the neighborhood window of the Gaussian pyramid image where each spatial extreme feature point is located are searched. Histograms are used to statistically analyze the gradient direction and magnitude of pixels in the neighborhood of the feature point. The histogram has 10 bars, each representing a 36° angle. The horizontal axis is the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient direction. A Gaussian function is used to smooth the gradient direction histogram to enhance the influence of neighborhood points on the feature point orientation and reduce the impact of abrupt changes. Then, parabolic interpolation is performed on the three bars closest to the main peak of the gradient direction histogram. When the main peak reaches 80%, it is used as the auxiliary orientation of the feature point to enhance the robustness of the feature point orientation information matching.

[0077] Step 105: Feature point description. A descriptor is created for each DOG function extremum point (spatial extremum feature point), that is, a set of feature vectors is used to describe this feature point so that it does not change with various factors.

[0078] Specifically, with the feature point as the center, the coordinate axis is rotated to the main direction of the feature point. An 8×8 window is selected with the main direction as the center, and each small grid is a pixel in the scale space of the feature point's neighborhood. The gradient magnitude and gradient direction of each pixel are calculated, and a Gaussian window is used to perform weighted calculations on them. Finally, a gradient histogram in 8 directions is drawn on each 4×4 small block, and the cumulative value of each gradient direction is calculated to form a seed point. Each feature point has 4 seed points, and each seed point has gradient information in 8 directions. These 4×4×8=128 gradient information are the feature vector of the feature point.

[0079] Step 106: Perform feature point matching on the spatial extreme feature point set to estimate the motion pose data of the wheel.

[0080] Specifically, the similarity is achieved by calculating the 128-dimensional Euclidean distance between two sets of spatial extreme feature points (DOG function extreme points) in the spatial extreme feature point set. The smaller the Euclidean distance, the higher the similarity. When the Euclidean distance is less than a set threshold, the matching is considered successful. By matching the feature points of each keyframe, the motion and pose relationship of the wheel are estimated. Keyframe matching features are inserted to calculate the localization of the frame. The wheel landmark points are calculated using the triangulation method. All keyframes and landmark points are allocated to estimate the motion and pose data of the wheel.

[0081] Step 200: Perform nonlinear least squares optimization and loop closure detection on the motion pose image data of the wheel in sequence to obtain globally consistent wheel motion trajectory data and wheel two-dimensional image data.

[0082] Step 201: Optimization using nonlinear least squares method.

[0083] Due to the presence of noise, the wheel motion and pose obtained in the above steps will not be an exact correspondence. Therefore, a nonlinear least squares method is used to optimize the wheel motion and pose data. First, the conditional distribution of the wheel state variables is estimated in batches using Bayes' rule to obtain the maximum likelihood estimate, i.e., the optimal wheel motion and pose estimates. Then, the wheel motion and pose estimates are substituted into the motion and observation equations of SLAM, and the PNP algorithm is used to provide the iterative initial values ​​for the wheel motion and pose estimates. Finally, the Gauss-Newton method is used to iteratively fine-tune the wheel motion and pose estimates, solving for a minimum value, completing the optimization, and obtaining locally consistent wheel motion trajectory data and two-dimensional wheel image data.

[0084] Step 202: Perform loop closure detection on the wheel motion and pose images.

[0085] Specifically, the K-means algorithm is used to cluster locally consistent wheel motion trajectory data and wheel 2D image data. Then, all clustered wheel motion and pose images are described by description vectors to ensure that the description vectors do not change. The L1 norm of each description vector is then calculated, and the similarity of each description vector is compared to define the similarity between each image. A similarity greater than 90% is considered a successful loop closure detection. The detected loop closure may be from multiple frames. The clustering algorithm is used to group similar loop closures into one class, so that the algorithm does not repeatedly detect loop closures of the same class. Finally, loop closure detection is completed, and globally consistent wheel motion trajectory data and wheel 2D image data are obtained.

[0086] Step 300: Perform monocular dense reconstruction based on wheel motion trajectory data and wheel two-dimensional image data to establish wheel spatial three-dimensional image data.

[0087] Specifically, based on wheel motion trajectory data and 2D wheel image data, the depth values ​​of the wheel motion and pose images are calculated using triangulation. The uncertainty of the depth information is calculated based on geometric relationships. Then, the current observed depth is fused into the previous estimate. Each pixel of the current wheel motion and pose image is traversed. Taking the first captured wheel pose image as a reference frame, the pixel coordinates of the reference frame are converted from pixel coordinates to 3D coordinates in the camera coordinate system. The 3D coordinates are multiplied by a rotation matrix and converted to the camera coordinate system in the current frame. Then, they are projected onto the pixel coordinates of the current frame. Considering the variance of the depth, the converted 3D coordinates in the reference frame are projected twice under the condition of maximum and minimum depth to obtain two projected coordinates. The line connecting these two points is the epipolar line to be searched. The best matching block on the epipolar line is searched using NCC. After a successful search, the depth map is updated, and the 3D image of the wheel space is completed.

[0088] In this invention, a three-dimensional monocular dense reconstruction of the wheel is performed. Some outliers are removed by a point cloud filtering algorithm. In one embodiment, the number of feature points is reduced from 7,127,546 to 6,587,342, which eliminates the problems of poor reconstruction effect and low efficiency caused by outliers. The final reconstructed three-dimensional image of the wheel is accurate and comprehensive, and the details of the wheel tread are also well displayed.

[0089] Step 400: Calculate the distance between the point cloud of the three-dimensional image of the wheel and the standard wheel point cloud to obtain the wheel out-of-roundness information.

[0090] Specifically, the wheel spatial 3D image point cloud in the wheel spatial 3D image data is transformed to the world coordinate system with the standard wheel point cloud, and then their Euclidean distance is calculated to obtain the wheel's out-of-roundness information to assist in human decision-making.

[0091] In one embodiment, the distance between the three-dimensional image point cloud of the wheel space and the standard wheel point cloud is calculated. Finally, the wheel out-of-roundness data obtained by three-dimensional mapping in polar coordinates is compared with the real data. The relative error of the sampling points is 0.06mm, and the measurement results meet the requirements.

[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting out-of-roundness of train wheels using three-dimensional mapping, wherein feature points are marked on the wheels during train operation to obtain a two-dimensional image of the local state of the wheels during train operation, characterized in that... A wheel feature point marking device is used to mark feature points on the wheel; The wheel feature point marking device includes two tread feature marking devices arranged on the outer side of the train track and two wheel flange feature marking devices installed on the inner side of the track; The feature marking device includes a feature marking device box and several feature marking pens disposed in the feature marking device box, which marks feature points on the wheel when the wheel contacts the feature marking pens. Two-dimensional images of the local state of the train wheels during operation are acquired using a monocular camera group; The monocular camera assembly is mounted on the train track and located behind the wheel feature point marking device; When the wheel passes the feature marker pen closest to the monocular camera group, the monocular camera group is activated. The monocular camera group starts to capture a two-dimensional image of the local state of the wheel during train operation. When the train has completely passed the wheel feature point marking device, the wheel has been in contact with the feature marker pen for a longer than the set time, and the monocular camera group is turned off. The real-time running speed of the train and the wheelbase of the bogie are calculated based on the time difference between the first and last feature markers that the wheel contacts, and the distance between the two feature markers. The shooting frequency of the monocular camera group is determined based on the real-time operating speed of the train. The shooting frequency of the monocular camera group must ensure that when a wheel completely passes over the wheel feature point marking device, 24 local state two-dimensional images can be captured by the monocular camera group. The number of captured local state two-dimensional images must ensure strong robustness of three-dimensional mapping while reducing the intensity of real-time image processing by the computer. The method further includes the following steps: S100. Generate a Gaussian difference pyramid based on the local state two-dimensional image. Based on the Gaussian difference pyramid, perform spatial extreme value feature point detection, precise localization of feature points, feature point orientation information matching, feature point description, and feature point matching in sequence to obtain the motion pose data of the wheel between adjacent images. S200. The nonlinear least squares method is sequentially used to optimize the motion pose image data of the wheel and to detect loop closure, so as to obtain globally consistent wheel motion trajectory data and wheel two-dimensional image data. S300: Perform monocular dense reconstruction based on wheel motion trajectory data and wheel two-dimensional image data to establish wheel spatial three-dimensional image data; S400: Calculate the distance between the 3D point cloud of the wheel space image and the standard wheel point cloud to obtain the wheel out-of-roundness information.

2. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 1, characterized in that, Before proceeding to step S100, based on the wheelbase of the train bogie, the local state two-dimensional images of different groups of wheels are captured at intervals using the same wheel, and the images are then classified, stored, and compiled into a database.

3. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 1, characterized in that, In step S100, the specific method for generating the Gaussian difference pyramid includes: performing Gaussian blurring on the two-dimensional image of the local state of the wheel at different scales, calculating the blur template using the Gaussian function, and performing a convolution operation between the template and the original two-dimensional image of the local state of the wheel to blur the two-dimensional image of the local state of the wheel, and then performing multiple downsampling operations on the blurred two-dimensional image of the local state, with each downsampling yielding one layer of the Gaussian pyramid image, and several images in each layer being combined to form a group of Gaussian pyramids, and subtracting the adjacent upper and lower layers of the Gaussian pyramid in each group to obtain the Gaussian difference pyramid.

4. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 3, characterized in that, In step S100: Spatial extreme feature point detection identifies potential scale- and rotation-invariant feature points using the difference of Gaussian pyramid and performs spatial extreme point detection to obtain a spatial extreme feature point set; precise feature point localization further filters the local extreme points detected in the scale space of the spatial extreme feature point set, i.e., accurately determines the position and scale of the feature points by fitting a three-dimensional quadratic function to the spatial extreme feature point set, while removing low-contrast feature points and unstable edge response points; feature point orientation information matching uses the image gradient method to obtain the stable orientation of the spatial extreme feature point set; feature point description establishes a descriptor for each spatial extreme feature point, which is a set of feature vectors; feature point matching estimates the motion pose data of the wheel based on the spatial extreme feature point set.

5. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 4, characterized in that, In step S200, the nonlinear least squares optimization includes: using Bayes' rule to estimate the conditional distribution of wheel state variables in batches, obtaining the maximum likelihood estimate, and obtaining better wheel motion and pose estimates; substituting the better wheel motion and pose estimates into the motion and observation equations of SLAM, using the PNP algorithm to provide iterative initial values ​​for the wheel motion and pose estimates, and finally using the Gauss-Newton method to iteratively fine-tune the wheel motion and pose estimates, solving for a minimum value, and obtaining locally consistent wheel motion trajectory data and two-dimensional wheel image data.

6. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 5, characterized in that, In step S200, loop closure detection includes: using the K-means algorithm to cluster locally consistent wheel motion trajectory data and wheel 2D image data; then describing all clustered wheel motion and pose images with description vectors; calculating the L1 norm of each description vector; comparing the similarity of each description vector; defining the similarity between each image; a similarity greater than 90% is considered a successful loop closure detection. Detected loops may be from multiple frames; clustering algorithms are used to group similar loops into one class, preventing the algorithm from repeatedly detecting loops of the same class, ultimately obtaining globally consistent wheel motion trajectory data and wheel 2D image data.

7. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 6, characterized in that, Step S300 specifically includes: calculating the depth value of the wheel motion and pose images using triangulation based on the wheel motion trajectory data and the wheel 2D image data; calculating the uncertainty of the depth information based on geometric relationships; then fusing the current observed depth into the previous estimate; traversing every pixel of the current wheel motion and pose images; using the first captured wheel pose image as a reference frame; converting the pixel coordinates of the reference frame from pixel coordinates to 3D coordinates in the camera coordinate system; multiplying the 3D coordinates by a rotation matrix and converting them to the camera coordinate system in the current frame; then projecting them onto the pixel coordinates of the current frame; and then projecting the converted 3D coordinates in the reference frame twice under the maximum and minimum depth conditions to obtain two projected coordinates. The line connecting these two points is the epipolar line to be searched. Using NCC to search for the best matching block on the epipolar line, after a successful search, updating the depth map and completing the establishment of the wheel space 3D image data.

8. The method for detecting out-of-roundness of train wheels in three-dimensional mapping according to claim 7, characterized in that, Step S400 specifically includes: transforming the wheel space three-dimensional image point cloud and the standard wheel point cloud in the wheel space three-dimensional image data to the world coordinate system, and then calculating their Euclidean distance to obtain the wheel's out-of-roundness information.

Images