Track feature registration method and device based on semantic constraint

Abstract

The invention provides a track feature registration method and device based on semantic constraint, and the method comprises the steps: carrying out the semantic segmentation of a source point cloud and a target point cloud based on track features, and determining a track semantic tag of each data point; forming a semantic source point cloud and a semantic target point cloud; according to the track semantic label, performing rough transformation on the semantic source point cloud and the semantic target point cloud to obtain a first transformation matrix; according to the first transformation matrix, performing fine transformation on the source point cloud and the target point cloud to obtain a second transformation matrix; and determining a registration transformation matrix between the source point cloud and the target point cloud. According to the method, the geometric fuzzy characteristic in the track registration process is broken through the semantic tags of the data points obtained through semantic segmentation, and the accuracy in the registration process is improved; and meanwhile, rough transformation is carried out by utilizing semantic characteristics, and fine transformation is realized by utilizing geometric characteristics on the basis, so that the registration speed is effectively improved, and the requirement of real-time track detection in train operation is met.

Description

Technical Field

[0001] This disclosure relates to the field of rail transit infrastructure detection technology, and in particular to a method, apparatus, storage medium and electronic device for track feature registration based on semantic constraints. Background Technology

[0002] In existing rail transit systems, the track serves as the fundamental carrier for train operation, and its condition directly determines the train's operational safety, comfort, and line efficiency. Especially with the widespread adoption of driverless rail train technology, achieving rapid and accurate track data acquisition, condition detection, and scene modeling is crucial for ensuring the long-term stable operation of rail transit systems. Currently, the commonly used track data acquisition method involves collecting point cloud data of the track and then using an integrated point cloud registration (ICP) algorithm to construct the track model. However, due to the high geometric similarity of parallel rails, traditional ICP algorithms are prone to high track mismatch rates, and the computational load of standard iterations is high, failing to meet the real-time requirements of track detection during train operation. Summary of the Invention

[0003] The purpose of this disclosure is to provide a method, apparatus, storage medium, and electronic device for track feature registration based on semantic constraints, so as to solve the problems existing in the prior art.

[0004] The embodiments of this disclosure adopt the following technical solution: a trajectory feature registration method based on semantic constraints, comprising: performing semantic segmentation of a source point cloud and a target point cloud based on trajectory features to determine the trajectory semantic label of each data point in the source point cloud and the target point cloud; selecting data points whose trajectory semantic labels meet preset conditions in the source point cloud and the target point cloud respectively, and forming a semantic source point cloud and a semantic target point cloud; performing a coarse transformation on the semantic source point cloud and the semantic target point cloud according to the trajectory semantic labels to obtain a first transformation matrix; performing a fine transformation on the source point cloud and the target point cloud according to the first transformation matrix to obtain a second transformation matrix; and determining the registration transformation matrix between the source point cloud and the target point cloud according to the first transformation matrix and the second transformation matrix.

[0005] This disclosure also provides a track feature registration device based on semantic constraints, comprising: a semantic segmentation module for performing semantic segmentation of a source point cloud and a target point cloud based on track features, and determining the track semantic label for each data point in the source point cloud and the target point cloud; a filtering module for filtering data points in the source point cloud and the target point cloud whose track semantic labels meet preset conditions, and forming a semantic source point cloud and a semantic target point cloud; a coarse transformation module for performing a coarse transformation on the semantic source point cloud and the semantic target point cloud according to the track semantic labels, to obtain a first transformation matrix; a fine transformation module for performing a fine transformation on the source point cloud and the target point cloud according to the first transformation matrix, to obtain a second transformation matrix; and an output module for determining the registration transformation matrix between the source point cloud and the target point cloud according to the first transformation matrix and the second transformation matrix.

[0006] This disclosure also provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described semantically constrained orbit feature registration method.

[0007] This disclosure also provides an electronic device, including at least a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the above-described semantically constrained orbit feature registration method.

[0008] The beneficial effects of this embodiment are as follows: the semantic labels of each data point obtained by semantic segmentation break the geometric ambiguity in the track registration process, thereby improving the accuracy of the registration process; at the same time, the semantic features are used for coarse transformation, and the geometric features are used to achieve fine transformation, which can effectively improve the registration speed, reduce the amount of data calculation, and meet the needs of real-time track detection during train operation. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart of the orbit feature registration method based on semantic constraints in the first embodiment of this disclosure;

[0011] Figure 2 This is a schematic diagram of the orbit feature registration device based on semantic constraints in the second embodiment of this disclosure. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0013] In existing rail transit systems, the track serves as the fundamental carrier for train operation, and its condition directly determines the train's operational safety, comfort, and line efficiency. Especially with the widespread adoption of driverless rail train technology, achieving rapid and accurate track data acquisition, condition detection, and scene modeling is crucial for ensuring the long-term stable operation of rail transit systems. Currently, the commonly used track data acquisition method involves collecting point cloud data of the track and then using an integrated point cloud registration (ICP) algorithm to construct the track model. However, due to the high geometric similarity of parallel rails, traditional ICP algorithms are prone to high track mismatch rates, and the computational load of standard iterations is high, failing to meet the real-time requirements of track detection during train operation.

[0014] To address the aforementioned issues, the first embodiment of this disclosure provides a track feature registration method based on semantic constraints. This method can be applied to track inspection vehicles, mobile track inspection instruments, and track SLAM systems. By cooperating with the aforementioned equipment to collect real-time point cloud data of train tracks, it enables track feature registration of train running tracks, and completes functions such as track model establishment and status monitoring.

[0015] Figure 1 A flowchart of a semantically constrained orbit feature registration method according to an embodiment of this disclosure is shown, which mainly includes steps S10 to S50:

[0016] S10: Perform semantic segmentation based on orbital features on the source point cloud and the target point cloud to determine the orbital semantic label for each data point in the source point cloud and the target point cloud.

[0017] S20: Select data points whose track semantic labels meet the preset conditions from the source point cloud and the target point cloud respectively, and form a semantic source point cloud and a semantic target point cloud.

[0018] In this embodiment, both the source point cloud and the target point cloud are point cloud data of the track collected by the device. They can be two consecutive frames of point cloud data to be stitched together. The purpose of registration in this embodiment is to stitch together multiple consecutive frames of point cloud data into a whole to construct the track model. To improve the accuracy of track feature registration and prevent misjudgment caused by parallel tracks with similar shapes during the registration process, this embodiment performs semantic segmentation on the point cloud to classify and describe the geometric features of the track in the point cloud, and assigns corresponding track semantic labels to data points with different geometric features in the point cloud according to the type identification results.

[0019] Specifically, the semantic labeling system in this embodiment is a highly customized and refined classification based on the geometric features of the track. It is not a general object classification system, but rather classifies the most stable, clearly defined, and least affected parts of the track structure by external environmental factors (such as water accumulation and weeds), providing high-quality input for subsequent registration steps. In this embodiment, track semantic labels include at least label categories and label weights. Label categories represent the type of track feature to which the data point belongs, including background, left railhead, right railhead, sleeper, frog, fastening system, and other track components. In practice, label IDs can also be added to the track semantic labels, with each label ID corresponding to a label category, facilitating code implementation in subsequent steps. Label weights represent the degree of influence of the label category in registration, specifically determined by the geometric stability and importance of the label category. Different label categories have different label weights. Table 1 shows an example of a track semantic labeling system; the specific label categories and suggested weight values ​​can be adjusted according to actual needs.

[0020] Table 1

[0021]

[0022] In some embodiments, before actually performing semantic segmentation, a semantic segmentation model and a three-dimensional orbital feature dataset should be constructed. The semantic segmentation model is then trained using the three-dimensional orbital feature dataset, enabling it to perform accurate and fast semantic segmentation of the source and target point clouds and output orbital semantic labels for the corresponding data points. In actual implementation, the semantic segmentation model can be built based on PointNet++, and the three-dimensional orbital feature dataset includes 3.2 × 10⁻⁶ points. 8 The data augmentation includes random point dropping and gamma transformation.

[0023] After determining the orbital semantic labels for each data point in the source and target point clouds, each data point in both clouds can be traversed to check if it meets preset conditions. Data points that meet the preset conditions are filtered out to form a semantic point cloud, while those that do not meet the preset conditions are left unprocessed. In this embodiment, to improve the speed and accuracy of registration, the selected point clouds that meet the preset conditions should have specific and clear geometric features and stability. Therefore, the preset condition can be set to "the label category is not the background," meaning that all data points with the label category of "background" do not meet the preset condition and can be filtered out from the source and target point clouds. Data points with the label categories of the remaining categories form the semantic source and semantic target point clouds.

[0024] Preferably, the semantic label also includes a label category confidence score, used to characterize the credibility of the label category obtained from semantic segmentation. After semantic segmentation is completed, the accuracy requirements of subsequent registration can be ensured by evaluating the quality of semantic segmentation. Specifically, after performing semantic segmentation based on track features on the source point cloud and the target point cloud, and determining the track semantic label for each data point in the source and target point clouds, the semantic segmentation quality of the source and target point clouds is checked to see if it meets the valid conditions. If the semantic segmentation quality does not meet the valid conditions, the current source or target point cloud is discarded, or a warning is triggered. For the discarded point cloud, semantic segmentation can be performed again, and its quality can be evaluated again, or the frame corresponding to the current point cloud can be discarded, and other adjacent frames can be used as the source or target point cloud for semantic segmentation again. In some embodiments, the valid conditions for evaluating the semantic segmentation results include: the proportion of valid data points in the point cloud is greater than 80% or the number of core feature points in the point cloud is greater than or equal to the valid threshold; wherein, valid data points are data points with a confidence score greater than 0.9, and core feature points are data points with the label category of left track head or right track head.

[0025] S30. Based on the orbital semantic labels, perform a coarse transformation on the semantic source point cloud and the semantic target point cloud to obtain the first transformation matrix.

[0026] After selecting the semantic source point cloud and semantic target point cloud, a coarse transformation is performed on them by combining the orbital semantic labels of their data points. This achieves preliminary registration based on semantic labels, resulting in the first transformation matrix. The first transformation matrix is ​​the optimal rotation and translation transformation matrix for transforming the semantic source point cloud into the semantic target point cloud. In this embodiment, only data points with practical significance are retained in the selected semantic source and semantic target point clouds. The first round of matching is performed based on these data points, which reduces the amount of data processing and eliminates background noise interference, thus improving the registration effect.

[0027] Specifically, the steps for performing a coarse transformation on the semantic source point cloud and semantic target point cloud based on the orbital semantic labels to obtain the first transformation matrix include:

[0028] S31, Based on the first search radius, determine the corresponding point in the semantic target point cloud for each data point in the semantic source point cloud to form a coarse transformation point pair; wherein, due to the large initial position deviation, the first search radius can be set to any distance between 0.1m and 0.5m to ensure the formation effect of the coarse transformation point pair;

[0029] S32, calculate the semantic weight of the coarse-transformed point pair based on the label weight and distance weight of the two data points in the pair. The label weight is the individual label weight of each data point in the pair. If both data points are left rail heads, the pair is more likely to be a correct match. If one data point is a left rail head and the other is a sleeper, the confidence level is lower. The distance weight is the distance between the two data points. The closer the two points are, the more likely they are to be a correct match, and the higher the distance weight. For example, it can be calculated using the formula exp(-d 2 / 2σ 2 ), where d represents the distance between two data points, and σ represents the standard deviation of the distance distribution of all data pairs; the product of the two label weights and one distance weight is used as the semantic weight of the coarse transformation point pair;

[0030] S33, based on all coarse transformation point pairs and their semantic weights, the current transformation matrix is ​​adjusted using singular value decomposition to determine the first optimization matrix; based on the above weighted coarse transformation point pairs, an optimal rotation and translation transformation matrix is ​​calculated using the SVD (singular value decomposition) mathematical method as the first optimization matrix, and then it is used to move the semantic source point cloud to make it closer to the semantic target point cloud; the current transformation matrix is ​​the basis for this iteration adjustment of the first optimization matrix. In the first iteration, the current transformation matrix can be the default unit matrix;

[0031] S34, using the first optimized matrix determined in the current iteration as the current transformation matrix for the next iteration, repeat the above steps until the first convergence condition is met, and use the first optimized matrix determined in the last iteration as the first transformation matrix. Typically, the optimal transformation matrix cannot be obtained through a single iteration; therefore, the above steps can be repeated to continuously optimize the transformation matrix until the first convergence condition is met. The first optimized matrix determined in the last iteration is then used as the first transformation matrix. Specifically, the first convergence condition can be that the number of coarse transformation iterations reaches a pre-set first threshold, such as 20 times, or that the change in the first optimized matrix determined in M ​​consecutive iterations is less than the first convergence threshold. In this case, continuing the iteration has little effect on optimizing the final output, and the algorithm can be considered converged. The iteration is terminated early to save computational resources. Generally, M is a positive integer less than the first preset threshold, typically 3.

[0032] S40. Based on the first transformation matrix, perform a fine transformation on the source point cloud and the target point cloud to obtain the second transformation matrix.

[0033] After the initial registration using coarse transformation, the source and target point clouds can be quickly narrowed down using the first transformation matrix. Then, a second registration using fine transformation is performed to achieve precise matching of the point cloud data in terms of geometric features. The specific steps are as follows:

[0034] S41, Based on the second search radius, determine the corresponding point in the target point cloud for each data point in the source point cloud to form a fine transformation point pair; wherein, when matching the fine transformation point pairs, it is performed on the source point cloud and the target point cloud that have not been filtered, and the fine transformation is set with a second search radius smaller than the first search radius, for example, between 0.02m and 0.1m, because after the coarse transformation, the data points in the source point cloud have been processed by the first transformation matrix, and the corresponding data points in the target point cloud should be in a relatively close position, thus eliminating residual noise and improving the matching accuracy;

[0035] S42, based on the distance weights of the two data points in the refined transform point pair and the refined transform point pair itself, singular value decomposition is used to adjust the current transform matrix to determine the second optimized matrix. It's important to note that the refined transform process is performed after the coarse transform has been completed. Therefore, in the first refined transform iteration, the current transform matrix used should be the first transform matrix to ensure the distance between the source and target point clouds is reduced, thus accelerating the matching speed and accuracy of the refined transform. Furthermore, during the refined transform iteration, it's no longer necessary to use semantic labels for weighting. When the data points of the two point clouds are very close, using semantic labels may introduce noise from the label boundaries, which could negatively impact registration accuracy. In this case, only distance weights are needed, such as the distance weight ω in the refined transform process.geo =1 / (1+α·d) 2 ), where d represents the distance between two data points, and α is a parameter that adjusts the rate of weight decrease, usually set to 100;

[0036] S43, using the second optimized matrix determined in the current iteration as the current transformation matrix for the next iteration, repeat the above steps until the second convergence condition is met, and use the second optimized matrix determined in the last iteration as the second transformation matrix. Similar to the coarse transformation, the optimal second transformation matrix cannot be obtained through only one iteration. Therefore, the transformation matrix can be continuously optimized by repeatedly executing the above steps until the second convergence condition is met, and the second optimized matrix determined in the last iteration is used as the second transformation matrix.

[0037] It is important to note that the second convergence condition can be any one of the following conditions, or in other words, if any one of the following conditions is met, the second convergence condition can be considered satisfied, and convergence is achieved. Specifically, these include:

[0038] When the number of fine transformation iterations reaches a pre-set second preset threshold, such as 30 times, its core function is to avoid infinite iterations caused by the algorithm getting stuck in local minima through hard stopping. At the same time, in order to improve the registration accuracy, the second preset threshold can be limited to at least 1.5 times the first preset threshold.

[0039] When the change in the second optimization matrix determined by N consecutive iterations is less than the second convergence threshold, N is generally a positive integer less than the second preset threshold, usually 3. At this time, the convergence is based on whether the transformation parameters are stable. If the matrix correction is extremely small, it means that the point cloud alignment state has not changed significantly, and continuing the iteration cannot improve the accuracy.

[0040] When the change in the cost function determined by N consecutive iterations is less than the third convergence threshold; specifically, the cost function is used to characterize the optimization objective of the registration process, and it can be expressed by the following formula:

[0041] E total =λ sem ·E sem +λ geo ·E geo (1)

[0042] Among them, E total E represents the cost function; sem E represents the coarse transformation matching error, meaning points with the same or similar semantic labels should be as close as possible. This error can be calculated as the difference between the target point cloud and the source point cloud after processing with the first transformation matrix. geoλ represents the fine transformation matching error, meaning that all corresponding points on the geometric shape should be as close as possible. It can be calculated by the difference between the target point cloud and the source point cloud after processing by the second transformation matrix; sem For the coarse transformation error weights, λ geo The two error weights mentioned above are adaptive adjustment parameters, specifically determined based on the current total number of iterations (i.e., the sum of the number of coarse transformation iterations and the number of fine transformation iterations). For example:

[0043]

[0044] λ geo (i)=1-λ sem (i); (3)

[0045] Where i represents the current total number of iterations, which is the sum of the coarse transformation iterations and the fine transformation iterations. In the actual iteration process, in the early stages of the iteration, the registration process is mainly dominated by semantic information, at which point λ sem The value of (i) is relatively large, indicating that the coarse transformation matching error accounts for a significant portion of the cost function. This forces the algorithm to overcome the large deviation in the initial position, avoid getting trapped in local minima, and achieve fast coarse alignment. As the number of iterations increases, λ... sem The value of (i) will continuously decrease. At this point, the optimization process is dominated by geometric information to avoid the inaccuracies (semantic noise) that may exist at the semantic label boundaries from interfering with the final high-precision result, allowing the algorithm to focus on the perfect matching of geometric details. Typically, the value of the cost function is mainly affected by the second transformation matrix output by the fine transformation. As the number of iterations increases, the magnitude of change in the cost function continuously decreases. When the change in the cost function determined by N consecutive iterations is less than the third convergence threshold, it indicates that the current solution is close to a local minimum, the optimization objective has stabilized, and the error reduction benefit of continuing iterations is negligible. At this point, the iteration can be terminated to save computational resources.

[0046] It should be noted that all preset values ​​involved in this embodiment, including but not limited to the first search radius, the second search radius, the first preset threshold, the first convergence threshold, the second preset threshold, the second convergence threshold, and the third convergence threshold, can be reasonably set in combination with the actual registration accuracy requirements. This embodiment does not impose any specific restrictions.

[0047] S50, determine the registration transformation matrix between the source point cloud and the target point cloud based on the first transformation matrix and the second transformation matrix.

[0048] After determining the first transformation matrix and the second transformation matrix, their matrix product is used as the registration transformation matrix between the source point cloud and the target point cloud, i.e.:

[0049] T opt =ΔT sem ·ΔT geo ;

[0050] Among them, T opt For the registration transformation matrix, ΔT sem Let ΔT be the first transformation matrix. geo This is the second transformation matrix.

[0051] This embodiment utilizes the semantic labels of each data point obtained by semantic segmentation to break the geometric ambiguity in the track registration process, thereby improving the accuracy of the registration process. At the same time, it uses semantic features for coarse transformation and then uses geometric features to achieve fine transformation, which can effectively improve the registration speed, reduce the amount of data computation, and meet the needs of real-time track detection during train operation.

[0052] Based on the same inventive concept, the second embodiment of this disclosure provides a track feature registration device based on semantic constraints, the structural schematic diagram of which is shown below. Figure 2 As shown, it mainly includes: a semantic segmentation module 10, used to perform semantic segmentation of the source point cloud and the target point cloud based on orbital features, and determine the orbital semantic label of each data point in the source point cloud and the target point cloud; a filtering module 20, used to filter out data points in the source point cloud and the target point cloud whose orbital semantic labels meet preset conditions, and form a semantic source point cloud and a semantic target point cloud; a coarse transformation module 30, used to perform a coarse transformation on the semantic source point cloud and the semantic target point cloud according to the orbital semantic labels, and obtain a first transformation matrix; a fine transformation module 40, used to perform a fine transformation on the source point cloud and the target point cloud according to the first transformation matrix, and obtain a second transformation matrix; and an output module 50, used to determine the registration transformation matrix between the source point cloud and the target point cloud according to the first transformation matrix and the second transformation matrix.

[0053] Specifically, the track semantic labels include at least: label category and label weight; the label categories include at least: background, left rail head, right rail head, sleeper, frog, fastening system and other track components; different label categories have different label weights; the preset condition includes: the label category is not background.

[0054] In some embodiments, the coarse transformation module 30 is specifically used to: determine the corresponding point in the semantic target point cloud for each data point in the semantic source point cloud according to the first search radius, forming a coarse transformation point pair; calculate the semantic weight of the coarse transformation point pair according to the label weight and distance weight of the two data points in the coarse transformation point pair; adjust the current transformation matrix using singular value decomposition according to all coarse transformation point pairs and their semantic weights to determine the first optimization matrix; use the first optimization matrix determined in the current iteration as the current transformation matrix for the next iteration, repeat the above steps until the first convergence condition is met, and use the first optimization matrix determined in the last iteration as the first transformation matrix; wherein, the first convergence condition includes: the number of coarse transformation iterations reaches a first preset threshold, or the change in the first optimization matrix determined in M ​​consecutive iterations is less than the first convergence threshold.

[0055] In some embodiments, the fine transformation module 40 is specifically used to: determine the corresponding point in the target point cloud for each data point in the source point cloud according to the second search radius, forming a fine transformation point pair, wherein the second search radius is smaller than the first search radius; adjust the current transformation matrix using singular value decomposition to determine the second optimization matrix according to the distance weight of the two data points in the fine transformation point pair and the fine transformation point pair, wherein the current transformation matrix is ​​the first transformation matrix in the first iteration; repeat the above steps with the second optimization matrix determined in the current iteration as the current transformation matrix for the next iteration until the second convergence condition is met, and use the second optimization matrix determined in the last iteration as the second transformation matrix; wherein the second convergence condition includes any one of the following: the number of fine transformation iterations reaches a second preset threshold, the change in the second optimization matrix determined in N consecutive iterations is less than the second convergence threshold, and the change in the cost function determined in N consecutive iterations is less than the third convergence threshold.

[0056] In some embodiments, the cost function is expressed based on the following formula:

[0057] E total =λ sem ·E sem +λ geo ·E geo (1)

[0058]

[0059] λ geo (i)=1-λ sem (i); (3)

[0060] Among them, E total Let E represent the cost function. sem E represents the coarse transform matching error. geo λ represents the fine transform matching error. semFor the coarse transformation error weights, λ geo The error weight for the fine transformation is denoted as i, which is the current total number of iterations, equal to the sum of the number of iterations for the coarse transformation and the number of iterations for the fine transformation.

[0061] In some embodiments, the semantic segmentation module 10 is further configured to: construct a semantic segmentation model and a three-dimensional orbital feature dataset; train the semantic segmentation model using the three-dimensional orbital feature dataset; and use the trained semantic segmentation model to perform semantic segmentation of the source point cloud and the target point cloud based on orbital features.

[0062] In some embodiments, the semantic label also includes label category confidence; the filtering module 20 is further configured to: detect whether the semantic segmentation quality of the semantic source point cloud and the semantic target point cloud meets the valid conditions respectively, and discard the current semantic source point cloud or the current semantic target point cloud or trigger a warning if the semantic segmentation quality does not meet the valid conditions; wherein, the valid conditions include: the proportion of valid data points in the point cloud is greater than 80% or the number of core feature points in the point cloud is greater than or equal to the valid threshold; valid data points are data points with a confidence score greater than 0.9, and core feature points are data points with a label category of left rail head or right rail head.

[0063] This embodiment utilizes the semantic labels of each data point obtained by semantic segmentation to break the geometric ambiguity in the track registration process, thereby improving the accuracy of the registration process. At the same time, it uses semantic features for coarse transformation and then uses geometric features to achieve fine transformation, which can effectively improve the registration speed, reduce the amount of data computation, and meet the needs of real-time track detection during train operation.

[0064] Based on the same inventive concept, the third embodiment of this disclosure provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the orbit feature registration method based on semantic constraints described in the first embodiment of this disclosure.

[0065] Based on the same inventive concept, the fourth embodiment of this disclosure provides an electronic device, including at least a memory and a processor. The memory stores a computer program. The processor, when executing the computer program in the memory, implements the steps of the semantically constrained track feature registration method described in the first embodiment of this disclosure. This electronic device can actually be a track inspection vehicle, a mobile track inspection instrument, or a track SLAM system, etc.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for track feature registration based on semantic constraints, characterized in that, The method comprises the following steps: performing track feature-based semantic segmentation on a source point cloud and a target point cloud to determine a track semantic label of each data point in the source point cloud and the target point cloud; filtering out data points with track semantic labels meeting preset conditions from the source point cloud and the target point cloud respectively to form a semantic source point cloud and a semantic target point cloud; performing coarse transformation on the semantic source point cloud and the semantic target point cloud according to the track semantic labels to obtain a first transformation matrix; performing fine transformation on the source point cloud and the target point cloud according to the first transformation matrix to obtain a second transformation matrix; determining a registration transformation matrix between the source point cloud and the target point cloud according to the first transformation matrix and the second transformation matrix.

2. The track feature registration method of claim 1, wherein, The track semantic label at least comprises a label category and a label weight; the label category at least comprises background, left rail head, right rail head, sleeper, frog, fastener system and other track components; different label categories have different label weights; the preset condition comprises that the label category is not background.

3. The track feature registration method of claim 2, wherein, The coarse transformation on the semantic source point cloud and the semantic target point cloud according to the track semantic labels to obtain the first transformation matrix comprises: determining a corresponding point of each data point in the semantic source point cloud in the semantic target point cloud according to a first search radius to form a coarse transformation point pair; calculating a semantic weight of the coarse transformation point pair according to the label weight and distance weight of two data points in the coarse transformation point pair; adjusting a current transformation matrix by singular value decomposition according to all the coarse transformation point pairs and their semantic weights to determine a first optimization matrix; taking the first optimization matrix determined in the current iteration as the current transformation matrix in the next iteration, and repeating the above steps until a first convergence condition is met, and taking the first optimization matrix determined in the last iteration as the first transformation matrix; wherein the first convergence condition comprises that the number of coarse transformation iterations reaches a first preset threshold, or the variation of the first optimization matrix determined in M consecutive iterations is less than a first convergence threshold.

4. The track feature registration method of claim 3, wherein, The fine transformation on the source point cloud and the target point cloud according to the first transformation matrix to obtain the second transformation matrix comprises: determining a corresponding point of each data point in the source point cloud in the target point cloud according to a second search radius to form a fine transformation point pair, wherein the second search radius is smaller than the first search radius; adjusting a current transformation matrix by singular value decomposition according to the distance weight of two data points in the fine transformation point pair and the fine transformation point pair, wherein in the first iteration, the current transformation matrix is the first transformation matrix; taking the second optimization matrix determined in the current iteration as the current transformation matrix in the next iteration, and repeating the above steps until a second convergence condition is met, and taking the second optimization matrix determined in the last iteration as the second transformation matrix; The second convergence condition comprises any one of the following: the number of iterations of the fine transformation reaches a second preset threshold, the variation of the second optimization matrix determined by N consecutive iterations is less than a second convergence threshold, and the variation of the cost function determined by N consecutive iterations is less than a third convergence threshold.

5. The track feature registration method of claim 4, wherein, The cost function is represented based on the following formula: E total = λ sem · E sem + λ geo · E geo ; (1) λ geo (i) = 1 - λ sem (i); (3) where E total represents the cost function, E sem represents the coarse transform matching error, E geo represents the fine transform matching error, λ sem is the coarse transform error weight, λ geo is the fine transform error weight, i is the current total iteration number, which is the sum of the coarse transform iteration number and the fine transform iteration number.

6. The track feature registration method of any one of claims 2 to 5, wherein, Before performing the track feature-based semantic segmentation on the source point cloud and the target point cloud, the method further comprises: constructing a semantic segmentation model and a track three-dimensional feature dataset; training the semantic segmentation model by using the track three-dimensional feature dataset, and performing the track feature-based semantic segmentation on the source point cloud and the target point cloud by using the trained semantic segmentation model.

7. The track feature registration method of claim 6, wherein, The semantic label further comprises a label category confidence; after performing the track feature-based semantic segmentation on the source point cloud and the target point cloud and determining the track semantic label of each data point in the source point cloud and the target point cloud, the method further comprises: respectively detecting whether the semantic segmentation quality of the semantic source point cloud and the semantic target point cloud meets an effective condition, and discarding the current semantic source point cloud or the current semantic target point cloud or triggering a warning in a case where the semantic segmentation quality does not meet the effective condition; The effective condition comprises that the proportion of effective data points in the point cloud is greater than 80% or the number of core feature points in the point cloud is greater than or equal to an effective threshold; the effective data point is a data point with a confidence greater than 0.9, and the core feature point is a data point with a label category of left rail head or right rail head.

8. A device for registering track features based on semantic constraints, characterized in that comprises: a semantic segmentation module configured to perform track feature-based semantic segmentation on a source point cloud and a target point cloud, and determine a track semantic label of each data point in the source point cloud and the target point cloud; a screening module configured to screen out data points with a track semantic label meeting a preset condition from the source point cloud and the target point cloud respectively, and form a semantic source point cloud and a semantic target point cloud; a fine transformation module configured to perform fine transformation on the semantic source point cloud and the semantic target point cloud according to the track semantic label, and obtain a first transformation matrix; a fine transformation module configured to perform fine transformation on the source point cloud and the target point cloud according to the first transformation matrix, and obtain a second transformation matrix; an output module configured to determine a registration transformation matrix between the source point cloud and the target point cloud according to the first transformation matrix and the second transformation matrix.

9. A storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the track feature registration method based on semantic constraints in any one of claims 1 to 7.

10. An electronic device comprising at least a memory, a processor, said memory having stored thereon a computer program, characterized in that, The processor implements the steps of the track feature registration method based on semantic constraints in any one of claims 1 to 7 when executing the computer program on the memory.

Images