Inspection robot semantic information positioning enhancement method, device and equipment and storage medium

Abstract

The invention provides an inspection robot semantic information positioning enhancement method and device, equipment and a storage medium. Relates to the technical field of robot positioning. The method comprises the following steps: acquiring three-dimensional point cloud data; segmenting the three-dimensional point cloud data by using a Point Net + + network to obtain an axle point cloud; acquiring axle center line data based on the axle point cloud, fitting the axle center line data to determine the axle center line position, and determining the axle position according to the axle center line position and odometer data. According to the method, the problem that the position information of the key component cannot be accurately obtained in a complex vehicle bottom environment is solved.

Description

Technical Field

[0001] The present application relates to the field of robot positioning technology, and in particular to a method, device, equipment and storage medium for enhancing semantic information positioning of an inspection robot. Background Art

[0002] The operation and maintenance of urban rail transit is of vital importance. It is not only related to the efficient operation of the urban transportation network, but also directly related to the safety and comfort experience of passengers. With the continuous expansion of the scale of urban rail transit networks, the high-load operation of vehicles and the continuous growth of passenger flow have put forward higher requirements for operation and maintenance guarantee work. The introduction of intelligent technologies, such as big data, cloud computing, the Internet of Things, and artificial intelligence, has become a key means to improve operation and maintenance efficiency and ensure operational safety. According to existing data, the parking error of subway trains in parking and train inspection depots is usually about 1 meter. This uncertainty in the location of each stop, especially in the case of complex undercarriage environment, is very unfavorable to the normal operation and maintenance of trains.

[0003] In the above background, the research on inspection robots based on laser SLAM is one of the current research hotspots. The precise positioning of inspection points by inspection robots is crucial in maintenance operations. However, under the complex conditions of the vehicle bottom environment, the original point cloud data alone is not enough to determine the specific parking position of the train. This not only reduces the inspection efficiency, but also may cause the risk of the mechanical arm of the maintenance robot colliding with the raised part of the bottom of the train due to the inability to accurately obtain key component information. On the other hand, the train carriages are softly connected through the through-channel device, making it difficult to determine the precise position of the entire train only by the position of a single carriage. Therefore, during the robot inspection process, the position of the train stop on the inspection channel must be accurately located in order to safely carry out the inspection operation.

[0004] The commonly used RANSAC point cloud segmentation technology can only extract point clouds with relatively regular shapes, and its application is limited when the point clouds are sparse or the scenes are complex. Through preliminary research and testing, it is impossible to fit the correct axle point cloud using only the RANSAC algorithm, which is unacceptable in practical applications. Traditional SLAM methods have limitations in point cloud processing and fail to fully utilize the potential of point cloud information. Summary of the invention

[0005] The present application provides a method, device, equipment and storage medium for enhancing semantic information positioning of an inspection robot to solve the problem of being unable to accurately obtain the location information of key components in a complex undercarriage environment. By introducing semantic information, point cloud information can be more fully utilized, especially the detection of the location of train parking points in a complex undercarriage environment. Therefore, the introduction of the undercarriage positioning method can enable the robot to accurately position the train above the inspection channel, thereby improving the safety and efficiency of the inspection operation. In this way, a series of challenges brought about by the uncertainty of the train stop position can be effectively solved, ensuring the efficient and safe operation of the inspection robot.

[0006] In a first aspect, the present application provides a method for enhancing semantic information positioning of an inspection robot, comprising:

[0007] Obtain 3D point cloud data;

[0008] Using the PointNet++ network to segment the three-dimensional point cloud data to obtain the axle point cloud;

[0009] Axle centerline data is acquired based on the axle point cloud, the axle centerline data is fitted to determine the axle centerline position, and the axle position is determined according to the axle centerline position and odometer data.

[0010] In a possible design, three-dimensional point cloud data is obtained, including:

[0011] Use the solid-state 3D laser radar carried by the inspection robot to collect field data to obtain point cloud data;

[0012] Based on the set region of interest, the point cloud data is subjected to spatial coordinate screening to obtain three-dimensional point cloud data.

[0013] In a possible design, the three-dimensional point cloud data is segmented using a PointNet++ network to obtain an axle point cloud, including:

[0014] After receiving the three-dimensional point cloud data accumulated for a set time, performing ROI segmentation on the accumulated three-dimensional point cloud data to obtain an original point cloud, and inputting the original point cloud into the PointNet++ network;

[0015] The PointNet++ network responds to the input original point cloud and selects a set of centroid points of local areas from the original point cloud using the farthest point sampling algorithm;

[0016] Taking the centroid point as the center, searching for neighboring points around the centroid point to form a local area set;

[0017] Feature extraction is performed on the local area set to obtain a feature vector as the axle point cloud; wherein, when feature extraction is performed on the local area set, the distribution characteristics of the point cloud are adapted by extracting feature vectors at different levels.

[0018] In a possible design, acquiring axle centerline data based on the axle point cloud, and fitting the axle centerline data to determine the axle centerline position includes:

[0019] Set the number of iterations and error threshold;

[0020] A subset is randomly selected from the axle point cloud, and a target model is fitted; wherein the distance from any point P on the surface of the target model to the axis is always equal to the radius, expressed as:

[0021] (x-x0) 2 +(y-y0) 2 +(z-z0) 2 +[a(x-x0)+b(y-y0)+c(z-z0)] 2

[0022] =r 2

[0023] Where x, y, z are the x-coordinate, y-coordinate and z-coordinate of any point on the surface of the target model, r is the radius of the bottom circle, x0, y0, z0 are the x-coordinate, y-coordinate and z-coordinate on the axis, a, b, c represent the x, y, z axis vectors;

[0024] Calculate the error between the target model and all points, and classify the points with errors less than a threshold as an inlier set;

[0025] If the number of inlier sets is greater than the preset value, the target model is updated.

[0026] When the number of iterations is reached or the target model converges, the axis x position and radius of the target model are output.

[0027] In one possible design, the error between the target model and all points is calculated by the following error equation:

[0028] f=(x-x0) 2 +(y-y0) 2 +(z-z0) 2 +

[0029] [a(x-x0)+b(y-y0)+c(z-z0)] 2 -r 2

[0030] In the formula, f represents the error;

[0031] If the number of internal point sets is greater than the preset value, the optimization target model is updated with the goal of minimizing s; where s is the result of the derivative calculation of the error equation, and the calculation formula is:

[0032]

[0033] Where f0 is the initial value of the error equation.

[0034] In a possible design, after determining the axle position according to the axle centerline position and the odometer data, the method further includes performing navigation error correction based on a correction coefficient of dynamically adjusting the navigation error; wherein the correction coefficient is calculated as:

[0035]

[0036] Where c is the correction coefficient and d is the distance between the axle and the robot.

[0037] In a second aspect, the present application provides a semantic information positioning enhancement device for an inspection robot, comprising:

[0038] A data acquisition module is configured to acquire three-dimensional point cloud data;

[0039] A semantic segmentation module is configured to segment the three-dimensional point cloud data using a PointNet++ network to obtain an axle point cloud;

[0040] The axle positioning module is configured to obtain axle centerline data based on the axle point cloud, fit the axle centerline data to determine the axle centerline position, and determine the axle position according to the axle centerline position and odometer data.

[0041] In a third aspect, the present application provides an electronic device comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the inspection robot semantic information positioning enhancement method described in the first aspect and various possible designs of the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer execution instructions. When a processor executes the computer execution instructions, the inspection robot semantic information positioning enhancement method described in the first aspect and various possible designs of the first aspect is implemented.

[0043] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the inspection robot semantic information positioning enhancement method as described in the first aspect and various possible designs of the first aspect.

[0044] The inspection robot semantic information positioning enhancement method, device, equipment and storage medium provided in this application introduces three-dimensional point cloud semantic information extraction, improves the accuracy of axle positioning, achieves centimeter-level positioning accuracy, improves the working safety and efficiency of the inspection robot, and at the same time does not require redundant hardware to locate the vehicle body above the robot, simplifies the robot hardware structure and reduces costs. Through the deep learning characteristics of the PointNet++ network, it is possible to extract high-precision axle point clouds in complex vehicle bottom environments, which has significant performance improvements over the traditional RANSAC algorithm. By removing the normalization operation and adding the resampling algorithm, the problem that the PointNet++ network output data cannot be directly used for positioning is solved, and a dense axle point cloud is obtained, which is convenient for later fitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] Figure 1 A flowchart of a method for enhancing semantic information positioning of an inspection robot provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of hierarchical feature extraction of PointNet++ provided in an embodiment of the present application;

[0048] Figure 3 IOU data diagram of different Epochs provided in the embodiment of this application;

[0049] Figure 4 A graph showing the semantic segmentation time test results provided by an embodiment of the present application;

[0050] Figure 5 A flow chart of RANSAC fitting of semi-cylindrical point cloud provided in an embodiment of the present application;

[0051] Figure 6 A flowchart of a three-dimensional semantic navigation error correction experiment provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of a semantic information positioning enhancement device for an inspection robot provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0054] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0056] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0057] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0058] The embodiment of the present application also provides a method for enhancing the semantic information positioning of an inspection robot. It is intended to achieve the positioning of train axles. The shortcomings of traditional SLAM technology in point cloud analysis limit its capabilities, and the present invention introduces semantic information to more effectively mine the value of point cloud data, especially in detecting the parking position of trains in complex undercarriage environments. The main reason for using the axle as a segmentation target is that the components of a single carriage undercarriage are rigidly connected. By accurately obtaining the position of the axle, the position of the remaining components relative to the inspection robot can be solved.

[0059] Figure 1 This is a flow chart of the inspection robot semantic information positioning enhancement method provided in the embodiment of the present application. Figure 1 As shown, the inspection robot semantic information positioning enhancement method includes:

[0060] S1. Obtain three-dimensional point cloud data.

[0061] In this embodiment, LIVOX-MID 70 solid-state laser radar is used to collect point cloud data. Due to the lack of public data sets, field data collection is used to build it. In order to improve the feasibility and processing efficiency of point cloud segmentation, the data is preprocessed to filter out the point cloud of the area of ​​interest.

[0062] For example, in a subway parking inspection depot, a solid-state 3D laser radar carried by a robot was used to collect field data. In order to improve the availability and processing efficiency of the data, we adopted the straight-through filtering technology. By defining the region of interest (ROI), the point cloud data was filtered for spatial coordinates, and only the data points with x-axis coordinates within 5 meters, y-axis coordinates within ±0.6 meters, and z-axis within the range of 1-1.8 meters were retained. This screening strategy not only ensures the relevance and accuracy of the data, but also significantly reduces the computing resources required for subsequent processing.

[0063] In the absence of datasets in the subway undercarriage environment, we independently created annotated datasets through CloudCompare to meet research needs. Based on data augmentation, a total of 300 point cloud data were obtained, including 240 for training sets, 30 for validation sets, and 30 for test sets.

[0064] S2. Use the PointNet++ network to segment the three-dimensional point cloud data and obtain the axle point cloud.

[0065] In this embodiment, the PointNet++ network segments the 3D point cloud and extracts the axle point cloud. The network uses deep learning technology to extract global features of the point cloud, especially in the complex environment of the subway bogie axle, which is easy to identify and has less occlusion, which helps to determine the relative position of the inspection points.

[0066] For example, the PointNet++ network is trained using PyTorch, an open source framework widely used in the field of deep learning. PointNet++ is an improved version of PointNet. PointNet++ draws on the layered receptive field concept of convolutional neural network (CNN) and gradually enriches the information processed by each unit by expanding the perception range of each processing unit layer by layer.

[0067] The specific process of the network is as follows: after receiving the radar 3D point cloud accumulated for 1 second, the ROI is segmented and then input into the network.

[0068] Sampling: Use the Farthest Point Sampling (FPS) algorithm to select a set of centroid points of local areas from the original point cloud.

[0069] Grouping: Taking the centroid point obtained by the sampling layer as the center, the Ball Query algorithm is used to find the "neighboring" points around the centroid point to form a local area set.

[0070] Feature extraction: Use the PointNett++ network to extract features from the local region set and encode the local region pattern into a feature vector.

[0071] In addition, in order to solve the problem of uneven sampling density of point cloud data, the Multi-resolution Grouping (MRG) algorithm is used: a computationally more efficient strategy that adapts to the distribution characteristics of the point cloud by extracting feature vectors at different levels, avoiding the high computational cost of feature extraction of large-scale neighborhoods at the lowest level. Figure 2 shown.

[0072] The PointNet++ network uses normalization technology when processing point cloud data, which helps convergence during the optimization process and makes it easier for the model to find the global optimal solution. However, the normalized point cloud data loses the original spatial position information, making it difficult to directly use it for subsequent precise fitting. In addition, since PointNet++ only processes 4096 points in a single prediction, the resulting point cloud data is relatively sparse and insufficient for effective axle fitting through the RANSAC algorithm.

[0073] In order to solve these problems, the output of the original PointNet++ was improved. First, the normalization step was removed to retain the original coordinate information of the point cloud. Then, a batch processing algorithm was introduced. Since the LIVOX-MID 70 radar used in the embodiment of the present application has a petal-shaped non-repetitive scanning characteristic, the point cloud density in each area is different. When the radar data is collected at a distance of 4 meters from the axle, about 1000-2000 points on the axle of the accumulated point cloud in one second, and the total number of points after ROI segmentation is about 30,000, so each index point is allocated times, and the number of points in each batch is less than 4096 points, which can ensure that each point is classified and processed to generate a dense axle point cloud. This improvement not only restores the spatial information of the point cloud, but also increases the density of the point cloud, thereby providing a more sufficient data basis for subsequent axle fitting. The hardware and software configuration of the semantic segmentation training system is shown in Table 1.

[0074] Table 1. Software and hardware configuration of semantic segmentation training system

[0075]

[0076] The embodiment of this application uses the ACC accuracy and IOU commonly used in segmentation networks to evaluate the performance of the model. As shown in Table 2, the accuracy of PointNet++ is 99.786%, and the IOU is 97.282%. This result shows that the PointNet++ network meets the accuracy requirements of point cloud semantic segmentation when processing point cloud segmentation tasks. The IOU data of different rounds are as follows: Figure 3As shown, considering the system efficiency, the point cloud semantic segmentation time is tested. Figure 4 shown.

[0077] Table 2 Semantic segmentation training results

[0078]

[0079] S3. Acquire axle centerline data based on the axle point cloud, fit the axle centerline data to determine the axle centerline position, and determine the axle position according to the axle centerline position and the odometer data.

[0080] In this embodiment, the RANSAC method is used to fit the axle centerline and determine the axle position in combination with the odometer data. The RANSAC method estimates the axle centerline position in data containing outliers through an iterative process.

[0081] Exemplarily, in order to obtain the data of the axle centerline x, the axle point cloud with label 1 is extracted in the previous step S2, and the segmented point cloud data containing semantic information is fitted by the random sampling consensus algorithm (RANSAC), and the noise and outliers in the point cloud are removed to obtain the axle centerline data at the same time. The RANSAC algorithm is based on the assumption that the data set contains both internal points that conform to the model and external points that deviate from the model. The target model is estimated from the observed data containing outliers through an iterative process. The target model is used to characterize the axle. The axle is generally a cylindrical structure. For a cylindrical structure, the distance from any point P on its surface to the axis is always equal to the radius r. Let (x, y, z) be any point on the surface of the cylinder, r be the radius of the bottom circle, (x0, y0, z0) be a point on the axis, and (a, b, c) be the axis vector. This property can be expressed by a general expression:

[0082] (x-x0) 2 +(y-y0) 2 +(z-z0) 2 +[a(x-x0)+b(y-y0)+c(z-z0)] 2

[0083] =r 2

[0084] The parameter value is solved by establishing an error equation, where the error equation is expressed as:

[0085] f=(x-x0) 2 +(y-y0) 2 +(z-z0) 2 +

[0086] [a(x-x0)+b(y-y0)+c(z-z0)] 2 -r 2

[0087] By taking the derivative of the error equation, we get:

[0088]

[0089] Where f0 is the initial value of the error equation. The optimal model is obtained by minimizing s.

[0090] The specific steps of using the RANSAC method to solve the parameters of the semi-cylinder in the point cloud are as follows: Figure 5 As shown:

[0091] The specific steps are as follows:

[0092] Step 1. Initialize parameters, set the number of iterations and error threshold.

[0093] Step 2. Randomly select a subset from the point cloud data and fit the target model.

[0094] Step 3. Calculate the error between the model and all points, and classify the points whose error is less than the threshold as the inliers.

[0095] Step 4. If the number of inlier sets is greater than a preset value, update the target model.

[0096] Step 5. Repeat steps 2 to 4 above until the number of iterations is reached or the model converges.

[0097] Step 6. Output the target model axis x position and radius.

[0098] To prove the feasibility and progress of the present application, the present application embodiment also carried out axle positioning test and verification.

[0099] By testing the data collected on site, four point cloud datasets at different locations were selected, and the axle point cloud was segmented using the PointNet++ network.

[0100] After segmentation, the distance between the axle centerline and the robot coordinate system is calculated to achieve accurate detection of the axle position. The axle positioning results are shown in Table 3.

[0101] Table 3 Axle positioning test of the actual point cloud collected from the bottom of the subway train

[0102]

[0103]

[0104] Table 3 shows the experimental results of the repeated run accuracy test. The robot detects the axle at various positions and records the distance of the axle relative to the robot along the inspection channel. Since it is difficult to directly obtain the true value of the distance between the robot and the axle in the inspection channel, the average of the four measurement results is used as the reference true value.

[0105] Through the method provided in the embodiments of the present application, the present application improves the accuracy of axle positioning and provides technical support for the maintenance and inspection of trains.

[0106] In some embodiments, the inspection robot semantic information positioning enhancement method further includes step S4, navigation error correction.

[0107] According to the axle positioning test and verification conducted as above, it is proved that this embodiment can accurately locate the axle, but inspection in a narrow channel with repeated features (pillars and inspection channel walls) is a major challenge for the robot's SLAM technology, and its accuracy depends not only on the careful design of the algorithm, but also on the processing performance of the control unit. Especially when the robot is under high temperature and harsh working conditions, the accuracy of its positioning and navigation will be more challenged, which affects the convergence accuracy of the SLAM algorithm and leads to the accuracy of the robot's navigation distance measurement between standard carriages. Table 4 is a comparison of the true value and the measured value of the wheel spacing. It can be seen that the SLAM scale wheel spacing error is large in the inspection channel environment, which puts higher requirements on the navigation algorithm.

[0108] Table 4 Comparison between the true value and the measured value of wheel spacing

[0109]

[0110]

[0111] Due to the small field of view of solid-state laser radar, the axle positioning accuracy is highest when the laser radar is 3-4 meters away from the axle through field test experience data. The navigation compensation algorithm is used in a simulated inspection channel scene (the two axles are 10 meters apart) to improve the navigation positioning accuracy of the inspection robot. By detecting the axle, the robot can calculate the error based on the distance traveled and correct it in time, and can design a correction coefficient based on this error, which makes navigation based on the three-dimensional point cloud more accurate. The specific inspection process is as follows: Figure 6 shown.

[0112] When the robot is directly under the first axle, let it travel 7 meters and collect static point cloud data for 1 second. Then, use the semantic segmentation technology of 3D point cloud to identify the axle. By analyzing the labels provided by the algorithm, confirm whether the axle is successfully identified. Once the axle label is detected, the RANSAC algorithm is used to fit the identified axle point cloud, and then the radius r of the axle and the distance d between the axle and the robot are measured. The effective range of the axle radius r should be between 0.07 and 0.11 meters. If the measured value fits this range, it is determined that the axle has been accurately extracted. Subsequently, the distance d is used to reversely calculate the actual forward distance of the robot, which is (10-d) meters. Finally, a correction coefficient for dynamically adjusting the navigation error is constructed:

[0113]

[0114] The robot was controlled to move forward d·c meters to reach the position directly below the second axle, and the experiment was repeated five times. The results were recorded in Table 5.

[0115] Table 5 Three-dimensional semantic navigation error correction experiment table Unit (meter)

[0116]

[0117] Analyzing the data, the robot’s positioning error using 3D semantic information within 10 meters is no more than 0.014 meters. With improved positioning accuracy, 3D semantic information is more flexible and adaptable.

[0118] Figure 7 This is a schematic diagram of the structure of the inspection robot semantic information positioning enhancement device provided in the embodiment of the present application. The embodiment of the present application also provides an inspection robot semantic information positioning enhancement device. Figure 7 As shown, the inspection robot semantic information positioning enhancement device includes:

[0119] A data acquisition module 701 is configured to acquire three-dimensional point cloud data;

[0120] The semantic segmentation module 702 is configured to segment the three-dimensional point cloud data using a PointNet++ network to obtain an axle point cloud;

[0121] The axle positioning module 703 is configured to obtain axle centerline data based on the axle point cloud, fit the axle centerline data to determine the axle centerline position, and determine the axle position according to the axle centerline position and odometer data.

[0122] In some embodiments, the data acquisition module is further configured to collect field data using a solid-state three-dimensional laser radar carried by the inspection robot to obtain point cloud data;

[0123] Based on the set region of interest, the point cloud data is subjected to spatial coordinate screening to obtain three-dimensional point cloud data.

[0124] In some embodiments, the semantic segmentation module is further configured to:

[0125] After receiving the three-dimensional point cloud data accumulated for a set time, performing ROI segmentation on the accumulated three-dimensional point cloud data to obtain an original point cloud, and inputting the original point cloud into the PointNet++ network;

[0126] The PointNet++ network responds to the input original point cloud and selects a set of centroid points of local areas from the original point cloud using the farthest point sampling algorithm;

[0127] Taking the centroid point as the center, searching for neighboring points around the centroid point to form a local area set;

[0128] Feature extraction is performed on the local area set to obtain a feature vector as the axle point cloud; wherein, when feature extraction is performed on the local area set, the distribution characteristics of the point cloud are adapted by extracting feature vectors at different levels.

[0129] In some embodiments, the axle positioning module is further configured to:

[0130] Set the number of iterations and error threshold;

[0131] A subset is randomly selected from the axle point cloud, and a target model is fitted; wherein the distance from any point P on the surface of the target model to the axis is always equal to the radius, expressed as:

[0132] (x-x0) 2 +(y-y0) 2 +(z-z0) 2 +[a(x-x0)+b(y-y0)+c(z-z0)] 2

[0133] =r 2

[0134] Where x, y, z are the x-coordinate, y-coordinate and z-coordinate of any point on the surface of the target model, r is the radius of the bottom circle, x0, y0, z0 are the x-coordinate, y-coordinate and z-coordinate on the axis, a, b, c represent the x, y, z axis vectors;

[0135] Calculate the error between the target model and all points, and classify the points with errors less than a threshold as an inlier set;

[0136] If the number of inlier sets is greater than the preset value, the target model is updated.

[0137] When the number of iterations is reached or the target model converges, the axis x position and radius of the target model are output.

[0138] In some embodiments, the axle positioning module is further configured to calculate the error between the target model and all points by the following error equation:

[0139] f=(x-x0) 2 +(y-y0) 2 +(z-z0) 2 +

[0140] [a(x-x0)+b(y-y0)+c(z-z0)] 2 -r 2

[0141] In the formula, f represents the error;

[0142] If the number of internal point sets is greater than the preset value, the optimization target model is updated with the goal of minimizing s; where s is the result of the derivative calculation of the error equation, and the calculation formula is:

[0143]

[0144] Where f0 is the initial value of the error equation.

[0145] In some embodiments, the device further includes an error correction module, wherein the error correction module is configured to perform navigation error correction based on a correction coefficient of dynamically adjusted navigation error; wherein the calculation formula of the correction coefficient is:

[0146]

[0147] Where c is the correction coefficient and d is the distance between the axle and the robot.

[0148] The inspection robot semantic information positioning enhancement device provided in the embodiment of the present application can be used to execute the technical solution of the inspection robot semantic information positioning enhancement method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0149] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown, the electronic device may include: a processor 81 and a memory 82 , wherein the processor 81 and the memory 82 may communicate with each other; illustratively, the processor 81 and the memory 82 communicate with each other via a communication bus 83 .

[0150] The processor 81 executes the computer execution instructions stored in the memory 82, so that the processor 81 executes the scheme in the above embodiment. The processor 81 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0151] The communication bus 83 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0152] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0153] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the semantic information positioning enhancement method of the inspection robot in the above embodiment.

[0154] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the semantic information positioning enhancement method of the inspection robot in the above embodiment.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0156] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0157] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0158] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0159] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0160] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0161] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0162] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0163] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0164] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for enhancing semantic information positioning of an inspection robot, characterized in that: include: Obtain 3D point cloud data; Using the PointNet++ network to segment the three-dimensional point cloud data to obtain the axle point cloud; Axle centerline data is acquired based on the axle point cloud, the axle centerline data is fitted to determine the axle centerline position, and the axle position is determined according to the axle centerline position and odometer data.

2. The method according to claim 1, characterized in that Acquiring three-dimensional point cloud data, including using a solid-state three-dimensional laser radar carried by an inspection robot to collect field data to obtain point cloud data; Based on the set region of interest, the point cloud data is subjected to spatial coordinate screening to obtain three-dimensional point cloud data.

3. The method according to claim 1, characterized in that The three-dimensional point cloud data is segmented using the PointNet++ network to obtain the axle point cloud, including: After receiving the three-dimensional point cloud data accumulated for a set time, performing ROI segmentation on the accumulated three-dimensional point cloud data to obtain an original point cloud, and inputting the original point cloud into the PointNet++ network; The PointNet++ network responds to the input original point cloud and selects a set of centroid points of local areas from the original point cloud using the farthest point sampling algorithm; Taking the centroid point as the center, searching for neighboring points around the centroid point to form a local area set; Feature extraction is performed on the local area set to obtain a feature vector as the axle point cloud; wherein, when feature extraction is performed on the local area set, the distribution characteristics of the point cloud are adapted by extracting feature vectors at different levels.

4. The method according to claim 1, characterized in that: Acquiring axle centerline data based on the axle point cloud, and fitting the axle centerline data to determine the axle centerline position, including: Set the number of iterations and error threshold; A subset is randomly selected from the axle point cloud, and a target model is fitted; wherein the distance from any point P on the surface of the target model to the axis is always equal to the radius, expressed as: (x-x0) 2 +(y-y0) 2 +(z-z0) 2 +[a(x-x0)+b(y-y0)+c(z-z0)] 2 =r 2 Where x, y, z are the x-coordinate, y-coordinate and z-coordinate of any point on the surface of the target model, r is the radius of the bottom circle, x0, y0, z0 are the x-coordinate, y-coordinate and z-coordinate on the axis, a, b, c represent the x, y, z axis vectors; Calculate the error between the target model and all points, and classify the points with errors less than a threshold as an inlier set; If the number of inlier sets is greater than the preset value, the target model is updated. When the number of iterations is reached or the target model converges, the axis x position and radius of the target model are output.

5. The method according to claim 4, characterized in that The error between the target model and all points is calculated by the following error equation: f=(x-x0) 2 +(y-y0) 2 +(z-z0) 2 +[a(x-x0)+b(y-y0)+c(z-z0)] 2 -r 2 In the formula, f represents the error; If the number of internal point sets is greater than the preset value, the optimization target model is updated with the goal of minimizing s; where s is the result of the derivative calculation of the error equation, and the calculation formula is: Where f0 is the initial value of the error equation.

6. The method according to claim 5, characterized in that After determining the axle position according to the axle centerline position and the odometer data, the method further includes performing navigation error correction based on dynamically adjusting the correction coefficient of the navigation error; wherein the calculation formula of the correction coefficient is: Where c is the correction coefficient and d is the distance between the axle and the robot.

7. A semantic information positioning enhancement device for an inspection robot, characterized in that: include: A data acquisition module is configured to acquire three-dimensional point cloud data; A semantic segmentation module is configured to segment the three-dimensional point cloud data using a PointNet++ network to obtain an axle point cloud; The axle positioning module is configured to obtain axle centerline data based on the axle point cloud, fit the axle centerline data to determine the axle centerline position, and determine the axle position according to the axle centerline position and odometer data.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images