Loopback detection method and device based on tunnel inspection object, and electronic equipment

Abstract

The invention discloses a loopback detection method and device based on a tunnel inspection object and electronic equipment, and relates to the field of intelligent inspection. The method comprises the following steps: determining current pose data of a tunnel inspection object at a current time point based on sensor data, and when the tunnel inspection object establishes a current pose node based on the current pose data, determining a target pose node to construct a loop with the current pose node, judging whether a loopback constraint between the current pose node and the target pose node is constructed or not, and determining the similarity between the current image feature of the current time point and the image feature of the target time point indicated by the target pose node; and when it is determined that the loopback constraint between the current pose node and the target pose node is constructed and the similarity is greater than a preset similarity threshold, loopback detection is performed based on the loopback constraint. According to the method and the device, the technical problem of relatively low accuracy of loopback detection caused by easy establishment of wrong loopback constraints in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection, and in particular to a loop detection method based on tunnel inspection objects, a device thereof, and electronic equipment. Background Art

[0002] The working environment faced by tunnel inspection robots is far more complex than that of conventional inspection robots, because they must work in tunnels with changing and complex conditions. In such special scenarios, inspection robots need to deal with multiple challenges such as moving vehicles, pedestrians, and objects thrown on the road. In addition, since there is usually a lack of satellite positioning signals inside the tunnel, the robot must rely on other navigation and positioning technologies to ensure accurate actions. In addition, there are many abnormal situations that may occur inside the tunnel. For example, the ventilation system or fire protection system may fail, the tunnel may seep water or the dome may collapse, and there may even be safety hazards such as excessive toxic gases. If these abnormalities are not discovered and handled in time, they may lead to serious consequences.

[0003] Therefore, the inspection robot needs to have a high degree of autonomous navigation and obstacle avoidance capabilities to ensure safe and effective bypassing of obstacles in complex tunnel environments. At the same time, the robot should also be equipped with high-precision sensors and detection equipment to promptly identify and record various abnormal situations. Once an abnormality is detected, the robot needs to quickly collect the type and key parameters of the abnormality, conduct on-site video shooting, and be able to accurately locate the abnormal point. In order to achieve these functions, the robot must have powerful data processing and transmission capabilities, as well as an efficient communication system, so that the collected information can be transmitted to the tunnel management center in real time for managers to make quick decisions and take countermeasures.

[0004] However, due to the long tunnel length and similar feature points, the subgraphs created are not much different, which may lead to incorrect loop constraints in loop detection, and then produce wrong position and map information in the backend optimization.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a loop detection method based on tunnel inspection objects, a device and an electronic device thereof, so as to at least solve the technical problem in the related art that erroneous loop constraints are easily constructed, resulting in low accuracy of loop detection.

[0007] According to one aspect of an embodiment of the present invention, a loop detection method based on a tunnel inspection object is provided, comprising: determining current posture data of the tunnel inspection object at a current time point based on sensor data of the tunnel inspection object; when the tunnel inspection object establishes a current posture node based on the current posture data, determining a target posture node to be looped with the current posture node based on posture data at any time point before the current time point, the current posture data and the sensor data, and judging whether to construct a loop constraint between the current posture node and the target posture node; acquiring current image features at the current time point, and determining the similarity between the current image features and the image features at the target time point indicated by the target posture node; and performing loop detection based on the loop constraint when it is determined to construct the loop constraint between the current posture node and the target posture node and the similarity is greater than a preset similarity threshold.

[0008] Further, the sensor data includes: angular velocity data and linear acceleration data of the tunnel inspection object at each time point. The step of determining the current posture data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object includes: determining a first position matrix based on the angular velocity data and the linear acceleration data at the previous time point, and determining a second position matrix based on the angular velocity data and the linear acceleration data at the next time point; determining a velocity vector over a time period from the previous time point to the next time point based on the first position matrix, the second position matrix, the previous time point and the next time point; determining a current position at the current time point using a linear interpolation algorithm based on the velocity vector, the first position matrix, the previous time point and the current time point; acquiring the posture data and the angular velocity data of the tunnel inspection object at the previous time point, and determining the current posture data at the current time point based on the posture data, the angular velocity data, the previous time point and the current time point using the linear interpolation algorithm; determining the current posture data based on the current position and the current posture data.

[0009] Further, based on the pose data at any time point before the current time point, the current pose data and the sensor data, a target pose node to be looped with the current pose node is determined, and a step of judging whether to construct a loop constraint between the current pose node and the target pose node includes: determining the time point indicated by the pose data consistent with the current pose data as the target time point, and determining the pose node established at the target time point as the target pose node; based on the sensor data, determining a first pose change relationship value and a second pose change relationship value from the target time point to the current time point; based on the current pose data and the target pose data corresponding to the target pose node, determining a first loop point value and a second loop point value; calculating the product value between the first pose change relationship value, the second pose change relationship value, the first loop point value and the second loop point value, and comparing the product value with a preset loop threshold; and when the product value is less than or equal to the preset loop threshold, determining to construct the loop constraint between the current pose node and the target pose node.

[0010] Furthermore, the step of obtaining the current image features at the current time point and determining the similarity between the current image features and the image features at the target time point indicated by the target pose node includes: acquiring the environmental image at the current time point, and using a preset convolutional neural network to extract the current image features on the environmental image, wherein the preset convolutional neural network is a neural network trained using a set of historical environmental images and feature information associated with each historical environmental image; determining a current feature vector corresponding to the current image feature and a feature vector of the image feature; and calculating the similarity between the current feature vector and the feature vector.

[0011] Furthermore, based on the loop constraint, the step of performing loop detection includes: obtaining the first pose data at the previous time point and the second pose data at the next time point; determining a first velocity value based on the first pose data, the current pose data, the previous time point and the current time point, and determining a second velocity value based on the second pose data, the current pose data, the next time point and the current time point; determining an acceleration value based on the first velocity value and the second velocity value; determining an optimized position at the current time point based on the first velocity value, the acceleration value and the time difference between the previous time point and the current time point; determining an optimized posture at the current time point based on the angular velocity and angular acceleration at the previous time point and the time difference; and updating the current pose data based on the optimized position and the optimized posture.

[0012] Furthermore, after determining the current posture data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object, it also includes: collecting laser point cloud data at each time; based on the posture data at each time point, performing distortion removal processing on the laser point cloud data to obtain the target laser point cloud data at each time point; constructing an objective function, and based on the objective function, optimizing the posture data at each time point.

[0013] Furthermore, after performing loop detection based on the loop constraint, it also includes: determining a loop count threshold and accumulating the loop count after each loop detection; when the loop count is equal to the loop count threshold, constructing an inspection map of the tunnel inspection object based on the target laser point cloud data and the posture data at each time point.

[0014] According to another aspect of an embodiment of the present invention, a loop detection device based on a tunnel inspection object is also provided, comprising: a first determination unit, for determining current posture data of the tunnel inspection object at a current time point based on sensor data of the tunnel inspection object; a second determination unit, for determining a target posture node to be looped with the current posture node based on posture data at any time point before the current time point, the current posture data and the sensor data, when the tunnel inspection object establishes a current posture node based on the current posture data, and determining whether to construct a loop constraint between the current posture node and the target posture node; a third determination unit, for acquiring current image features at the current time point, and determining the similarity between the current image features and the image features at the target time point indicated by the target posture node; a detection unit, for performing loop detection based on the loop constraint when it is determined that the loop constraint between the current posture node and the target posture node is constructed and the similarity is greater than a preset similarity threshold.

[0015] Further, the sensor data includes: angular velocity data and linear acceleration data of the tunnel inspection object at each time point, and the first determination unit includes: a first determination module, which is used to determine the first position matrix based on the angular velocity data and the linear acceleration data at the previous time point, and determine the second position matrix based on the angular velocity data and the linear acceleration data at the next time point; a second determination module, which is used to determine the velocity vector in the time period from the previous time point to the next time point based on the first position matrix, the second position matrix, the previous time point and the next time point; a third determination module, which is used to determine the current position of the current time point by using a linear interpolation algorithm based on the velocity vector, the first position matrix, the previous time point and the current time point; a fourth determination module, which is used to obtain the posture data and the angular velocity data of the tunnel inspection object at the previous time point, and determine the current posture data of the current time point by using the linear interpolation algorithm based on the posture data, the angular velocity data, the previous time point and the current time point; a fifth determination module, which is used to determine the current posture data based on the current position and the current posture data.

[0016] Furthermore, the second determination unit includes: a sixth determination module, used to determine the time point indicated by the posture data consistent with the current posture data as the target time point, and determine the posture node established at the target time point as the target posture node; a seventh determination module, used to determine the first posture change relationship value and the second posture change relationship value from the target time point to the current time point based on the sensor data; an eighth determination module, used to determine the first loop point value and the second loop point value based on the current posture data and the target posture data corresponding to the target posture node; a first calculation module, used to calculate the product value between the first posture change relationship value, the second posture change relationship value, the first loop point value and the second loop point value, and compare the product value with a preset loop threshold; a ninth determination module, used to determine the construction of the loop constraint between the current posture node and the target posture node when the product value is less than or equal to the preset loop threshold.

[0017] Furthermore, the third determination unit includes: a first extraction module, used to collect the environmental image at the current time point, and use a preset convolutional neural network to extract the current image features on the environmental image, wherein the preset convolutional neural network is a neural network trained using a set of historical environmental images and feature information associated with each historical environmental image; a tenth determination module, used to determine a current feature vector corresponding to the current image feature and a feature vector of the image feature; and a second calculation module, used to calculate the current feature vector and the similarity between the feature vectors.

[0018] Further, the detection unit includes: a first acquisition module, used to acquire the first posture data of the previous time point and the second posture data of the next time point; an eleventh determination module, used to determine the first speed value based on the first posture data, the current posture data, the previous time point and the current time point, and determine the second speed value based on the second posture data, the current posture data, the next time point and the current time point; a twelfth determination module, used to determine the acceleration value based on the first speed value and the second speed value; a thirteenth determination module, used to determine the optimized position of the current time point based on the first speed value, the acceleration value and the time difference between the previous time point and the current time point; a fourteenth determination module, used to determine the optimized posture of the current time point based on the angular velocity and angular acceleration of the previous time point and the time difference; a first update module, used to update the current posture data based on the optimized position and the optimized posture.

[0019] Furthermore, the loop detection device also includes: a first acquisition module, which is used to collect laser point cloud data at each time after determining the current posture data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object; a first processing module, which is used to perform distortion removal processing on the laser point cloud data based on the posture data at each time point to obtain target laser point cloud data at each time point; a first optimization module, which is used to construct an objective function and optimize the posture data at each time point based on the objective function.

[0020] Furthermore, the loop detection device also includes: a fifteenth determination module, which is used to determine a loop number threshold after performing loop detection based on the loop constraint, and accumulate the number of loops after each loop detection; a first construction module, which is used to construct an inspection map of the tunnel inspection object based on the target laser point cloud data and the posture data at each time point when the number of loops is equal to the loop number threshold.

[0021] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the loop detection method based on tunnel inspection objects described in any one of the above is implemented.

[0022] According to another aspect of an embodiment of the present invention, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned loop detection methods based on tunnel inspection objects.

[0023] In the present invention, based on the sensor data of the tunnel inspection object, the current posture data of the tunnel inspection object at the current time point is determined. When the tunnel inspection object establishes the current posture node based on the current posture data, based on the posture data at any time point before the current time point, the current posture data and the sensor data, the target posture node to be looped with the current posture node is determined, and it is judged whether to construct a loop constraint between the current posture node and the target posture node, the current image feature at the current time point is obtained, and the similarity between the current image feature and the image feature at the target time point indicated by the target posture node is determined. When it is determined that a loop constraint is constructed between the current posture node and the target posture node, and the similarity is greater than a preset similarity threshold, loop detection is performed based on the loop constraint, thereby solving the technical problem in the related art that erroneous loop constraints are easily constructed, resulting in low accuracy of loop detection.

[0024] In the present invention, the current posture data of the tunnel inspection object at the current time point can be determined based on the sensor data of the tunnel inspection object. Then, when the tunnel inspection object establishes a new posture node based on the current posture data, the target posture node that can construct a loop with the current posture node is determined, and it is determined whether a loop constraint between the current posture node and the target posture node can be constructed. At the same time, it is also necessary to compare the similarity between the current image features at the current time point and the image features at the target time point indicated by the target posture node. If the similarity is greater than a preset similarity threshold and it is determined to construct a loop constraint, loop detection can be performed based on the constructed loop constraint, which can avoid establishing an erroneous loop constraint, thereby improving the accuracy of loop detection and achieving the technical effect of accurate loop detection of the tunnel environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flow chart of an optional loop detection method based on tunnel inspection objects according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of an optional loop judgment using an improved Lazy Decis ion algorithm according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of an optional similarity calculation process according to an embodiment of the present invention;

[0029] Figure 4 is a schematic diagram of an optional process of constructing an inspection map based on loop detection according to an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of an optional process of combining feature matching and loop detection to construct an inspection map according to an embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of an optional loop detection device based on tunnel inspection objects according to an embodiment of the present invention;

[0032] Figure 7 The present invention is a hardware structure block diagram of an electronic device (or mobile device) for a loop detection method based on tunnel inspection objects according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0036] SLAM (Simultaneous Localization and Mapping): simultaneous positioning and map construction.

[0037] Loop closure detection: is the process of identifying areas that the robot has already visited and updating the map accordingly.

[0038] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) collected and involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated, and corresponding operation entrances are provided for users to choose to authorize or refuse. For example, an interface is set between the system and the relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0039] The present invention proposes a SLAM loop detection method based on a neural network and an improved Lazy Decis ion (i.e., a delay strategy used in SLAM) in a tunnel environment. The loop condition is judged by improving the Lazy Decis ion, and feature matching is judged based on the neural network. Then, loop detection is performed according to the judgment results of the two. The method can solve the problem that, due to the long length of the tunnel and the almost same feature points at various locations, the subgraphs established are not very different, so it is easy to obtain wrong loop constraints during loop detection, thereby obtaining wrong postures and wrong maps during back-end optimization.

[0040] The present invention is described in detail below in conjunction with various embodiments.

[0041] Embodiment 1

[0042] According to an embodiment of the present invention, an embodiment of a loop detection method based on tunnel inspection objects is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] Figure 1 is a flow chart of an optional loop detection method based on tunnel inspection objects according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0044] Step S101, based on the sensor data of the tunnel inspection object, determine the current posture data of the tunnel inspection object at the current time point.

[0045] In the embodiment of the present invention, laser SLAM is particularly suitable for dark environments such as tunnels because it does not rely on visible light for environmental recognition, and laser radar has high precision and long-distance ranging capabilities, and can provide accurate depth information. Through the integration with IMU (Inertial Measurement Unit), odometer and other sensors, laser SLAM can provide reliable environmental perception for robots. Therefore, sensors such as laser radar, IMU, odometer, etc. can be deployed on the tunnel inspection object (i.e., tunnel inspection robot) to collect sensor data such as point cloud data, angular velocity and linear acceleration.

[0046] In step S101, a posture predictor may be used to predict current posture data of the tunnel inspection object at the current time point based on collected sensor data of the tunnel inspection object.

[0047] In this embodiment, the posture predictor can predict the approximate posture of the robot at any time by linear interpolation method based on the angular velocity and linear acceleration collected at each time point.

[0048] Step S102, when the tunnel inspection object establishes the current pose node based on the current pose data, based on the pose data at any time point before the current time point, the current pose data and the sensor data, determine the target pose node to be looped with the current pose node, and determine whether to construct a loop constraint between the current pose node and the target pose node.

[0049] In step S102, the tunnel inspection object establishes a new posture node (i.e., the current posture node) according to the current posture data, and constructs a loop constraint with the previously established subgraph (i.e., the subgraph composed of multiple posture nodes). Since the subgraphs established when the robot walks in the tunnel are almost identical, it is easy to get an erroneous loop by direct loop detection. Therefore, in order to avoid loop errors caused by performing a loop detection for each newly added posture node, an improved Lazy Decis ion algorithm can be used. First, according to the posture data at any time point at the current time point, the current posture data, and the sensor data, the target posture node to be looped with the current posture node is determined, and according to the posture change relationship between the target posture data corresponding to the target posture node and the current posture data, it is determined whether to construct a loop constraint between the current posture node and the target posture node. In this way, it is possible to avoid establishing erroneous loop constraints.

[0050] Step S103, obtaining the current image features at the current time point, and determining the similarity between the current image features and the image features at the target time point indicated by the target pose node.

[0051] In step S103, a camera device is deployed on the tunnel inspection object, and the current image features at the current time point and the image features at the target time point indicated by the target posture node are obtained based on the tunnel environment image captured by the camera device. Then, the similarity between the current image features and the image features is calculated to identify whether the positions of the robot at these two time points are the same through the similarity, thereby determining whether a loop occurs.

[0052] Step S104, when it is determined that a loop constraint is constructed between the current pose node and the target pose node, and the similarity is greater than a preset similarity threshold, loop detection is performed based on the loop constraint.

[0053] In an embodiment of the present invention, if it is determined that a loop constraint can be constructed between the current pose node and the target pose node, and the similarity between the current image feature and the image feature at the target time point indicated by the target pose node is greater than a preset similarity threshold (which can be set according to actual conditions), loop detection can be performed based on the constructed loop constraint to achieve optimization of the map.

[0054] In summary, the current posture data of the tunnel inspection object at the current time point can be determined based on the sensor data of the tunnel inspection object. When the tunnel inspection object establishes the current posture node based on the current posture data, the target posture node to be looped with the current posture node is determined based on the posture data at any time point before the current time point, the current posture data and the sensor data, and it is determined whether to construct a loop constraint between the current posture node and the target posture node, obtain the current image feature at the current time point, and determine the similarity between the current image feature and the image feature at the target time point indicated by the target posture node. When it is determined that the loop constraint between the current posture node and the target posture node is constructed, and the similarity is greater than a preset similarity threshold, loop detection is performed based on the loop constraint, which can avoid establishing an erroneous loop constraint, thereby improving the accuracy of loop detection and achieving the technical effect of accurate loop detection of the tunnel environment, thereby solving the technical problem that erroneous loop constraints are easily constructed in related technologies, resulting in low accuracy of loop detection.

[0055] In an embodiment of the present invention, the sensor data includes: angular velocity data and linear acceleration data of the tunnel inspection object at each time point. In order to improve the accuracy of determining the current posture data, in the loop detection method based on the tunnel inspection object provided in the first embodiment of the present application, the first position matrix is ​​determined based on the angular velocity data and linear acceleration data at the previous time point, and the second position matrix is ​​determined based on the angular velocity data and linear acceleration data at the next time point; based on the first position matrix, the second position matrix, the previous time point and the next time point, the velocity vector in the time period from the previous time point to the next time point is determined; based on the velocity vector, the first position matrix, the previous time point and the current time point, a linear interpolation algorithm is used to determine the current position at the current time point; the posture data and angular velocity data of the tunnel inspection object at the previous time point are obtained, and based on the posture data, the angular velocity data, the previous time point and the current time point, a linear interpolation algorithm is used to determine the current posture data at the current time point; based on the current position and the current posture data, the current posture data is determined.

[0056] In the embodiment of the present invention, the IMU can directly measure the angular velocity and linear acceleration of the robot, but the linear velocity error obtained by integrating the linear acceleration is large, and the accuracy is not as good as the linear velocity measured by the odometer. When measuring the angular velocity, the angular velocity directly measured by the three-axis gyroscope has a higher accuracy than the accuracy of the angular velocity measured by the odometer. Therefore, a posture predictor can be used to process the sensor data to obtain the current posture data. Specifically, when predicting the robot posture, the posture at the time to be predicted (i.e., the current time point) can be solved by linear interpolation. Determine the last moment (i.e., the last time point) t s The robot position matrix is ​​X s(i.e., the first position matrix, which is a position matrix constructed based on the angular velocity data and linear acceleration data at the previous time point, and the matrix elements in the position matrix contain angular velocity and linear acceleration), and determine the distance from the time to be predicted t i Most recent and greater than t i The next time point t e The position matrix is ​​X e (i.e., the second position matrix, which is a position matrix constructed based on the angular velocity data and linear acceleration data at the next time point), and then according to the first position matrix X s , the second position matrix X e , the previous time point t s and the next time point t e , determine the velocity vector V in the time period from the previous time point to the next time point i (i.e. from t s to e The odometer linear velocity vector for this period of time), where the velocity vector V i The calculation formula is as follows:

[0057]

[0058] Then, according to the velocity vector V i , the first position matrix X s , the previous time point t s and the current time point t i , and t is obtained by linear interpolation i The position at the moment (i.e. the current position at the current time point X i ), where the current position X i The calculation formula is as follows:

[0059] X i =X s +V i ·(t i -t s );

[0060] In addition, the posture data Q of the tunnel inspection object at the previous time point can be obtained at the same time k And the angular velocity data ω k (can be measured by IMU), and according to the attitude data Q k , angular velocity data ω k , the previous time point t k and the current time point t i , and t is obtained by linear interpolation i The posture at the moment (i.e. the current posture data Q at the current time point) i ), where the current posture data Q iThe calculation formula is as follows:

[0061] Q i =Q k ·ω k ·(t i -t k );

[0062] In this embodiment, the current posture data is determined based on the current position at the current time point and the current posture data.

[0063] In this embodiment, the posture predictor can predict the approximate posture of the robot at any time by linear interpolation method.

[0064] In order to accurately determine whether to construct a loop constraint, in the loop detection method based on tunnel inspection objects provided in Example 1 of the present application, the time point indicated by the posture data consistent with the current posture data is determined as the target time point, and the posture node established at the target time point is determined as the target posture node; based on the sensor data, the first posture change relationship value and the second posture change relationship value from the target time point to the current time point are determined; based on the current posture data and the target posture data corresponding to the target posture node, the first loop point value and the second loop point value are determined; the product value between the first posture change relationship value, the second posture change relationship value, the first loop point value and the second loop point value is calculated, and the product value is compared with the preset loop threshold; when the product value is less than or equal to the preset loop threshold, it is determined to construct a loop constraint between the current posture node and the target posture node.

[0065] In the embodiment of the present invention, when the robot moves forward to establish a new posture node, it will construct loop constraints with the previously established subgraph. However, when the robot walks in the tunnel, the established subgraphs are almost identical, and direct loop detection can easily result in an erroneous loop. Therefore, in order to avoid loop errors caused by performing a loop detection for each newly added posture node, this embodiment uses an improved Lazy Decision algorithm to determine whether a loop constraint has been constructed. Specifically:

[0066] The time point indicated by the pose data consistent with the current pose data can be first determined as the target time point, and the pose node established at the target time point can be determined as the target pose node. For example, the current pose data is compared with the previous pose data. If consistent pose data (such as consistent position coordinates, angles, etc.) appear, the robot may have returned to the area that has been visited, and the pose node corresponding to the consistent pose data can be determined as the target pose node that can construct a loop constraint with the current pose node. Then, the first pose change relationship value T1 and the second pose change relationship value T2 from the target time point to the current time point can be determined based on the sensor data. Among them, the first pose change relationship value T1 and the second pose change relationship value T2 refer to the position change value and angle change value actually measured. Afterwards, the first loop point value T3 and the second loop point value T4 are determined based on the current pose data and the target pose data corresponding to the target pose node, wherein the first loop point value T3 and the second loop point value T4 are the position change value and angle change value obtained by prediction. After that, the product value I between the first pose change relationship value T1, the second pose change relationship value T2, the first loop point value T3 and the second loop point value T4 can be calculated, that is, T1·T2·T3·T4=I, and the product value I is compared with the preset loop threshold Th (which can be set according to actual conditions). If the product value is greater than the preset loop threshold, that is, ||T1·T2·T3·T4||>Th, it is considered to be an incorrect loop and no loop detection is performed; if the product value is less than or equal to the preset loop threshold, that is, ||T1·T2·T3·T4||≤Th, it is considered to be a correct loop, a loop detection can be performed, and a loop constraint between the current pose node and the target pose node is constructed.

[0067] Figure 2 FIG. 1 is a schematic diagram of an optional loopback judgment using an improved Lazy Decision algorithm according to an embodiment of the present invention. Figure 2 As shown in the figure, when the robot moves forward to establish a new posture node, it will construct loop constraints with the previously established subgraph. In order to avoid loop errors caused by performing a loop detection for each new posture node, the improved Lazy Decision algorithm is used to first determine the posture change values ​​such as T1, T2, T3, and T4, where T1 and T2 are the actual recorded values, and T3 and T4 are the predicted values. Then, the product value between T1, T2, T3, and T4 is calculated. If the product value is less than or equal to the preset loop threshold Th, it indicates that the current loop is correct, and a loop detection can be performed to construct loop constraints.

[0068] In order to improve the accuracy of calculating the similarity between image features, in the loop detection method based on tunnel inspection objects provided in Example 1 of the present application, an environmental image at the current time point is collected, and a preset convolutional neural network is used to extract the current image features on the environmental image, wherein the preset convolutional neural network is a neural network trained using a set of historical environmental images and feature information associated with each historical environmental image; the current feature vector corresponding to the current image feature and the feature vector of the image feature are determined; and the current feature vector and the similarity between the feature vectors are calculated.

[0069] In an embodiment of the present invention, a set of historical environmental images within a historical time period (such as within the past year) can be first collected, and then feature information (i.e., object information existing on the image, such as fire hydrants, ventilation ducts, domes, etc.) can be annotated for each historical environmental image. The set of historical environmental images annotated with feature information is then used to train a convolutional neural network until the convolutional neural network converges to obtain a preset convolutional neural network.

[0070] In an embodiment of the present invention, a current image feature on an environment image at a current time point can be extracted by a preset convolutional neural network, and image features on an environment image at any time point before the current time point can be extracted. Then, a current feature vector corresponding to the current image feature and a feature vector corresponding to each image feature are determined. After that, a similarity measure between the current feature vector and each feature vector is calculated to identify whether the two frames of images are at the same position, thereby determining whether a loop occurs.

[0071] In an optional embodiment, the similarity between two feature vectors can be calculated by Euclidean distance. For example, the feature vectors can be converted into points in a multidimensional space, and then the distance between the two points is calculated, and the distance is represented as the similarity. The distance between two points can be calculated by the following formula:

[0072]

[0073] Among them, x and y are points in multidimensional space, n represents the number of dimensions of the feature vector, which is consistent with the number of dimensions of the space, and x i ,y i Respectively represent the coordinate values ​​on the i-th dimension.

[0074] Figure 3 is a schematic diagram of an optional similarity calculation process according to an embodiment of the present invention, such as Figure 3 As shown, feature extraction is performed on the data at the i-th moment to obtain the feature of the i-th moment, and feature extraction is performed on the data at the previous i-1 moment to obtain the feature matrix of the previous i-1 moment, and then similarity calculation is performed on the feature of the i-th moment and each feature of the previous i-1 moment in the feature matrix of the previous i-1 moment to obtain the target moment indicated by the highest similarity.

[0075] In order to accurately perform loop detection, in the loop detection method based on tunnel inspection objects provided in Example 1 of the present application, the first posture data of the previous time point and the second posture data of the next time point are obtained; based on the first posture data, the current posture data, the previous time point and the current time point, the first speed value is determined, and based on the second posture data, the current posture data, the next time point and the current time point, the second speed value is determined; based on the first speed value and the second speed value, the acceleration value is determined; based on the first speed value, the acceleration value and the time difference between the previous time point and the current time point, the optimized position of the current time point is determined; based on the angular velocity and angular acceleration of the previous time point and the time difference, the optimized posture of the current time point is determined; based on the optimized position and the optimized posture, the current posture data is updated.

[0076] In the embodiment of the present invention, the posture nodes X1, X2, and X3 corresponding to the adjacent time points t1, t2, and t3 respectively (i.e., the first posture data at the previous time point, the current posture data at the current time point, and the second posture data at the next time point) can be obtained, and then the speed and acceleration between the two adjacent nodes are calculated respectively, that is, based on the first posture data X1, the current posture data X2, the previous time point t1, and the current time point t2, the first speed value V is determined. 12 , and determine the second velocity value V based on the second posture data X3, the current posture data X2, the next time point t3 and the current time point t2 23 , then, based on the first speed value V 12 and the second speed value V 23 , determine the acceleration value a 13 , the calculation formula is as follows:

[0077]

[0078] Then, based on the first speed value, the acceleration value, and the time difference between the previous time point and the current time point, the optimal position X at the current time point is determined. i , where the optimal position X at the current time point i The calculation formula is:

[0079]

[0080] Where Δt is the time close to t i moment (i.e. the current time point) but less than t i The moment of time t k (i.e. the previous time point) and time t i The time difference between the previous time point and the current time point, that is, Δt = t i -t k , Vk Indicates the first speed value (ie V 12 ), a k Indicates the acceleration value (i.e. a 13 ).

[0081] Afterwards, based on the angular velocity ω at the previous time point k and angular acceleration β k , time difference Δt, determine the optimal posture Q at the current time point i , the calculation formula is as follows:

[0082]

[0083] After that, the current posture data is updated according to the optimized position and optimized posture.

[0084] In order to further improve the accuracy of the posture data, in the loop detection method based on the tunnel inspection object provided in Example 1 of the present application, after the current posture data of the tunnel inspection object at the current time point is determined based on the sensor data of the tunnel inspection object, the laser point cloud data at each time point is collected; based on the posture data at each time point, the laser point cloud data is distorted and processed to obtain the target laser point cloud data at each time point; an objective function is constructed, and based on the objective function, the posture data at each time point is optimized.

[0085] In the embodiment of the present invention, the collected laser point cloud data is distorted by the robot's movement and cannot be directly used for posture fusion and map construction. Therefore, it is necessary to remove the motion distortion of the laser point cloud. The robot posture set corresponding to each laser point at the corresponding time can be obtained by calling the posture predictor: P = {p1, p2, ..., p n}, where p1, p2, … p n is the predicted pose matrix of the robot at each laser point. Then convert all laser point coordinates to the robot coordinate system, that is: x' i =p i T ·x i , where x i is the original laser point cloud data without distortion removal, p i is the pose data at the corresponding time point, x' i It is the laser point cloud data with distortion removed, that is, based on the pose data p at each time point i , for the laser point cloud data x i Perform distortion removal to obtain the target laser point cloud data x' at each time point i .

[0086] In an embodiment of the present invention, in a scenario where high positioning accuracy is required, the posture data at each time point can be further optimized. Specifically, an objective function can be constructed first, and the objective function is: Where E(T) is the robot’s estimated pose with the smallest error, T is the robot’s local pose, and T = (T x ,T y ,T θ ), T x ,T y Represents position data, T θ Indicates angle data, S i (T) is the position of the laser data transformed from the robot position to the local map coordinate system, M s (x) is the occupancy probability of the smoothed map at x. Then, according to the objective function, the pose data at each time point is optimized, and the pose data with the smallest objective function value is the optimal pose data.

[0087] In order to improve the accuracy of the constructed inspection map, in the loop detection method based on the tunnel inspection object provided in Example 1 of the present application, after loop detection is performed based on the loop constraint, a loop count threshold is determined, and the loop count is accumulated after each loop detection; when the loop count is equal to the loop count threshold, an inspection map of the tunnel inspection object is constructed based on the target laser point cloud data and posture data at each time point.

[0088] In the embodiment of the present invention, the loop count threshold can be dynamically adjusted according to the complexity of the tunnel environment and the uncertainty of the robot's motion. For example, if the robot moves in an environment with rich tunnel turning features, the threshold is lowered because it is easier to find a reliable loop. If the robot moves in an environment with a straight tunnel lacking features, the threshold is increased to reduce misjudgment.

[0089] In the embodiment of the present invention, after each loop detection, the number of loops is accumulated. If the number of loops is equal to the loop number threshold, it indicates that the current loop detection is completed, and an inspection map of the tunnel inspection object can be constructed based on the target laser point cloud data and position data at each time point.

[0090] Figure 4 FIG. 1 is a schematic diagram of an optional process of constructing an inspection map based on loop detection according to an embodiment of the present invention. Figure 4As shown, part of the pose nodes and subgraphs can be obtained first, and inserted into the pose queue and subgraph queue respectively. Then, when constructing the loop constraint between the current pose node and the previous subgraph, it is determined whether the loop condition of the improved LazyDecision is met. If not, the part of the pose nodes and subgraphs are re-acquired. If they are met, the branch and bound method is used to obtain a relatively accurate initial pose value of the current pose on the completed subgraph. Then, it is determined whether the number of loops is equal to the set threshold. If not, the part of the pose nodes and subgraphs are re-acquired. If they are equal, the map is constructed.

[0091] Figure 5 FIG. 1 is a schematic diagram of an optional process of combining feature matching and loop detection to construct an inspection map according to an embodiment of the present invention. Figure 5 As shown, feature extraction can be performed on the data at the i-th moment to obtain the feature at the i-th moment, and feature extraction can be performed on the data at the previous i-1 moment to obtain the feature matrix at the previous i-1 moment, and then the similarity calculation is performed on the feature at the i-th moment and each feature at the previous i-1 moment in the feature matrix at the previous i-1 moment. At the same time, partially obtained pose nodes and subgraphs are obtained, and inserted into the pose queue and subgraph queue respectively, and then when the current pose node and the previous subgraph are constructed with a loop constraint, it is determined whether the loop condition of the improved Lazy Decis ion is met. If the loop condition is not met, the partially obtained pose nodes and subgraphs are re-obtained. If the loop condition is met and the similarity is greater than the similarity threshold, the branch and bound method is used to obtain a relatively accurate pose initial value of the current pose on the completed subgraph. Then, it is determined whether the number of loops is equal to the set threshold. If not, the partially obtained pose nodes and subgraphs are re-obtained. If they are equal, the map is constructed.

[0092] In the embodiment of the present invention, an improved Lazy Decis ion loop detection strategy is proposed to address the loop error problem that often occurs in tunnel scenes. At the same time, the pose predictor is optimized, and the pose of the inspection robot is more accurately predicted by using the quadratic interpolation method under the uniform acceleration assumption, which can improve the robustness of the tunnel inspection robot SLAM system, so that it can perform inspection tasks more stably and accurately in complex tunnel environments. In addition, by adding neural network feature extraction and feature matching, a more stable and more generalized feature representation capability can be obtained through a learning-based approach, which is conducive to obtaining a more accurate map.

[0093] The following is a detailed description in conjunction with another embodiment.

[0094] Embodiment 2

[0095] A loop detection device based on tunnel inspection objects provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1.

[0096] Figure 6 is a schematic diagram of an optional loop detection device based on tunnel inspection objects according to an embodiment of the present invention, such as Figure 6 As shown, the loop detection device may include: a first determination unit 60, a second determination unit 61, a third determination unit 62, and a detection unit 63.

[0097] Wherein, the first determination unit 60 is used to determine the current position data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object;

[0098] A second determination unit 61 is used to determine a target pose node to be looped with the current pose node based on pose data at any time point before the current time point, the current pose data and sensor data, and determine whether to construct a loop constraint between the current pose node and the target pose node when the tunnel inspection object establishes the current pose node based on the current pose data;

[0099] A third determining unit 62, configured to obtain a current image feature at a current time point, and determine a similarity between the current image feature and an image feature at a target time point indicated by a target pose node;

[0100] The detection unit 63 is used to perform loop detection based on the loop constraint when it is determined that a loop constraint is constructed between the current pose node and the target pose node and the similarity is greater than a preset similarity threshold.

[0101] The above-mentioned loop detection device can determine the current posture data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object through the first determination unit 60, and determine the target posture node to be looped with the current posture node based on the posture data at any time point before the current time point, the current posture data and the sensor data through the second determination unit 61 when the tunnel inspection object establishes the current posture node based on the current posture data, and judge whether to construct a loop constraint between the current posture node and the target posture node, obtain the current image feature at the current time point through the third determination unit 62, and determine the similarity between the current image feature and the image feature at the target time point indicated by the target posture node, and perform loop detection based on the loop constraint when it is determined to construct a loop constraint between the current posture node and the target posture node and the similarity is greater than a preset similarity threshold through the detection unit 63, so as to avoid establishing an erroneous loop constraint, thereby improving the accuracy of loop detection, and achieving the technical effect of accurate loop detection of the tunnel environment, thereby solving the technical problem that erroneous loop constraints are easily constructed in the related technology, resulting in low accuracy of loop detection.

[0102] Optionally, the sensor data includes: angular velocity data and linear acceleration data of the tunnel inspection object at each time point, and the first determination unit includes: a first determination module, which is used to determine the first position matrix based on the angular velocity data and linear acceleration data at the previous time point, and determine the second position matrix based on the angular velocity data and linear acceleration data at the next time point; a second determination module, which is used to determine the velocity vector in the time period from the previous time point to the next time point based on the first position matrix, the second position matrix, the previous time point and the next time point; a third determination module, which is used to determine the current position at the current time point by using a linear interpolation algorithm based on the velocity vector, the first position matrix, the previous time point and the current time point; a fourth determination module, which is used to obtain the posture data and angular velocity data of the tunnel inspection object at the previous time point, and determine the current posture data at the current time point by using a linear interpolation algorithm based on the posture data, the angular velocity data, the previous time point and the current time point; a fifth determination module, which is used to determine the current posture data based on the current position and the current posture data.

[0103] Optionally, the second determination unit includes: a sixth determination module, used to determine the time point indicated by the posture data consistent with the current posture data as the target time point, and determine the posture node established at the target time point as the target posture node; a seventh determination module, used to determine the first posture change relationship value and the second posture change relationship value from the target time point to the current time point based on the sensor data; an eighth determination module, used to determine the first loop point value and the second loop point value based on the current posture data and the target posture data corresponding to the target posture node; a first calculation module, used to calculate the product value between the first posture change relationship value, the second posture change relationship value, the first loop point value and the second loop point value, and compare the product value with a preset loop threshold; a ninth determination module, used to determine the construction of a loop constraint between the current posture node and the target posture node when the product value is less than or equal to the preset loop threshold.

[0104] Optionally, the third determination unit includes: a first extraction module, used to collect the environmental image at the current time point, and use a preset convolutional neural network to extract current image features on the environmental image, wherein the preset convolutional neural network is a neural network trained using a set of historical environmental images and feature information associated with each historical environmental image; a tenth determination module, used to determine a current feature vector corresponding to the current image feature and a feature vector of the image feature; and a second calculation module, used to calculate the current feature vector and the similarity between feature vectors.

[0105] Optionally, the detection unit includes: a first acquisition module, used to acquire the first posture data at the previous time point and the second posture data at the next time point; an eleventh determination module, used to determine the first speed value based on the first posture data, the current posture data, the previous time point and the current time point, and determine the second speed value based on the second posture data, the current posture data, the next time point and the current time point; a twelfth determination module, used to determine the acceleration value based on the first speed value and the second speed value; a thirteenth determination module, used to determine the optimized position of the current time point based on the first speed value, the acceleration value and the time difference between the previous time point and the current time point; a fourteenth determination module, used to determine the optimized posture of the current time point based on the angular velocity and angular acceleration and the time difference at the previous time point; a first update module, used to update the current posture data based on the optimized position and the optimized posture.

[0106] Optionally, the loop detection device also includes: a first acquisition module, which is used to collect laser point cloud data at each time after determining the current posture data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object; a first processing module, which is used to remove distortion from the laser point cloud data based on the posture data at each time point to obtain target laser point cloud data at each time point; a first optimization module, which is used to construct an objective function and optimize the posture data at each time point based on the objective function.

[0107] Optionally, the loop detection device also includes: a fifteenth determination module, used to determine a loop count threshold after performing loop detection based on loop constraints, and accumulate the number of loops after each loop detection; a first construction module, used to construct an inspection map of the tunnel inspection object based on the target laser point cloud data and posture data at each time point when the number of loops is equal to the loop count threshold.

[0108] The above-mentioned loop detection device may also include a processor and a memory. The above-mentioned first determination unit 60, the second determination unit 61, the third determination unit 62, the detection unit 63, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.

[0109] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. The kernel can be set to one or more, and the kernel parameters are adjusted to determine the loop constraint between the current pose node and the target pose node, and the similarity is greater than a preset similarity threshold, and then perform loop detection based on the loop constraint.

[0110] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0111] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, any one of the above-mentioned loop detection methods based on tunnel inspection objects is implemented.

[0112] When the computer program product is executed on a data processing device, it is suitable for executing a program initialized with the following method steps: based on the sensor data of the tunnel inspection object, determine the current posture data of the tunnel inspection object at the current time point; when the tunnel inspection object establishes a current posture node based on the current posture data, determine the target posture node to be looped with the current posture node based on the posture data at any time point before the current time point, the current posture data and the sensor data, and judge whether to construct a loop constraint between the current posture node and the target posture node, obtain the current image feature at the current time point, and determine the similarity between the current image feature and the image feature at the target time point indicated by the target posture node; when it is determined to construct a loop constraint between the current posture node and the target posture node and the similarity is greater than a preset similarity threshold, perform loop detection based on the loop constraint.

[0113] According to another aspect of an embodiment of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned loop detection method based on tunnel inspection objects.

[0114] Figure 7 1 is a hardware structure block diagram of an electronic device (or mobile device) for a loop detection method based on tunnel inspection objects according to an embodiment of the present invention. Figure 7 As shown, the electronic device may include one or more processors (e.g., Figure 7 The processor 702a, processor 702b, ..., processor 702n, etc., which may include but are not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 704 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 7 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 7 More or fewer components as shown, or with Figure 7 Different configurations shown.

[0115] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0116] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0117] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0118] In the several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0120] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0122] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A loop detection method based on tunnel inspection objects, characterized in that: include: Determining current position data of the tunnel inspection object at a current time point based on sensor data of the tunnel inspection object; When the tunnel inspection object establishes a current pose node based on the current pose data, a target pose node to be looped with the current pose node is determined based on pose data at any time point before the current time point, the current pose data and the sensor data, and a loop constraint between the current pose node and the target pose node is determined; Acquire a current image feature at the current time point, and determine a similarity between the current image feature and an image feature at a target time point indicated by the target pose node; When it is determined that the loop constraint between the current pose node and the target pose node is constructed and the similarity is greater than a preset similarity threshold, loop detection is performed based on the loop constraint.

2. The loop detection method according to claim 1, characterized in that: The sensor data includes: angular velocity data and linear acceleration data of the tunnel inspection object at each time point. Based on the sensor data of the tunnel inspection object, the step of determining current posture data of the tunnel inspection object at the current time point includes: Determine a first position matrix based on the angular velocity data and the linear acceleration data at a previous time point, and determine a second position matrix based on the angular velocity data and the linear acceleration data at a next time point; Determine a velocity vector over a time period from the previous time point to the next time point based on the first position matrix, the second position matrix, the previous time point, and the next time point; Based on the velocity vector, the first position matrix, the previous time point and the current time point, a linear interpolation algorithm is used to determine the current position of the current time point; Acquire the posture data and the angular velocity data of the tunnel inspection object at the previous time point, and determine the current posture data at the current time point using the linear interpolation algorithm based on the posture data, the angular velocity data, the previous time point and the current time point; Based on the current position and the current posture data, the current posture data is determined.

3. The loop detection method according to claim 1, characterized in that: The step of determining a target pose node to be looped with the current pose node based on pose data at any time point before the current time point, the current pose data, and the sensor data, and determining whether to construct a loop constraint between the current pose node and the target pose node comprises: Determine the time point indicated by the posture data consistent with the current posture data as the target time point, and determine the posture node established at the target time point as the target posture node; Based on the sensor data, determining a first pose change relationship value and a second pose change relationship value from the target time point to the current time point; Determine a first loop point value and a second loop point value based on the current pose data and the target pose data corresponding to the target pose node; Calculate the product value of the first posture change relationship value, the second posture change relationship value, the first loop point value, and the second loop point value, and compare the product value with a preset loop threshold; When the product value is less than or equal to the preset loop closure threshold, it is determined to construct the loop closure constraint between the current pose node and the target pose node.

4. The loop detection method according to claim 1, characterized in that: The step of obtaining the current image feature at the current time point and determining the similarity between the current image feature and the image feature at the target time point indicated by the target pose node comprises: Collecting the environment image at the current time point, and using a preset convolutional neural network to extract the current image features on the environment image, wherein the preset convolutional neural network is a neural network trained using a set of historical environment images and feature information associated with each historical environment image; Determine a current feature vector corresponding to the current image feature and a feature vector of the image feature; The similarity between the current feature vector and the feature vector is calculated.

5. The loop detection method according to claim 1, characterized in that: Based on the loop constraint, the step of performing loop detection includes: Obtain the first pose data at the previous time point and the second pose data at the next time point; Determine a first speed value based on the first posture data, the current posture data, the previous time point and the current time point, and determine a second speed value based on the second posture data, the current posture data, the next time point and the current time point; determining an acceleration value based on the first velocity value and the second velocity value; Determining an optimized position of the current time point based on the first speed value, the acceleration value, and a time difference between the previous time point and the current time point; Determining the optimized posture at the current time point based on the angular velocity and angular acceleration at the previous time point and the time difference; Based on the optimized position and the optimized posture, the current posture data is updated.

6. The loop detection method according to claim 1, characterized in that: After determining the current position data of the tunnel inspection object at the current time point based on the sensor data of the tunnel inspection object, the method further includes: Collect laser point cloud data at every time; Based on the posture data at each time point, the laser point cloud data is subjected to distortion removal processing to obtain target laser point cloud data at each time point; An objective function is constructed, and based on the objective function, the posture data at each time point is optimized.

7. The loop detection method according to claim 6, characterized in that: After performing loop detection based on the loop constraint, the method further includes: Determine the loop count threshold and accumulate the loop count after each loop detection; When the number of loops is equal to the loop number threshold, an inspection map of the tunnel inspection object is constructed based on the target laser point cloud data and the posture data at each time point.

8. A loop detection device based on tunnel inspection objects, characterized in that: include: A first determining unit, configured to determine current position data of the tunnel inspection object at a current time point based on sensor data of the tunnel inspection object; A second determination unit is used to determine, when the tunnel inspection object establishes a current pose node based on the current pose data, a target pose node to be looped with the current pose node based on pose data at any time point before the current time point, the current pose data and the sensor data, and to determine whether to construct a loop constraint between the current pose node and the target pose node; A third determining unit, configured to obtain a current image feature at the current time point, and determine a similarity between the current image feature and an image feature at a target time point indicated by the target pose node; The detection unit is used to perform loop detection based on the loop constraint when it is determined that the loop constraint between the current pose node and the target pose node is constructed and the similarity is greater than a preset similarity threshold.

9. A computer program product, characterized in that It includes a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the loop detection method based on tunnel inspection objects as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the loop detection method based on tunnel inspection objects as described in any one of claims 1 to 7.

Images