Multi-sensor information fusion bridge detection robot positioning method

Abstract

The invention relates to the technical field of data processing, in particular to a multi-sensor information fusion bridge detection robot positioning method. The method comprises the following steps: respectively constructing a building point cloud coordinate system and a carrier point cloud coordinate system according to point cloud data of a target bridge and point cloud data of an object in an environment where the target bridge is located; rotating and aligning the building point cloud coordinate system and the carrier point cloud coordinate system according to the similarity between the relative distance between each point cloud in the carrier point cloud coordinate system and other point clouds and the relative distance between each point cloud of the bridge pier in the building point cloud coordinate system and other point clouds; according to the similar situation of the positions of all real-time point clouds collected by the robot at the current moment in the coordinate system after rotation alignment and the position of the point cloud of each pier of the target bridge, and the position distribution similarity of angular points in the image and angular points in the coordinate system after rotation alignment, a real shooting area is screened; and positioning the robot in combination with the orientation of the robot. The positioning precision of the bridge detection robot is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a positioning method for a bridge inspection robot based on multi-sensor information fusion. Background Art

[0002] A bridge inspection robot is an intelligent device specifically used for bridge structure inspection. It can move flexibly at various structural parts of the bridge and is equipped with a variety of sensors and detection devices to achieve comprehensive, efficient, and accurate inspection of the bridge.

[0003] Due to the large and complex bridge structure, and the need to accurately locate abnormal positions during inspection, such as cracks on the surface of bridge piers, fractures on the surface of ropes, etc., it is necessary to accurately position the bridge inspection robot during its operation to evaluate the impact of abnormal points on the bridge structure, and it is also beneficial to plan the inspection path of the robot to avoid colliding with the bridge structure and causing unnecessary losses.

[0004] Existing bridge inspection robot positioning methods are mainly achieved through methods such as GPS, visual positioning, and lidar positioning. However, these positioning methods are often affected by insufficient signal strength and dim and dusty environments, resulting in a large impact on the positioning accuracy of the robot. Summary of the Invention

[0005] In order to solve the problem of low positioning accuracy when existing methods are used to position robots in bridge inspection environments, the purpose of the present invention is to provide a positioning method for a bridge inspection robot based on multi-sensor information fusion. The specific technical solutions adopted are as follows: The present invention provides a positioning method for a bridge inspection robot based on multi-sensor information fusion. The method includes the following steps: Obtain the point cloud data of the target bridge, the point cloud data of objects in the environment where the target bridge is located, and the images captured by the target bridge inspection robot; construct a building point cloud coordinate system according to the point cloud data of the target bridge, and construct a carrier point cloud coordinate system according to the point cloud data of objects in the environment where the target bridge is located; Align the building point cloud coordinate system and the carrier point cloud coordinate system by rotation according to the similarity between the relative distances of each point cloud in the carrier point cloud coordinate system and other point clouds and the relative distances of each point cloud of the bridge pier in the building point cloud coordinate system and other point clouds; screen suspected shooting areas according to the similarity between the positions of all real-time point clouds collected by the robot at the current moment in the rotation-aligned coordinate system and the positions of the point clouds of each bridge pier of the target bridge; Screen the real shooting areas from all suspected shooting areas according to the similarity of the position distribution of the corner points in the image and the corner points in each suspected shooting area; Locate the robot according to the position distribution of the point cloud in the actual shooting area and the orientation of the robot.

[0006] Preferably, rotating and aligning the building point cloud coordinate system and the carrier point cloud coordinate system according to the similarity between the relative distances of each point cloud in the carrier point cloud coordinate system from other point clouds and the relative distances of each point cloud of the pier in the building point cloud coordinate system from other point clouds includes: Calculate the first average distance between each point cloud in the carrier point cloud coordinate system and all other point clouds in the carrier point cloud coordinate system respectively; For any point cloud on any pier in the building point cloud coordinate system: calculate the second average distance between the any point cloud on the any pier and all other point clouds on the any pier; obtain the first similarity between each point cloud in the carrier point cloud coordinate system and the any point cloud on the any pier according to the first difference between the first average distance and the second average distance, and the first difference is negatively correlated with the first similarity; Match the point clouds in the carrier point cloud coordinate system with the point clouds on any pier of the target bridge according to the magnitude relationship of the first similarity between each point cloud in the carrier point cloud coordinate system and all the point clouds on the any pier, and determine the pier in the carrier point cloud coordinate system; Rotate and align the building point cloud coordinate system and the carrier point cloud coordinate system according to the positions of the piers in the carrier point cloud coordinate system and the positions of the piers in the building point cloud coordinate system.

[0007] Preferably, matching the point clouds in the carrier point cloud coordinate system with the point clouds on any pier of the target bridge according to the magnitude relationship of the first similarity between each point cloud in the carrier point cloud coordinate system and all the point clouds on the any pier, and determining the pier in the carrier point cloud coordinate system includes: For any point cloud in the building point cloud coordinate system: if the maximum value of the first similarity between the any point cloud in the building point cloud coordinate system and all the point clouds in the carrier point cloud coordinate system is greater than a preset first similarity threshold, then use the any point cloud in the building point cloud coordinate system as the matching point of the point cloud in the carrier point cloud coordinate system corresponding to the maximum value; All the matching points in the carrier point cloud coordinate system form the pier in the carrier point cloud coordinate system.

[0008] Preferably, screening the suspected shooting area according to the similarity between the positions of all the real-time point clouds collected by the robot at the current moment in the rotated and aligned coordinate system and the positions of the point clouds of each pier of the target bridge includes: The coordinate information of all real-time point clouds collected by the robot at the current moment in the carrier point cloud coordinate system after rotational alignment constitutes the first position vector; all the point clouds on each pier in the building point cloud coordinate system after rotational alignment constitute the second position vector of each pier; calculate the second similarity between the first position vector and each second position vector; Use the second similarity to screen the suspected shooting areas.

[0009] Preferably, the step of using the second similarity to screen the suspected shooting areas includes: Determine the area where the pier is located when the second similarity is greater than the preset second similarity threshold as the suspected shooting area.

[0010] Preferably, the step of screening the real shooting areas from all the suspected shooting areas according to the similarity of the position distribution of the corner points in the image and the corner points in each suspected shooting area includes: The coordinate information of all the corner points in the image constitutes the third position vector; All the corner points in each suspected shooting area constitute the reference point set of each suspected shooting area; project all the points in each reference point set in the camera shooting direction to obtain the corner points of the projection area of each reference point set, and the coordinate information of the corner points of the projection area of each reference point set constitutes the fourth position vector of each reference point set; Take the similarity between the third position vector and each fourth position vector as the matching coefficient between the corner points in the image and the reference point set of each suspected shooting area; Use the matching coefficient to screen the real shooting areas.

[0011] Preferably, the step of using the matching coefficient to screen the real shooting areas includes: Determine the suspected shooting area corresponding to the maximum value of the matching coefficient as the real shooting area.

[0012] Preferably, the step of positioning the robot according to the position distribution of the point cloud in the real shooting area and the orientation of the robot includes: Obtain the radial distance, polar angle, and azimuth angle of each point cloud in the real shooting area respectively; where the radial distance is the distance from the point cloud to the sensor; Based on the minimum and maximum values of the radial distances, the minimum and maximum values of the polar angles, and the minimum and maximum values of the azimuth angles of all the point clouds in the real shooting area, determine the suspected area where the robot is located; Position the robot according to the position distribution of the point cloud in the suspected area where the robot is located and the unit vector of the robot direction.

[0013] Preferably, positioning the robot according to the position distribution of the point cloud in the suspected area where the robot is located and the unit vector of the robot's direction includes: Obtaining the vector pointing from each point cloud in the suspected area where the robot is located to the origin of the polar coordinate as the feature vector of each point cloud; Calculating the cosine similarity between the unit vector of the robot's direction and the feature vector of each point cloud; determining the point cloud corresponding to the maximum cosine similarity as the position of the robot.

[0014] Preferably, determining the suspected area where the robot is located based on the minimum and maximum radial distances, the minimum and maximum polar angles, and the minimum and maximum azimuth angles of all point clouds in the real shooting area includes: Obtaining the radius range of the suspected area where the robot is located based on the minimum and maximum radial distances of all point clouds in the real shooting area; Obtaining the polar angle range of the suspected area where the robot is located based on the minimum and maximum polar angles of all point clouds in the real shooting area; Obtaining the azimuth angle range of the suspected area where the robot is located based on the minimum and maximum azimuth angles of all point clouds in the real shooting area; Determining the suspected area where the robot is located according to the radius range, the polar angle range, and the azimuth angle range.

[0015] The present invention has at least the following beneficial effects: The present invention uses different sensors to collect the point cloud data of the target bridge and the point cloud data of the objects in the environment where the target bridge is located, constructs a building point cloud coordinate system according to the point cloud data of the target bridge, and constructs a carrier point cloud coordinate system according to the point cloud data of the objects in the environment where the target bridge is located. The building point cloud coordinate system only contains the point cloud data of the structure of the target bridge itself, while the carrier point cloud coordinate system contains not only the point cloud data of the structure of the target bridge itself but also the point cloud data of other sundries in the environment where the target bridge is located. It is necessary to perform an alignment operation on the positions in the two coordinate systems. According to the similarity between the positions of all real-time point clouds collected by the robot at the current moment in the rotated and aligned coordinate system and the positions of the point clouds of each pier of the target bridge, the suspected shooting areas are screened, and the similarity of the position distribution of the corner points in the image captured by the detection robot and the corner points in each suspected shooting area is evaluated, and the real shooting area is screened from all suspected shooting areas, and then the robot is accurately positioned in combination with the orientation of the robot. The method provided in this embodiment combines the data information collected by multiple sensors, so as to complement the data between different sensors in dusty, dim and other environments, avoid the situation that the positioning is inaccurate due to environmental influence of a single sensor, ensure the anti-interference ability of the positioning while improving the positioning accuracy of the bridge detection robot. Description of the Drawings

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 It is a flowchart of a positioning method for a bridge inspection robot with multi-sensor information fusion provided by an embodiment of the present invention; Figure 2 It is a structural block diagram of a positioning system for a bridge inspection robot with multi-sensor information fusion provided by an embodiment of the present invention. Specific embodiments

[0018] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will, in conjunction with the drawings and preferred embodiments, detail a positioning method for a bridge inspection robot with multi-sensor information fusion proposed according to the present invention as follows.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0020] The following will specifically describe the specific solution of a positioning method for a bridge inspection robot with multi-sensor information fusion provided by the present invention in conjunction with the drawings.

[0021] An embodiment of a positioning method for a bridge inspection robot with multi-sensor information fusion: The specific scenario targeted by this embodiment is as follows: The bridge pier is an important part in the bridge design process. Its main function is to support the upper structure of the bridge, including the bridge span structure (such as beams, arches, suspension cables, etc.) and the bridge deck system (such as bridge deck paving, railings, etc.). Since a large part of the bridge pier is usually underwater and is constantly impacted and corroded by water flow all year round, the inspection of the bridge pier is usually relatively frequent to ensure the safety of the bridge. During the process of using a bridge inspection robot to inspect the bridge, it is often necessary to plan the inspection path. Before planning it, it is necessary to accurately position the bridge inspection robot. This embodiment provides a positioning method for a bridge inspection robot with multi-sensor information fusion to achieve the accurate positioning of the bridge inspection robot and ensure the accurate accuracy of the subsequent path planning.

[0022] This embodiment proposes a positioning method for a bridge inspection robot with multi-sensor information fusion, as Figure 1As shown in the figure, a positioning method for a bridge detection robot with multi-sensor information fusion in this embodiment includes the following steps: Step S1, obtain the point cloud data of the target bridge, the point cloud data of the objects in the environment where the target bridge is located, and the images captured by the target bridge detection robot; construct a building point cloud coordinate system according to the point cloud data of the target bridge, and construct a carrier point cloud coordinate system according to the point cloud data of the objects in the environment where the target bridge is located.

[0023] A variety of sensors are installed on the bridge detection robot, namely: a vision sensor, a lidar, and an inertial measurement unit. Among them, the vision sensor uses a camera to obtain environmental information for identifying the feature points of the bridge pier and obstacles; the lidar emits laser beams and receives reflected signals to obtain the three-dimensional point cloud data of the objects in the environment where the bridge is located for constructing an environmental map and precise ranging; the inertial measurement unit includes an accelerometer and a gyroscope, which measure the acceleration and angular velocity of the robot for attitude estimation and short-term position calculation.

[0024] Before the bridge detection robot starts working, first load the structural map of the target bridge into the internal system of the robot and pre-associate the point cloud data of the target bridge. Subsequently, start all the sensors on the bridge detection robot and ensure that all sensors can work normally. Further, deploy the bridge detection robot on the bridge pier and start detecting the appearance of the bridge pier. During the working process of the bridge detection robot, continuously collect the surrounding environmental information of the robot through the above-mentioned multiple sensors. The surrounding environmental information includes the point cloud data of the objects in the environment where the target bridge is located, and use the camera to capture the environmental image in real time. At the same time, upload the collected data to the control terminal of the inspection personnel through wireless / wired communication methods.

[0025] Further, all the point cloud data of the target bridge collected are mapped into a three-dimensional rectangular coordinate system, and the mapped coordinate system is denoted as the building point cloud coordinate system. The building point cloud coordinate system only includes the point cloud of the bridge itself. Among them, the point cloud of the bridge itself contains the accurate three-dimensional coordinate information of each part of the bridge (such as bridge piers, bridge decks, etc.). When constructing the building point cloud coordinate system, the implementer can determine the origin position and the axis direction of the building point cloud coordinates according to the structural characteristics and detection requirements of the bridge. For example, a certain key point of the bridge pier can be selected as the origin, the length direction of the bridge is taken as the X-axis, the width direction is taken as the Y-axis, and the height direction is taken as the Z-axis; all the point cloud data of the target bridge are mapped into this coordinate system, and thus the building point cloud coordinate system is obtained. In specific applications, the implementer sets the coordinate axes and the coordinate origin of this three-dimensional rectangular coordinate system according to the specific situation. Since there may be sundries such as stones and tree trunks underwater, and this information is not available in the building point cloud coordinate system, while the point cloud of the objects in the environment where the target bridge is located collected contains information about such objects, therefore, the current position and attitude information of the bridge inspection robot are obtained through an inertial measurement unit or other positioning devices. The attitude information includes the orientation of the robot (such as pitch angle, yaw angle, roll angle), etc. Taking the geometric center of the robot or the installation position of the lidar as the coordinate origin, taking the orientation of the robot as the positive direction of the X-axis, taking the vertical direction as the Z-axis direction, and taking the direction perpendicular to both the X-axis and the Z-axis as the Y-axis, all the point cloud data of the objects in the environment where the target bridge is located collected by the robot are mapped into a new three-dimensional rectangular coordinate system, and this coordinate system is denoted as the carrier point cloud coordinate system. The point cloud data of the objects in the environment where the target bridge is located exist in the carrier point cloud coordinate system. In specific applications, the implementer sets the coordinate axes and the coordinate origin of the carrier point cloud coordinate system according to the specific situation.

[0026] Step S2, according to the similarity between the relative distances of each point cloud in the carrier point cloud coordinate system from other point clouds and the relative distances of each point cloud of the bridge pier in the building point cloud coordinate system from other point clouds, the building point cloud coordinate system and the carrier point cloud coordinate system are rotationally aligned; according to the similarity between the positions of all the real-time point clouds collected by the robot at the current moment in the rotationally aligned coordinate system and the positions of the point clouds of each bridge pier of the target bridge, the suspected shooting areas are screened.

[0027] Since the actual positions represented by the same coordinates in the carrier point cloud coordinate system and the building point cloud coordinate system may be different, the acquired point cloud data and the constructed point cloud coordinate system are both affected by the robot's pose, which may lead to the misalignment between the carrier coordinate system and the building point cloud coordinate system. At the same time, in the underwater environment, the data collected by the robot in real time is affected by underwater sediment and floating objects different from the pre-loaded map, and directly comparing the data of the two coordinate systems is also inaccurate. Therefore, it is first necessary to perform a matching operation on the two coordinate systems to align the building point cloud coordinate system and the carrier point cloud coordinate system for subsequent positioning. The existing method is to directly compare the data collected by the lidar with the pre-loaded map data, that is, to directly compare the two point cloud coordinate systems, usually for all data (3D point clouds), but this method has a large amount of calculation and a slow processing speed. In this embodiment, a partial feature area of the pre-loaded map is intercepted, that is, a small area is matched, so as to realize the overall alignment of the two coordinate systems, with less processing volume and better calculation efficiency than the existing method.

[0028] Specifically, calculate the average distance between each point cloud in the carrier point cloud coordinate system and all other point clouds in the carrier point cloud coordinate system, and record this average distance as the first average distance. Each point cloud in the carrier point cloud coordinate system has a corresponding first average distance.

[0029] For any point cloud on any pier in the building point cloud coordinate system: calculate the average distance between this point cloud and all other point clouds on this pier, and record this average distance as the second average distance; according to the first difference between the first average distance and the second average distance, obtain the first similarity between each point cloud in the carrier point cloud coordinate system and this point cloud on this pier in the building point cloud coordinate system, and the first difference is negatively correlated with the first similarity.

[0030] In this embodiment, a specific calculation formula for the first similarity is given. The first similarity between the k-th point cloud in the carrier point cloud coordinate system and the j-th point cloud of the i-th pier in the building point cloud coordinate system can be expressed as: Where represents the first similarity between the k-th point cloud in the carrier point cloud coordinate system and the j-th point cloud of the i-th pier in the building point cloud coordinate system, represents the second average distance between the j-th point cloud of the i-th pier in the building point cloud coordinate system and all other point clouds on the i-th pier, represents the first average distance between the k-th point cloud in the carrier point cloud coordinate system and all other point clouds in the carrier point cloud coordinate system, represents a preset first adjustment parameter, represents the absolute value symbol, represents a normalization function.

[0031] In this embodiment, a preset first adjustment parameter is introduced into the calculation formula of the first similarity to prevent the denominator from being 0. In this embodiment, the preset first adjustment parameter is 0.01. In specific applications, the implementer can set it according to specific circumstances. It is used to represent the first difference. The greater the first difference, the lower the similarity between the two point clouds, that is, the smaller the first similarity between the k-th point cloud in the carrier point cloud coordinate system and the j-th point cloud of the i-th pier in the building point cloud coordinate system.

[0032] By using the above method, the first similarity between each point cloud in the carrier point cloud coordinate system and each point cloud on the pier in the building point cloud coordinate system can be obtained. The greater the first similarity, the higher the similarity degree between the corresponding two point clouds.

[0033] For any point cloud in the building point cloud coordinate system: If the maximum value of the first similarity between this point cloud and all point clouds in the carrier point cloud coordinate system is greater than the preset first similarity threshold, then this point cloud in the building point cloud coordinate system is used as the matching point of the point cloud in the carrier point cloud coordinate system corresponding to the maximum value. By using this method, the point clouds in the building point cloud coordinate system can be matched, and the matching points of each point cloud in the building point cloud coordinate system can be obtained in the carrier point cloud coordinate system. All the matching points in the carrier point cloud coordinate system form the pier in the carrier point cloud coordinate system. In this embodiment, the preset first similarity threshold is 0.8. In specific applications, the implementer can set it according to specific circumstances.

[0034] Further, according to the position of the pier in the carrier point cloud coordinate system and the position of the pier in the carrier point cloud coordinate system, determine the included angle of the pier in the carrier point cloud coordinate system and the pier in the building point cloud coordinate system in the X-axis direction and the included angle in the Y-axis direction and the included angle in the Z-axis direction , and then rotate and align the building point cloud coordinate system and the carrier point cloud coordinate system. After the rotation and alignment, each same coordinate information in the two coordinate systems represents the same position of the pier. The specific rotation and alignment method is a prior art and will not be elaborated here.

[0035] Next, combine the distance and angle information feedback by the real-time point cloud data to screen the suspected shooting area.

[0036] Specifically, in the preset order, splice the coordinate information of all real-time point clouds collected by the robot at the current moment in the rotated and aligned carrier point cloud coordinate system to form a first position vector. For example, the coordinate information of the real-time point cloud in the carrier point cloud coordinate system is respectively , , …, , the first position vector formed thereby is , where represents the coordinates of the first real-time point cloud in the carrier point cloud coordinate system, represents the coordinates of the second real-time point cloud in the carrier point cloud coordinate system, represents the coordinates of the nth real-time point cloud in the carrier point cloud coordinate system. Similarly, in the preset order, all the point clouds on each pier in the rotated and aligned building point cloud coordinate system are spliced to form the second position vector of each pier. Each pier has a corresponding second position vector. The construction method of the second position vector is similar to that of the first position vector, and will not be elaborated here; since the number of all real-time point clouds collected by the robot at the current moment may not be equal to the number of point clouds on each pier in the building point cloud coordinate system, when the two numbers are not equal, the vacant positions are filled with 0 to make the lengths of the first position vector and the second position vector equal. It should be noted that: the preset order can be from top to bottom and from left to right. In specific applications, the implementer can set according to the specific situation.

[0037] Calculate the cosine similarity between the first position vector and each second position vector respectively, and record this cosine similarity as the second similarity, that is, there is a second similarity between the first position vector and each second position vector. The calculation method of the cosine similarity is prior art and will not be elaborated here.

[0038] The greater the second similarity, the higher the position similarity between the real-time point cloud collected by the robot at the current moment and the point cloud on the corresponding pier in the rotated and aligned building point cloud coordinate system, and the more likely the area where the corresponding pier is located in the building point cloud coordinate system is the suspected shooting area. Based on this feature, the suspected shooting areas are screened using the second similarity next. Specifically, the area where the pier is located when the second similarity is greater than the preset second similarity threshold is determined as the suspected shooting area. In this embodiment, the preset second similarity threshold is 0.6. In specific applications, the implementer sets according to the specific situation.

[0039] So far, using the above method, multiple suspected shooting areas have been screened out.

[0040] Step S3, according to the position distribution similarity between the corner points in the image and the corner points in each suspected shooting area, screen the real shooting area from all suspected shooting areas.

[0041] Perform corner detection on the image captured by the target bridge detection robot at the current moment through a corner detection algorithm (Harris corner detection algorithm), and at the same time, perform corner detection on each suspected shooting area through a corner detection algorithm (Harris corner detection algorithm); the Harris corner detection algorithm is prior art and will not be elaborated here.

[0042] According to the preset order, splice the coordinate information of all corner points in the image to form a third position vector; according to the preset order, splice all corner points in each suspected shooting area to form a reference point set for each suspected shooting area, project all points in each reference point set in the camera shooting direction to obtain the corner points of the projection area of each reference point set, and the coordinate information of the corner points of the projection area of each reference point set forms the fourth position vector of each reference point set. It should be noted that: the construction methods of the third position vector and the fourth position vector are similar to those of the first position vector and the second position vector, except that the coordinate information of each point is two-dimensional information, and the construction methods of the third position vector and the fourth position vector will not be elaborated here too much.

[0043] Calculate the cosine similarity between the third position vector and each fourth position vector respectively, record the cosine similarity as the similarity between the third position vector and each fourth position vector, and then use this similarity as the matching coefficient between the corner points in the image and the reference point set of each suspected shooting area. Since in this embodiment, the cosine similarity between the third position vector and the fourth position vector is used as the similarity between the two, therefore, when the lengths of the third position vector and the fourth position vector are not equal, they are padded with 0 to make the lengths of the supplemented third position vector and the fourth position vector equal.

[0044] By using the above method, the matching coefficient between the corner points in the image and the reference point set of each suspected shooting area can be obtained. The larger the matching coefficient, the more likely the corresponding suspected shooting area is the real shooting area. Therefore, next, use the matching coefficient to screen the real shooting area from all suspected shooting areas. Specifically, determine the suspected shooting area corresponding to the maximum value of the matching coefficient as the real shooting area.

[0045] Step S4, position the robot according to the position distribution of the point cloud in the real shooting area and the orientation of the robot.

[0046] In this embodiment, the real shooting area of the camera is determined in step S4. Next, the robot will be accurately positioned based on the position distribution of the point cloud in the real shooting area and the current orientation of the robot.

[0047] Since the data collected by the lidar is affected by the underwater environment, resulting in certain distortion of the data, it is impossible to determine the position of the robot through the intersection of the polar coordinates of all point cloud data. Therefore, the true shooting area is determined through the polar coordinates of all point clouds, thereby narrowing the area for robot positioning. Place the robot at the position of each point cloud for direction comparison. If the direction vector of the shooting angle of the robot has the highest correlation with the shooting angle of a certain point cloud position, then the above image was probably taken at this position, and the relative position of the point cloud is the position of the robot. It should be noted that the origin of the polar coordinates is the central position of the image captured by the robot.

[0048] Specifically, respectively obtain the radial distance, polar angle, and azimuth angle of each point cloud within the true shooting area; among them, the radial distance is the distance from the point cloud to the sensor, the polar angle is the angle with the positive Z-axis, and the azimuth angle is the angle between the projection on the XOY plane and the positive X-axis.

[0049] Take the minimum value of the radial distances of all point clouds within the true shooting area as the lower limit value of the radius range of the suspected area, and take the maximum value of the radial distances of all point clouds within the true shooting area as the upper limit value of the radius range of the suspected area, thereby obtaining the radius range of the suspected area where the robot is located.

[0050] Take the minimum value of the polar angles of all point clouds within the true shooting area as the lower limit value of the polar angle range of the suspected area, and take the maximum value of the polar angles of all point clouds within the true shooting area as the upper limit value of the polar angle range of the suspected area, thereby obtaining the polar angle range of the suspected area where the robot is located.

[0051] Take the minimum value of the azimuth angles of all point clouds within the true shooting area as the lower limit value of the azimuth angle range of the suspected area, and take the maximum value of the azimuth angles of all point clouds within the true shooting area as the upper limit value of the azimuth angle range of the suspected area, thereby obtaining the azimuth angle range of the suspected area where the robot is located.

[0052] Take the origin of the coordinate system as the center point of the suspected area where the robot is located, and determine the suspected area where the robot is located according to the radius range, the polar angle range, and the azimuth angle range.

[0053] Furthermore, according to the real-time data collected by the inertial measurement unit, the real-time pose of the robot can be calculated, where the pose mainly includes the current orientation of the robot, that is, the direction of the robot. Obtain the vector pointing from each point cloud in the suspected area of the robot to the origin of the polar coordinates as the feature vector of each point cloud in the suspected area of the robot. Calculate the cosine similarity between the unit vector of the robot's direction and the feature vector of each point cloud; determine the point cloud corresponding to the maximum value of the cosine similarity as the position of the robot.

[0054] So far, the accurate positioning of the bridge inspection robot has been completed by using the method provided in this embodiment.

[0055] In this embodiment, the point cloud data of the target bridge and the point cloud data of the objects in the environment where the target bridge is located are collected by different sensors. The building point cloud coordinate system is constructed according to the point cloud data of the target bridge, and the carrier point cloud coordinate system is constructed according to the point cloud data of the objects in the environment where the target bridge is located. The building point cloud coordinate system only contains the point cloud data of the structure of the target bridge itself, while the carrier point cloud coordinate system contains not only the point cloud data of the structure of the target bridge itself but also the point cloud data of other sundries in the environment where the target bridge is located. It is necessary to perform an alignment operation on the positions in the two coordinate systems. According to the similarity between the positions of all the real-time point clouds collected by the robot at the current moment in the rotated and aligned coordinate system and the positions of the point clouds of each pier of the target bridge, the suspected shooting areas are screened. The similarity of the position distribution of the corner points in the images captured by the inspection robot and the corner points in each suspected shooting area is evaluated, and the real shooting areas are screened from all the suspected shooting areas. Furthermore, the robot is accurately positioned in combination with the orientation of the robot. The method provided in this embodiment fuses the data information collected by multiple sensors, thereby complementing the data between different sensors in dusty, dim and other environments, avoiding the situation of inaccurate positioning caused by the influence of the environment on a single sensor, and improving the positioning accuracy of the bridge inspection robot while ensuring the anti-interference ability of the positioning.

[0056] An embodiment of a positioning system for a bridge inspection robot with multi-sensor information fusion: Refer to Figure 2 , which shows the structural block diagram of a positioning system for a bridge inspection robot with multi-sensor information fusion provided by an embodiment of the present invention. The system includes a data acquisition module, a first screening module, a second screening module and a positioning module; Among them, the data acquisition module is used to obtain the point cloud data of the target bridge, the point cloud data of the objects in the environment where the target bridge is located, and the images captured by the target bridge inspection robot; construct the building point cloud coordinate system according to the point cloud data of the target bridge, and construct the carrier point cloud coordinate system according to the point cloud data of the objects in the environment where the target bridge is located; The first screening module is used to rotate and align the building point cloud coordinate system and the carrier point cloud coordinate system according to the similarity between the relative distances of each point cloud in the carrier point cloud coordinate system and other point clouds and the relative distances of each point cloud of the pier in the building point cloud coordinate system and other point clouds; screen the suspected shooting areas according to the similarity between the positions of all the real-time point clouds collected by the robot at the current moment in the rotated and aligned coordinate system and the positions of the point clouds of each pier of the target bridge; A second screening module, configured to screen real shooting areas from all suspected shooting areas according to the similarity of the position distribution of the corner points in the image and the corner points in each suspected shooting area; A positioning module, configured to position the robot according to the position distribution of the point cloud in the real shooting area and the orientation of the robot.

[0057] It should be understood that Figure 2 The structural block diagram and modules of a bridge inspection robot positioning system for multi-sensor information fusion shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated designed hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer-executable instructions and / or included in processor control code. For example, such code is provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules in this specification can be implemented not only by hardware circuits of programmable hardware devices such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable logic devices such as field programmable gate arrays, but also by software executed by various types of processors, or by a combination of the above hardware circuits and software (e.g., firmware).

[0058] For more details about the above-mentioned various modules, reference can be made to other parts of this specification, and no further elaboration will be provided here.

[0059] In other embodiments, a medium is further provided. The medium stores at least one program that can be run by a computer. When the at least one program is run by the computer, the computer is made to execute the steps in the method for positioning a bridge inspection robot for multi-sensor information fusion in the above embodiments. The medium can be a computer-readable storage medium.

[0060] Among them, the provided system and medium are both used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and no further elaboration will be provided here.

[0061] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-sensor information fusion bridge inspection robot positioning method, characterized in that: The method comprises the following steps: Obtain the point cloud data of the target bridge, the point cloud data of the objects in the environment where the target bridge is located, and the images captured by the target bridge detection robot; construct a building point cloud coordinate system based on the point cloud data of the target bridge, and construct a carrier point cloud coordinate system based on the point cloud data of the objects in the environment where the target bridge is located; According to the similarity between the relative distance between each point cloud and other point clouds in the carrier point cloud coordinate system and the relative distance between each point cloud of the bridge pier and other point clouds in the building point cloud coordinate system, the building point cloud coordinate system and the carrier point cloud coordinate system are rotated and aligned; according to the similarity between the position of all real-time point clouds collected by the robot at the current moment in the rotationally aligned coordinate system and the position of the point cloud of each bridge pier of the target bridge, the suspected shooting area is screened; Filtering a real shooting area from all the suspected shooting areas according to the position distribution similarity between the corner points in the image and the corner points in each suspected shooting area; The robot is positioned according to the position distribution of the point cloud in the actual shooting area and the orientation of the robot.

2. The method for positioning a bridge inspection robot based on multi-sensor information fusion according to claim 1, characterized in that: The method of rotating and aligning the building point cloud coordinate system and the carrier point cloud coordinate system according to the similarity between the relative distance between each point cloud and other point clouds in the carrier point cloud coordinate system and the relative distance between each point cloud of the bridge pier and other point clouds in the building point cloud coordinate system comprises: Calculate the first average distance between each point cloud in the carrier point cloud coordinate system and all other point clouds in the carrier point cloud coordinate system respectively; For any point cloud on any bridge pier in the building point cloud coordinate system: calculate the second average distance between any point cloud on any bridge pier and all other point clouds on any bridge pier; obtain a first similarity between each point cloud in the carrier point cloud coordinate system and any point cloud on any bridge pier according to a first difference between the first average distance and the second average distance, wherein the first difference is negatively correlated with the first similarity; According to the magnitude relationship of the first similarity between each point cloud in the carrier point cloud coordinate system and all point clouds on any one of the bridge piers, the point cloud in the carrier point cloud coordinate system is matched with the point cloud on any one of the bridge piers of the target bridge to determine the bridge pier in the carrier point cloud coordinate system; According to the position of the bridge pier in the carrier point cloud coordinate system and the position of the bridge pier in the building point cloud coordinate system, the building point cloud coordinate system and the carrier point cloud coordinate system are rotated and aligned.

3. The method for positioning a bridge inspection robot based on multi-sensor information fusion according to claim 2, characterized in that: According to the magnitude relationship of the first similarity between each point cloud in the carrier point cloud coordinate system and all point clouds on any bridge pier, matching the point cloud in the carrier point cloud coordinate system with the point cloud on any bridge pier of the target bridge, and determining the bridge pier in the carrier point cloud coordinate system, comprises: For any point cloud in the building point cloud coordinate system: if the maximum value of the first similarity between any point cloud in the building point cloud coordinate system and all point clouds in the carrier point cloud coordinate system is greater than a preset first similarity threshold, then any point cloud in the building point cloud coordinate system is used as a matching point of the point cloud in the carrier point cloud coordinate system corresponding to the maximum value; All matching points in the carrier point cloud coordinate system constitute the bridge pier in the carrier point cloud coordinate system.

4. The multi-sensor information fusion bridge inspection robot positioning method according to claim 1 is characterized in that: The screening of the suspected shooting area according to the similarity between the position of all the real-time point clouds collected by the robot at the current moment in the rotationally aligned coordinate system and the position of the point cloud of each pier of the target bridge includes: The coordinate information of all the real-time point clouds collected by the robot at the current moment in the carrier point cloud coordinate system after rotation and alignment constitutes a first position vector; all the point clouds on each pier in the building point cloud coordinate system after rotation and alignment constitute a second position vector of each pier; calculate the second similarity between the first position vector and each second position vector; The second similarity is used to screen suspected shooting areas.

5. The method for positioning a bridge inspection robot using multi-sensor information fusion according to claim 4, characterized in that: The using the second similarity to screen the suspected shooting areas includes: The area where the bridge pier is located when the second similarity is greater than a preset second similarity threshold is determined as a suspected shooting area.

6. The multi-sensor information fusion bridge inspection robot positioning method according to claim 1, characterized in that: The screening of the real shooting area from all the suspected shooting areas according to the position distribution similarity between the corner points in the image and the corner points in each suspected shooting area includes: The coordinate information of all corner points in the image constitutes a third position vector; All corner points in each suspected shooting area constitute a reference point set of each suspected shooting area; all points in each reference point set are projected in the camera shooting direction to obtain corner points of the projection area of ​​each reference point set, and the coordinate information of the corner points of the projection area of ​​each reference point set constitutes a fourth position vector of each reference point set; Using the similarity between the third position vector and each fourth position vector as a matching coefficient between the corner point in the image and a set of reference points of each suspected shooting area; The matching coefficient is used to screen the real shooting area.

7. The multi-sensor information fusion bridge inspection robot positioning method according to claim 6 is characterized in that: Using the matching coefficient to screen the real shooting area includes: The suspected shooting area corresponding to the maximum matching coefficient is determined as the real shooting area.

8. The multi-sensor information fusion bridge inspection robot positioning method according to claim 1, characterized in that: Positioning the robot according to the position distribution of the point cloud of the real shooting area and the orientation of the robot includes: Get the radial distance, polar angle and azimuth of each point cloud in the real shooting area respectively; the radial distance is the distance from the point cloud to the sensor; Determine the suspected area where the robot is located based on the minimum and maximum values ​​of the radial distance, the minimum and maximum values ​​of the polar angle, and the minimum and maximum values ​​of the azimuth angle of all point clouds in the real shooting area; The robot is positioned according to the position distribution of the point cloud in the area where the robot is suspected to be located and the unit vector of the robot's direction.

9. The multi-sensor information fusion bridge inspection robot positioning method according to claim 8, characterized in that: Positioning the robot according to the position distribution of the point cloud in the area where the robot is suspected to be located and the unit vector of the robot direction includes: Obtain a vector pointing from each point cloud in the area suspected to be where the robot is located to the origin of the polar coordinates as a feature vector of each point cloud; The cosine similarity between the unit vector of the robot direction and the feature vector of each point cloud is calculated; the point cloud corresponding to the maximum value of the cosine similarity is determined as the position of the robot.

10. The multi-sensor information fusion bridge inspection robot positioning method according to claim 8, characterized in that: The method of determining the suspected area where the robot is located based on the minimum and maximum values ​​of the radial distances, the minimum and maximum values ​​of the polar angles, and the minimum and maximum values ​​of the azimuth angles of all point clouds in the real shooting area includes: The radius range of the suspected area where the robot is located is obtained based on the minimum and maximum values ​​of the radial distances of all point clouds in the real shooting area; The polar angle range of the suspected area where the robot is located is obtained based on the minimum and maximum polar angles of all point clouds in the real shooting area; The azimuth range of the area where the robot is suspected to be located is obtained based on the minimum and maximum azimuth values ​​of all point clouds in the real shooting area; The suspected area where the robot is located is determined according to the radius range, the polar angle range, and the azimuth angle range.

Images