Positioning method, device and electronic equipment for industrial inspection robot

Abstract

This application discloses a positioning method, device, and electronic device for an industrial inspection robot. The method includes: acquiring a three-dimensional reference model, wherein the three-dimensional reference model is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions; acquiring environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process; extracting visual feature points from the environmental images corresponding to each inspection point, and performing feature matching between the visual feature points and three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs, wherein each feature pair is a point pair composed of a visual feature point and its corresponding three-dimensional structural feature point; and performing positioning calculation processing on the feature pairs to obtain the positioning information of the industrial inspection robot in the target industrial scene. This application solves the technical problem that inspection robots rely on static environment modeling and are susceptible to dynamic interference (such as equipment movement and changes in lighting) in indoor environments without GPS, leading to positioning drift.

Description

Technical Field

[0001] This application relates to the field of industrial inspection, and more specifically, to a positioning method, device, and electronic equipment for an industrial inspection robot. Background Technology

[0002] Factories have an urgent need for digital assets (3D environmental models, equipment status data). Simultaneously, high-risk inspection scenarios involving toxic or hazardous substances exist within factories. Traditional inspection methods cannot simultaneously generate dynamic, high-precision environmental digital twin data (for instrument identification and temperature measurement in specific areas), making it difficult to support subsequent production line optimization and maintenance decisions. The core requirements for inspection work in industrial indoor scenarios (such as factory workshops, equipment rooms, and storage areas) are high precision, full coverage, and dynamic adaptation. However, traditional inspection models have long faced multiple technical bottlenecks:

[0003] Indoor environments lack stable Global Positioning System (GPS) signals. Traditional positioning methods (inertial navigation, QR code positioning, 4G / 5G, etc.) are easily affected by equipment obstructions, narrow passages, and changes in lighting, resulting in significant positioning accuracy drift and failing to meet the baseline requirements for millimeter-level inspections. Factory indoor environments exhibit dynamic changes (equipment relocation, material stacking adjustments, temporary obstacle additions, facility aging and wear). Static map modeling requires periodic manual resurveying and updates, leading to high maintenance costs, slow response times, and potential missed inspections and positioning failures. Manual inspections using portable handheld devices are limited by accessibility to high-risk areas (toxic, flammable, high-voltage equipment areas, confined spaces) and complex terrain (equipment undersides, narrow passages), and rely on human experience, resulting in inconsistent inspection accuracy and low efficiency. Inspection robots, while relying on static environmental modeling, are susceptible to dynamic interference (such as equipment movement and changes in lighting) in GPS-free indoor environments, leading to positioning drift.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a positioning method, device, and electronic device for an industrial inspection robot, which at least solves the technical problem that the inspection robot relies on static environment modeling and is susceptible to dynamic interference (such as equipment movement and changes in lighting) in indoor environments without GPS, leading to positioning drift.

[0006] According to one aspect of the embodiments of this application, a positioning method for an industrial inspection robot is provided, comprising: acquiring a three-dimensional reference model, wherein the three-dimensional reference model is a three-dimensional model obtained by modeling a target industrial scene from multiple dimensions; acquiring environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process; extracting visual feature points from the environmental images corresponding to each inspection point, and performing feature matching between the visual feature points and three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs, wherein the feature pairs are point pairs composed of visual feature points and three-dimensional structural feature points corresponding to the visual feature points; and performing positioning calculation processing on the feature pairs to obtain positioning information of the industrial inspection robot in the target industrial scene.

[0007] Optionally, the 3D baseline model is constructed as follows: receiving parameter configuration instructions and adjusting the equipment parameters of the industrial inspection robot according to the instructions, wherein the industrial inspection robot is a multi-legged robot equipped with multiple types of sensors, including a 3D scanner, a multi-line LiDAR, and a panoramic camera; the industrial inspection robot moves in the target industrial scene according to a preset scanning path, and collects multiple types of field data through the multiple sensors during the movement, including first-type point cloud data collected by the 3D scanner, second-type point cloud data collected by the multi-line LiDAR, and RGB image data collected by the panoramic camera. The first-type point cloud data is used to characterize the structural details of the industrial equipment in the target industrial scene, and the second-type point cloud data is used to characterize the global spatial contour of the industrial environment in which the target industrial scene is located; and 3D modeling is performed based on the multiple types of field data to obtain the 3D baseline model.

[0008] Optionally, a 3D baseline model is obtained by performing 3D modeling based on multiple types of field data, including: performing unified data processing on multiple types of field data to obtain processed first-type point cloud data, processed second-type point cloud data, and processed RGB image data, wherein the processed first-type point cloud data, processed second-type point cloud data, and processed RGB image data are located in the same coordinate system; stitching together multiple frames of point clouds from the processed first-type point cloud data to obtain a complete point cloud, and constructing an initial 3D model based on the complete point cloud; using multiple frames of point clouds from the processed second-type point cloud data to perform contour correction processing on the areas to be improved in the initial 3D model to obtain a corrected 3D model, wherein the areas to be improved are the missing or distorted areas in the initial 3D model; and performing texture mapping and color rendering processing on the corrected 3D model using the processed RGB image data to obtain a 3D baseline model.

[0009] Optionally, feature matching is performed between visual feature points and three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. This includes: based on the extrinsic calibration relationship between the three-dimensional scanner and the panoramic camera, multiple three-dimensional structural feature points in the three-dimensional reference model are projected onto the image coordinate system of the panoramic camera to obtain a theoretical projection point set, wherein each projected three-dimensional structural feature point in the theoretical projection point set is a comparison feature point; feature matching is performed between visual feature points and the theoretical projection point set to obtain feature pairs, wherein the feature pairs are valid feature pairs, and the distance between the feature descriptors corresponding to the visual feature points and the three-dimensional structural feature points that make up the feature pairs is less than a distance threshold.

[0010] Optionally, feature matching is performed between visual feature points and the theoretical projection point set to obtain feature pairs, including: obtaining the target feature descriptor corresponding to the visual feature point and the feature descriptors corresponding to multiple contrast feature points in the theoretical projection point set; calculating the distance between the target feature descriptor and the feature descriptor corresponding to each contrast feature point to obtain multiple distance values; determining the minimum value among the multiple distance values ​​as the target distance value, and determining the contrast feature point corresponding to the target distance value as the matching feature point corresponding to the visual feature point; determining the three-dimensional structural feature point corresponding to the matching feature point as the target matching feature point; determining the visual feature point and the target matching feature point as an initial matching pair, and determining the initial matching pair as a feature pair if the comparison value corresponding to the initial matching pair is less than the distance threshold, wherein the comparison value is the quotient of the target distance value and the second smallest value among the multiple distance values.

[0011] Optionally, the feature pairs are processed for localization to obtain the localization information of the industrial inspection robot in the target industrial scene. This includes: acquiring the camera intrinsic parameter matrix of the panoramic camera used to collect environmental images, wherein the industrial inspection robot is equipped with multiple types of sensors, including a 3D scanner, a multi-line LiDAR, and a panoramic camera; acquiring the image pixels corresponding to the visual feature points in the feature pairs and the 3D point cloud spatial coordinates of the target matching feature points; determining the initial pose data based on the camera intrinsic parameter matrix, the image pixels corresponding to the visual feature points, and the 3D point cloud spatial coordinates of the target matching feature points, wherein the initial pose data is the rotation matrix and translation vector of the panoramic camera in the target industrial scene; and determining the 3D coordinates and posture data of the industrial inspection robot in the target industrial scene based on the initial pose data.

[0012] Optionally, determining the positioning information of the industrial inspection robot in the target industrial scene based on the initial pose data includes: acquiring a rigid extrinsic transformation matrix, wherein the rigid extrinsic transformation matrix is ​​a fixed spatial transformation relationship between the panoramic camera and the industrial inspection robot body, and the rigid extrinsic transformation matrix is ​​determined after pre-calibrating the installation positions of the panoramic camera and the robot body; converting the rotation matrix and translation vector in the initial pose data into the target rotation matrix and target translation vector of the industrial inspection robot in the target industrial scene based on the rigid extrinsic transformation matrix, and determining the target rotation matrix and target translation vector as the posture data of the industrial inspection robot in the target industrial scene; determining the three-dimensional coordinates of the industrial inspection robot in the target industrial scene based on the three spatial coordinate components of the target translation vector; and determining the three-dimensional coordinates and posture data of the industrial inspection robot in the target industrial scene as positioning information.

[0013] Optionally, the method further includes: at preset update intervals or upon receiving an update instruction, an industrial inspection robot moves along a preset scanning path in the target industrial scene, and during the movement, re-collects multiple types of field data through multiple sensors; by comparing the re-collected multiple types of field data with the three-dimensional reference model, the changed area is obtained, wherein the changed area includes at least one of the following: equipment relocation, addition of obstacles, or marking of changed areas; the changed area is updated based on the re-collected multiple types of field data to obtain an updated three-dimensional reference model.

[0014] According to another aspect of the embodiments of this application, a positioning device for an industrial inspection robot is also provided, comprising: a first acquisition module for acquiring a three-dimensional reference model, wherein the three-dimensional reference model is a three-dimensional model obtained by modeling a target industrial scene from multiple dimensions; a second acquisition module for acquiring environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process; a matching module for extracting visual feature points from the environmental images corresponding to each inspection point, and performing feature matching between the visual feature points and three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs, wherein the feature pairs are point pairs composed of visual feature points and three-dimensional structural feature points corresponding to the visual feature points; and a positioning module for performing positioning calculation processing on the feature pairs to obtain positioning information of the industrial inspection robot in the target industrial scene.

[0015] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, and the program controls the device where the non-volatile storage medium is located to execute the above-mentioned positioning method of the industrial inspection robot when it runs.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the program executes the above-described positioning method for an industrial inspection robot when it runs.

[0017] According to another aspect of the embodiments of this application, a computer program product is also provided, including computer instructions, which, when executed by a processor, implement the above-described positioning method for an industrial inspection robot.

[0018] In this embodiment, a three-dimensional reference model is acquired, which is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions. Environmental images corresponding to each inspection point are acquired by the industrial inspection robot during the inspection process. Visual feature points are extracted from the environmental images corresponding to each inspection point, and feature matching is performed between the visual feature points and the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. Each feature pair is a point pair composed of a visual feature point and its corresponding three-dimensional structural feature point. The feature pairs are then processed for positioning to obtain the positioning information of the industrial inspection robot in the target industrial scene. This method solves the technical problem of the inspection robot relying on static environment modeling and being susceptible to dynamic interference (such as equipment movement or changes in lighting) in indoor environments without GPS, leading to positioning drift. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a hardware structure block diagram of a computer terminal for implementing a positioning method for an industrial inspection robot, according to an embodiment of this application.

[0021] Figure 2 This is a flowchart of a positioning method for an industrial inspection robot according to an embodiment of this application;

[0022] Figure 3 This is a flowchart of another positioning method for an industrial inspection robot provided according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a first type of industrial inspection robot provided according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of a second type of industrial inspection robot provided according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the positioning device for an industrial inspection robot according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] The information collected in this application embodiment is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant regions, and necessary confidentiality measures have been taken. It does not violate public order and good morals, and provides corresponding operation entry points for users to choose to authorize or reject the automated decision results. If the user chooses to reject, the process will proceed to the expert decision-making process.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained below:

[0030] Quadruped robots: Bionic quadruped mobile devices equipped with sensors such as inertial navigation, vision, and laser scanning, as well as intelligent control systems, can perform inspections in complex industrial areas, replacing manual labor in high-risk tasks and improving operational safety and efficiency.

[0031] 3D modeling: 3D point cloud data of industrial scenes are collected by equipment such as 3D laser scanners. After algorithm processing, feature extraction and model reconstruction, a digital 3D model corresponding to the physical environment is generated, providing a high-precision spatial reference for robot positioning and navigation and environmental change monitoring.

[0032] 3D scanner: A non-contact, high-precision device that uses lasers to collect three-dimensional point cloud data of objects or scenes, and then uses algorithms to reconstruct and generate three-dimensional models to support industrial inspection and positioning, environmental modeling, and defect detection.

[0033] Three-dimensional feature points, such as equipment corners, structural outlines, fixed markings, pipeline interfaces, and beam-column nodes, are spatially unique three-dimensional features in the environment, providing core basis for positioning and comparison.

[0034] SLAM modeling: Simultaneous Localization and Mapping (SLAM) enables robots to autonomously locate themselves while building a 3D map of their environment, providing a spatial reference for navigation in indoor scenes without GPS.

[0035] ORB-SLAM is a visual synchronous localization and mapping technology based on Oriented FAST and Rotated BRIEF (ORB) feature points. It can realize real-time pose estimation of robots in unknown environments and creation of 3D environment maps through image information acquired by cameras.

[0036] IMU data consists of motion parameters such as angular velocity and linear acceleration of a carrier, collected by the Inertial Measurement Unit (IMU) through core sensors such as built-in gyroscopes and accelerometers. It can be used to calculate the carrier's real-time attitude, position, and trajectory.

[0037] Robustness: A core indicator that measures the stability of a system, device, or algorithm in maintaining its original functions and performance under disturbances such as external disturbances, changes in internal parameters, and environmental uncertainties.

[0038] Random Sampling Consensus Algorithm (RANSAC): This is a robust model fitting method that achieves iterative sampling verification. It can accurately fit the optimal mathematical model from a dataset containing a large amount of noise and outliers, and is widely used in model solving scenarios for computer vision and intelligent equipment.

[0039] Perspective-n-Point (PnP) algorithm: This is a pose estimation algorithm based on the correspondence between 3D spatial points and 2D image projection points, which solves for the camera rotation matrix and translation vector. It is often combined with RANSAC to improve accuracy and is suitable for fields such as intelligent equipment positioning and navigation.

[0040] Target method: refers to using a "precisely sized and clearly defined standard reference object" to calibrate the spatial coordinate relationship between different devices.

[0041] Robot superstructure: All working equipment installed on the robot body / top, as opposed to the robot chassis (walking and driving parts).

[0042] In related technologies, the core requirements for inspection work in industrial indoor scenarios (such as factory workshops, equipment rooms, and storage areas) are high precision, full coverage, and dynamic adaptation. However, traditional inspection methods have long faced multiple technical bottlenecks:

[0043] Indoor environments lack stable Global Positioning System (GPS) signals, and traditional positioning methods (inertial navigation, QR code positioning, 4G / 5G, etc.) are easily affected by equipment obstructions, narrow passages, and changes in lighting, resulting in significant positioning accuracy drift and failing to meet the benchmark requirements for millimeter-level inspections. Factory indoor environments exhibit dynamic changes (equipment relocation, material stacking adjustments, temporary obstacle additions, facility aging and wear), requiring regular manual resurveying and updates of static map modeling, leading to high maintenance costs, slow response times, and potential missed inspections and positioning failures. Manual inspections using portable handheld devices are limited by accessibility to high-risk areas (toxic, flammable, high-voltage equipment areas, confined spaces) and complex terrain (equipment undersides, narrow passages), and rely on human experience, resulting in inconsistent inspection accuracy and low efficiency. Therefore, inspection robots rely on static environment modeling and are susceptible to dynamic interference (such as equipment movement and changes in lighting) in GPS-free indoor environments, leading to positioning drift. To address this issue, this application provides relevant solutions, detailed below.

[0044] According to an embodiment of this application, an embodiment of a positioning method for an industrial inspection robot is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0045] The methods and embodiments provided in this application can be executed on a computer terminal or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a positioning method for an industrial inspection robot is shown. Figure 1 As shown, the computer terminal 10 may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0046] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10. As involved in the embodiments of this application, the data processing circuits serve as processor control (e.g., selection of a variable resistor termination path connected to an interface).

[0047] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the positioning method of the industrial inspection robot in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned positioning method of the industrial inspection robot. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0048] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0049] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10.

[0050] Under the aforementioned operating environment, embodiments of this application provide a positioning method for an industrial inspection robot, such as... Figure 2 The diagram shows a flowchart of a positioning method for an industrial inspection robot according to an embodiment of this application, including:

[0051] Step S202: Obtain the three-dimensional reference model.

[0052] In the technical solution provided in step S202, the three-dimensional reference model is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions.

[0053] The 3D baseline model is constructed as follows: The industrial inspection robot receives parameter configuration instructions and adjusts its equipment parameters accordingly. The industrial inspection robot is a multi-legged robot equipped with multiple sensors, including a 3D scanner, a multi-line LiDAR, and a panoramic camera. The industrial inspection robot moves through the target industrial scene according to a preset scanning path, collecting various types of on-site data through the sensors during its movement. These on-site data include first-type point cloud data collected by the 3D scanner, second-type point cloud data collected by the multi-line LiDAR, and RGB image data collected by the panoramic camera. The first-type point cloud data is used to characterize the structural details of the industrial equipment in the target industrial scene, and the second-type point cloud data is used to characterize the global spatial contour of the industrial environment in which the target industrial scene is located. Based on these multiple types of on-site data, a 3D baseline model is obtained. There are several ways to achieve a 3D baseline model based on multiple types of field data. For example, one method involves uniformly processing the multiple types of field data to obtain processed first-type point cloud data, processed second-type point cloud data, and processed RGB image data. The processed first-type point cloud data, processed second-type point cloud data, and processed RGB image data are located in the same coordinate system. Multiple frames of point clouds from the processed first-type point cloud data are then stitched together to obtain a complete point cloud, and an initial 3D model is constructed based on this complete point cloud. Using multiple frames of point clouds from the processed second-type point cloud data, contour correction processing is performed on the areas to be improved in the initial 3D model to obtain a corrected 3D model. The areas to be improved are the missing or distorted areas in the initial 3D model. Finally, texture mapping and color rendering processing are performed on the corrected 3D model using the processed RGB image data to obtain the 3D baseline model. The construction process of the 3D baseline model is described in detail below.

[0054] In some embodiments of this application, the industrial inspection robot is a multi-legged robot (e.g., a quadruped robot, which is currently a more advanced inspection robot carrier) equipped with multiple types of sensors. These sensors include a 3D scanner, a multi-line LiDAR, and a panoramic camera. The panoramic camera is mounted on the industrial inspection robot body via a gimbal. It first receives parameter configuration instructions (e.g., remote control instructions sent by an engineer at the control terminal, which include adjustment values ​​for the industrial inspection robot's equipment parameters) and adjusts these parameters accordingly. These parameters include: the type of target device to be scanned, the scanning accuracy threshold (millimeter level), the robot's gait mode (adaptive modes such as normal walking, low crawling, and narrow passages), and safe operating boundaries (avoiding high-voltage equipment and flammable / explosive areas). After configuring the industrial inspection robot's parameters, the robot moves within the target industrial scene according to a preset scanning path, collecting various types of on-site data through the sensors during the movement. The target industrial scene can be a plant area, an indoor inspection device area, a reaction area, or a separation and purification area in a process industry.

[0055] Industrial inspection robots move along preset scanning paths within target industrial scenarios, collecting various types of on-site data through multiple sensors during their movement. The robot's posture can be adjusted, such as tilting up or down by 25°, to allow for changes in the angle and position of the gimbal (for mounting a panoramic camera) and 3D scanner. The collected data specifically comprises various on-site data for different feature reference points. These feature reference points are representative locations within a preset scanning path configured by the engineer. Examples of feature reference points include structurally stable and long-term unchanging physical features in the industrial scene, such as load-bearing columns, explosion-proof walls, and permanent marking devices; as well as geometrically identifiable points such as indoor pump rooms (process pumps, delivery pumps, booster pump rooms, etc.), compressor plants (process gas, hydrogen, nitrogen, air compression, etc.), equipment and instruments, signs, flange edges, and pipe welds. It is important to note that during the process of industrial inspection robots collecting various types of on-site data through multiple sensors while in motion, in order to further avoid monitoring blind spots, a real-time point cloud coverage analysis algorithm is used to dynamically monitor the scanning blind spots collected by multiple sensors. The industrial inspection robot automatically adjusts its gait parameters (stride length, turning angle, and travel speed) based on the location of the blind spots, and achieves all-round approach to the target area (various feature reference points) by climbing steps, crossing ditches, and getting close to equipment bases. At the same time, the two-axis stabilized gimbal on the 3D scanner automatically performs angle compensation according to the pose changes (pitch and roll) of the industrial inspection robot, and corrects the spatial pose of the 3D scanner in real time. During the scanning process, the quadruped robot dynamically adjusts its own position, and the 3D scanner synchronously adjusts its lifting height and scanning angle. This dual optimization ensures the integrity of the scanning coverage and the accuracy of the data, and avoids scanning angle deviation caused by the shaking of the industrial inspection robot.

[0056] The various types of field data collected through the above methods enable the robot's onboard equipment to acquire data from multiple angles and dimensions within the field area, adapting to the complex indoor environment of industrial scenarios, effectively avoiding positioning blind spots, and ensuring comprehensive data coverage. Simultaneously, a high-precision timestamp synchronization trigger mechanism achieves millisecond-level synchronization of LiDAR point cloud acquisition, 3D scanner high-density point cloud acquisition, and panoramic camera high-resolution RGB image acquisition, ensuring the spatiotemporal consistency of the three types of data. These multiple types of field data provide a solid data foundation for constructing a 3D benchmark model, forming a 3D benchmark model with dual guarantees of structural accuracy and spatial integrity. During subsequent inspections, visual feature points in the environmental images collected by the inspection robot can be matched with 3D structural feature points in this 3D benchmark model with high precision. This effectively overcomes feature loss or ambiguity caused by a single data source in dynamic industrial scenarios, significantly improving the robustness of feature matching and the stability of positioning calculations, ultimately achieving high-precision, adaptive positioning of industrial robots in complex and dynamic scenarios without GPS.

[0057] After obtaining the above-mentioned various types of field data, a 3D model was created based on the data to obtain a 3D baseline model. The specific modeling process is detailed below:

[0058] To achieve fusion modeling of multiple types of field data, the extrinsic parameters of the panoramic camera and 3D scanner were pre-calibrated using the target method. Target features were simultaneously observed by the 3D scanner and panoramic camera. Combining the camera's intrinsic parameter matrix with the scanner's factory calibration parameters, the rigid body transformation relationship between the 3D scanner and the panoramic camera relative to the robot's coordinate system was calculated. Furthermore, SLAM pose estimation using the inertial measurement unit (IMU) and LiDAR on the robot was employed to obtain the relative pose relationship between the multi-line LiDAR and the 3D scanner. Finally, a unified robot base coordinate system was established as the global reference coordinate system. The process of unifying the processing of multiple types of field data involves mapping various types of field data located in different coordinate systems (first-type point cloud data (high-density, high-precision scanned point cloud) acquired by the 3D scanner, second-type point cloud data (wide-area, medium-density LiDAR point cloud) acquired by the multi-line LiDAR, and RGB image data acquired by the panoramic camera) to the same coordinate system (the aforementioned robot base coordinate system). The processed first-type point cloud data, the processed second-type point cloud data, and the processed RGB image data are all located in the same coordinate system.

[0059] The first type of processed point cloud data comes from a 3D scanner and has high-density, high-precision (millimeter-level) geometric features. However, a single-frame scan is limited by the scanning viewpoint and robot pose, covering only a local area. To construct a globally consistent complete 3D structure, the following second step is performed (stitching together multiple frames of point clouds from the processed first type of point cloud data to obtain a complete point cloud): The processed first type of point cloud data is processed frame by frame according to the time series. Each frame of point cloud carries pose information (rotation matrix and translation vector) jointly estimated by the robot's IMU and multi-line LiDAR SLAM. This pose information provides an initial transformation estimate for point cloud registration. Subsequently, an improved Iterative ClosestPoint (ICP) algorithm is used to accurately register point clouds between adjacent frames: in the current frame point cloud, for each point, the nearest neighbor point in the previous frame point cloud is searched to construct an initial correspondence; based on the corresponding point pairs, the rigid body transformation matrix is ​​solved using the least squares method to minimize the sum of squared Euclidean distances between point pairs; the above process is repeated iteratively until the transformation matrix converges or the number of iterations reaches a preset upper limit, completing the inter-frame registration. All point cloud data that have completed inter-frame registration are unified into the robot base coordinate system to form an aligned point cloud set, thus obtaining a complete point cloud. The third step: based on the complete point cloud, a modeling method (e.g., the Poisson reconstruction algorithm) is used to perform 3D modeling to obtain an initial 3D model. This initial 3D model has the macroscopic geometric contours of the main structures in the industrial scene (such as walls, pipes, and equipment shells), but due to scanning occlusion, reflection interference, or the influence of moving objects, there are missing, sparse, or blurred areas in some regions that need to be improved.

[0060] The processed second type of point cloud data comes from multi-line LiDAR, which has a wide scanning range and strong penetration, making it particularly suitable for acquiring areas that cannot be covered by 3D scanners due to limited viewing angles (such as the bottom of equipment, narrow passages, the back of columns, pipe gaps, etc.). Therefore, multiple frames of point clouds in the processed second type of point cloud data can be used to perform contour correction processing on the areas to be improved in the initial 3D model (e.g., missing or distorted areas in the initial 3D model caused by occlusion or reflection) to obtain the corrected 3D model: the point clouds in the processed second type of point cloud data are accumulated point by point to the local mesh surface of the area to be improved according to their spatial positions. The fused local mesh is then smoothed by normals and seamlessly stitched at the boundaries to cover the missing or distorted parts in the original model, thus obtaining the corrected 3D model. Finally, high-fidelity texture reconstruction is performed using the processed RGB image data: texture mapping and color rendering are applied to the corrected 3D model using the processed RGB image data to obtain a 3D baseline model. This can be achieved by using the extrinsic parameter matrices of the calibrated panoramic camera and robot base to back-project the pixel coordinates of each RGB image onto the surface of the corrected 3D model, thus performing texture mapping and color rendering to obtain the 3D baseline model. This 3D baseline model is a high-precision colored 3D mesh model containing spatial coordinates, device outline, and structural features.

[0061] Step S204: Obtain the environmental image corresponding to each inspection point collected by the industrial inspection robot during the inspection process.

[0062] In the technical solution provided in step S204, during the industrial inspection process, an industrial inspection robot performs inspections according to a preset work path. The preset work path is a pre-configured inspection route for the robot, containing multiple inspection points (each inspection point corresponds to a location in the target industrial scene, where at least one of the aforementioned feature reference points can be observed). The preset work path can be modified, deleted, or added to at any time in response to the engineer's modification instructions. It is important to note that steps S204-S208 are a real-time, continuous process. That is, after acquiring the environmental image corresponding to each inspection point collected by the industrial inspection robot during the inspection process in step S204, the positioning calculation process in steps S206-S208 is immediately performed to calculate the true coordinate information of the industrial inspection robot in real time, until the entire inspection process of the industrial inspection robot is completed. At each inspection point, the industrial inspection robot acquires multiple frames of images according to a preset posture sequence (e.g., tilting up 25°, tilting down 25°). These frames are captured in real-time by a panoramic camera mounted on the robot (located on a dual-light gimbal). After acquiring all images for that inspection point, the image with the highest clarity is selected as the corresponding environmental image. During the inspection process, the dual-light gimbal automatically performs image stabilization compensation based on data from the robot's posture sensors. Simultaneously, it adaptively adjusts the panoramic camera's exposure parameters according to ambient lighting conditions, ensuring clear and high-contrast images are captured even under complex lighting conditions such as strong light, backlight, and dim lighting. Furthermore, the integrated design of the 3D scanner and the robot body is not a simple assembly; its structure and function are deeply integrated, enabling it to adapt to complex terrain environments such as slopes and steps, achieving integrated inspection operations across all scenarios. This integrated design effectively avoids the compatibility defects of split structures, significantly improving product operational stability and overall system reliability, ensuring continuous and uninterrupted inspection.

[0063] Step S206: Extract visual feature points from the environmental image corresponding to each inspection point, and perform feature matching between the visual feature points and the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs.

[0064] In the technical solution provided in step S206, there are multiple ways to perform feature matching between visual feature points and three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. For example, based on the extrinsic calibration relationship between the three-dimensional scanner and the panoramic camera, multiple three-dimensional structural feature points in the three-dimensional reference model are projected onto the image coordinate system of the panoramic camera to obtain a theoretical projection point set. Each projected three-dimensional structural feature point in the theoretical projection point set is a comparison feature point. The visual feature points are then matched with the theoretical projection point set to obtain feature pairs. These feature pairs are valid feature pairs, and the distance between the feature descriptors corresponding to the visual feature points and the three-dimensional structural feature points that make up the feature pairs is less than a distance threshold.

[0065] In the above steps, there are several ways to perform feature matching between visual feature points and the theoretical projection point set to obtain feature pairs. For example: obtain the target feature descriptor corresponding to the visual feature point and the feature descriptors corresponding to multiple contrast feature points in the theoretical projection point set; calculate the distance between the target feature descriptor and the feature descriptor corresponding to each contrast feature point to obtain multiple distance values; determine the minimum value among the multiple distance values ​​as the target distance value, and determine the contrast feature point corresponding to the target distance value as the matching feature point corresponding to the visual feature point; determine the three-dimensional structural feature point corresponding to the matching feature point as the target matching feature point; determine the visual feature point and the target matching feature point as the initial matching pair, and if the comparison value corresponding to the initial matching pair is less than the distance threshold, determine the initial matching pair as the feature pair, where the comparison value is the quotient of the target distance value and the second smallest value among the multiple distance values.

[0066] In some embodiments of this application, to achieve high-precision positioning of the inspection robot in GPS-free environments, a geometric correspondence between the image and the model is constructed by accurately matching visual feature points with three-dimensional structural feature points in a three-dimensional reference model. Inverse positioning is achieved through projection alignment and descriptor matching, as follows:

[0067] Step S206.1: After completing the construction of the 3D reference model, extract all 3D structural feature points that can be used as positioning basis from the 3D reference model, including: endpoints of equipment and instrument scale lines, corners of signs, flange edge contour points, pipe weld joints, intersections of fixed beams and columns, and other feature points with stable geometric shapes that can be clearly identified in the image. Based on the extrinsic calibration relationship between the 3D scanner and the panoramic camera, project multiple 3D structural feature points in the 3D reference model onto the image coordinate system of the panoramic camera to obtain a theoretical projection point set: Obtain the extrinsic calibration relationship between the 3D scanner and the panoramic camera (i.e., the rigid body transformation matrix of the camera coordinate system relative to the 3D scanner coordinate system, which has been calibrated in the modeling stage using the target method), transform each 3D structural feature point (located in the 3D scanner coordinate system) from its world coordinate system (i.e., the 3D scanner coordinate system) to the panoramic camera coordinate system, and then, based on the intrinsic parameter matrix of the panoramic camera, project the 3D points in the camera coordinate system onto the image plane to obtain their theoretical projection points in the image coordinate system. All 3D structural feature points, after the above projection, constitute a theoretical projection point set. Each projected 3D structural feature point in the theoretical projection point set is a contrast feature point, representing the image position that should be observed when the robot is in an ideal pose under the current camera view.

[0068] Step S206.2: An improved ORB feature detection and description algorithm is used to extract visual feature points from the environmental image corresponding to each inspection point. Visual feature points are corner points and structural points detected in the environmental image that have high response values, high contrast, and clear edges. Then, the target feature descriptors corresponding to the visual feature points and the feature descriptors corresponding to multiple contrast feature points in the theoretical projection point set are obtained. Feature descriptors are numerical vectors used to characterize the visual characteristics of local regions in an image or video. These vectors can reflect information such as texture, gradient, and grayscale mode of the local image around the feature point, and possess scale invariance, rotation invariance, or illumination invariance, enabling robust matching between different images or frames.

[0069] Step S206.3: Calculate the distance between the target feature descriptor and the feature descriptor corresponding to each comparison feature point (e.g., Hamming distance; the smaller the Hamming distance, the more similar the two descriptors are, and the higher the matching probability). Obtain multiple distance values, determine the minimum value among the multiple distance values ​​as the target distance value, and determine the comparison feature point corresponding to the target distance value as the matching feature point corresponding to the visual feature point. Determine the 3D structural feature point corresponding to the matching feature point as the target matching feature point (i.e., the 3D structural feature point before the matching feature point is projected). Determine the visual feature point and the target matching feature point as the initial matching pair, and determine the matching degree of the initial matching pair: if the comparison value corresponding to the initial matching pair is less than the distance threshold, determine the initial matching pair as a feature pair. The second smallest distance value among multiple distance values ​​is determined as the second smallest value. The quotient of the target distance value corresponding to the initial matching pair and the second smallest value is determined as the comparison value. If the comparison value is less than a preset distance threshold (e.g., 0.75), the initial matching pair is determined as a feature pair. This feature pair is a valid feature pair. The comparison values ​​corresponding to the visual feature points and 3D structural feature points that make up the valid feature pair are respectively less than the distance threshold. It is called a valid feature pair because this matching determination can eliminate false matches (such as texture duplication, reflection interference, similar structures), ensuring that the matching result simultaneously meets the dual constraints of spatial projection consistency and descriptor similarity, significantly improving the accuracy and robustness of feature matching, and thus providing a highly reliable point correspondence for subsequent localization calculation, ultimately achieving high-precision, adaptive, and interference-resistant localization of the robot in indoor industrial scenarios without GPS. For the environmental image of each inspection point in step S204, after processing in step S206 to obtain its corresponding set of feature pairs, step S208 is continued.

[0070] Step S208: Perform localization calculation on the feature pairs to obtain the localization information of the industrial inspection robot in the target industrial scene.

[0071] In the technical solution provided in step S208, there are multiple ways to perform localization calculation on the feature pairs to obtain the localization information of the industrial inspection robot in the target industrial scene. For example: obtaining the camera intrinsic parameter matrix of the panoramic camera used to collect environmental images, wherein the industrial inspection robot is equipped with multiple types of sensors, including a 3D scanner, a multi-line LiDAR, and a panoramic camera; obtaining the image pixels corresponding to the visual feature points in the feature pairs and the 3D point cloud spatial coordinates of the target matching feature points; determining the initial pose data based on the camera intrinsic parameter matrix, the image pixels corresponding to the visual feature points, and the 3D point cloud spatial coordinates of the target matching feature points, wherein the initial pose data is the rotation matrix and translation vector of the panoramic camera in the target industrial scene; and determining the 3D coordinates and posture data of the industrial inspection robot in the target industrial scene based on the initial pose data.

[0072] Determining the positioning information of an industrial inspection robot in a target industrial scene based on initial pose data can be achieved in several ways, such as: obtaining a rigid extrinsic transformation matrix, where the rigid extrinsic transformation matrix represents the fixed spatial transformation relationship between the panoramic camera and the industrial inspection robot body, and is determined after pre-calibrating the installation positions of the panoramic camera and the robot body; converting the rotation matrix and translation vector in the initial pose data into the target rotation matrix and target translation vector of the industrial inspection robot in the target industrial scene based on the rigid extrinsic transformation matrix, and determining the target rotation matrix and target translation vector as the posture data of the industrial inspection robot in the target industrial scene; determining the three-dimensional coordinates of the industrial inspection robot in the target industrial scene based on the three spatial coordinate components of the target translation vector; and determining the three-dimensional coordinates and posture data of the industrial inspection robot in the target industrial scene as positioning information. The positioning calculation process of step S208 is described in detail below.

[0073] Step S208.1: Obtain the camera intrinsic parameter matrix K (containing the camera's focal length in pixels along the x and y axes of the image, and the pixel coordinates of the principal point (optical center) in the image coordinate system) of the panoramic camera used for acquiring environmental images. The camera intrinsic parameter matrix has been calibrated using the standard checkerboard calibration method during the initialization phase and remains fixed during subsequent operation. Obtain the image pixels (i.e., pixel coordinates) corresponding to the visual feature points in the feature pairs and the 3D point cloud spatial coordinates of the target matching feature points (the spatial coordinates of the 3D structural feature points in the 3D scanner coordinate system (this coordinate system is the global coordinate system of the 3D reference model, denoted as the world coordinate system).

[0074] Step S208.2: Determine the initial pose data based on the camera intrinsic parameter matrix, the image pixels corresponding to the visual feature points, and the 3D point cloud spatial coordinates of the target matching feature points: Solve for the initial pose data using the PnP algorithm (the initial pose data is the rotation matrix and translation vector of the panoramic camera in the target industrial scene, specifically the rotation matrix of the panoramic camera relative to the 3D scanner). Translation vector The formula is as follows:

[0075] p=K ( PS+ ),

[0076] Where p represents the image pixel coordinates (i.e., the image pixels corresponding to the visual feature points), PS represents the three-dimensional point cloud spatial coordinates of the target matching feature points, and K represents the camera intrinsic parameter matrix.

[0077] Step S208.3: After obtaining the initial pose data, given the pose with the panoramic camera as a reference, the next step is to transform it into the base coordinate system of the robot body, thereby obtaining the robot body's true spatial pose in the global environment. The rigid extrinsic transformation matrix between the panoramic camera and the industrial inspection robot body is obtained. This rigid extrinsic transformation matrix represents the fixed spatial transformation relationship between the panoramic camera and the industrial inspection robot body. The rigid extrinsic transformation matrix is ​​determined after pre-calibrating the installation positions of the panoramic camera and the robot body (hand-eye calibration).

[0078] Based on the rigid extrinsic transformation matrix, the rotation matrix and translation vector in the initial pose data are converted into the target rotation matrix and target translation vector of the industrial inspection robot in the target industrial scene. To convert the camera pose into the robot's actual pose, the rigid extrinsic transformation matrix is ​​first used to reverse-calculate the spatial position and orientation of the industrial inspection robot in the same global coordinate system. Specifically, the camera's position and orientation are "reverse-translated" and "reverse-rotated" according to the known installation offset relationship. That is, the installation offset of the camera relative to the robot is subtracted from the current camera pose (initial pose data), thereby restoring the robot's actual coordinates and orientation in the real environment. Through this transformation, the target rotation matrix and target translation vector of the robot in the target industrial scene are finally obtained. The target rotation matrix and target translation vector are determined as the attitude data of the industrial inspection robot in the target industrial scene. The three-dimensional coordinates of the industrial inspection robot in the target industrial scene are determined based on the three spatial coordinate components of the target translation vector: the three spatial coordinate components of the target translation vector (the three spatial coordinate components of the X, Y, and Z axes) are respectively determined as the three-dimensional coordinates of the industrial inspection robot in the target industrial scene. The three-dimensional coordinates and attitude data of the industrial inspection robot in the target industrial scene are determined as the positioning information. Furthermore, during the positioning calculation process, point cloud positioning data collected in real time by LiDAR can be integrated for cross-validation to ensure the stability and accuracy of the coordinate output.

[0079] The above-mentioned positioning and calculation process is actually a feature point reverse positioning and calculation (by capturing images of the scene, extracting visual feature points from the images, and accurately comparing them with feature points in the three-dimensional reference model, the robot's real-time spatial coordinates are calculated in reverse). It directly achieves millimeter-level precise positioning without deployment through three-dimensional digital reconstruction of the indoor environment and feature point reverse calculation, significantly reducing hardware costs and deployment complexity, and eliminating the need to deploy external positioning equipment.

[0080] In some embodiments of this application, the industrial inspection robot continuously performs coordinate calculation and updates during its movement. If a positioning deviation is detected that exceeds a set threshold (the deviation between the calculated position of the industrial inspection robot and the theoretical position of the corresponding inspection point exceeds the set threshold), a correction mechanism will be automatically triggered. On the one hand, the pan-tilt camera angle will be adjusted to collect more feature points and improve matching accuracy. On the other hand, the gait parameters will be fine-tuned through the body motion controller and reverse 3D modeling technology to make the industrial inspection robot return to the preset operation path. At the same time, the calculated 3D coordinate data will be synchronized to the platform in real time and combined with the previously generated 3D benchmark model to provide operation guidance for the robot, ensuring that inspection, data collection and other operations are accurately implemented.

[0081] In some embodiments of this application, an industrial inspection robot is used to achieve regular automatic inspections. The robot's onboard 3D scanner initiates scene information re-acquisition tasks according to a dynamically set cycle, thereby achieving iterative updates of the 3D model. Different scenes drive the industrial inspection robot's point-tracking and verification logic. The robot repeatedly adjusts its posture, such as raising and lowering itself by 25°, to achieve multi-angle and multi-dimensional data acquisition. This logic is used for verification, overlaying and covering the newly modeled data. This avoids robot positioning deviations caused by factors such as object displacement, new obstacles, and equipment layout adjustments within the scene, continuously ensuring operational positioning accuracy. The process is as follows:

[0082] At each preset update cycle (the 3D model update cycle is set differently based on the scene type; a high-frequency update strategy (e.g., once a week) is used for areas with dense equipment and high item mobility, while a low-frequency periodic update strategy (e.g., once a month) is used for areas with stable layouts) or when an update instruction is received (the re-collection task can be started immediately after the factory completes equipment modification and area function adjustment), the industrial inspection robot moves in the target industrial scene according to the preset scanning path, and re-collects various types of on-site data through multiple sensors during the movement. During the collection process, the industrial inspection robot uses its own pose sensor and LiDAR for real-time positioning to ensure that the collection path is accurately aligned with the initial modeling path, ensuring the spatial consistency of the old and new data. At the same time, it can automatically identify and accurately mark areas of change such as equipment relocation, new obstacles, sign replacement, and ground facility modifications.

[0083] By comparing the newly collected field data with the 3D baseline model, the changed areas are identified. These changed areas include at least one of the following: equipment relocation, addition of obstacles, or changes in signage. The changed areas are then updated based on the newly collected field data to obtain an updated 3D baseline model. During the comparison, the degree of difference is quantitatively assessed, distinguishing between "minor local changes" and "global structural changes." Differentiated update strategies are adopted for different types of differences. For areas with minor local changes, only the point cloud data and texture information of that area are locally corrected. For areas with global structural changes, a full-scene model reconstruction process is initiated, integrating the effective information from the old and new data to generate a completely new 3D baseline model. The updated 3D baseline model automatically supplements annotation information such as equipment position changes and newly added facility attributes, ensuring a complete match with the real scene. This avoids problems such as feature point matching errors and coordinate calculation deviations caused by scene changes. Simultaneously, by comparing the theoretical coordinates of the robot at known fixed points with the actual calculated coordinates, positioning accuracy verification tests are conducted to confirm that the positioning error meets operational requirements, continuously ensuring positioning accuracy during operations.

[0084] Addressing the shortcomings of existing inspection technologies in adapting to complex terrains and achieving accurate positioning in dynamic scenes, and considering the stringent requirements of industrial scenarios for the safety, stability, and efficiency of inspection operations, a novel inspection method integrating cutting-edge technologies such as robotics, 3D scanning, video acquisition, and positioning is urgently needed to overcome the limitations of traditional inspection solutions and achieve safe, stable, and efficient integrated inspection operations in industrial settings. This study focuses on continuous production, high-risk and complex operating conditions, and precise parameter control in industrial inspection scenarios. The specific characteristics and comparative analysis of inspection are as follows: Strong production continuity necessitates "uninterrupted online" inspection: Process industries, such as chemical, oil refining, metallurgy, and power generation, rely on continuous production. Interruptions in the production process can lead to material waste, equipment damage, and even safety accidents. Therefore, inspection operations must be conducted without halting production, placing stringent demands on the mobility and non-contact detection capabilities of inspection equipment. The operating environment is highly hazardous and complex, with safety as the top priority: The scenarios commonly involve high temperatures and pressures, flammable and explosive materials, and toxic and harmful media, along with potential risks such as pipeline leaks, equipment corrosion, and dust pollution. Furthermore, the plant areas are often enclosed spaces, multi-layered structures, and complex terrains such as slopes and steps, imposing mandatory requirements on the explosion-proof, corrosion-resistant, interference-resistant, and terrain-adaptable capabilities of the inspection equipment. The equipment and parameters are highly correlated, demanding stringent inspection accuracy: Equipment, pipelines, and instruments in process industries form a highly coupled system; even small fluctuations in a single parameter (such as temperature, pressure, or flow rate) can trigger a chain reaction. Inspections must not only monitor the physical state of equipment but also collect high-precision operating data, with data errors controlled to the millimeter level to avoid misjudgments due to data deviations. The scenarios are highly dynamic, requiring real-time updates of operating data: During production, materials continuously flow, operating conditions adjust with load, and equipment wear and media corrosion are constantly changing. The inspection scenarios are not static environments; traditional fixed-point inspections cannot adapt to these changes, necessitating inspection systems with dynamic perception and iterative model updates. The inspection coverage is extensive and densely packed with points, requiring meticulous coverage: the plant area includes diverse structures such as large pressure vessels, dense piping networks, elevated platforms, and underground valve chambers. Inspection points are numerous and scattered, necessitating both large-scale area coverage and precise inspection of critical valves, welds, instruments, and other minute details. Currently, existing inspection technologies and related patents generally have significant technical drawbacks in practical applications within process industry plants and indoor plant areas. In particular, technologies relying on traditional foot sensors, GPS positioning, and pure SLAM algorithms exhibit significant limitations under complex operating conditions.

[0085] The method in this embodiment relies on steps S202-S208 to achieve a routine inspection system of "model navigation - multimodal perception - data closed loop," realizing unmanned, high-precision inspection of the entire plant area and indoor equipment areas. The robot dog can automatically execute inspection tasks according to a preset cycle, or manually trigger special inspections based on equipment maintenance plans and environmental changes. During the inspection process, real-time inspection data is continuously fed back to the three-dimensional benchmark model, realizing dynamic linkage between the model and the physical world. This not only solves the pain points of incomplete coverage, low efficiency, and fragmented data in manual inspections, but also improves the accuracy and timeliness of defect identification through intelligent and standardized detection, providing all-time and all-round technical support for safe production and stable equipment operation in the plant area.

[0086] Figure 3 This is a flowchart illustrating another positioning method for an industrial inspection robot provided in an embodiment of this application. First, a remotely controlled quadruped robot (equipped with a 3D scanner and gimbal) enters the device scene, moving within the target industrial scene according to a preset scanning path. The 3D scanner is activated, the LiDAR collects point clouds, and the panoramic camera acquires images (i.e., multiple types of on-site data are collected through various sensors during the movement). Color rendering processing (point cloud and image fusion) is then performed, where the processed RGB image data is used to perform texture mapping and color rendering on the corrected 3D model. Accuracy optimization and adjustments are made during the acquisition process, with the quadruped robot dynamically adjusting its position in parallel, and the 3D scanner adjusting its lifting height / scanning angle. Finally, a 3D baseline model is generated and refined (through manual modeling optimization using industrial-grade reverse engineering software). Next, the quadruped robot's production operation involves capturing scene photos using a panoramic camera and extracting visual feature points (i.e., obtaining environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process, and extracting visual feature points from the environmental images corresponding to each inspection point). The quadruped robot's coordinates are then calculated in reverse (i.e., performing localization calculations on the feature pairs to obtain the industrial inspection robot's positioning information in the target industrial scene). Simultaneously, the latest scene information is periodically collected and the 3D baseline model is updated (to avoid the impact of changes in scene objects). As long as the operation continues, industrial inspections are performed by the quadruped robot; otherwise, the operation ends.

[0087] Figure 4 This is a schematic diagram of a first type of industrial inspection robot provided according to an embodiment of this application, showing the body of a quadruped industrial inspection robot, including a head 401, a body 403 (the head 401 and the body 403 are connected), and legs 405, wherein the legs include multiple legs (e.g., quadrupeds) of the robot, each leg is connected to a different part of the body 403, and each leg has an individual posture adjustment capability.

[0088] Figure 5 This is a schematic diagram of a second type of industrial inspection robot provided according to an embodiment of this application, illustrating its application in... Figure 4 The industrial inspection robot shown is equipped with multiple types of sensors, including a head 401, a body 403, and a body 503. The body 503 uses... Figure 4 The body 403 shown is equipped with a 3D scanner 409, a dual-light gimbal 405 with a panoramic camera, and a lidar 407.

[0089] This application also provides a schematic diagram of the structure of a positioning device for an industrial inspection robot, as shown in the embodiment. Figure 6 As shown, it includes:

[0090] The first acquisition module 602 is used to acquire a three-dimensional benchmark model, wherein the three-dimensional benchmark model is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions.

[0091] The second acquisition module 604 is used to acquire environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process.

[0092] The matching module 606 is used to extract visual feature points from the environmental image corresponding to each inspection point, and perform feature matching between the visual feature points and the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. The feature pairs are point pairs composed of visual feature points and the three-dimensional structural feature points corresponding to the visual feature points.

[0093] The positioning module 608 is used to perform positioning calculations on feature pairs to obtain the positioning information of the industrial inspection robot in the target industrial scene.

[0094] It should be noted that, Figure 6 The positioning device of the industrial inspection robot shown is used to perform... Figure 2 The positioning method of the industrial inspection robot shown is therefore Figure 2 The relevant explanations in the positioning method of industrial inspection robots also apply to the positioning device of this industrial inspection robot, and will not be repeated here.

[0095] It should be noted that the modules in the positioning device of the above-mentioned industrial inspection robot can be program modules (such as a set of program instructions to implement a certain function) or hardware modules. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0096] This application also provides a non-volatile storage medium, which includes a stored program, wherein, when the program is running, it controls the device where the non-volatile storage medium is located to execute the positioning method of the industrial inspection robot of any of the above embodiments.

[0097] This application also provides an electronic device, which includes a processor for running a program, wherein the positioning method of the industrial inspection robot of any of the above embodiments is executed when the program is running.

[0098] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the positioning method of the industrial inspection robot of any of the above embodiments.

[0099] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0104] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A positioning method for an industrial inspection robot, characterized in that, include: Obtain a three-dimensional benchmark model, wherein the three-dimensional benchmark model is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions; Acquire environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process; Visual feature points are extracted from the environmental image corresponding to each inspection point, and feature matching is performed between the visual feature points and the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. The feature pairs are point pairs composed of the visual feature points and the three-dimensional structural feature points corresponding to the visual feature points. The feature pairs are processed to obtain the positioning information of the industrial inspection robot in the target industrial scene.

2. The method according to claim 1, characterized in that, The three-dimensional benchmark model is constructed in the following way: The system receives parameter configuration instructions and adjusts the equipment parameters of the industrial inspection robot according to the parameter configuration instructions. The industrial inspection robot is a multi-legged robot equipped with multiple types of sensors, including a 3D scanner, a multi-line lidar, and a panoramic camera. The industrial inspection robot moves in the target industrial scene according to a preset scanning path, and collects multiple types of field data through multiple sensors during the movement. The multiple types of field data include a first type of point cloud data collected by the 3D scanner, a second type of point cloud data collected by the multi-line lidar, and RGB image data collected by the panoramic camera. The first type of point cloud data is used to characterize the structural details of industrial equipment in the target industrial scene, and the second type of point cloud data is used to characterize the global spatial contour of the industrial environment in which the target industrial scene is located. Based on the various types of field data, a three-dimensional model is created to obtain the three-dimensional baseline model.

3. The method according to claim 2, characterized in that, The process of performing three-dimensional modeling based on the multiple types of field data to obtain the three-dimensional baseline model includes: The various types of field data are processed in a unified manner to obtain processed first-type point cloud data, processed second-type point cloud data, and processed RGB image data, wherein the processed first-type point cloud data, the processed second-type point cloud data, and the processed RGB image data are located in the same coordinate system. The multiple frames of point cloud data in the processed first type of point cloud data are stitched together to obtain a complete point cloud, and an initial three-dimensional model is constructed based on the complete point cloud. Using multiple frames of point cloud data from the processed second type of point cloud data, contour correction processing is performed on the regions to be improved in the initial 3D model to obtain the corrected 3D model, wherein the regions to be improved are the missing or distorted regions in the initial 3D model. The modified 3D model is then subjected to texture mapping and color rendering using the processed RGB image data to obtain the 3D baseline model.

4. The method according to claim 2, characterized in that, The step of matching the visual feature points with the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs includes: Based on the extrinsic calibration relationship between the 3D scanner and the panoramic camera, multiple 3D structural feature points in the 3D reference model are projected onto the image coordinate system of the panoramic camera to obtain a theoretical projection point set, wherein each projected 3D structural feature point in the theoretical projection point set is a comparison feature point; The visual feature points are matched with the theoretical projection point set to obtain the feature pairs.

5. The method according to claim 4, characterized in that, The step of performing feature matching between the visual feature points and the theoretical projection point set to obtain the feature pairs includes: Obtain the target feature descriptor corresponding to the visual feature point and the feature descriptors corresponding to multiple contrast feature points in the theoretical projection point set; Calculate the distance between the target feature descriptor and the feature descriptor corresponding to each of the comparison feature points to obtain multiple distance values; The minimum value among the plurality of distance values ​​is determined as the target distance value, and the comparison feature point corresponding to the target distance value is determined as the matching feature point corresponding to the visual feature point; The three-dimensional structural feature points corresponding to the matching feature points are determined as target matching feature points; The visual feature point and the target matching feature point are determined as an initial matching pair. If the comparison value corresponding to the initial matching pair is less than the distance threshold, the initial matching pair is determined as the feature pair. The comparison value is the quotient of the target distance value and the second smallest value among the plurality of distance values.

6. The method according to claim 5, characterized in that, The feature pairs are processed to obtain the positioning information of the industrial inspection robot in the target industrial scene, including: Obtain the camera intrinsic parameter matrix of the panoramic camera used to acquire the environmental images, wherein the industrial inspection robot is equipped with multiple types of sensors, including a 3D scanner, a multi-line lidar, and the panoramic camera; Obtain the image pixels corresponding to the visual feature points in the feature pair and the three-dimensional point cloud spatial coordinates of the target matching feature points; Initial pose data is determined based on the camera intrinsic parameter matrix, the image pixels corresponding to the visual feature points, and the three-dimensional point cloud spatial coordinates of the target matching feature points. The initial pose data is the rotation matrix and translation vector of the panoramic camera in the target industrial scene. Based on the initial pose data, the three-dimensional coordinates and pose data of the industrial inspection robot in the target industrial scene are determined.

7. The method according to claim 6, characterized in that, Determining the positioning information of the industrial inspection robot in the target industrial scene based on the initial pose data includes: Obtain the rigid extrinsic transformation matrix, wherein the rigid extrinsic transformation matrix is ​​a fixed spatial transformation relationship between the panoramic camera and the industrial inspection robot body, and the rigid extrinsic transformation matrix is ​​determined by calibrating the installation positions of the panoramic camera and the robot body in advance; Based on the rigid extrinsic transformation matrix, the rotation matrix and translation vector in the initial pose data are converted into the target rotation matrix and target translation vector of the industrial inspection robot in the target industrial scene, and the target rotation matrix and target translation vector are determined as the pose data of the industrial inspection robot in the target industrial scene; The three-dimensional coordinates of the industrial inspection robot in the target industrial scene are determined based on the three spatial coordinate components of the target translation vector. The three-dimensional coordinates and posture data of the industrial inspection robot in the target industrial scene are determined as the positioning information.

8. The method according to claim 2, characterized in that, The method further includes: At each preset update cycle or when an update instruction is received, the industrial inspection robot moves in the target industrial scene according to the preset scanning path, and collects multiple types of field data again through the multiple sensors during the movement. By comparing the differences between the newly collected field data and the three-dimensional benchmark model, the changed areas are obtained, wherein the changed areas include at least one of the following: equipment relocation, addition of obstacles, or change of markings. The changed area is updated based on the newly collected multi-type field data to obtain an updated three-dimensional reference model.

9. A positioning device for an industrial inspection robot, characterized in that, include: The first acquisition module is used to acquire a three-dimensional benchmark model, wherein the three-dimensional benchmark model is a three-dimensional model obtained by modeling the target industrial scene from multiple dimensions; The second acquisition module is used to acquire environmental images corresponding to each inspection point collected by the industrial inspection robot during the inspection process. The matching module is used to extract visual feature points from the environmental image corresponding to each inspection point, and perform feature matching between the visual feature points and the three-dimensional structural feature points in the three-dimensional reference model to obtain feature pairs. The feature pairs are point pairs composed of the visual feature points and the three-dimensional structural feature points corresponding to the visual feature points. The positioning module is used to perform positioning calculations on the feature pairs to obtain the positioning information of the industrial inspection robot in the target industrial scene.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device containing the non-volatile storage medium to perform the positioning method of the industrial inspection robot according to any one of claims 1 to 8.

11. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the positioning method of the industrial inspection robot according to any one of claims 1 to 8.

12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the positioning method of the industrial inspection robot according to any one of claims 1 to 8.

Images