Pipeline inspection system based on AR intelligent wearable device

Abstract

The invention relates to a pipeline inspection system based on AR intelligent wearable equipment, and belongs to the technical field of industrial detection. The system comprises a sensing layer, a processing layer and an interaction layer, the sensing layer integrates laser projection, a flexible pressure sensor and an inertial measurement unit through a wearable glove, and pipeline data collection is achieved in combination with a millimeter wave radar, a multispectral camera and an ultrasonic thickness measurement module. The processing layer adopts a heterogeneous computing platform, dynamically distributes computing power and optimizes data processing efficiency; the interaction layer provides clear operation guidance in a complex environment through an adaptive projection engine and bone conduction feedback; the system can switch detection modes according to hand actions and automatically adjust projection parameters to adapt to different pipe diameters and illumination conditions; a distributed power supply design is adopted, a super capacitor and a lithium battery are combined, and long-time stable operation is guaranteed; according to the scheme, the inspection efficiency is improved, manual intervention is reduced, and the method is suitable for pipeline detection and maintenance in a complex industrial environment.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of industrial detection, in particular to a pipeline inspection system based on an AR intelligent wearable device. BACKGROUND

[0002] Pipeline inspection usually relies on manual inspection or fixed detection equipment, and the traditional method has problems such as low efficiency and delayed data feedback. Handheld detection equipment requires operators to frequently switch tools, affecting work efficiency. Most existing AR auxiliary systems rely on head-mounted displays, which require both hands to operate and cannot detect and interact at the same time. Some wearable devices use a single sensor, which cannot fully cover the multi-dimensional detection requirements of pipeline thickness, temperature and defects. Changes in environmental light or metal reflection may interfere with the display of visual information. In a noisy environment, the clarity of voice prompts is insufficient, affecting the accuracy of operation. In addition, the data processing of traditional systems is concentrated on remote servers, which has poor real-time performance and cannot meet the on-site rapid decision-making requirements. SUMMARY

[0003] The application aims to provide a pipeline inspection system based on an AR intelligent wearable device to solve the problems in the background art.

[0004] The pipeline inspection system based on the AR intelligent wearable device comprises a perception layer, a processing layer and an interaction layer. The perception layer comprises a hand interaction unit, an environment perception unit and a pipeline detection unit. The hand interaction unit is embedded with a solid laser projection module, a flexible pressure sensor and an inertial measurement unit. The processing layer comprises an edge computer node, a dynamic load balancing module and a laser switching controller. The interaction layer comprises an adaptive projection engine, a multi-modal feedback and a lightweight AR database.

[0005] Further, the laser module is embedded in the back of the wearable glove, the laser projection module comprises a 520nm MEMS laser module (4-array time division multiplexing), a liquid lens focusing mechanism (drive voltage 3.5-5.2V) and a miniature ToF sensor (VL53L5CX, ranging 30-80cm), the whole thermal map of the pipeline is projected when the five fingers are straight, the ultrasonic thickness detection data is switched to the curved index finger, the historical maintenance record is called when the middle finger is curved, the system automatically switches to the voice navigation mode when it detects that the laser module has no palm reflection signal for thirty seconds, and the wearer is prompted through a vibration motor. When working, the ToF sensor detects the palm distance (30cm to 50cm) -> the liquid lens curvature voltage is adjusted from 3.5V to 5.2V -> the micro-mirror is synchronously reduced in scanning amplitude angle (the projection size is kept constant) The head pressure sensor is a flexible strain sensor (attached to the knuckle of the glove, detecting the bending angle), and the nine-axis IMU (BMI270, hand posture tracking).

[0006] Furthermore, the environmental perception unit includes a radar array, a geomagnetic sensor, and a multispectral camera. The radar array is a 60GHz millimeter-wave radar array (TI IWR6843) with three transmit and four receive antennas. The multispectral camera is equipped with a dual-channel visible light (OV5640) + near-infrared (850nm LED) and a narrow-band filter (to suppress reflections on the pipeline surface). The pipeline detection unit includes an ultrasonic thickness measurement module and an infrared thermal imaging module. The ultrasonic side and rear module is equipped with an ultrasonic thickness gauge (0.1mm resolution, magnetic quick disassembly and assembly), and the infrared thermal imaging module is equipped with an infrared thermal imager (Lepton 3.5, temperature measurement range -10~400℃).

[0007] Furthermore, the edge computing node includes a main control chip and a storage module, wherein the main control chip is Xilinx Zynq-7020 (FPGA+ARM dual-core), and the storage module includes industrial-grade eMMC (64GB, storage pipeline BIM model) and FRAM (MB85RC256V, parameter configuration non-volatile storage). The inspection system also includes dual battery units, which are embedded in gloves and helmets respectively. The battery unit on the glove is a soft-pack lithium battery (3.7V / 800mAh, charge and discharge cycles >500 times), and the battery unit on the helmet is a supercapacitor (5V / 1F, to cope with the transient high power consumption of millimeter-wave radar).

[0008] Furthermore, the multimodal feedback includes a bone conduction headset and a tactile motor array. The bone conduction headset includes a vibration transducer (frequency range 300Hz-5kHz) and an environmental noise suppression ENC algorithm. The noise suppression ENC algorithm is specifically: math H_b=-\sum_{k=1}^{24}\frac{E_b(k)}{E_{total}}\cdot\log_2\left(\frac{E_b(k)}{E_{total}}\right) \quad\text{where:} (kth subband energy, fkf k is the Bark edge frequency)

[0009] Judgment condition: When Hb < 2.3 and lasts for > 200ms, it is judged as steady-state industrial noise; The bone conduction compensation: math Y_{enh}(f)=\begin{cases}X(f)\cdot\left[1+0.5\cdot\cos\left(\pi\frac{f-fc}{B}\right)\right]&|f-fc| <B\\X(f)&\text{其他}\end{cases} fc: fundamental frequency of jaw vibration detected by IMU (usually 150-300Hz) B: Enhanced bandwidth (take 0.2fc0.2f c).

[0010] Furthermore, the adaptive projection engine uses the surface mapping equation to establish the three-dimensional geometric relationship of palm-channel-projection: math \begin{cases} x_p=\frac{R\cdot\cos\theta}{1+\frac{D}{R}(1-\cos\phi)}\\y_p=\frac{R\cdot\sin\theta\cdot\sin\phi}{1+\frac{D}{R}(1-\cos\phi)}\end{cases}; (xp, yp): coordinates of the projected image after correction; R: Pipeline radius (obtained in real time by ToF sensor); D: distance from palm to pipe (30-80cm dynamic range); θ, Φ: Palm pitch / yaw angle (IMU data).

[0011] Furthermore, the adaptive projection engine uses bilinear interpolation optimization: I_{out}(x,y) = \sum_{i=0}^1 \sum_{j=0}^1 I_{in}(u+i,v+j) \cdot (1-|u-\lfloor u \rfloor|) \cdot (1-|v-\lfloor v \rfloor|) (u,v): non-integer coordinates calculated by inverse mapping (xp,yp)→(x,y) FPGA implementation: using 4 parallel multipliers, delay < 0.5ms; The operation of the adaptive projection engine for sensing ambient light is as follows: math I_{adjust} = \begin{cases} I_0 \cdot \left(\frac{L_{env}}{1000}\right)^{0.7} & L_{env}>10^4 \ lux \\I_0 & \text{Other}\end{cases} I0: Reference laser intensity (4mW / mm 2 ) Lenv: Ambient light intensity obtained by the multispectral camera.

[0012] Furthermore, the dynamic load balancing module includes a hardware counter array and a power consumption monitoring circuit. Each sensor interface of the hardware counter array deploys a 32-bit cycle counter (sampling rate / data volume statistics), which triggers a priority increase when the burst flow of ultrasonic thickness data exceeds 2MB / s; The power consumption monitoring circuit uses the INA226 current sensor to work, and the power consumption-temperature model is: math P_{pred}(t)=\sum_{i=1}^n\alpha_i\cdot fi(t)\cdot V_{dd}^2+\beta\cdot(Tj-25) fi: clock frequency of each subsystem Tj: Junction temperature (read via PMBus.

[0013] Furthermore, the query optimization of the lightweight AR database is specifically as follows: sql SELECT * FROM pipe_segments WHERE geohash LIKE'WS456%' -- The first 6 digits match the current region ORDER BY dna DESC LIMIT 10; Adopting adaptive LOD loading algorithm: math LOD_{level}=\begin{cases} 3&\text{if}\D>20m\\text{or}\\text{CPU load}>70\%\\ 2&\text{if}\5m <D\leq 20m\\ 1&\text{if}\D\leq 5m \end{cases} Distance D comes from the ToF sensor Data volume of each LOD level: LOD3: Displays only the pipe centerline (0.1KB / m) LOD2: Simplified point cloud + base texture (5KB / m) LOD1: Complete BIM model (20KB / m).

[0014] Furthermore, the laser switching controller adopts a time division multiplexing scheduling algorithm: math T_{slot}(n)=\begin{cases} \frac{T{cycle}}{4}\cdot(1+0.2\cdot\frac{P_{avg}-P_n}{P_{max}})&\text{Balanced mode}\\\frac{T_{cycle}}{2}\cdot\delta(n)&\text{Priority mode} \end{cases} Tcycle: total cycle (default 1ms) δ(n): Set to 1 when a defect is found in the area corresponding to laser n Hardware implementation: Generate precise timing via FPGA (Xilinx Artix-7); Fault detection is specifically as follows: math Fault=\begin{cases} 1&\text{if}\\frac{|I_{meas}-I_{set}|}{I_{set}}>0.3\\text{and last}\geq10\mu s\\ 1&\text{if}\\text{Pulse count}\geq 10^8\\text{times}\\ 0&\text{Other} \end{cases}.

[0015] By adopting this technical solution, which integrates laser projection and hand motion recognition into wearable gloves, operators can switch detection modes directly through finger movements, without the need for additional equipment. Extending five fingers displays the heat map, bending the index finger retrieves thickness data, and bending the middle finger to view maintenance records reduces the number of steps and improves work efficiency.

[0016] Compared with the existing technology, the beneficial effects of the present invention are: the pipeline inspection system based on AR smart wearable devices, Multi-spectral camera with narrow-band filter to suppress metal reflection, ensuring clear images. Millimeter wave radar and geomagnetic sensor enhance spatial awareness, enabling accurate positioning in low light or complex structural environments. Bone conduction earphones combined with noise suppression algorithms maintain clear speech in industrial noise environments; Edge computing nodes use FPGA+ARM architecture, with dynamic load balancing modules that allocate computing power in real time, ensuring fast processing of high-priority data such as ultrasonic thickness measurement. Light AR database loads pipeline models on demand, reducing data transmission pressure and improving response speed; Adaptive projection engine dynamically adjusts projection parameters based on pipeline curvature and distance, combined with liquid lens autofocus to ensure image integrity. Ambient light sensing module automatically adjusts laser intensity, enabling clear display in strong light environments; Distributed power supply design combines lithium batteries and supercapacitors to meet the power consumption needs of different modules. Power consumption monitoring circuit dynamically adjusts operating frequency to reduce idle power consumption and extend single-use time. Laser module is equipped with a fault detection mechanism that triggers protection in case of abnormal current or overuse. ToF sensor and IMU real-time calibration of hand position to avoid misoperation. Idle automatic switching to voice mode to prevent resource waste. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The block diagram of the pipeline inspection system based on AR intelligent wearable device of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0019] Please refer to Figure 1 The present application provides a technical solution: a pipeline inspection system based on AR intelligent wearable device, with a perception layer consisting of three parts. The hand interaction unit adopts a wearable glove structure, with a laser projection module embedded in the back of the glove, containing four 520nm MEMS laser modules that work in time division multiplexing mode. The liquid lens focusing mechanism has a driving voltage range of 3.5-5.2V, working with a VL53L5CX miniature ToF sensor to achieve 30-80cm ranging. When detecting changes in hand palm distance, the system automatically adjusts the lens curvature voltage and scan amplitude angle. Flexible pressure sensors use polyimide-based strain gauges attached to the knuckles, detecting bending angles. Nine-axis IMU uses BMI270 chip with a sampling rate of 200Hz, used to track hand spatial posture.

[0020] The environmental perception unit is equipped with millimeter wave radar arrays on both sides of the helmet, using TI IWR6843 chipsets operating at a frequency of 60 GHz. The geomagnetic sensor uses a HMC5983 three-axis magnetoresistance chip for indoor positioning correction. The multispectral camera combines an OV5640 visible light sensor and an 850 nm near-infrared module, with a narrow-band filter installed in front of the lens to suppress metal reflections. The pipe detection unit includes a magnetic ultrasonic thickness gauge with a measurement resolution of 0.1 mm, and a Lepton 3.5 infrared thermal imager with a temperature range of -10 to 400°C.

[0021] The processing layer core is a Xilinx Zynq-7020 heterogeneous computing platform, with the FPGA part responsible for sensor data preprocessing and the ARM core running a Linux system. The storage system uses a 64 GB industrial-grade eMMC storage pipe BIM model, with parameter configurations stored in an MB85RC256V FRAM chip. The dynamic load balancing module monitors the sensor data flow in real time through a 32-bit hardware counter, and automatically raises the processing priority when the ultrasonic data exceeds 2 MB / s. The laser switching controller implements a time division multiplexing scheduling algorithm in the FPGA, with a basic cycle of 1 ms and dynamic adjustment of the working time of each laser module in abnormal conditions.

[0022] The interaction layer uses bone conduction headphones and tactile motors to achieve multi-modal feedback. The bone conduction transducer has a working frequency band of 300 Hz-5 kHz, with built-in ENC algorithm for real-time analysis of environmental noise spectrum characteristics. When stable industrial noise is detected, the jawbone vibration compensation algorithm is automatically activated. The adaptive projection engine calculates the projection coordinate transformation matrix in real time based on the pipe radius and distance data measured by the ToF sensor, and implements bilinear interpolation optimization through FPGA, with a processing delay control within 0.5 ms.

[0023] The power system uses a distributed design, with a 3.7V / 800mAh soft pack lithium battery built into the glove, with a cycle life of more than 500 times. The helmet is equipped with a 5V / 1F super capacitor group, specifically for powering the millimeter wave radar. The system monitors the power consumption of each module through INA226 current sensors, and dynamically adjusts the operating frequency based on junction temperature data.

[0024] The lightweight AR database uses Geohash spatial indexing, with the current area pipe data loaded first during query. The LOD level is automatically switched according to distance, with only the centerline displayed 20 meters away and the complete BIM model loaded within 5 meters. The projection brightness is automatically adjusted according to the ambient light, with the laser power increased according to a non-linear curve when strong light is detected.

[0025] The fault detection system continuously monitors the working current of each laser module. If the deviation exceeds 30% for more than 10 microseconds, the protection mechanism is triggered. At the same time, the number of laser pulses is recorded, and when it reaches 100 million, maintenance and replacement are prompted. The system automatically switches to voice navigation mode after 30 seconds of idling, and reminds the operator through the tactile motor.

Claims

1. The pipeline inspection system based on AR smart wearable devices is characterized by: It includes a perception layer, a processing layer and an interaction layer; the perception layer includes a hand interaction unit, an environmental perception unit and a pipeline detection unit; the hand interaction unit is embedded with a laser projection module, a flexible pressure sensor and an inertial measurement unit, the processing layer includes an edge computer node, a dynamic load balancing module and a laser switching controller; the interaction layer includes an adaptive projection engine, multimodal feedback and a lightweight AR database.

2. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The laser module is embedded in the back of the wearable glove. The laser projection module includes a 520nm MEMS laser module (4-array time-division multiplexing), a liquid lens focusing mechanism (driving voltage 3.5-5.2V) and a micro ToF sensor (VL53L5CX, ranging from 30-80cm). When the five fingers are straightened, the overall thermal map of the pipeline is projected. Bend the index finger to switch to ultrasonic thickness detection data, and bend the middle finger to retrieve historical maintenance records. When the system detects that the laser module has no palm reflection signal for 30 consecutive seconds, it automatically switches to voice navigation mode and prompts the wearer through a vibration motor. During operation, the ToF sensor detects the palm distance (30cm→50cm) → the liquid lens curvature voltage is adjusted from 3.5V to 5.2V → the micromirror synchronously reduces the scanning angle (keeping the projection size constant). The head-shaped pressure sensor is a flexible strain sensor (attached to the glove knuckles to detect the bending angle) and a nine-axis IMU (BMI270, hand posture tracking).

3. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The environmental perception unit includes a radar array, a geomagnetic sensor, and a multispectral camera. The radar array is a 60GHz millimeter-wave radar array (TI IWR6843) with three transmit and four receive antennas. The multispectral camera is equipped with dual channels of visible light (OV5640) and near-infrared (850nm LED) and a narrow-band filter (to suppress reflections on the pipeline surface). The pipeline detection unit includes an ultrasonic thickness measurement module and an infrared thermal imaging module. The ultrasonic side and rear module is equipped with an ultrasonic thickness gauge (0.1mm resolution, magnetic quick disassembly and assembly), and the infrared thermal imaging module is equipped with an infrared thermal imager (Lepton 3.5, temperature measurement range -10~400°C).

4. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The edge computing node includes a main control chip and a storage module. The main control chip is Xilinx Zynq-7020 (FPGA+ARM dual-core), and the storage module includes industrial-grade eMMC (64GB, storage pipeline BIM model) and FRAM (MB85RC256V, parameter configuration non-volatile storage). The inspection system also includes dual battery units, embedded in gloves and helmets respectively. The battery unit on the glove is a soft-pack lithium battery (3.7V / 800mAh, charge and discharge cycles >500 times), and the battery unit on the helmet is a supercapacitor (5V / 1F, to cope with the transient high power consumption of millimeter-wave radar).

5. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The multimodal feedback system includes a bone conduction headset and a tactile motor array. The bone conduction headset includes a vibration transducer (frequency range 300Hz-5kHz) and an environmental noise suppression ENC algorithm. The ENC algorithm is specifically: math $H_b = -\sum_{k = 1}^{24}\frac{E_b(k)}{E_{total}}\cdot\log_2\left(\frac{E_b(k)}{E_{total}}\right)$ where: (kth subband energy, fkf k is the Bark edge frequency) ； Judgment condition: When $H_b < 2.3$ and it lasts for more than 200 ms, it is judged as steady industrial noise; The bone conduction compensation: math $Y_{enh}(f)=\begin{cases}X(f)\cdot\left[1 + 0.5\cdot\cos\left(\pi\frac{f - fc}{B}\right)\right]&|f - fc| < B\\X(f)&\text{otherwise}\end{cases}$ $fc$: Fundamental frequency of jaw vibration detected by IMU (usually 150 - 300 Hz) $B$: Enhancement bandwidth (take $0.2f_c$).

6. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The adaptive projection engine establishes a three - dimensional geometric relationship of palm - pipe - projection using a surface mapping equation: math $\begin{cases}$ $x_p=\frac{R\cdot\cos\theta}{1+\frac{D}{R}(1 - \cos\phi)}$\\$y_p=\frac{R\cdot\sin\theta\cdot\sin\phi}{1+\frac{D}{R}(1 - \cos\phi)}$\end{cases}$; $(x_p,y_p)$: Coordinates after projection image correction; $R$: Pipe radius (obtained in real - time by ToF sensor); $D$: Distance from palm to pipe (dynamic range of 30 - 80 cm); $\theta,\Phi$: Palm pitch / yaw angle (IMU data).

7. The pipeline inspection system based on AR smart wearable device according to claim 6 is characterized in that: The adaptive projection engine adopts bilinear interpolation optimization: $I_{out}(x,y)=\sum_{i = 0}^1\sum_{j = 0}^1I_{in}(u + i,v + j)\cdot(1 - |u-\lfloor u\rfloor|)\cdot(1 - |v-\lfloor v\rfloor|)$ $(u,v)$: Non - integer coordinates calculated by inverse mapping $(x_p,y_p)\to(x,y)$ FPGA implementation: Use 4 parallel multipliers, with delay $< 0.5$ ms; The operation of the adaptive projection engine for ambient light perception is: math $I_{adjust}=\begin{cases}I_0\cdot\left(\frac{L_{env}}{1000}\right)^{0.7}&L_{env}>10^4\ lux\\I_0&\text{otherwise}\end{cases}$ I0: Reference laser intensity (4mW / mm 2 ) $L_{env}$: Ambient light intensity obtained by multispectral camera.

8. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The dynamic load balancing module includes a hardware counter array and a power consumption monitoring circuit. Each sensor interface of the hardware counter array deploys a 32-bit cycle counter (sampling rate / data volume statistics). When the burst flow of ultrasonic thickness data exceeds 2MB / s, it triggers a priority increase. The power consumption monitoring circuit uses the INA226 current sensor to work, and the power consumption-temperature model is: math P_{pred}(t)=\sum_{i=1}^n\alpha_i\cdot fi(t)\cdot V_{dd}^2+\beta\cdot(Tj-25) fi: clock frequency of each subsystem Tj: Junction temperature (read via PMBus.

9. The pipeline inspection system based on AR smart wearable device according to claim 1 is characterized in that: The query optimization of the lightweight AR database is specifically as follows: sql SELECT * FROM pipe_segments WHERE geohash LIKE'WS456%' -- The first 6 digits match the current region ORDER BY dna DESC LIMIT 10; Adopting adaptive LOD loading algorithm: math LOD_{level}=\begin{cases} 3&\text{if}\D>20m\\text{or}\\text{CPU load}>70\%\\ 2&\text{if}\5m <D\leq 20m\\ 1&\text{if}\D\leq 5m \end{cases} Distance D comes from the ToF sensor Data volume of each LOD level: LOD3: Displays only the pipe centerline (0.1KB / m) LOD2: Simplified point cloud + base texture (5KB / m) LOD1: Complete BIM model (20KB / m).

10. The pipeline inspection system based on AR smart wearable device according to claim 1, characterized in that: The laser switching controller adopts a time division multiplexing scheduling algorithm: math T_{slot}(n)=\begin{cases} \frac{T{cycle}}{4}\cdot(1+0.2\cdot\frac{P_{avg}-P_n}{P_{max}})&\text{Balanced mode}\\\frac{T_{cycle}}{2}\cdot\delta(n)&\text{Priority mode} \end{cases} Tcycle: total cycle (default 1ms) δ(n): Set to 1 when a defect is found in the area corresponding to laser n Hardware implementation: Generate precise timing via FPGA (Xilinx Artix-7); Fault detection is specifically as follows: math Fault=\begin{cases} 1&\text{if}\\frac{|I_{meas}-I_{set}|}{I_{set}}>0.3\\text{and last}\geq 10\mu s\\ 1&\text{if}\\text{Pulse count}\geq 10^8\\text{times}\\ 0&\text{Other} \end{cases}.

Images