An ar patrol glasses and method based on adaptive knn positioning and thermal imaging

Abstract

The application discloses an AR inspection glasses based on adaptive KNN positioning and thermal imaging and a method thereof, relates to the field of industrial safety inspection, and comprises an AR glasses main body, an edge processing unit and an independent or integrated AR glasses host unit, further comprises a thermal imaging gas detection integrated module arranged on the AR glasses main body, and the thermal imaging gas detection integrated module is electrically connected with the edge processing unit through a circuit. The application realizes absolute alignment of a tens of thousands of pixel thermal imaging matrix and specific gas concentration spectrum characteristics on a physical timestamp, solves millisecond delay caused by traditional equipment relying on software polling scheduling, solves fatal misplacement and ghosting phenomenon between a device heating maintenance point in the AR glasses main body, a colorless gas leakage position alarm frame and a real physical coordinate in a dynamic operation moving state of an inspector, and ensures that an abnormal area can be accurately determined in an inspection operation.

Description

Technical Field

[0001] This application relates to the field of industrial safety inspection, and in particular to an AR inspection glasses and method based on adaptive KNN positioning and thermal imaging. Background Technology

[0002] Industrial indoor scenarios such as power rooms, chemical workshops, enclosed pipe corridors, and industrial plants are characterized by dense equipment, complex pipelines, and complex environmental features such as metal obstruction, electromagnetic interference, and lack of signal in enclosed spaces. These are the core, key, and difficult scenarios for industrial safety inspection.

[0003] A search revealed a Chinese patent document that discloses an infrared night vision AR smart glasses based on multispectral fusion (publication number: CN121121002A), but it still has the following defects.

[0004] Industrial indoor inspection environments are characterized by dense equipment, metal cabinet obstructions, crisscrossing pipelines, enclosed walls, and severe multipath reflection and attenuation of wireless signals. Traditional inspection and positioning technologies have fundamental adaptation defects, as detailed below:

[0005] (1) Satellite GPS / BeiDou positioning has no signal coverage in enclosed indoor spaces and deep in factory buildings. It can only achieve coarse positioning at the meter level and cannot accurately match specific equipment, pipelines, joints and other fine inspection points. It does not support fine indoor inspection operations at all.

[0006] (2) The traditional fixed K-value KNN fingerprint positioning algorithm uses K-nearest neighbor values ​​as preset fixed parameters. It cannot dynamically and adaptively adjust according to the real-time signal interference intensity, fingerprint neighborhood density, and occlusion situation. When there are metal equipment obstruction, personnel movement obstruction, or electromagnetic interference on site, Wi-Fi and Bluetooth RSSI signal fingerprints will experience severe drift, distortion, and jumps. Fixed K-value matching will filter out a large number of invalid and biased fingerprint samples, which directly leads to positioning coordinate shift, point jump, and trajectory disorder. The positioning error increases sharply and cannot achieve stable high-precision positioning.

[0007] (3) Traditional positioning technology lacks an inertial data fusion correction mechanism and relies solely on wireless fingerprint matching. Instantaneous signal fluctuations will cause frequent jitter in the positioning results, ultimately leading to the inspection system's inability to accurately identify whether the inspection personnel have reached the designated equipment location. Equipment work order triggering, location inspection records, and fault location binding all show misalignment, greatly reducing the accuracy of inspection location matching and failing to meet the core requirements of centimeter-level positioning inspection in industrial indoor environments. Summary of the Invention

[0008] To address the technical issues in existing technologies, such as low positioning accuracy of indoor inspection personnel, low efficiency in gas leak detection, and fragmented information presentation, this application provides an AR inspection glasses and method based on adaptive KNN positioning and thermal imaging.

[0009] The AR inspection glasses and method based on adaptive KNN positioning and thermal imaging provided in this application adopt the following technical solution:

[0010] An AR inspection glasses and method based on adaptive KNN positioning and thermal imaging includes an AR glasses main body, an edge processing unit, and an independent or integrated AR glasses host unit. It also includes a thermal imaging gas detection integrated module disposed on the AR glasses main body, and the thermal imaging gas detection integrated module is electrically connected to the edge processing unit through a circuit.

[0011] It also includes the AR glasses main body being connected to the AR glasses host unit via wires;

[0012] The edge processing unit includes a Wi-Fi fingerprint acquisition module, a thermal imaging gas detection integrated module, an edge processing module, and an AR optical display module;

[0013] The edge processing unit also has a built-in environment adaptive boundary dynamic switching mechanism and a neural network processor. The signal processing logic of the edge processing unit includes extracting the CSI channel state information of the underlying orthogonal frequency division multiplexing signal from the Wi-Fi fingerprint acquisition module.

[0014] The neural network processor calculates the subcarrier phase variance in the CSI matrix in real time, and uses the subcarrier phase variance as a physical indicator to quantify the severity of the current spatial multipath effect and electromagnetic scattering.

[0015] The logic of the environmental adaptive boundary dynamic switching mechanism is as follows:

[0016] The monitored phase variance is less than a preset threshold. At this time, the system determines that it is currently in a high signal stability region, and the hardware controller dynamically sets the nearest neighbor parameters of the KNN algorithm. ;

[0017] The monitored phase variance is greater than or equal to a preset threshold. When the system determines that it is in a region of severe signal attenuation due to severe metal obstruction or dense equipment, the hardware controller will forcibly trigger a boundary switch, expand the sampling nearest neighbor range, and simultaneously start physical space Euclidean distance threshold filtering to eliminate pseudo-nearest neighbor reference points in non-connected areas of the spatial topology.

[0018] Preferably, the Wi-Fi fingerprint acquisition module continuously scans all AP wireless access points on site at a fixed frequency of 1Hz to 2Hz, with each cycle lasting 0.5 seconds to 1 second. Simultaneously, it collects two types of fingerprint features: RSSI received signal strength and CSI channel status information. The fingerprint vector is then transmitted in real-time to the local processing module for location calculation. By fusing the RSSI surface signal and the CSI underlying channel features, the module calculates the real-time online fingerprint vector and compares it with the fingerprint in the offline fingerprint database. The weighted Euclidean distance between the fingerprint vectors of each reference point The physical mapping formula is as follows:

[0019]

[0020] In the above formula This represents the total number of valid access points (APs) that the Wi-Fi fingerprinting module can reliably detect within the current spatial grid. and These represent the data currently collected in real time by the edge processing unit from the [number]th [unit]. The received signal strength index and channel state information of each AP. and These represent the pre-loaded offline fingerprint database, the first... The corresponding physical reference point is the th The mean of the time-bound calibration characteristics of each AP;

[0021] In the above formula, and These are the feature fusion weight coefficients, which are set based on empirical multipath attenuation models in the field, and are subject to mandatory constraints during system initialization. + =1.

[0022] Preferably, the integrated thermal imaging gas detection module includes at least an infrared thermal imaging sensor and an infrared spectroscopy gas analysis unit, both of which are integrated into a single package structure.

[0023] The function of the infrared thermal imaging sensor is to capture infrared thermal radiation in the industrial indoor inspection environment and output the infrared thermal radiation as a real-time thermal imaging raw image.

[0024] The infrared spectroscopy gas analysis unit is used to capture the gas in the industrial indoor inspection environment based on the principle of gas absorption spectroscopy in a specific infrared band, and output the gas concentration and the coordinates of the leakage area.

[0025] The infrared spectroscopy gas analysis unit captures gases in the industrial indoor inspection environment, including at least methane, carbon monoxide, sulfur hexafluoride, and refrigerant.

[0026] Preferably, the infrared thermal imaging sensor and the infrared spectral gas analysis unit are electrically connected to each other and are synchronously acquired through hardware, thereby aligning the thermal imaging image with the gas detection data timestamp, and thus obtaining an image corresponding to the gas leak.

[0027] Preferably, the edge processing module includes at least an independent embedded processing unit, and the independent embedded processing unit includes at least an MCU and an NPU micro-computing unit, and performs all algorithm calculations locally without relying on real-time transmission from the cloud. The specific function of the independent embedded processing unit is as follows:

[0028] Stores an offline Wi-Fi fingerprint database;

[0029] The adaptive KNN positioning algorithm is executed to calculate the two-dimensional spatial coordinates of the inspection personnel in real time.

[0030] Thermal imaging image noise reduction, contrast enhancement, and wireless image transmission preprocessing;

[0031] It integrates multi-source data, including location coordinates, thermal imaging temperature field, and gas leak location data;

[0032] Output the overlay rendering data to the AR display module.

[0033] Preferably, the AR optical display module is used to transmit real-world images in real time and overlay personnel positioning coordinates, regional temperature cloud map, gas leak outline, concentration value, equipment number, and historical maintenance records onto the real-world field of view.

[0034] An AR inspection glasses method based on adaptive KNN localization and thermal imaging, characterized by the following steps:

[0035] S1: Offline fingerprint database construction and system initialization preparation;

[0036] S1.1: All modules start synchronously, the Wi-Fi fingerprint acquisition module continuously scans the AP signal, and the thermal imaging gas module enters real-time acquisition mode;

[0037] S2: Real-time edge localization solution based on adaptive KNN;

[0038] S2.1: Adaptive KNN real-time localization;

[0039] S2.2: The inspection personnel will align the integrated thermal imaging gas detection module at the front of the glasses with the corresponding area;

[0040] S2.3: Multi-data AR overlay visualization;

[0041] S3: Multimodal synchronous acquisition of infrared thermal imaging and spectral gas detection;

[0042] S4: Spatiotemporal coordinate alignment and spatial mapping of multi-source data;

[0043] S5: AR layered rendering of virtual and real objects and hazard tracing.

[0044] Preferably, the feature is that: in S1, the front-end preparation specifically involves dividing the inspection room area into uniform grid reference points, marking the real physical coordinates of each reference point, and collecting RSSI and CSI data of surrounding APs multiple times at each reference point, taking the average value as the offline fingerprint feature vector of that point, and generating a cloud database. The method for constructing the offline fingerprint database is as follows: after dividing the inspection area into uniform grid reference points, collecting RSSI and CSI features of surrounding APs at each point multiple times and taking the average value, first uploading the feature key-value pairs of coordinates and fingerprint vectors and aggregating them on the cloud server to construct a global offline fingerprint database.

[0045] Preferably, in step S2, the real-time edge localization solution based on adaptive KNN includes fingerprint distance calculation, which involves calculating the distance between the feature vector and the fingerprint features of all reference points in the fingerprint database. Using all reference points in the cloud database, the feature distance between the online fingerprint and each offline fingerprint is calculated, and then the result is obtained through adaptive KNN. Values ​​are dynamically selected and are adaptive. The criteria for dynamic value selection are:

[0046] (1) Dense fingerprint grid in high signal stability region: ;

[0047] (2) Severe signal obstruction: .

[0048] The adaptive The logic for dynamically selecting values ​​is to automatically distinguish region density based on the fingerprint similarity of adjacent reference points and update it dynamically in real time. value.

[0049] Preferably, the real-time dynamic update The values ​​need to be weighted and calculated, specifically by filtering for the values ​​with the smallest distances. The final personnel location coordinates are calculated using a distance-weighted inverse calculus of three nearest neighbor reference points, as shown in the following expression:

[0050]

[0051] in the formula For the first The fingerprint distance between the nearest neighbors is used, with closer distances carrying greater weight, to ensure that the positioning coordinates closely approximate the actual location. To provide the inspection personnel with the final estimate of the two-dimensional positioning coordinates, For the first The weighted average of the nearest neighbor reference points is assigned using the reciprocal of the distance. For the first in the database The actual offline physical coordinates of the nearest reference point. The summation operator represents summation over all... Accumulate the nearest neighbor points. The smaller the fingerprint distance, the larger the weight value; the closer the reference point, the stronger its influence on the final positioning coordinates.

[0052] And two-dimensional spatial coordinates Solve the mathematical formula and standardize the parameter references. The specific expression is as follows:

[0053]

[0054] in the formula For the first The fingerprint distance between the nearest neighbors This refers to the system's final estimate of the current real-time two-dimensional physical coordinates of the inspection personnel. For the first The weighted values ​​are assigned to the nearest neighbor reference points, using the inverse of the distance. For the offline database The calibration physical coordinates of the nearest reference point This represents the total number of neighboring reference points dynamically selected under the current environmental conditions.

[0055] In summary, this application includes at least one of the following beneficial technical effects:

[0056] 1. This invention achieves absolute alignment of a thermal imaging matrix of tens of thousands of pixels with specific gas concentration spectral characteristics on physical timestamps, solving the millisecond-level latency caused by traditional equipment relying on software polling scheduling. It also solves the fatal misalignment and ghosting phenomenon between the equipment heating and maintenance points, colorless gas leak location alarm boxes in the AR glasses body and the real physical coordinates when inspection personnel are in a dynamic operation and moving state, ensuring that abnormal areas can be accurately identified during inspection operations.

[0057] 2. This invention is designed for industrial inspection environments. By fusing the CSI status information of the underlying physical channel with the surface RSSI signal strength as fingerprint features, an environmental adaptive boundary dynamic switching mechanism is established. This enables the system to automatically expand the sampling neighborhood range to reduce singular errors when facing local electromagnetic blind zones or refraction interference from dense metal pipe corridors. This reduces the probability of location jumps in severely metal-obscured areas to a controllable tolerance range, achieving truly industrial-grade indoor decimeter or even centimeter-level fine-point inspection.

[0058] 3. This invention achieves absolute physical blockage and isolation between the core precision photoelectric sensor and external corrosive gases and high-dust industrial media, greatly improving the system's adaptability to harsh working conditions and maintenance cycle. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the structure of this application;

[0060] Figure 2 This is a schematic diagram of the AR inspection glasses method based on adaptive KNN localization and thermal imaging in this application;

[0061] Figure 3 This is a schematic diagram of the edge processing unit module structure of this application.

[0062] Reference numerals: 1. AR glasses main body; 101. Thermal imaging gas detection integrated module; 2. Edge processing unit; 3. AR glasses main unit. Detailed Implementation

[0063] The following is in conjunction with the appendix Figure 1 - Figure 3 This application will be described in further detail.

[0064] This application discloses an AR inspection glasses and method based on adaptive KNN positioning and thermal imaging.

[0065] Example 1

[0066] Reference Figure 1 and Figure 3 An AR inspection glasses based on adaptive KNN positioning and thermal imaging includes an AR glasses body 1, a thermal imaging gas detection integrated module 101 integrated at the front end of the AR glasses body 1, and an edge processing unit 2 connected to the AR glasses body 1 via an external cable.

[0067] The thermal imaging gas detection integrated module 101 is electrically connected to the edge processing unit 2 via an FPC flexible cable;

[0068] The AR glasses main body 1 is connected to the independently worn AR glasses host unit 3 via a data cable;

[0069] The edge processing unit 2 integrates a Wi-Fi fingerprint acquisition module, a thermal imaging gas detection module, an edge processing module, and an AR optical display module. The AR optical display module is located on the inside of the AR glasses body 1 and is connected to an independent embedded processor.

[0070] The integrated thermal imaging gas detection module 101 includes infrared thermal imaging sensors and infrared spectral gas analyzers arranged in a side-by-side array. The integrated thermal imaging gas detection module 101 is covered by a rigid protective shell, and the rigid protective shell has a germanium glass infrared transmission window embedded at the position of the optical path axis facing the infrared thermal imaging sensors and infrared spectral gas analyzers.

[0071] The standalone AR glasses host unit 3 has a built-in high-capacity battery and extended storage. It is connected to the edge processing unit 2 via a physical cable to provide power to the system and store the high-definition video stream and multi-source hidden danger data packets required for backtracking audits after the operation is completed. The specific real-time spatial calculation and image rendering are all completed independently by the edge processing unit 2 locally.

[0072] Edge processing unit 2 is equipped with local flash memory, which is used to synchronize and download the offline fingerprint database from the cloud database to local storage before the inspection task begins. During the real-time inspection operation, the edge processing module only relies on the locally stored database and the local MCU and NPU computing power chip to perform adaptive KNN distance calculation and coordinate solution, which fully realizes low-latency edge computing without relying on real-time network transmission in the cloud.

[0073] Edge processing unit 2 also has a built-in environment adaptive boundary dynamic switching mechanism and neural network processor. The signal processing logic of edge processing unit 2 includes extracting the CSI channel state information of the underlying orthogonal frequency division multiplexing signal using the Wi-Fi fingerprint acquisition module.

[0074] The neural network processor calculates the subcarrier phase variance in the CSI matrix in real time, and uses the subcarrier phase variance as a physical indicator to quantify the severity of the current spatial multipath effect and electromagnetic scattering.

[0075] The logic of the environment adaptive boundary dynamic switching mechanism is as follows:

[0076] The monitored phase variance is less than a preset threshold. At this time, the system determines that it is currently in a high signal stability region, and the hardware controller dynamically sets the nearest neighbor parameters of the KNN algorithm. ;

[0077] Monitoring phase variance greater than or equal to When the system determines that it is in a region of severe signal attenuation due to severe metal obstruction or dense equipment, the hardware controller will forcibly trigger a boundary switch, expand the sampling nearest neighbor range, and simultaneously start physical space Euclidean distance threshold filtering to eliminate pseudo-nearest neighbor reference points in non-connected areas of the spatial topology.

[0078] The Wi-Fi fingerprint acquisition module works as follows: it continuously scans all AP wireless access points on site at a fixed frequency of 1Hz to 2Hz (i.e., 0.5 seconds to 1 second per cycle), and simultaneously collects two types of fingerprint features: RSSI received signal strength and CSI channel status information. The real-time fingerprint vector is then transmitted to the local processing module for location calculation. By fusing the RSSI surface signal and the CSI underlying channel features, it calculates the real-time online fingerprint vector and compares it with the fingerprint in the offline fingerprint database. The weighted Euclidean distance between the fingerprint vectors of each reference point The physical mapping formula is as follows:

[0079]

[0080] In the above formula This represents the total number of valid access points (APs) that the Wi-Fi fingerprinting module can reliably detect within the current spatial grid. and These represent the data currently collected in real time by edge processing unit 2 from the [missing data]. The received signal strength index and channel state information of each AP. and These represent the pre-loaded offline fingerprint database, the first... The corresponding physical reference point is the th The mean of the time-bound calibration characteristics of each AP;

[0081] In the above formula, and These are the feature fusion weight coefficients, which are set based on empirical multipath attenuation models in the field, and are subject to mandatory constraints during system initialization. + =1.

[0082] The integrated thermal imaging gas detection module includes at least an infrared thermal imaging sensor and an infrared spectral gas analysis unit. The two share the same hardware crystal clock source, and their control trigger terminals are connected in parallel to the same hardware interrupt pin output by the independent embedded processor. This interrupt pin is used to receive a synchronous interrupt signal and perform hardware-level synchronous acquisition. The interrupt pin is connected to the external trigger ports of the two sensors. In operation, the independent embedded processor sends a hard pulse level signal at a fixed frequency through the EXT_INT pin. Within a nanosecond delay of receiving the rising edge, the thermal imaging and gas sensors force the enable terminals of their internal integrators to go high simultaneously, performing absolutely synchronized physical exposure and sampling.

[0083] The function of an infrared thermal imaging sensor is to capture infrared thermal radiation in an industrial indoor inspection environment and output the infrared thermal radiation as a real-time thermal imaging raw image.

[0084] The function of the infrared spectroscopy gas analysis unit is to detect the gas in the industrial indoor inspection environment based on the principle of gas absorption spectroscopy in a specific infrared band, and output the gas concentration and the coordinates of the leak area.

[0085] The infrared spectroscopy gas analysis unit captures gases in industrial indoor inspection environments, including at least methane, carbon monoxide, sulfur hexafluoride, and refrigerants.

[0086] The infrared thermal imaging sensor and the infrared spectral gas analysis unit are electrically connected to each other and acquire data synchronously through hardware. This allows the thermal imaging image to be aligned with the timestamp of the gas detection data, thereby obtaining an image that corresponds to the gas leak.

[0087] The edge processing module includes at least a microcontroller (MCU) and a neural network processor (NPU). The NPU is preloaded with an offline Wi-Fi fingerprint database containing physical coordinate grids and offline fingerprint vectors. The edge processing module is configured to call an adaptive KNN positioning algorithm to perform two-dimensional spatial coordinate calculation on the spatial fingerprint feature vectors, and output the fused personnel positioning coordinates, thermal imaging temperature field image, and gas leak point concentration data to the AR optical display module.

[0088] The AR optical display module is used to transmit real-world images in real time and overlay personnel location coordinates, area temperature cloud map, gas leak outline, concentration value, equipment number, and historical maintenance records onto the real-world field of view.

[0089] The AR optical display module also supports high-brightness red alerts and flashing warnings for high-risk leakage locations.

[0090] Example 2

[0091] Based on Embodiment 1, this embodiment aims to provide an AR inspection glasses method based on adaptive KNN positioning and thermal imaging, referring to... Figure 2 This includes the following steps:

[0092] S1: Offline fingerprint database construction and system initialization preparation;

[0093] S1.1: All modules start synchronously, the Wi-Fi fingerprint acquisition module continuously scans the AP signal, and the thermal imaging gas module enters real-time acquisition mode;

[0094] S1.2: Adaptive KNN real-time localization;

[0095] S1.3: The inspection personnel align the integrated thermal imaging gas detection module 101 at the front of the glasses with the corresponding area;

[0096] S1.4: Multi-data AR overlay visualization;

[0097] S2: The edge real-time localization solution algorithm based on adaptive KNN is as follows: the Wi-Fi fingerprint acquisition module obtains online fingerprint vectors in real time at a sampling frequency of 1Hz to 2Hz. Then, the edge processing module executes the adaptive KNN localization algorithm locally to calculate the Euclidean distance between the online fingerprint and each reference point in the offline feature library. By analyzing the fingerprint similarity of adjacent reference points, spatial occlusion and grid density are dynamically evaluated in real time. The nearest neighbor value is adaptively and dynamically selected, and the current two-dimensional spatial coordinates of the inspection personnel are calculated in real time using the distance reciprocal weighted formula.

[0098] S2.2: Adaptive KNN real-time localization;

[0099] S3.3: The inspection personnel will align the integrated thermal imaging gas detection module 101 at the front of the glasses with the corresponding area;

[0100] S4.4: Multi-data AR overlay visualization.

[0101] S3: Multimodal synchronous acquisition of infrared thermal imaging and spectral gas detection. Specifically, the operator moves the line of sight to make the thermal imaging gas integrated module 101 at the front of the glasses target the inspection area. The infrared thermal imaging sensor captures the infrared thermal radiation on site and performs Gaussian noise reduction and temperature threshold segmentation preprocessing.

[0102] The infrared spectral analysis unit synchronously detects characteristic spectral absorption, and the infrared thermal imaging sensor and the infrared spectral gas analysis unit achieve precise time stamp alignment through a hardware clock, thereby acquiring the abnormal high temperature field and the type, concentration and pixel area of ​​gas leakage in real time.

[0103] S4: Spatiotemporal coordinate alignment and multi-source data spatial mapping, specifically: combining the real-time global two-dimensional spatial coordinates of the inspection personnel obtained in step S2. The head spatial pose matrix and the infrared module intrinsic parameter matrix are synchronously extracted by the nine-axis inertial sensor contained in the AR glasses body 1. The pixel coordinates of the abnormal heat map center and the gas leak area captured in step S3 are reverse-projected to generate directional geometric rays corresponding to the real space.

[0104] S5: AR layered rendering and hazard tracing. Specifically, the edge processing module fuses the mapped multi-source data and sends it to the AR optical display module to perform three-dimensional spatial layered rendering on the real-world perspective image. The bottom layer is the real-world scene image, the middle layer is a semi-transparent thermal imaging temperature cloud map, and the top layer is a precisely anchored and drawn red high-risk leak outline. A floating window displays the real-time gas concentration value, location coordinates, and equipment ledger. When an abnormal alarm threshold is triggered, the system automatically records and saves a four-in-one hazard package (timestamp - global coordinates - thermal image screenshot - spectral data) to the local flash memory for retrospective auditing after the operation is completed.

[0105] In S1, the Wi-Fi fingerprint acquisition module scans and generates a real-time online fingerprint vector, which contains the RSSI online data of all currently detectable APs.

[0106] In S3, Gaussian noise reduction and temperature threshold segmentation are first performed on the original thermal image to extract high-temperature abnormal areas for infrared thermal imaging preprocessing. In addition, the infrared unit collects gas absorption band signals, matches them with the gas characteristic spectrum library, and outputs the gas type, concentration, and leakage pixel area to complete gas spectrum identification. Finally, the leakage pixels in the thermal image are combined with the glasses pose and personnel positioning coordinates to convert them into global two-dimensional coordinates of the inspection scene and mark the leakage point.

[0107] In S4, the multi-source data AR fusion rendering algorithm specifically maps the real-time location coordinates of personnel, the thermal field distribution of equipment temperature, and the location information of gas leaks into a unified spatial coordinate system, and then transmits them to the AR display module for layered rendering.

[0108] Bottom layer: Real-world perspective view of the scene;

[0109] Middle layer: Overlay of semi-transparent thermal imaging temperature cloud map;

[0110] Upper layer: Leakage outline, gas concentration value, location coordinates, equipment ledger information;

[0111] Alarm strategy: Overheating and high concentration leak points are highlighted in red and simultaneously trigger a local buzzer alarm.

[0112] In S1.2, the adaptive KNN real-time positioning generates a set of online Wi-Fi fingerprints every 0.5 to 1 second, and the processing module automatically calculates the fingerprint distance and adaptively selects the fingerprints. The nearest neighbor, weighted solution, uses real-time spatial coordinates of the personnel to continuously update the positioning results.

[0113] In S1 and S3, the inspector aligns the integrated thermal imaging gas detection module 101 at the front of the glasses with the corresponding area, outputs an infrared temperature image in real time, identifies overheating defects, and then uses the infrared spectral unit to detect whether there is a leak of flammable or toxic gas, outputting the leak area and concentration.

[0114] In S1 and S4, when excessive high temperature or gas leakage is detected, the equipment automatically stores the information. The stored information includes at least the current location coordinates, thermal imaging screenshot, gas concentration data, and timestamp. It also supports exporting inspection records and hazard point distribution maps after the inspection is completed.

[0115] In S1, the front-end preparation specifically involves: dividing the inspection room area into uniform grid reference points, marking each reference point with its actual physical coordinates, and repeatedly collecting RSSI and CSI data from surrounding APs at each reference point, taking the average as the offline fingerprint feature vector for that point, and generating a cloud database. The offline fingerprint database construction method specifically involves: after dividing the inspection area into uniform grid reference points, repeatedly collecting RSSI and CSI features from surrounding APs at each point and taking the average, uploading the feature key-value pairs of coordinates and fingerprint vectors, and aggregating them on a cloud server to construct a global offline fingerprint database.

[0116] In S1, the edge processing unit 2 synchronizes and completely downloads the offline fingerprint database from the cloud to the local flash memory. During subsequent real-time inspection operations, the edge processing module only retrieves the local flash memory data as the positioning and matching benchmark, and interacts with the cloud network in real time without any connection.

[0117] The cloud database is only used for receiving and storing the mean values ​​of RSSI and CSI feature vectors collected from multiple reference points during the offline phase, and for constructing and training a global offline fingerprint database.

[0118] The CSI data amplitude extraction and phase dewinding preprocessing logic is as follows: For CSI channel state information, since the underlying subcarrier data contains complex amplitude and phase values, the system first extracts the CSI amplitude matrix by calculating the absolute value of the subcarrier. Then, it calls a linear transformation function to dewind the original CSI phase, eliminating random phase offset caused by transceiver clock asynchrony. The system then performs a weighted average of the stable RSSI scalar values ​​of multiple frames and the dewinding CSI amplitude and phase matrix within a time window, and takes this average value as the final offline fingerprint feature vector for that point.

[0119] In S2, fingerprint distance calculation involves comparing the feature vector with the fingerprint features of all reference points in the fingerprint database. Using all reference points in the cloud database, the feature distance between the online fingerprint and each offline fingerprint is calculated, and then adaptive... Values ​​are dynamically selected. Adaptive. The criteria for dynamic value selection are:

[0120] High-signal stable region fingerprint dense grid: ;

[0121] Severe signal blockage: .

[0122] Severe signal obstruction occurs in areas with sparse fingerprints, corners, or areas blocked by devices.

[0123] Adaptive The logic for dynamically selecting values ​​is to automatically distinguish region density based on the fingerprint similarity of adjacent reference points and update it dynamically in real time. value.

[0124] Real-time dynamic updates The values ​​need to be weighted and calculated, specifically by filtering for the values ​​with the smallest distances. The final personnel location coordinates are calculated using a distance-weighted inverse calculus of three nearest neighbor reference points, as shown in the following expression:

[0125]

[0126] in the formula For the first The fingerprint distance between the nearest neighbors is used, with closer distances carrying greater weight, to ensure that the positioning coordinates closely approximate the actual location. To provide the inspection personnel with the final estimate of the two-dimensional positioning coordinates, For the first The weighted average of the nearest neighbor reference points is assigned using the reciprocal of the distance. For the first in the database The actual offline physical coordinates of the nearest reference point. The summation operator represents summation over all... Accumulate the nearest neighbor points. The smaller the fingerprint distance, the larger the weight value; the closer the reference point, the stronger its influence on the final positioning coordinates.

[0127] And two-dimensional spatial coordinates Solve the mathematical formula and standardize the parameter references. The specific expression is as follows:

[0128]

[0129] in the formula For the first The fingerprint distance between the nearest neighbors This refers to the system's final estimate of the current real-time two-dimensional physical coordinates of the inspection personnel. For the first The weighted values ​​are assigned to the nearest neighbor reference points, using the inverse of the distance. For the offline database The calibration physical coordinates of the nearest reference point This represents the total number of neighboring reference points dynamically selected under the current environmental conditions.

[0130] When severe signal obstruction and sparse fingerprint grid are detected, by... The value increases to To mitigate the singular feature errors caused by multipath reflection, and to add a physical threshold filtering mechanism based on Euclidean distance before weighted calculation: if the feature distance of a candidate nearest neighbor point... Greater than the preset dynamic threshold If the physical space topology is in a non-connected region, it will be automatically removed from the K nearest neighbors to ensure that the sampling range is expanded without introducing out-of-bounds topology errors.

[0131] The above are merely optional embodiments of this disclosure and are not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An AR inspection glasses based on adaptive KNN positioning and thermal imaging, comprising an AR glasses main body (1), an edge processing unit (2), and an independent or integrated AR glasses host unit (3), characterized in that: It also includes a thermal imaging gas detection integrated module (101) disposed on the AR glasses body (1), the thermal imaging gas detection integrated module (101) being electrically connected to the edge processing unit (2) via a line; It also includes the AR glasses main body (1) connected to the AR glasses host unit (3) via a line; The edge processing unit (2) includes a Wi-Fi fingerprint acquisition module, a thermal imaging gas detection integrated module, an edge processing module, and an AR optical display module; The edge processing unit (2) also has an environment adaptive boundary dynamic switching mechanism and a neural network processor. The signal processing logic of the edge processing unit (2) includes extracting the CSI channel state information of the underlying orthogonal frequency division multiplexing signal from the Wi-Fi fingerprint acquisition module. The neural network processor calculates the subcarrier phase variance in the CSI matrix in real time, and uses the subcarrier phase variance as a physical indicator to quantify the severity of the current spatial multipath effect and electromagnetic scattering. The logic of the environmental adaptive boundary dynamic switching mechanism is as follows: The monitored phase variance is less than a preset threshold. At this time, the system determines that it is currently in a high signal stability region, and the hardware controller dynamically sets the nearest neighbor parameters of the KNN algorithm. ; The monitored phase variance is greater than or equal to a preset threshold. When the system determines that it is in a region of severe signal attenuation due to severe metal obstruction or dense equipment, the hardware controller will forcibly trigger a boundary switch, expand the sampling nearest neighbor range, and simultaneously start physical space Euclidean distance threshold filtering to eliminate pseudo-nearest neighbor reference points in non-connected areas of the spatial topology.

2. The AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 1, characterized in that: The Wi-Fi fingerprint acquisition module continuously scans all AP wireless access points on site at a fixed frequency of 1Hz to 2Hz, with each cycle lasting 0.5 to 1 second. Simultaneously, it collects two types of fingerprint features: RSSI received signal strength and CSI channel status information. The fingerprint vector is then transmitted in real-time to the local processing module for location calculation. By fusing the RSSI surface signal and the CSI underlying channel features, the module calculates the real-time online fingerprint vector and compares it with the fingerprint in the offline fingerprint database. The weighted Euclidean distance between the fingerprint vectors of each reference point The physical mapping formula is as follows: In the above formula This represents the total number of valid access points (APs) that the Wi-Fi fingerprinting module can reliably detect within the current spatial grid. and These represent the data currently collected in real time by the edge processing unit (2) from the first... The received signal strength index and channel state information of each AP. and These represent the pre-loaded offline fingerprint database, the first... The corresponding physical reference point is the th The mean of the time-bound calibration characteristics of each AP; In the above formula, and These are the feature fusion weight coefficients, which are set based on empirical multipath attenuation models in the field, and are subject to mandatory constraints during system initialization. + =1.

3. The AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 1, characterized in that: The integrated thermal imaging gas detection module includes at least an infrared thermal imaging sensor and an infrared spectral gas analysis unit, both of which are configured in an integrated package structure. The function of the infrared thermal imaging sensor is to capture infrared thermal radiation in the industrial indoor inspection environment and output the infrared thermal radiation as a real-time thermal imaging raw image. The infrared spectroscopy gas analysis unit is used to capture gases in the industrial indoor inspection environment based on the principle of gas absorption spectroscopy in a specific infrared band, and output the gas concentration and the coordinates of the leak area. The infrared spectroscopy gas analysis unit captures gases in the industrial indoor inspection environment, including at least methane, carbon monoxide, sulfur hexafluoride, and refrigerant.

4. The AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 3, characterized in that: The infrared thermal imaging sensor and the infrared spectral gas analysis unit are electrically connected to each other and are synchronously acquired through hardware, thereby aligning the thermal imaging image with the gas detection data timestamp, and thus obtaining an image corresponding to the gas leak.

5. An AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 1, characterized in that: The edge processing module includes at least an independent embedded processing unit, which includes at least an MCU and an NPU micro-computing unit. All algorithm calculations are performed locally without relying on real-time cloud transmission. The specific function of the independent embedded processing unit is as follows: Stores an offline Wi-Fi fingerprint database; The adaptive KNN positioning algorithm is executed to calculate the two-dimensional spatial coordinates of the inspection personnel in real time. Thermal imaging image noise reduction, contrast enhancement, and wireless image transmission preprocessing; It integrates multi-source data, including location coordinates, thermal imaging temperature field, and gas leak location data; Output the overlay rendering data to the AR display module.

6. An AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 1, characterized in that: The AR optical display module is used to transmit real-world images in real time and overlay personnel location coordinates, regional temperature cloud maps, gas leak outlines, concentration values, equipment numbers, and historical maintenance records onto the real-world view.

7. An AR inspection glasses method based on adaptive KNN positioning and thermal imaging as described in claims 1-6, characterized in that: Includes the following steps: S1: Offline fingerprint database construction and system initialization preparation; S1.1: All modules start synchronously, the Wi-Fi fingerprint acquisition module continuously scans the AP signal, and the thermal imaging gas module enters real-time acquisition mode; S2: Real-time edge localization solution based on adaptive KNN; S2.1: Adaptive KNN real-time localization; S2.2: The inspection personnel will align the integrated thermal imaging gas detection module (101) at the front of the glasses with the corresponding area; S2.3: Multi-data AR overlay visualization; S3: Multimodal synchronous acquisition of infrared thermal imaging and spectral gas detection; S4: Spatiotemporal coordinate alignment and spatial mapping of multi-source data; S5: AR layered rendering of virtual and real objects and hazard tracing.

8. The AR inspection glasses method based on adaptive KNN localization and thermal imaging according to claim 7, characterized in that: In S1, the front-end preparation specifically involves dividing the inspection room area into uniform grid reference points, marking each reference point with its actual physical coordinates, and collecting RSSI and CSI data of surrounding APs multiple times at each reference point. The average value is then used as the offline fingerprint feature vector for that point, and a cloud database is generated. The method for constructing the offline fingerprint database is as follows: after dividing the inspection area into uniform grid reference points, the RSSI and CSI features of surrounding APs at each point are collected multiple times and the average value is taken. First, the feature key-value pairs of coordinates and fingerprint vectors are uploaded and aggregated on the cloud server to construct a global offline fingerprint database.

9. An AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 7, characterized in that: In S2, the real-time edge localization solution based on adaptive KNN includes fingerprint distance calculation. This involves calculating the distance between the feature vector and the fingerprint features of all reference points in the fingerprint database. Using all reference points in the cloud database, the feature distance between the online fingerprint and each offline fingerprint is calculated, and then adaptive KNN is applied. Values ​​are dynamically selected and are adaptive. The criteria for dynamic value selection are: (1) Dense fingerprint grid in high signal stability region: ; (2) Severe signal obstruction: . The adaptive The logic for dynamically selecting values ​​is to automatically distinguish region density based on the fingerprint similarity of adjacent reference points and update it dynamically in real time. value.

10. An AR inspection glasses based on adaptive KNN positioning and thermal imaging according to claim 9, characterized in that: The real-time dynamic update The values ​​need to be weighted and calculated, specifically by filtering for the values ​​with the smallest distances. The final personnel location coordinates are calculated using a distance-weighted inverse calculus of three nearest neighbor reference points, as shown in the following expression: in the formula For the first The fingerprint distance between the nearest neighbors is used, with closer distances carrying greater weight, to ensure that the positioning coordinates closely approximate the actual location. To provide the inspection personnel with the final estimate of the two-dimensional positioning coordinates, For the first The weighted average of the nearest neighbor reference points is assigned using the reciprocal of the distance. For the first in the database The actual offline physical coordinates of the nearest reference point. The summation operator represents summation over all... Accumulate the nearest neighbor points. The smaller the fingerprint distance, the larger the weight value; the closer the reference point, the stronger its influence on the final positioning coordinates. And two-dimensional spatial coordinates Solve the mathematical formula and standardize the parameter references. The specific expression is as follows: in the formula For the first The fingerprint distance between the nearest neighbors This refers to the system's final estimate of the current real-time two-dimensional physical coordinates of the inspection personnel. For the first The weighted values ​​are assigned to the nearest neighbor reference points, using the inverse of the distance. For the offline database The calibration physical coordinates of the nearest reference point This represents the total number of neighboring reference points dynamically selected under the current environmental conditions.

Images