A control method for smart AR glasses

Abstract

This invention belongs to the field of AR glasses control technology, specifically disclosing a smart AR glasses control method. This method continuously senses extreme environmental parameters before and during power inspection by real-time acquisition of high-voltage electromagnetic field spectrum, ambient light, and wind speed data in the power inspection area, solving the problems of existing AR glasses not considering environmental resistance and functional failure in extreme scenarios. Simultaneously, it utilizes Fast Fourier Transform to extract the dominant frequency characteristics of the electromagnetic field, combines this with ambient light statistics to adjust the AR glasses settings, and activates jitter compensation when wind speed exceeds limits, enabling the glasses to effectively avoid signal interference and ensure basic functional stability in power inspection scenarios such as strong magnetic fields and complex lighting. Furthermore, by synchronously monitoring the pupil diameter change rate and eyelid occlusion ratio, it verifies eye movement status in real-time throughout the inspection process and generates a precise iris gaze trajectory after dynamic judgment and calibration, ensuring the accuracy of eye-tracking interaction under complex working postures.

Description

Technical Field

[0001] This invention belongs to the field of AR glasses control technology, and more specifically, relates to a smart AR glasses control method. Background Technology

[0002] During high-voltage cable tower climbing and inspection, inspectors face challenges such as repeated climbing and repositioning, and looking down to check equipment, leading to distractions. Against this industry backdrop, AR glasses have become a key technology for the intelligent upgrading of power inspection. To ensure the accuracy and efficiency of inspection work, the control and management of AR glasses is particularly important.

[0003] Existing technologies, such as the AR glasses and AR glasses control method disclosed in Chinese invention patent application No. 202311585115.7, capture bone conduction speech signals through a contact microphone and collect ambient sound in conjunction with a non-contact microphone. After fusing the dual signals to improve the signal-to-noise ratio, the external speech is converted into text overlay display in real time, solving the problems of low recognition rate of a single microphone and failure of human-machine voice separation in noisy environments, and realizing the robustness enhancement of voice commands and real-time subtitle assistance function.

[0004] Existing technologies, such as the AR smart glasses disclosed in Chinese invention patent application No. 201911079793.X, rely on a split battery compartment / control compartment architecture and a three-level link between the glasses, front-end devices, and central station. Environmental data is collected by the camera on the glasses, pre-processed by the control compartment, and then transmitted to the front-end devices. Finally, the central station integrates multi-source data and feeds it back to the AR display terminal, breaking through the single-device computing power bottleneck and cross-terminal collaboration latency barrier, achieving multi-terminal virtual and real fusion information synchronization and improved operation efficiency.

[0005] Patent 1 solves the voice input quality problem through dual microphones, and Patent 2 solves the data processing problem through multi-level transmission. Obviously, the current use cases of AR glasses are mostly concentrated in the interaction layer, while the pain points of power inspection are outside the current concentration points. That is, they are not suitable for operational places like power inspection. There are also the following shortcomings: 1. Neither of them considers the problem of environmental resistance. In extreme scenarios, such as when the magnetic field is too strong, it will cause the basic functions to fail. Moreover, multi-level transmission will cause great interference in signal blind spots in mountainous areas.

[0006] 2. Currently, voice interaction is mainly used to compensate for hand inconvenience in certain scenarios. However, pure voice solutions cannot be used when climbing and exerting force, making it difficult to guarantee the accuracy of commands and interactions, and thus failing to guarantee detection results. Summary of the Invention

[0007] In view of this, in order to solve the above problems, a control method for smart AR glasses is proposed.

[0008] The objective of this invention can be achieved through the following technical solution: This invention provides a smart AR glasses control method, which includes: real-time acquisition of high-voltage electromagnetic field spectrum data, ambient light data and wind speed data of the current power inspection area.

[0009] The frequency and intensity of the dominant frequency component in the spectrum are extracted by Fast Fourier Transform as electromagnetic field spectrum features. The mean and standard deviation of the real-time ambient light intensity are calculated to obtain ambient light features. The AR glasses settings are adjusted by combining the electromagnetic field spectrum features and ambient light features, and shake compensation is activated when the wind speed continues to exceed the standard.

[0010] Simultaneously monitor the rate of change in pupil diameter and the proportion of eyelid occlusion to determine whether dynamic calibration is triggered. If triggered, generate a calibrated iris gaze trajectory.

[0011] It continuously acquires electromyographic signals of the forearm flexor muscles, and activates eye-tracking mode when the grip strength value continuously exceeds the set threshold.

[0012] Based on the calibrated iris gaze trajectory, when the gaze focus stays on the surface of the object for too long, the glasses camera zooms to capture the image of the defective area, and then feeds back the image after superimposing historical fault comparison data.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention can continuously sense extreme environmental parameters before and during operation by collecting high-voltage electromagnetic field spectrum, ambient light and wind speed data in the power inspection area in real time, which solves the problem that the existing AR glasses do not consider environmental resistance and the function fails in extreme scenarios. At the same time, the main frequency characteristics of the electromagnetic field are extracted by using fast Fourier transform, and the AR glasses settings are adjusted by combining the ambient light statistics. The shaking compensation is activated when the wind speed exceeds the standard, so that the glasses can effectively avoid signal interference and ensure the stability of basic functions in power inspection scenarios such as strong magnetic fields and complex lighting.

[0014] (2) This invention calibrates the iris gaze trajectory by synchronously monitoring the pupil diameter change rate and the eyelid occlusion ratio, which solves the problem of inaccurate eye tracking caused by climbing force and environmental interference in power inspection, thus affecting the interaction and detection effect. It can ensure the accuracy of eye interaction under complex working postures and provide a reliable gaze positioning basis for subsequent defect detection.

[0015] (3) This invention continuously acquires electromyographic signals of the forearm flexor muscle group and activates eye-tracking mode when the grip force exceeds the threshold, which makes up for the shortcomings of voice interaction and realizes effective interaction without relying on voice in force exertion scenarios such as climbing. This can ensure the accuracy of command issuance and interaction and support the stable progress of the detection process. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the overall implementation steps of the method of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the adjustment process for the AR glasses of the present invention.

[0019] Figure 3 A schematic diagram illustrating the transmission power adjustment judgment process for this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 As shown, the present invention provides a smart AR glasses control method, which includes: S1, real-time acquisition of high-voltage electromagnetic field spectrum data, ambient light data and wind speed data of the current power inspection area.

[0022] S2. The frequency and intensity of the main frequency component in the spectrum are extracted by fast Fourier transform as electromagnetic field spectrum features. The mean and standard deviation of the real-time ambient light intensity are calculated to obtain ambient light features. The AR glasses settings are adjusted by combining the electromagnetic field spectrum features and ambient light features. When the wind speed continues to exceed the standard, shake compensation is activated.

[0023] This invention, through real-time acquisition of high-voltage electromagnetic field spectrum, ambient light, and wind speed data in power inspection areas, can continuously perceive extreme environmental parameters before and during operations, solving the problems of existing AR glasses not considering environmental resistance and functional failure in extreme scenarios. Simultaneously, it utilizes Fast Fourier Transform to extract the dominant frequency characteristics of the electromagnetic field, combines this with ambient light statistics to adjust the AR glasses settings, and activates jitter compensation when wind speed exceeds limits, enabling the glasses to effectively avoid signal interference and ensure stable basic functions in power inspection scenarios such as strong magnetic fields and complex lighting.

[0024] S3. Simultaneously monitor the pupil diameter change rate and eyelid occlusion ratio to determine whether dynamic calibration is triggered. If triggered, generate the calibrated iris gaze trajectory.

[0025] S4. Continuously acquire electromyographic signals of the forearm flexor muscles, and activate eye-tracking mode when the grip strength value continuously exceeds the set threshold.

[0026] S5. Based on the calibrated iris gaze trajectory, when the gaze focus stays on the surface of the object for too long, control the glasses camera to zoom and capture the image of the defective part, and then provide feedback after superimposing historical fault comparison data.

[0027] The high-voltage electromagnetic field spectrum data mentioned in step S1 is acquired by a dual-axis electromagnetic field sensor integrated into the end of the mirror temple. The electromagnetic field sensor includes a high-frequency loop antenna and a low-frequency fluxgate sensor.

[0028] The ambient light data is collected by a multi-channel ambient light sensor at the edge of the lens.

[0029] The wind speed data is collected by a miniature ultrasonic anemometer on the outside of the nose pad bracket, and the anemometer measures the wind speed vector.

[0030] Please see Figure 2 As shown, the AR glasses settings adjustment in step S2 is determined by the following steps: S21, calculate the spectral energy ratio of each main frequency component as its weight.

[0031] S22. Extract the maximum and minimum intensity values ​​of all frequency components in the current spectrum, and output the normalized intensity value of each main frequency component after normalizing the intensity value of each main frequency component through Min-Max.

[0032] S23. The normalized intensity value of each main frequency component and its weighted linear sum are used as the comprehensive power index, and the transmission power adjustment is determined accordingly.

[0033] S24. If it is determined that the transmission power needs to be adjusted, confirm the adjustment of the transmission power and perform the transmission power adjustment based on the adjusted transmission power.

[0034] S25. Otherwise, the current power inspection area image is collected by the camera mounted on the AR glasses and recorded as the target image. The target image is then analyzed by performing ROI positioning, grayscale conversion and dynamic threshold segmentation, and the ratio of bright and dark pixels is statistically analyzed.

[0035] S26. Based on ambient light characteristics and the ratio of bright to dark pixels, determine whether any of the following conditions are triggered. If triggered, adjust the display settings and perform the corresponding display adjustment operation: the standard deviation exceeds the preset threshold.

[0036] The average ambient light intensity exceeded the experimentally calibrated range of human eye comfort.

[0037] The ratio of bright pixels or the ratio of dark pixels exceeds the corresponding preset threshold.

[0038] S27. Otherwise, do not adjust the display settings.

[0039] Intelligibly, the linear weighted sum refers to the sum of the normalized intensity values ​​of each dominant frequency component multiplied by its weight.

[0040] In one specific embodiment, the image recognition technology is a relatively mature existing technology, and the process of ROI positioning, grayscale conversion and dynamic threshold segmentation of the target image and then statistically analyzing the distribution ratio of bright and dark pixels through image recognition technology are all common processing procedures. The specific processing steps will not be shown in detail.

[0041] In one specific embodiment, the calibration of the human eye comfort range needs to be combined with the actual operation scenario of power inspection. This involves selecting typical working conditions such as strong sunlight on sunny days, diffused light on cloudy days, indoor artificial lighting, and weak light at night as test scenarios, and using a spectroradiometer to accurately measure the range of ambient light intensity. Then, several operators with power inspection experience are selected as subjects to wear AR glasses and perform simulated inspection tasks in various scenarios, such as defect identification and parameter reading. Simultaneously, physiological indicators such as pupil changes and blink frequency are monitored using an eye tracker, and subjective scoring methods, such as the NASA-TLX scale, are used to record the degree of visual fatigue and comfort. The collected subjective and objective data are statistically analyzed, with a maximum subjective comfort score of 5 points. An ambient light intensity range where 95% of subjects have a subjective comfort score greater than or equal to 4 points and no significant abnormalities in eye movement physiological indicators, such as pupil diameter fluctuation less than or equal to 15%, is used as the basic comfort threshold. This is combined with the brightness adjustment range of the AR display screen and the reflective characteristics of the power equipment surface, and the threshold boundary is optimized through dynamic matching experiments. Finally, a human eye comfort light intensity range suitable for inspection scenarios is calibrated.

[0042] It should be added that the preset threshold for the brightness / dark pixel ratio is determined by collecting equipment images under various working conditions during power inspection, such as strong light, shadow, and nighttime. After ROI positioning and dynamic threshold segmentation, the proportion of bright / dark pixels is statistically analyzed. The mean and standard deviation are calculated according to the scene classification. The sum of the mean and twice the standard deviation can be selected as the initial threshold. Then, combined with human visual comfort and the reflective characteristics of the equipment material, the threshold is optimized through subjective scoring and defect recognition accuracy testing. Finally, the preset threshold for the brightness / dark pixel ratio is determined to balance the presentation of image details and visual comfort. For example, the threshold for the proportion of bright pixels is usually in the range of 65% to 75%, and the threshold for the proportion of dark pixels is usually in the range of 30% to 40%.

[0043] Further, please refer to Figure 3As shown, the specific steps for determining whether to adjust the transmission power in step S23 include: dividing the real-time collected magnetic field spectrum data into time windows based on a preset time interval, and extracting the comprehensive power index under each time window.

[0044] If the overall power index exceeds the preset overall power index threshold for N consecutive time windows, and meets any of the following conditions, then the transmission power will be adjusted:

[0045] The interference power of adjacent channel exceeds the preset threshold.

[0046] The received signal strength exceeds the target value by an amount that exceeds the receiver's dynamic range allowable threshold.

[0047] Otherwise, it is determined that no transmission power adjustment will be performed.

[0048] In one specific embodiment, N is a positive integer greater than or equal to 2.

[0049] Understandably, the preset threshold for adjacent channel interference power is set comprehensively based on the Adjacent Channel Selectivity (ACS) index, industry standards, and interference characteristics of the actual scenario. It first collects historical interference power data of adjacent channels within the inspection area using a spectrum analyzer and statistically analyzes the interference power distribution within a 95% confidence interval. This is then combined with receiver hardware parameters, such as calculating the critical interference power for adjacent channels with an ACS greater than or equal to 50dB. This critical power is the power value that causes a 3dB decrease in the receiver's signal-to-noise ratio. Finally, a 3-5dB safety margin is added to form the final threshold, such as -50dBm, to balance anti-interference capability and transmission power efficiency.

[0050] The specific superposition value of the final safety margin is determined based on the amplitude of the disturbance fluctuation. That is, the disturbance amplitude is mapped to the range of 0 to 1 through the minimum-maximum normalization method, which is used as the superposition ratio. The product of the superposition ratio and the safety margin is used as the final superposition value.

[0051] Understandably, the preset comprehensive power index threshold is based on historical data statistics of the magnetic field spectrum of the inspection scenario and the receiver's anti-interference threshold setting, and is dynamically corrected according to the interference fluctuation amplitude of the scenario, such as a 3 times standard deviation margin in mountainous scenarios, and the degree of equipment aging, such as an additional 20% margin for old equipment.

[0052] Understandably, the aforementioned permission thresholds are set differently based on the dynamic range of the AR glasses receiver in a strong electromagnetic environment, such as reserving a 15%-25% safety margin to prevent signal saturation, combined with typical signal scenarios in the inspection route.

[0053] It should be added that the target reception strength is dynamically issued by the upper-layer protocol based on the quality of service requirements, or can be set autonomously by the terminal based on the channel status.

[0054] Furthermore, the specific process of confirming the adjustment of transmission power in step S24 includes: A1. When the comprehensive power index is within the preset value range, the basic adjustment range is calculated according to the preset unit deviation attenuation coefficient and the difference between the comprehensive power index and its threshold. Otherwise, the calculated basic adjustment range is corrected by the strong interference enhancement coefficient to obtain the final basic adjustment range.

[0055] A2. If the weight of a certain main frequency component exceeds the preset first weight threshold, and the normalized intensity value of the main frequency component exceeds the preset interference value, the relative deviation between the normalized intensity value and the preset interference value shall be used as the compensation ratio.

[0056] A3. The compensation adjustment range is obtained by multiplying the preset additional compensation adjustment range and the compensation ratio, and then the base adjustment range is added to obtain the final adjustment range of the main frequency component.

[0057] A4. If the weight of a certain main frequency component is lower than the preset second weight threshold, the preset minimum adjustment range will be used as the final adjustment range of that main frequency component.

[0058] A5. If the weight of a certain main frequency component exceeds the preset first weight threshold and the normalized intensity value of the main frequency component does not exceed the preset interference value, or the weight of the main frequency component is between the preset second weight threshold and the first weight threshold, the basic adjustment range shall be used as the final adjustment range of the main frequency component.

[0059] A6. The transmission power of each main frequency component is adjusted based on the adjustment range of each main frequency component and the current set transmission power.

[0060] Understandably, the basic adjustment range is obtained by multiplying the difference between the comprehensive power index and its threshold by a preset unit deviation attenuation coefficient, and the correction is performed by multiplying the basic adjustment range by the strong interference enhancement coefficient.

[0061] It is important to note that when the overall power index exceeds the preset range, it indicates that the level is far beyond the safe range, and the attenuation effect needs to be amplified by increasing the enhancement factor. The default value is 1.2 to 1.5. This can be optimized based on the adjustment effect in historical strong interference scenarios, such as using an enhancement factor of 1.5 in high electromagnetic interference areas like substations.

[0062] Understandably, the preset value range consists of a preset threshold and a preset multiple threshold. The preset multiple threshold is calculated by analyzing historical comprehensive power index data from high-frequency interference areas such as substations and transmission lines to determine the average fluctuation range of the comprehensive power index under strong interference scenarios, such as 1.2 times. This ensures that the threshold can both avoid false triggering in weak interference scenarios and promptly initiate aggressive adjustment strategies under strong interference, balancing anti-interference effect and adjustment frequency.

[0063] It should be added that the unit deviation attenuation coefficient represents the basic power adjustment range when each unit of the comprehensive power index exceeds the threshold. Its value is determined by combining the linear adjustment accuracy of the AR glasses power amplifier and the sensitivity of power grid inspection to signal quality. Specifically, it refers to the minimum hardware adjustment step size, such as 1dB or 2dB, and the power and bit error rate curves under typical interference scenarios to ensure that each unit of comprehensive power index deviation corresponds to a reasonable attenuation amount. This avoids insufficient effect due to too small an adjustment range while preventing signal distortion caused by too large an adjustment range.

[0064] Understandably, the preset interference value is a critical value for determining whether strong interference exists in the main frequency component. Its value is determined by testing the normalized intensity distribution of electromagnetic signals near high-voltage equipment such as transformers and circuit breakers. For example, if it is determined that an intensity greater than or equal to 0.8 may cause image transmission distortion or frame loss, 0.8 is used as the preset interference value to trigger additional compensation adjustments, ensuring the transmission stability of the core frequency band under strong interference.

[0065] It should be added that the preset additional compensation is mainly used to strengthen and adjust high-weight, strong interference components. Its value is determined by combining the quality requirements of power grid inspection for core services with the hardware's anti-interference capabilities. When encountering strong interference in the core frequency band, additional attenuation is required to suppress the interference. The additional attenuation value is determined by simulating the peak signal-to-noise ratio of images under different interference intensities, such as 3dB. This ensures that the adjusted signal-to-noise ratio remains above 35dB, i.e., without significant distortion.

[0066] The minimum adjustment margin is the minimum power attenuation that low-weighted components must perform. Its value is based on the minimum hardware adjustment step size of the AR glasses power amplifier and basic anti-interference requirements. For example, at least 1dB adjustment is required for non-core frequency bands to reduce out-of-band radiation, so 1dB is usually set as the default minimum adjustment margin.

[0067] The preset first weight threshold is the dividing line between core service frequency bands and non-core frequency bands. Its value is determined by the service priority of power grid inspection. That is, the main data frequency band carrying real-time transmission of high-definition images can be assigned a weight greater than or equal to 0.6, with a default value of 0.6. This ensures that such frequency bands receive priority processing during adjustments.

[0068] The preset second weight threshold is used to define low-weight frequency bands that require the minimum adjustment. Its value is based on the functional positioning of non-core frequency bands in power grid inspection. That is, for frequency bands that frequently experience interference but are not critical for business, such as temporary communication channels, if the statistical weight is less than or equal to 0.2, only the minimum anti-interference measures are required, and the preset second weight threshold can be assigned to 0.2.

[0069] Furthermore, the specific steps for performing the corresponding display adjustment operation in step S26 include: if the standard deviation exceeds a preset threshold, using contrast as a display adjustment item.

[0070] Enable the Reinhard global tone mapping algorithm to compress the dynamic range of highlights until the standard deviation reaches the preset permitted standard deviation.

[0071] If other conditions are triggered, brightness will be included as a display adjustment item, and brightness adjustment will be performed.

[0072] After adjustment, wait for the preset time, then re-detect the brightness standard deviation and ambient light intensity. If they still do not exceed the preset threshold, trigger expert mode to prompt the user to adjust manually.

[0073] It should be added that the Reinhard global tone mapping algorithm is a mature existing algorithm, and its specific execution process will not be described further.

[0074] Understandably, the preset threshold for the standard deviation is pre-set based on the needs of the power grid inspection task, for example, by determining the optimal standard deviation threshold through subjective evaluation experiments. Furthermore, the preset waiting time is primarily set based on a combination of equipment response stability and human-machine interaction efficiency, requiring the time for the detection equipment to complete hardware parameter convergence after brightness adjustment. For example, CMOS sensor automatic gain control requires 5-10 seconds to stabilize, and the empirical value for the observation period before manual intervention is set at 1-2 minutes. Considering both factors, 1 minute can be selected as the preset waiting time.

[0075] Furthermore, the process of performing brightness adjustment includes: B1. If the average ambient light intensity is within the experimentally calibrated range of human eye comfort, and if the proportion of bright area pixels exceeds the standard, identify the overexposed area in the target image.

[0076] B2. After enhancing the overexposed area, the foreground and background are separated by an image segmentation algorithm. A Gaussian blur brightness mask is generated for the background area, and the background brightness is attenuated according to a preset ratio coefficient.

[0077] B3. If the proportion of dark area pixels exceeds the standard, the dark area is segmented using the Otsu threshold method, and noise areas with an area smaller than the preset pixel are removed by combining 8-neighbor connected component analysis.

[0078] B4. First, perform guided filtering on the dark area. If the average brightness of the dark area is lower than the preset threshold, perform adaptive gamma correction; otherwise, use the default gamma value for correction.

[0079] B5. When the ambient light intensity is higher than the upper limit of the comfort range, if the proportion of bright pixels exceeds the standard, the display brightness is reduced by a preset exponential decay ratio. The decay ratio is dynamically calculated based on the ratio of the proportion of bright pixels to a preset threshold, and the minimum brightness is not lower than the preset minimum display brightness.

[0080] B6. If the brightness pixel ratio does not exceed the standard, the display brightness will be directly adjusted to the upper limit of the brightness range that is comfortable for the human eye.

[0081] B7. When the ambient light intensity is lower than the lower limit of the comfort range, if the proportion of dark area pixels exceeds the standard, dark area enhancement processing will be performed first, and then the global brightness will be increased by a preset amount, and the maximum brightness after the increase will not exceed the lower limit of the comfort range.

[0082] B8. If the percentage of pixels in the dark area does not exceed the standard, the display brightness will be directly adjusted to the lower limit of the brightness range that is comfortable for the human eye.

[0083] In one specific embodiment, by converting the image to the CIELAB color space, consecutive pixel regions in the luminance channel with luminance values ​​exceeding 20,000 nits can be identified and thus recorded as overexposed areas. Here, 20,000 nits is a commonly used empirical value.

[0084] Understandably, the image enhancement can be performed by adaptive histogram equalization of the exposed area using a 3×3 grid. The image segmentation algorithm can specifically use the GrabCut algorithm to separate the foreground and background. Both image enhancement and image segmentation are common existing processing methods, and the specific execution process will not be shown in detail.

[0085] Understandably, overexposed areas are typically represented in an image as continuous bright spots. A kernel size of 15×15 can cover the geometric scale of common overexposed areas. The default blur kernel size for the solid brightness mask is set to 15×15. Gaussian blur processing is an existing method and will not be described in detail.

[0086] It should be added that, according to the Weber-Fechner law, the human eye's perception of brightness is logarithmic. An attenuation factor of 0.8 can reduce the subjective brightness of overly bright areas, such as 20,000 nits, by nearly 50%, thus eliminating glare while preserving background details. If the attenuation factor is reduced to below 0.5, it may lead to the loss of information in dark areas. Therefore, the preset ratio factor can be set to 0.8.

[0087] In one specific embodiment, the Otsu thresholding method for segmenting dark areas is a common existing technique, and its specific segmentation process will not be described in detail.

[0088] It should be added that adaptive gamma correction refers to dynamically adjusting the gamma value based on the brightness distribution of the dark area. For example, when the average brightness of the dark area is lower than... When using an adaptive gamma value such as 1.2, a power function with a gamma value greater than 1 is used to compress the range of pixel values ​​in dark areas while amplifying the relative contrast. For example, an input of 0.1 to 0.3 becomes 0.08 to 0.26 after being raised to the power of 1.2. This increases the proportion of differences between adjacent pixels, highlighting details such as cracks. If the brightness is higher than the threshold, a standard gamma value of 2.2 is used to avoid over-enhancing and amplifying noise.

[0089] It is important to note that the specific values ​​in adaptive gamma correction are as follows: The threshold is determined by the typical brightness range of the dark areas in power line inspection and the dynamic range of the industrial camera. Below this value, the sensor's detail capture capability decreases and needs to be enhanced. A gamma value of 1.2 is derived through subjective visual experiments and optimization using objective indicators such as SSIM and PSNR, improving the relative contrast of dark areas while controlling noise amplification. The standard gamma value of 2.2 is derived from Stevens' power law of human eye brightness perception and display device industry standards, ensuring that the image display conforms to visual habits and is compatible with mainstream devices.

[0090] In one specific embodiment, the enhancement processing in step B7 is the same as the enhancement processing in step B2, and will not be described again.

[0091] It should also be noted that the preset amplitude setting is determined comprehensively based on human visual comfort, device performance, and ambient light differences. By defining the lower limit of the comfortable range to which ambient light needs to be increased according to industry standards, when the ambient light is below this lower limit, the difference between the current light intensity and the lower limit is calculated. Then, considering the degree of excessive pixel ratio in dark areas and device noise control requirements, the amplitude value is obtained through multiple sets of experiments. For example, the increase amplitude is set to 60%-80% of the difference.

[0092] The specific process of activating jitter compensation when the wind speed continuously exceeds the standard in step S2 is as follows: C1. If the wind speed continuously exceeds the safety threshold and the duration reaches the set window time, it is marked as the wind speed continuously exceeding the standard.

[0093] C2. When the wind speed continues to exceed the standard, the helmet angular acceleration and linear acceleration are obtained through a nine-axis IMU. Based on the spatial coupling relationship between the wind speed vector and the acceleration, a three-axis reverse compensation matrix is ​​generated, and then the horizontal plane jitter compensation amount and the vertical plane jitter compensation amount are output.

[0094] It should be added that the safety threshold is determined by pre-testing the relationship between the shaking amplitude of the AR glasses and image stability under different wind speeds, and the vibration amplitude is monitored in real time by a built-in accelerometer. Image stability can be judged by image blur, that is, the minimum wind speed value corresponding to the image blur exceeding the acceptable threshold for the human eye is determined as the safety threshold.

[0095] Image blur is determined by calculating the variance of the Laplacian operator convolution result after grayscale processing of the acquired image. When the variance is 0, the image blur is assigned a value of 1; otherwise, the reciprocal of the variance value is used as the image blur.

[0096] In one specific embodiment, the formula for the three-axis inverse compensation matrix is: , This indicates the amount of horizontal displacement that needs to be compensated for, corresponding to the amount of horizontal plane shake compensation. This indicates the amount of displacement that needs to be compensated in the vertical direction of the lens, corresponding to the vertical plane shake compensation amount. This indicates the rotation angle that the lens needs to compensate for, and the corresponding angular velocity compensation is used to correct image tilt caused by helmet yaw / roll. Represents the transfer function coefficient matrix. This represents a scalar value for wind speed. Indicates the angle between the wind direction and the plane of the lens's optical axis. This indicates the helmet angular acceleration value. and These represent the projected components of the wind speed vector in the lens coordinate system, respectively. The horizontal and vertical plane jitter compensation amounts are proportional to the wind speed components in the lens coordinate system, and the angular velocity compensation amount is proportional to the angular acceleration measured by the IMU.

[0097] It should be added that the sub-coefficients in the transfer function coefficient matrix are determined by simulating wind speeds in the range of 5 to 25 m / s in a wind tunnel laboratory, measuring the actual lens offset under different wind speed / angle combinations, and then using the least squares method to fit and determine the sub-coefficients in the transfer function coefficient matrix.

[0098] Regarding step S3, it should be added that the monitoring of the pupil diameter change rate and the eyelid occlusion ratio is achieved by an infrared eye camera set inside the glasses. The infrared eye camera is equipped with a 940nm wavelength filter and combines image processing algorithms to identify the pupil edge and eyelid contour. The eyelid occlusion ratio is defined as the ratio of the vertical distance of the eyelid covering the upper edge of the iris to the iris diameter.

[0099] The specific steps for determining whether dynamic calibration is triggered in step S3 are as follows: If the following conditions are met simultaneously, dynamic calibration is triggered: the pupil diameter change rate exceeds the preset dynamic response threshold.

[0100] The proportion of eyelid occlusion is lower than the preset first effective threshold.

[0101] The ambient light intensity is within the preset effective response range.

[0102] Otherwise, it is determined that dynamic calibration will not be triggered.

[0103] Understandably, the dynamic response threshold for pupil diameter change rate originates from the combination of human visual physiology research and the dynamic response capability of equipment. It is determined by collecting pupil diameter change data from a large number of people under different lighting conditions and visual tasks, and combining this with the response delay of display device brightness adjustment to statistically determine the reasonable range of natural pupil change rates. When the rate of change exceeds this range, it often represents abnormal visual stimulation or physiological response. Using this threshold, effective brightness adjustment needs can be accurately distinguished from noise interference, ensuring the accuracy of closed-loop control. Its value has been determined through multiple rounds of human-machine optical physiology experiments and equipment integration tests.

[0104] Understandably, the first effective threshold for eyelid occlusion ratio is based on research into normal human visual behavior and image recognition reliability. By analyzing image data of different eyelid states, such as open eyes, blinking, and slightly closed eyes, and considering the need for effective pupil detection during brightness adjustment, it was determined that when the eyelid occlusion ratio is below this threshold, the pupil detection results are reliable and can be used for brightness adjustment decisions. The value was verified through eye motion capture experiments and image segmentation algorithms, balancing the extraction of the effective pupil region with the risk of misjudgment, ensuring a stable input for brightness adjustment.

[0105] It is also understandable that the preset effective response range for ambient light intensity is defined based on the comfortable visual environment for the human eye and the adjustment capability of the display device. Referring to the standard of comfortable ambient light for the human eye, and combined with the brightness adjustment limits of the display device under different lighting conditions, such as the ambient light threshold corresponding to the minimum / maximum adjustable brightness, a large number of ambient light and visual experience tests are conducted to screen out the range of ambient light in which the device can effectively compensate for brightness and the human eye can experience no obvious discomfort. This range is then used as the effective range for adjustment triggering.

[0106] Furthermore, the specific execution process of dynamic calibration in step S3 includes: querying the pre-stored pupil diameter-light intensity mapping relationship to obtain the expected pupil diameter corresponding to the current ambient light.

[0107] Calculate the deviation between the actual pupil diameter and the expected value, and decompose the deviation into planar coordinate offset components according to the gaze direction.

[0108] The pupil diameter change rate is multiplied by a preset unit fixed compensation coefficient to generate a compensation amount, and the coordinate offset component and the compensation amount are superimposed on the original iris feature point coordinates.

[0109] Data frames with eyelid occlusion exceeding a preset discard threshold are filtered out. After performing motion smoothing on the calibration coordinates of consecutive valid frames, the calibrated iris gaze trajectory is output.

[0110] It should be added that the pre-stored pupil diameter-light intensity mapping relationship was obtained by collecting multiple sets of physiological data and performing statistical analysis through controlled illumination experiments.

[0111] It should be added that the preset unit fixed compensation coefficient for the pupil diameter change rate is derived based on the linear relationship between brightness adjustment requirements and pupil light response. Physiological experiments were conducted to clarify the correlation between the pupil diameter change rate and the required brightness compensation. Combined with the engineering design of the display device's brightness adjustment step size, such as the minimum perceptible change in brightness during a single adjustment, this coefficient was determined through simulation calculations and subjective visual verification. This ensures that the compensation amount converted from pupil changes both conforms to the physiological needs of the human eye and is compatible with the hardware adjustment precision of the device, achieving accurate brightness compensation.

[0112] This invention calibrates the iris gaze trajectory by synchronously monitoring the rate of change in pupil diameter and the proportion of eyelid occlusion. This solves the problem of inaccurate eye tracking caused by climbing exertion and environmental interference in power line inspection, which affects the interaction and detection results. It can ensure the accuracy of eye movement interaction under complex working postures and provide a reliable gaze positioning basis for subsequent defect detection.

[0113] Regarding step S4, it should be added that the continuous acquisition of electromyographic (EMG) signals from the forearm flexor muscles is achieved through a flexible EMG patch sensor worn on the user's forearm. This EMG patch sensor contains 8 differential electrodes, with a signal acquisition frequency of 1000Hz. The grip strength value is obtained by bandpass filtering, rectification, and sliding window integration of the EMG signal, and the set threshold for the grip strength value is...

[0114] Bandpass filtering, rectification, and sliding window integration are all existing processing methods and will not be described further.

[0115] This invention continuously acquires electromyographic signals of the forearm flexor muscles and activates eye-tracking mode when grip strength exceeds a threshold, thus compensating for the shortcomings of voice interaction and enabling effective interaction without relying on voice in force exertion scenarios such as climbing. This ensures the accuracy of command issuance and interaction and supports the stable progress of the detection process.

[0116] Regarding step S5, it is necessary to add that the gaze focus stays on the surface of the object for an extended period of time. The extended period of time is defined as the duration of the gaze focus in the insulator area exceeding 2 seconds. The location of the surface of the object is defined by the coordinate range of the pre-imported three-dimensional model of the power equipment.

[0117] This invention addresses the problems of inaccurate defect location capture and difficulty in synchronously referencing historical information in power line inspections by monitoring the duration of gaze focus based on calibrated iris gaze trajectory. Furthermore, by controlling camera zoom for image capture and overlaying historical fault data, defects can be accurately located, assisting inspection personnel in rapid comparative analysis and thus improving the efficiency of defect identification and judgment.

[0118] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A method for controlling intelligent AR glasses, characterized in that, The method comprises: Real-time acquisition of high-voltage electromagnetic field spectrum data, ambient light data and wind speed data of the current power inspection area; Extract the frequency and intensity value of the main frequency component in the spectrum by fast Fourier transform as the electromagnetic field spectrum feature, and calculate the mean and standard deviation value of the real-time ambient light intensity to obtain the ambient light feature. The electromagnetic field spectrum feature and the ambient light feature are combined to adjust the AR glasses setting, and the dithering compensation is activated when the wind speed continuously exceeds the standard; Synchronously monitor the pupil diameter change rate and the eyelid shielding ratio to determine whether to trigger dynamic calibration, and generate the calibrated iris gaze trajectory if triggered; Continuously acquire forearm flexor muscle group electromyographic signals, and start the eye movement tracking mode when the grip strength value continuously exceeds the set threshold value; Based on the calibrated iris gaze trajectory, when the gaze focus stays on the surface of the detected object for more than a certain time, control the zoom of the camera of the glasses to capture the image of the defect position, and superimpose the historical fault comparison data for feedback. 2.The intelligent AR glasses control method of claim 1, wherein: The high-voltage electromagnetic field spectrum data is collected by a dual-axis electromagnetic field sensor integrated at the end of the glasses leg, and the electromagnetic field sensor includes a high-frequency loop antenna and a low-frequency fluxgate sensor; The ambient light data is collected by a multi-channel ambient light sensor at the edge of the lens; The wind speed data is collected by a miniature ultrasonic wind speed meter outside the nose support bracket, and the wind speed meter measures the wind speed vector. 3.The intelligent AR glasses control method of claim 1, wherein: The AR glasses setting adjustment is determined by the following steps: Calculate the spectrum energy proportion of each main frequency component as its weight; Extract the maximum and minimum intensity values of all frequency components in the current spectrum, and normalize the intensity values of each main frequency component to output the normalized intensity values of each main frequency component; The linear weighted sum of the normalized intensity values of each main frequency component and its weight is used as the comprehensive power index to determine whether to adjust the transmission power setting; If it is determined to adjust the transmission power setting, the transmission power is adjusted based on the adjusted transmission power; Otherwise, collect the current power inspection area image through the camera mounted on the AR glasses, denoted as the target image, and perform ROI positioning, grayscale conversion and dynamic threshold segmentation on the target image through image recognition technology to count the light and dark pixel distribution ratio; Based on the ambient light feature and the light and dark pixel distribution ratio, determine whether to trigger any of the following conditions, and perform display setting adjustment if triggered to perform the corresponding display adjustment operation: The standard deviation value exceeds the preset threshold value; The mean value of the ambient light intensity exceeds the eye comfort interval calibrated in the experiment; The light pixel distribution ratio or the dark pixel distribution ratio exceeds the corresponding preset threshold value; Otherwise, no display setting adjustment is performed.

4. The intelligent AR glasses control method of claim 3, wherein: The specific steps of determining whether to adjust the transmission power setting include: Based on the preset division time interval, the real-time collected magnetic field spectrum data is time window segmented to extract the comprehensive power index under each time window; If the comprehensive power index exceeds the preset comprehensive power index threshold value for N consecutive time windows, and any of the following conditions is met, it is determined to adjust the transmission power setting: The adjacent frequency channel interference power exceeds the preset threshold value; The amplitude of the received signal strength exceeding the target value exceeds the receiver dynamic range permission threshold value; Otherwise, it is determined not to adjust the transmission power.

5. The intelligent AR glasses control method of claim 3, wherein: The specific process of confirming the adjustment of the transmission power comprises: When the comprehensive power index is within a preset value interval, a preset unit deviation attenuation coefficient and a difference between the comprehensive power index and a threshold value are used to calculate a basic adjustment amplitude, otherwise, the calculated basic adjustment amplitude is modified by a strong interference intensification coefficient to obtain a final basic adjustment amplitude; If the weight of a certain main frequency component exceeds a preset first weight threshold value and the normalized intensity value of the main frequency component exceeds a preset interference value, a relative deviation value between the normalized intensity value and the preset interference value is used as a compensation ratio; A compensation adjustment amplitude is obtained based on a product of a preset additional compensation adjustment amplitude and the compensation ratio, and the final adjustment amplitude of the main frequency component is obtained by superimposing the basic adjustment amplitude; If the weight of a certain main frequency component is lower than a preset second weight threshold value, a preset minimum adjustment amplitude is used as the final adjustment amplitude of the main frequency component; If the weight of a certain main frequency component exceeds the preset first weight threshold value and the normalized intensity value of the main frequency component does not exceed the preset interference value or the weight of the main frequency component is between the preset second weight threshold value and the first weight threshold value, the basic adjustment amplitude is used as the final adjustment amplitude of the main frequency component; The adjustment amplitudes of the main frequency components are used to obtain the adjustment transmission power of each main frequency component based on the current setting transmission power.

6. The intelligent AR glasses control method of claim 3, wherein: The specific steps of executing the corresponding display adjustment operation comprise: If the standard deviation value exceeds a preset threshold value, the contrast is used as a display adjustment item; The Reinhard global tone mapping algorithm is enabled to compress the high light dynamic range until the standard deviation value reaches a preset permitted standard deviation value; If other conditions are triggered, the brightness is used as a display adjustment item, and the brightness adjustment is executed; After the adjustment is completed, the brightness standard deviation and the ambient light intensity are re-detected for a preset time length, and if they still do not exceed the preset threshold value, the expert mode is triggered to prompt the user to manually adjust.

7. The intelligent AR glasses control method of claim 6, wherein: The process of executing the brightness adjustment comprises: If the average ambient light intensity is within an experimentally calibrated human eye comfort interval, if the bright area pixel ratio exceeds a standard, overexposed regions in the target image are identified; After the overexposed regions are subjected to image enhancement, the foreground and the background are separated by an image segmentation algorithm, a Gaussian blur processed brightness mask is generated for the background region, and the background brightness is attenuated by a preset proportion coefficient; If the dark area pixel ratio exceeds a standard, the dark area is segmented by the Otsu threshold value method, and noise regions with an area smaller than a preset pixel are removed; The dark area is subjected to guided filter processing, and if the average brightness of the dark area is lower than a preset threshold value, adaptive gamma correction is performed, otherwise, a default gamma value is used for correction; When the ambient light intensity is higher than the upper limit of the comfort interval, if the bright area pixel ratio exceeds a standard, the display brightness is reduced by a preset exponential attenuation ratio, the attenuation ratio is dynamically calculated according to the ratio of the bright area pixel ratio to a preset threshold value, and the minimum brightness is not lower than a preset minimum display brightness; If the brightness pixel ratio does not exceed a standard, the display brightness is directly adjusted to the upper limit value of the brightness in the human eye comfort interval; When the ambient light intensity is lower than the lower limit of the comfort interval, if the dark area pixel ratio exceeds a standard, dark area enhancement processing is first performed, and then the global brightness is increased by a preset amplitude, and the maximum brightness after the increase does not exceed the lower limit value of the comfort interval. If the proportion of dark pixels does not exceed the threshold, the display brightness is directly adjusted to the lower limit of the comfortable range of human eyes. 8.The intelligent AR glasses control method of claim 1, wherein: The specific process of activating the jitter compensation when the wind speed continuously exceeds the threshold is as follows: If the wind speed continuously exceeds the safety threshold and the duration reaches the set window time, it is marked as continuous over-standard wind speed. When the wind speed continuously exceeds the threshold, the helmet angular acceleration and linear acceleration are obtained through the nine-axis IMU, and based on the spatial coupling relationship between the wind speed vector and the acceleration, a three-axis reverse compensation matrix is generated, and then the horizontal plane jitter compensation amount and the vertical plane jitter compensation amount are output. 9.The intelligent AR glasses control method of claim 1, wherein: The specific steps of determining whether to trigger dynamic calibration are as follows: If the following conditions are met, dynamic calibration is triggered: The pupil diameter change rate exceeds the preset dynamic response threshold; The eyelid shielding ratio is lower than the preset first effective threshold; The ambient light intensity is in the preset effective response interval; Otherwise, it is determined that dynamic calibration is not triggered. 10.The intelligent AR glasses control method of claim 1, wherein: The specific execution process of the dynamic calibration includes: Query the pre-stored pupil diameter-illumination intensity mapping relationship to obtain the expected pupil diameter corresponding to the current ambient light; Calculate the deviation of the actual pupil diameter from the expected value, and decompose the deviation into a plane coordinate offset component according to the gaze direction; Multiply the pupil diameter change rate by a preset unit fixed compensation coefficient to generate a compensation amount, and superimpose the coordinate offset component and the compensation amount on the original iris feature point coordinates; Filter the data frames whose eyelid shielding ratio exceeds the preset discard threshold, and output the calibrated iris gaze trajectory after performing motion smoothing processing on the calibration coordinates of the continuous valid frames.

Images