Image processing method and system of intelligent glasses

Abstract

The invention discloses an image processing method and system for intelligent glasses, and relates to the technical field of image recognition. The method comprises the following steps: synchronizing and preprocessing data collected by heterogeneous sensors in the intelligent glasses; constructing a global space-time coordinate system; a high-risk area is determined through dynamic saliency analysis, a high-resolution detection ROI is generated, full-pixel analysis is performed on a focus area, and an obstacle list with 3D positions, categories and danger levels is output; and generating a personalized safety path through a safety corridor generation algorithm based on the 3D position of the obstacle, the motion model of the visually impaired person and the real-time position. Data collected by heterogeneous sensors in the intelligent glasses are synchronized, a global space-time coordinate system is constructed, path planning is performed after obstacles are identified, high-frequency IMU angular velocity data and a self-adaptive optical flow correction network are fused, the image reverse compensation precision in a dynamic scene is improved, and navigation safety in a complex scene is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and particularly relates to an image processing method and system for intelligent glasses. Background Art

[0002] With the development of computer vision and Internet of Things technologies, intelligent devices that assist visually impaired people in traveling have gradually become a research hotspot. However, existing blind guidance systems still have significant deficiencies in dynamic environment perception, multi-modal decision-making efficiency, and user personalization adaptation, specifically manifested as follows:

[0003] 1. Perception limitations of traditional blind guidance devices

[0004] (1) Dominance of static perception: Current mainstream products (such as ultrasonic blind canes, RGB camera glasses) mainly rely on single-modal sensors and cannot effectively capture the movement trends of dynamic obstacles.

[0005] (2) Depth perception error: Based on the stereo matching algorithm of binocular vision (such as SGBM), the depth estimation error exceeds 40% in low-light or texture-lacking scenarios, leading to the risk of misjudgment by users for the height of steps and the depth of potholes.

[0006] 2. Real-time deficiencies in multi-sensor fusion

[0007] Data synchronization bottleneck: Existing fusion schemes (such as Kalman filtering) do not consider the temporal alignment problems of RGB cameras (30fps), ToF sensors (60fps), and IMUs (200Hz). Asynchronous data streams result in a dynamic obstacle trajectory prediction deviation rate as high as 22%.

[0008] 3. Disconnection between path planning and user behavior

[0009] Existing navigation algorithms (such as A*, Dijkstra) only consider the static topology of the environment and do not model user gait characteristics (such as stride fluctuations, turning inertia). Experiments show that the execution error of visually impaired users for mechanical voice instructions (such as "turn left 90 degrees") reaches ±35% because the system does not adapt to individual movement ability differences.

[0010] 4. Information overload and ambiguity in the feedback system

[0011] Most commercially available products use a single voice prompt (such as "obstacle ahead"), but they cannot convey the spatial orientation and risk level of obstacles. Tactile feedback devices (such as vibrating belts) have problems with chaotic coding rules. The same vibration pattern may correspond to "steps" or "moving vehicles", leading to user misoperations. Summary of the Invention

[0012] The object of the present invention is to provide an image processing method and system for smart glasses, which synchronize the data collected by heterogeneous sensors in the smart glasses, construct a global spatio-temporal coordinate system, identify obstacles and then perform path planning, solving the problems of lag in response to existing dynamic obstacles and inefficient utilization of multi-modal data.

[0013] To solve the above technical problems, the present invention is realized through the following technical solutions:

[0014] The present invention is an image processing method for smart glasses, including the following steps:

[0015] Step S1: Synchronize and preprocess the data collected by heterogeneous sensors in the smart glasses;

[0016] Step S2: Taking the IMU as the origin, convert the RGB-D camera and millimeter-wave radar to the head coordinate system of visually impaired persons through the extrinsic matrix, and construct a global spatio-temporal coordinate system;

[0017] Step S3: Perform dynamic light compensation according to the environmental brightness, switch the sensor mode, and use the IMU angular velocity data to perform reverse motion compensation on the RGB image;

[0018] Step S4: Input the DVS event stream into a bionic pulsed neural network, simulate the response of retinal ganglion cells, and the output is a spatio-temporal pulsed feature map;

[0019] Step S5: Determine the high-risk area through dynamic saliency analysis, generate a high-resolution detection ROI, perform full-pixel analysis on the focus area, and output a list of obstacles with 3D positions, categories and danger levels;

[0020] Step S6: Generate a personalized safe path based on the obstacle 3D position, the motion model of the visually impaired person, and the real-time position through a safe corridor generation algorithm;

[0021] Step S7: Based on the path generation result, perform multi-channel feedback and emergency response based on a 3D audio navigation engine.

[0022] As a preferred technical solution, in the step S1, the heterogeneous sensors installed in the smart glasses include a dynamic vision sensor, an RGB-D camera, a millimeter-wave radar and an inertial navigation unit; the specific process of spatio-temporal synchronization of the heterogeneous sensor data is as follows:

[0023] Step S11: Use a dual-frequency GNSS receiver (GPS L1 / L5 + Beidou B1 / B2) to generate a 1PPS (pulse per second) signal as the global time reference source;

[0024] Step S12: Each heterogeneous sensor (dynamic vision sensor, RGB-D camera, millimeter-wave radar, and inertial navigation unit) is built-in with an oven-controlled crystal oscillator (OCXO), synchronized with 1PPS through a PLL phase-locked loop, and the clock deviation is controlled within ±50 ns;

[0025] Step S13: Based on the 100 Hz angular velocity data of the IMU, construct the motion trajectory models of each sensor; the motion trajectory models are as follows:

[0026] w radar (t + Δt) = R IMU→radar × w IMU (t) + b time ;

[0027] In the formula, w radar (t + Δt) is the angular velocity measurement value at time t + Δt, Δt is the time offset, R is the rotation extrinsic parameter matrix, solved by least squares iteration, R IMU→radar is the rotation extrinsic parameter matrix from the IMU coordinate system to the radar coordinate system, w IMU (t) is the original angular velocity measurement value of the IMU at time t, b time represents the systematic deviation term of the event offset;

[0028] Step S14: Deploy a cold atom interferometer to measure the absolute acceleration. When an abnormal motion is detected, trigger multi-sensor data resampling, verify the time series consistency through quantum sensor data, and correct the crystal oscillator drift error;

[0029] Step S15: Establish a quaternion interpolation model and convert the data of each sensor to the global coordinates in real time.

[0030] As a preferred technical solution, in the step S14, after correcting the crystal oscillator offset error, drive the user's head to perform online calibration of preset actions, and calculate the rigid body transformation matrix between sensors; the calculation formula is as follows:

[0031] R,t = argmin R,t ∑||R × p sensor + t - p IMU || 2 ;

[0032] In the formula, R is the rotation matrix, t is the translation vector, p sensor is the point coordinate in the sensor coordinate system, p IMU is the corresponding reference point coordinate in the IMU coordinate system, p IMU is the reference point in the IMU coordinate system;

[0033] The specific processing flow is as follows:

[0034] Step S141: Calculate the centroids of two point sets. The specific formula is as follows:

[0035]

[0036] Perform centering processing to eliminate the influence of translation. The specific formula is as follows:

[0037]

[0038] Step S142: Calculate the covariance matrix and construct the covariance matrix of the centered point set:

[0039]

[0040] Step S143: Singular value decomposition. Perform SVD decomposition on the covariance matrix:

[0041] H = U∑V T ;

[0042] In the formula, U and V represent 3×3 orthogonal matrices, and ∑ represents a 3×3 diagonal matrix;

[0043] Step S144: Calculate the rotation matrix and determine the optimal rotation matrix according to the SVD result:

[0044]

[0045] In the formula, det(VU T ) represents ensuring the right-handed system of the rotation matrix and avoiding mirror reflection;

[0046] Step S145: Calculate the translation vector. Use the centroid difference to calculate the translation vector. The specific formula is as follows:

[0047] t = μ IMU -R×μ sensor .

[0048] As a preferred technical solution, in step S15, establish a quaternion interpolation model and convert the data of each sensor to the global coordinate in real time. The specific process is as follows:

[0049] Step S151: Collect the synchronous motion data of the IMU and each sensor through preset actions (nodding up and down, shaking the head left and right), and use the Kabsch algorithm to solve the rotation matrix R sensor→IMU ;

[0050] Step S152: Establish the extrinsic quaternion q ext , and convert the rotation matrix to the quaternion form of the traveling quaternion;

[0051] Step S153: Based on the IMU trigger signal, align the time stamps of each sensor, and use the least squares method to optimize the time offset Δt;

[0052] Step S154: Real-time attitude update settlement and calculate quaternion interpolation; the real-time attitude update settlement obtains the current IMU quaternion q IMU (t) through gyroscope integration, and the formula is:

[0053]

[0054] In the formula, w IMU is the angular velocity, is quaternion multiplication;

[0055] Quaternion interpolation: Perform Slerp interpolation on high-frequency IMU data (such as 200Hz) and low-frequency sensor data (such as camera 30Hz). The specific formula is:

[0056] In the formula, θ is the angle between q1 and q2, and α ∈ [0, 1] is the interpolation coefficient;

[0057] Step S155: Convert the sensor data p sensor in quaternion form to the global coordinate system:

[0058]

[0059] In the formula, R ext and t ext are preset external parameters;

[0060] Step S156: Introduce a velocity vector V for moving targets (such as vehicles, pedestrians) to correct position offset:

[0061] p corrected = p global + V × Δt;

[0062] Step S157: Adopt a sliding window to dynamically match timestamps to ensure spatio-temporal alignment of multi-sensor data.

[0063] As a preferred technical solution, in step S2, the specific process of constructing a global spatio-temporal coordinate system with the IMU as the origin is as follows:

[0064] Step S21: Customize a dynamic calibration board: Use an active calibration board with an LED array to generate a spatio-temporal encoded event stream; each LED flashes at a specific frequency (such as 10Hz) to form a spatio-temporal code that can be recognized by the event camera, and the calibration board has a built-in QR code for the RGB camera to recognize;

[0065] Step S22, Calibration Data Acquisition: The visually impaired person wears the smart glasses and performs a preset linear motion (translation + rotation), and records synchronously; The synchronously recorded data includes: the accelerometer and gyroscope data of the IMU, the LED event stream of the event camera, the QR code pose of the RGB camera, and the point cloud data of the radar.

[0066] Step S23, External Parameter Solution: Optimize the external parameters of the event camera through the spatio-temporal correlation between the LED flashing event and the IMU motion trajectory; The specific formula is as follows:

[0067]

[0068] Step S24, Radar-IMU Calibration: Use the millimeter-wave feature points reflected by the corner points of the calibration board to construct an ICP objective function, and the specific formula is as follows:

[0069]

[0070] Step S25, Environmental Feature Extraction: Extract ORB feature points from the RGB image, extract plane features from the radar point cloud, and establish cross-modal feature associations.

[0071] Step S26, Graph-based External Parameter Update: Construct a factor graph to jointly optimize the external parameters and poses; The optimization objective function is:

[0072]

[0073] In the formula, ξ is the external parameter of the sensor represented by the Lie group.

[0074] Step S27, Spatio-temporal Synchronization Algorithm: Use the FPGA to generate a global clock signal (such as 100 MHz), and synchronize all sensor samplings through the hardware trigger line, such as:

[0075] IMU: The main clock source, sending a synchronization pulse every 10 ms.

[0076] Event Camera: Only transmits event packets when receiving a trigger signal.

[0077] RGB Camera: The global shutter is aligned with the trigger signal, and the exposure time is dynamically adjusted.

[0078] Adaptive Timestamp Correction:

[0079] For sensors that cannot be hardware-synchronized (such as millimeter-wave radar), use the two-way PTP (Precision Time Protocol):

[0080] The main controller (IMU) exchanges timestamp messages with the radar to calculate the transmission delay.

[0081] Correct the radar data timestamp: tcorrected = t radar + d;

[0082] Step S28, Motion Compensation Interpolation: When there is a small time deviation in the sensor data, interpolation is performed using the IMU motion model. The specific formula is as follows:

[0083]

[0084] In the formula, exp(·) and log(·) represent Lie algebra mappings;

[0085] Step S29, Dynamic Transformation of the Head Coordinate System: Coordinate transformation is performed on each sensor. When an external parameter error is detected, online optimization is triggered;

[0086] First, input data: IMU real-time pose and sensor raw data: pixel coordinates (U, V) and timestamp of the event camera, image frame of the RGB camera, radar point cloud p radar ∈R 3 ; Coordinate transformation of the sensor includes:

[0087] Event camera data transformation: In the formula, π -1 is the camera inverse projection model; Radar data transformation: Global coordinate mapping:

[0088] Step S10, Verification and Accuracy Improvement: Establish a polynomial regression model of temperature - external parameter offset:

[0089] ΔT = a×(T - T0) + b×(T - T0) 2 , when the temperature sensor detects that ΔT > 5, compensation is automatically applied;

[0090] At the same time, multi-modal cross-verification is also required: project the edge features of event detection onto the radar point cloud and calculate the consistency score:

[0091]

[0092] If the score Score < 85%, trigger recalibration of the external parameters.

[0093] As a preferred technical solution, in step S3, the specific process of performing reverse motion compensation on the RGB image using the IMU angular velocity data is as follows:

[0094] Step S31: Use FPGA to generate a global hardware synchronization pulse (10MHz) to formulate a trigger mechanism; Implementation: achieve microsecond-level alignment of IMU data (1000Hz) and the RGB camera (30fps); perform interpolation compensation on the IMU data during the exposure period of each frame of the image;

[0095] Step S32: Interpolate the IMU data to the pose at the image center moment and correct the timestamp. The timestamp correction formula is as follows:

[0096]

[0097] Interpolate the IMU data to the pose at the image center moment;

[0098] Step S33: Calibrate the extrinsic parameters of each sensor; for the calibration of the extrinsic parameters, a designed 3D checkerboard calibration target is used, with an IMU sensor embedded on its surface. Calculate the relative pose through the correlation of IMU data between the target body and the IMU of the glasses during the movement of the target body:

[0099] argmin Tc←I Σ‖w I -R C←I ‖ 2 ;

[0100] where T C←I ∈SE(3) is the transformation matrix from the camera to the IMU;

[0101] Step S34: Generate a rotation prior by integrating the angular velocity and perform event stream motion compensation; for the preprocessing of the angular velocity, an adaptive Kalman filter is used to eliminate the IMU zero bias:

[0102] w = w raw -b w ,

[0103] where τ = 10s is the time-varying zero bias time constant;

[0104] Calculation of the rotation increment: Integrate the IMU angular velocity within the image frame interval Δt:

[0105]

[0106] Step S35: Model the non-linear motion trajectory and perform optical flow correction pixel by pixel; perform motion correction on the time stream and model the non-linear motion trajectory using B-spline pose interpolation: Convert the discrete IMU pose estimation to a continuous-time trajectory:

[0107]

[0108] In the formula, B i (t) is the cubic B-spline basis function, and ξ i ∈se(3) is the Lie algebra of the control node;

[0109] Step S36: Generate a motion-compensated image through inverse bilinear transformation. For each pixel (x, y) in the original image I, calculate the compensated coordinates:

[0110] x′ = x + u(x, y), y′ = y + v(x, y);

[0111] Generate a deblurred image I′ using differentiable bilinear interpolation;

[0112] Step S37: Through visual and inertial joint optimization and online calibration:

[0113] Construct a sliding window optimization model:

[0114]

[0115] In the formula, is the IMU pre-integration residual, r vis = I′(W(x; ξ)) - I t+1 (x) is the photometric residual.

[0116] As a preferred technical solution, in step S4, the specific steps of inputting the DVS event stream into the bionic spiking neural network to simulate the response of retinal ganglion cells are as follows:

[0117] Step S41: Use bio-inspired difference-of-Gaussians filtering to eliminate event noise, split the event stream into positive / negative polarity channels, perform log-polar coordinate transformation, and simulate the foveal resolution distribution of the retina;

[0118] Preprocess the DVS events, and the noise elimination formula is as follows:

[0119]

[0120] In the formula, σ can be dynamically adjusted, σ = 0.8 pixels for high event rates, and σ = 1.5 pixels for low event rates;

[0121] Simulate the foveal resolution distribution of the retina, and the formula is as follows:

[0122] θ = arctan(y / x);

[0123] Step S42: Use the leaky integrate-and-fire model to simulate the response of ganglion cells;

[0124] Step S43: Construct a 3D convolution kernel to extract spatio-temporal features, and the specific formula is:

[0125]

[0126] Use direction selectivity to enhance edge motion features;

[0127] Step S44: Generate a feature map using time-space pooling and optimize the feature map;

[0128]

[0129] In the formula, F(x,y) represents the weighted pulse density at the position (x,y) of the moment pulse feature map, T represents the total length of the event window, τ represents the time decay constant, t represents the discrete time variable, S(x,y,t) represents the membrane potential state of the neuron, V th represents the neuron firing threshold, and δ(·) represents the pulse trigger function;

[0130] When optimizing the feature map, perform normalization processing using the pulse density and use a pulse-driven CRF (conditional random field) to smooth the feature boundaries;

[0131] Step S45: Construct a retinal eccentricity mapping function, increase the resolution of the central region feature map by 4 times, and introduce a dopaminergic modulation signal to control the pulse firing mode:

[0132] V th ←V th ×(1 + γ × DA level ) ;

[0133] In the formula, V th represents the threshold voltage, γ represents the modulation coefficient, and DA level represents the modulation signal strength.

[0134] As a preferred technical solution, in the step S6, the specific process of generating a personalized safe path based on the 3D position of the obstacle, the motion model of the visually impaired person, and the real-time position through the safe corridor generation algorithm is as follows:

[0135] Step S61: Input the data of the current pose of the visually impaired person, the target point, and the list of obstacle positions;

[0136] Step S62: Generate random points in the free space, find the node in the tree that is closest to the random point, extend a certain step length in the direction of the random point to generate a new node, check whether the path from the new node to the neighboring node is better, and update the parent node to shorten the total path length. Stop when the new node reaches near the target point (such as the distance < threshold);

[0137] Step S63: Output the list of path points path, such as [pt1, pt2, pt3, pt4,..., ptn];

[0138] Step S64: Obtain the list of distances [d1, d2, d3, d4,..., dn] from the current point to all nearby obstacles. The closer the obstacle is, the greater its contribution to the risk;

[0139] Step S65: Generate a polygonal area with a width centered on the path point; when the risk is low (e.g., there are no obstacles around), the width approaches 0.8 + 0.7 / 0.001 ≈ 1.5 (the upper limit is restricted to 1.5 meters); when the risk is high (e.g., obstacles are dense), the width shrinks, and the minimum is not less than 0.8 + 0.7 / (extreme risk) ≈ 0.8 meters;

[0140] Step S66: Calculate the curvature of the corridor centerline;

[0141] Step S67: If the curvature of any point in the path exceeds the preset threshold, trigger a speed reduction, such as the maximum turning ability of a wheelchair or a guide dog;

[0142] Step S68: Output the corridor structure composed of multiple polygons, and each path point corresponds to a safety area with a dynamic width for navigation and obstacle avoidance.

[0143] As a preferred technical solution, in the step S7, the 3D audio navigation engine uses the head-related transfer function to generate azimuth cue sounds; when constructing the head-related transfer function, a spherical array microphone is used to collect impulse response data in an anechoic chamber at horizontal azimuths from 0° to 360° and elevation angles from -45° to +45°, covering a distance gradient from 1m to 10m. For each azimuth and distance combination, two signals are recorded: the left-ear HRIR and the right-ear HRIR, with a sampling rate of 48 kHz. The original HRIR is converted to the HRTF in the frequency domain and stored as a three-dimensional tensor;

[0144] Among them, horizontal azimuth control adopts ITD dynamic compensation and ILD gain control;

[0145] The ITD dynamic compensation: Calculate the theoretical ITD according to the target azimuth angle θ;

[0146]

[0147] In the formula, r = 8.75 is the average head radius of an adult, and c is the speed of sound;

[0148] The ILD gain control obtains the target gain difference through the azimuth-ILD mapping table.

[0149] The present invention is an image processing system for smart glasses, including a perception layer hardware module, a processing layer software module, and an interaction layer output module;

[0150] The perception layer hardware module includes a quantum inertial navigation unit, an environmental perception unit, and a physiological signal sensor; the quantum inertial navigation unit includes an interferometer and MEMS-IMU assistance; the interferometer is used for sub-millimeter-level attitude measurement; the MEMS-IMU assistance is used for high-frequency motion capture; the environmental perception unit includes a dynamic vision sensor, an RGB-D camera, and a millimeter-wave radar;

[0151] The processing layer software module includes an edge computing unit, an obstacle recognition unit, and a text-to-speech conversion unit; the edge computing unit includes a time data synchronization and path planning engine;

[0152] The output module of the interaction layer includes an audio navigation unit, a multi-channel tactile feedback unit, and a voice interaction unit.

[0153] The present invention has the following beneficial effects:

[0154] (1) By synchronizing the data collected by heterogeneous sensors in the smart glasses, the present invention constructs a global spatio-temporal coordinate system, recognizes obstacles and then conducts path planning, integrates high-frequency IMU angular velocity data with an adaptive optical flow correction network, and achieves an image reverse compensation accuracy of 0.1 pixel level in dynamic scenarios, ensuring navigation safety in complex scenarios.

[0155] (2) Based on IMU angular velocity data, the present invention realizes RGB image reverse motion compensation. Combining non-linear filtering, depth homography estimation, and event camera assistance, it deeply fuses IMU high-frequency angular velocity data with visual features, constructs a motion compensation framework based on Lie group manifold, and breaks through the limitations of traditional pure vision or pure inertial methods.

[0156] (3) By deeply fusing IMU high-frequency angular velocity data with visual features, the present invention constructs a motion compensation framework based on Lie group manifold, breaks through the limitations of traditional pure vision or pure inertial methods. Through B-spline continuous time modeling, it improves the motion blur MTF50, and the event camera assistance reduces the IMU integration error, enhancing the robustness.

[0157] (4) The present invention adopts a retinal ganglion cell simulation + attention focusing mechanism, inputs the DVS event stream into a bionic spiking neural network, simulates the response of retinal ganglion cells, determines high-risk areas through dynamic saliency analysis, and calculates the collision time of moving objects based on the radar velocity vector field, improving the efficiency and accuracy of obstacle recognition.

[0158] (5) The present invention adopts a time asynchronous processing architecture, enabling event-driven computing to skip the processing of empty areas, reducing energy consumption by 83% compared with traditional CNNs. At the same time, it adopts a pulse-frequency dual-channel fusion to retain precise time encoding (microsecond level) and frequency encoding information, realizing efficient bio-inspired processing of the DVS event stream and achieving real-time performance on mobile devices.

[0159] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. Description of the Drawings

[0160] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0161] Figure 1 It is a flowchart of an image processing method for an intelligent glasses of the present invention;

[0162] Figure 2 It is a schematic structural diagram of an image processing system for an intelligent glasses of the present invention. Detailed implementation manners

[0163] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0164] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0165] To make the purpose, technical solutions and advantages of the present application clearer, the following will further describe the present application in detail with reference to the Figure 1-2 accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0166] Embodiment 1

[0167] Please refer to Figure 1 As shown, the present invention is an image processing method for an intelligent glasses, including the following steps:

[0168] Step S1: Synchronize and preprocess the data collected by the heterogeneous sensors in the intelligent glasses;

[0169] Step S2: Taking the IMU as the origin, convert the RGB-D camera and millimeter-wave radar to the head coordinate system of the visually impaired person through the external parameter matrix, and construct a global spatio-temporal coordinate system;

[0170] Step S3: Perform dynamic light compensation according to the environmental brightness, switch the sensor mode, and use the IMU angular velocity data to perform reverse motion compensation on the RGB image;

[0171] Step S4: Input the DVS event stream into the bionic pulse neural network, simulate the response of retinal ganglion cells, and the output is a spatio-temporal pulse feature map;

[0172] Step S5: Determine the high-risk areas through dynamic saliency analysis, generate high-resolution detection ROIs, perform full-pixel analysis on the focus areas, and output a list of obstacles with 3D positions, categories, and danger levels;

[0173] Step S6: Generate a personalized safe path based on the 3D positions of the obstacles, the movement model of the visually impaired person, and the real-time position through the safe corridor generation algorithm;

[0174] Step S7: Based on the path generation result, perform multi-channel feedback and emergency response based on the 3D audio navigation engine.

[0175] In Step S1, the heterogeneous sensors installed in the smart glasses include a dynamic vision sensor, an RGB-D camera, a millimeter-wave radar, and an inertial navigation unit; the specific process of spatio-temporal synchronization of the heterogeneous sensor data is as follows:

[0176] Step S11: Use a dual-frequency GNSS receiver (GPS L1 / L5 + Beidou B1 / B2) to generate a 1PPS (pulse per second) signal as the global time reference source;

[0177] Step S12: Each heterogeneous sensor (dynamic vision sensor, RGB-D camera, millimeter-wave radar, and inertial navigation unit) is equipped with an oven-controlled crystal oscillator (OCXO), which is synchronized with the 1PPS through a PLL phase-locked loop to control the clock deviation within ±50 ns;

[0178] Step S13: Based on the 100Hz angular velocity data of the IMU, construct the motion trajectory models of the sensors; the motion trajectory models are as follows:

[0179] w radar (t + Δt) = R IMU→radar × w IMU (t) + b time ;

[0180] In the formula, w radar (t + Δt) is the angular velocity measurement value at time t + Δt, Δt is the time offset, R is the rotation extrinsic parameter matrix, which is solved iteratively by the least squares method, and R IMU→radar is the rotation extrinsic parameter matrix from the IMU coordinate system to the radar coordinate system, w IMU (t) is the original angular velocity measurement value of the IMU at time t, and b time represents the systematic deviation term of the event offset;

[0181] By iteratively solving R IMU→radar and Δt, the spatio-temporal unification of the IMU and radar data can be achieved, and it can be further optimized. The optimization objective function is:

[0182]

[0183] In dynamic target detection, the calibrated parameters are used to compensate for the influence of head movement on radar measurement;

[0184] Step S14: Deploy a cold atom interferometer to measure absolute acceleration. When abnormal movement is detected, trigger multi-sensor data resampling, verify the time series consistency through quantum sensor data, and correct the crystal oscillator drift error;

[0185] Step S15: Establish a quaternion interpolation model and convert the data of each sensor to the global coordinate in real time.

[0186] In step S14, after correcting the crystal oscillator offset error, drive the user's head to perform online calibration of preset actions, and calculate the rigid body transformation matrix between sensors; the calculation formula is as follows:

[0187] R,t=argmin R,t ∑||R×p sensor +t-p IMU || 2 ;

[0188] In the formula, R is the rotation matrix, t is the translation vector, p sensor is the point coordinate in the sensor coordinate system, p IMU is the corresponding reference point coordinate in the IMU coordinate system, p IMU is the reference point in the IMU coordinate system;

[0189] The specific processing flow is as follows:

[0190] Step S141: Calculate the centroids of two point sets. The specific formula is:

[0191]

[0192] Perform centering processing to eliminate the influence of translation. The specific formula is:

[0193]

[0194] Step S142: Calculate the covariance matrix and construct the covariance matrix of the centered point set:

[0195]

[0196] Step S143: Singular value decomposition, perform SVD decomposition on the covariance matrix:

[0197] H=U∑V T ;

[0198] In the formula, U and V represent 3*3 orthogonal matrices, and ∑ represents a 3*3 diagonal matrix;

[0199] Step S144: Calculate the rotation matrix and determine the optimal rotation matrix according to the SVD result:

[0200]

[0201] In the formula, det(VU T ) represents ensuring the right-handed system of the rotation matrix and avoiding mirror reflection;

[0202] Step S145: Calculate the translation vector. Use the centroid difference to calculate the translation vector. The specific formula is as follows:

[0203] t = μ IMU - R × μ sensor .

[0204] For example, calculate the centroids of two point sets: μ radar = (2.0, 0.2, 1.0), μ IMU = (1.9, 0.3, 1.17);

[0205] Perform centering processing to eliminate the influence of translation. Center the point set:

[0206]

[0207] Covariance matrix:

[0208]

[0209] After SVD decomposition, we get:

[0210]

[0211] Translation vector: t = (0.9, 0.1, 0.17) T .

[0212] In step S15, establish a quaternion interpolation model to convert the data of each sensor to the global coordinate in real time. The specific process is as follows:

[0213] Step S151: Collect the synchronous motion data of the IMU and each sensor through preset actions (nodding up and down, shaking the head left and right), and use the Kabsch algorithm to solve the rotation matrix R sensor→IMU ;

[0214] Step S152: Establish the extrinsic quaternion q ext , and convert the rotation matrix to the quaternion form of the driving quaternion;

[0215] Step S153: Based on the IMU trigger signal, align the time stamps of each sensor, and use the least squares method to optimize the time offset Δt;

[0216] Step S154: Real-time attitude update settlement and calculation of quaternion interpolation; real-time attitude update settlement obtains the current IMU quaternion q IMU (t) through gyroscope integration, and the formula is:

[0217]

[0218] In the formula, w IMU is the angular velocity, and

[0219] is quaternion multiplication;

[0220] Quaternion interpolation: Perform Slerp interpolation on high-frequency IMU data (such as 200Hz) and low-frequency sensor data (such as camera 30Hz). The specific formula is:

[0221] Step S155: Convert the sensor data p sensor in quaternion form to the global coordinate system:

[0222]

[0223] In the formula, R ext and t ext are preset external parameters;

[0224] Step S156: Introduce the velocity vector V for moving targets (such as vehicles, passers-by) to correct the position offset:

[0225] p corrected = p global + V × Δt;

[0226] Step S157: Use a sliding window to dynamically match timestamps to ensure spatio-temporal alignment of multi-sensor data.

[0227] In Step S2, the specific process of constructing the global spatio-temporal coordinate system with the IMU as the origin is as follows:

[0228] Step S21: Customize a dynamic calibration board: Use an active calibration board with an LED array to generate a spatio-temporal encoded event stream; each LED flashes at a specific frequency (such as 10Hz) to form a spatio-temporal code that can be recognized by the event camera, and the calibration board has a built-in QR code for the RGB camera to recognize;

[0229] Step S22, Calibration Data Acquisition: The visually impaired person wears the smart glasses and performs a preset linear motion (translation + rotation), and records synchronously; The synchronously recorded data includes: the accelerometer and gyroscope data of the IMU, the LED event stream of the event camera, the QR code pose of the RGB camera, and the point cloud data of the radar.

[0230] Step S23, External Parameter Solving: Optimize the external parameters of the event camera through the spatio-temporal correlation between the LED flashing event and the IMU motion trajectory; The specific formula is as follows:

[0231]

[0232] Step S24, Radar-IMU Calibration: Use the millimeter-wave feature points reflected by the corner points of the calibration board to construct the ICP objective function. The specific formula is as follows:

[0233]

[0234] Step S25, Environmental Feature Extraction: Extract ORB feature points from the RGB image, extract plane features from the radar point cloud, and establish cross-modal feature associations.

[0235] Step S26, Graph-Based External Parameter Update: Construct a factor graph to jointly optimize the external parameters and poses; The optimization objective function is:

[0236]

[0237] In the formula, ξ is the external parameter of the sensor represented by the Lie group.

[0238] Step S27, Spatio-Temporal Synchronization Algorithm: Use the FPGA to generate a global clock signal (such as 100MHz), and synchronize all sensor samplings through the hardware trigger line. For example:

[0239] IMU: The main clock source, sending a synchronization pulse every 10ms.

[0240] Event Camera: Only transmits event packets when receiving a trigger signal.

[0241] RGB Camera: The global shutter is aligned with the trigger signal, and the exposure time is dynamically adjusted.

[0242] Adaptive Timestamp Correction:

[0243] For sensors that cannot be hardware-synchronized (such as millimeter-wave radar), use the two-way PTP (Precision Time Protocol):

[0244] The main controller (IMU) exchanges timestamp messages with the radar to calculate the transmission delay.

[0245] Correct the radar data timestamp: tcorrected = t radar + d;

[0246] Step S28, Motion Compensation Interpolation: When there is a small time deviation in the sensor data, use the IMU motion model for interpolation. The specific formula is as follows:

[0247]

[0248] In the formula, exp(·) and log(·) represent Lie algebra mappings;

[0249] Step S29, Dynamic Transformation of the Head Coordinate System: Perform coordinate transformation on each sensor. When an external parameter error is detected, trigger online optimization;

[0250] First, input data: the real-time pose of the IMU and the original data of the sensors: the pixel coordinates (U, V) and timestamp of the event camera, the image frame of the RGB camera, and the radar point cloud p radar ∈ R 3 ; Coordinate transformation of the sensors includes:

[0251] Event camera data transformation: In the formula, π -1 is the camera inverse projection model; Radar data transformation: Global coordinate mapping:

[0252] Step S10, Verification and Accuracy Improvement: Establish a polynomial regression model of temperature - external parameter offset:

[0253] ΔT = a × (T - T0) + b × (T - T0) 2 , when the temperature sensor detects that ΔT > 5, automatically apply compensation;

[0254] At the same time, multi-modal cross-verification is also required: project the edge features of event detection onto the radar point cloud and calculate the consistency score:

[0255]

[0256] If the score Score < 85%, trigger re-calibration of the external parameters.

[0257] In step S3, the specific process of using the IMU angular velocity data for reverse motion compensation of the RGB image is as follows:

[0258] Step S31: Use the FPGA to generate a global hardware synchronization pulse (10MHz) to formulate a triggering mechanism; Achieve: microsecond-level alignment of the IMU data (1000Hz) and the RGB camera (30fps); Interpolate and compensate the IMU data during the exposure period of each frame of the image;

[0259] Step S32: Interpolate the IMU data to the pose at the image center moment, and correct the timestamps. The timestamp correction formula is as follows:

[0260]

[0261] Interpolate the IMU data to the pose at the image center moment;

[0262] Step S33: Calibrate the extrinsic parameters of each sensor; for the calibration of the extrinsic parameters, a designed 3D checkerboard calibration target is used, with an IMU sensor embedded on the surface. Calculate the relative pose through the correlation of the IMU and the glasses IMU data when the target moves:

[0263] argmin Tc←I ∑||w I -R C←I || 2 ;

[0264] where T C←I ∈ SE(3) is the transformation matrix from the camera to the IMU;

[0265] Step S34: Generate a rotation prior by integrating the angular velocity and perform event stream motion compensation; for the preprocessing of the angular velocity, an adaptive Kalman filter is used to eliminate the IMU zero bias:

[0266] w = w raw -b w ,

[0267] where τ = 10s is the time-varying zero bias time constant;

[0268] Calculation of the rotation increment: Integrate the IMU angular velocity within the image frame interval Δt:

[0269]

[0270] Step S35: Model the non-linear motion trajectory and perform optical flow correction pixel by pixel; perform motion correction on the time flow and use B-spline pose interpolation for modeling the non-linear motion trajectory: Convert the discrete IMU pose estimation to a continuous-time trajectory:

[0271]

[0272] In the formula, B i (t) is the cubic B-spline basis function, and ξ i ∈ se(3) is the Lie algebra of the control node;

[0273] Step S36: Generate a motion compensation image through inverse bilinear transformation. For each pixel (x, y) in the original image I, calculate the compensated coordinates:

[0274] x′ = x + u(x, y), y′ = y + v(x, y);

[0275] Generate the deblurred image I′ using differentiable bilinear interpolation;

[0276] Step S37: Through visual and inertial joint optimization and online calibration:

[0277] Construct a sliding window optimization model:

[0278]

[0279] Where, is the IMU pre-integration residual, r vis = I′(W(x; ξ)) - I t+1 (x) is the photometric residual.

[0280] In step S4, the specific steps of inputting the DVS event stream into the bionic spiking neural network to simulate the response of retinal ganglion cells are as follows:

[0281] Step S41: Use bio-inspired difference Gaussian filtering to eliminate event noise, split the event stream into positive / negative polarity channels, perform logarithmic polar coordinate transformation, and simulate the resolution distribution of the retinal fovea;

[0282] Preprocess the DVS events, and the noise elimination formula is as follows:

[0283]

[0284] Where, σ can be dynamically adjusted, σ = 0.8 pixels at high event rates, and σ = 1.5 pixels at low event rates;

[0285] Simulate the resolution distribution of the retinal fovea, and the formula is as follows:

[0286] θ = arctan(y / x);

[0287] Step S42: Use the leaky integrate-and-fire model to simulate the response of ganglion cells;

[0288] Step S43: Construct a 3D convolution kernel to extract spatio-temporal features, and the specific formula is:

[0289]

[0290] Use direction selectivity to enhance edge motion features;

[0291] Step S44: Use spatio-temporal pooling to generate a feature map and optimize the feature map;

[0292]

[0293] In the formula, F(x, y) represents the weighted pulse density at the position (x, y) of the moment pulse feature map, T represents the total length of the event window, τ represents the time decay constant, t represents the discrete time variable, S(x, y, t) represents the membrane potential state of the neuron, V th represents the neuron firing threshold, and δ(·) represents the pulse trigger function;

[0294] When optimizing the feature map, pulse density is used for normalization processing, and a pulse-driven CRF (conditional random field) is used to smooth the feature boundary;

[0295] Step S45: Construct a retinal eccentricity mapping function, increase the resolution of the central region feature map by 4 times, and introduce a dopaminergic modulation signal to control the pulse firing pattern:

[0296] V th ←V th ×(1 + γ × DA level );

[0297] In the formula, V th represents the threshold voltage, γ represents the modulation coefficient, and DA level represents the modulation signal strength.

[0298] In step S6, the specific process of generating a personalized safe path based on the 3D position of the obstacle, the movement model of the visually impaired person, and the real-time position through the safe corridor generation algorithm is as follows:

[0299] Step S61: Input the data of the current pose of the visually impaired person, the target point, and the list of obstacle positions;

[0300] Step S62: Generate random points in the free space, find the node in the tree that is closest to the random point, extend a certain step length in the direction of the random point to generate a new node, check whether the path from the new node to the neighboring node is better, and update the parent node to shorten the total path length. Stop when the new node reaches near the target point (such as the distance < threshold);

[0301] Step S63: Output the list of path points path, such as [pt1, pt2, pt3, pt4,..., ptn];

[0302] Step S64: Obtain the list of distances [d1, d2, d3, d4,..., dn] from the current point to all nearby obstacles. The closer the obstacle, the greater the contribution to the risk;

[0303] Step S65: Generate a polygonal area with a width centered on the path point; when the risk is low (e.g., there are no obstacles around), the width approaches 0.8 + 0.7 / 0.001 ≈ 1.5 (the upper limit is restricted to 1.5 meters); when the risk is high (e.g., obstacles are dense), the width shrinks, with a minimum of not less than 0.8 + 0.7 / (extreme risk) ≈ 0.8 meters;

[0304] Step S66: Calculate the curvature of the corridor centerline;

[0305] Step S67: If the curvature of any point on the path exceeds the preset threshold, trigger deceleration, such as the maximum turning ability of a wheelchair or a guide dog;

[0306] Step S68: Output a corridor structure composed of multiple polygons, with each path point corresponding to a safety area with a dynamic width for navigation and obstacle avoidance.

[0307] In Step S7, the 3D audio navigation engine uses the head-related transfer function to generate azimuth cue sounds; when constructing the head-related transfer function, impulse response data at horizontal azimuths from 0° to 360° and elevation angles from -45° to +45° are collected using a spherical array microphone in an anechoic chamber, covering a distance gradient from 1m to 10m. For each azimuth and distance combination, two signals are recorded: the left-ear HRIR and the right-ear HRIR, with a sampling rate of 48 kHz. The original HRIR is converted to the HRTF in the frequency domain and stored as a three-dimensional tensor;

[0308] Among them, horizontal azimuth control adopts ITD dynamic compensation and ILD gain control;

[0309] ITD dynamic compensation: Calculate the theoretical ITD according to the target azimuth angle θ;

[0310]

[0311] In the formula, r = 8.75 is the average head radius of an adult, and c is the speed of sound;

[0312] ILD gain control obtains the target gain difference through the azimuth-ILD mapping table.

[0313] Embodiment 2

[0314] Refer to Figure 2 As shown, the present invention is an image processing system for smart glasses, which can be used to execute and complete the method content of Embodiment 1 of the present invention, including: a perception layer hardware module, a processing layer software module, and an interaction layer output module;

[0315] The hardware module of the perception layer includes a quantum inertial navigation unit, an environmental perception unit, and a physiological signal sensor; the quantum inertial navigation unit includes an interferometer and MEMS-IMU assistance; the interferometer is used for sub-millimeter attitude measurement with an angular velocity accuracy of 0.001° / s; the MEMS-IMU assistance is used for high-frequency motion capture, 1000Hz high-frequency motion capture, to compensate for the delay of the atomic sensor; combined with UWB and visual SLAM, the indoor positioning error is <10cm; the environmental perception unit includes a dynamic vision sensor, an RGB-D camera, and a millimeter-wave radar.

[0316] The software module of the processing layer includes an edge computing unit, an obstacle recognition unit, and a text-to-speech conversion unit; the edge computing unit includes time data synchronization and a path planning engine; the time data synchronization realizes hardware-level timestamp alignment (FPGA realizes μs-level synchronization).

[0317] The output module of the interaction layer includes an audio navigation unit, a multi-channel tactile feedback unit (such as bone conduction headphones, smart bracelets), and a voice interaction unit.

[0318] It should be noted that in the above system embodiments, the various units included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0319] In addition, those of ordinary skill in the art can understand that all or part of the steps in implementing the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0320] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and utilize the present invention well. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method for smart glasses, characterized in that, It includes the following steps: Step S1: Synchronize and preprocess the data collected by the heterogeneous sensors in the smart glasses; Step S2: Taking the IMU as the origin, convert the RGB-D camera and millimeter-wave radar to the head coordinate system of the visually impaired person through the extrinsic matrix to construct a global spatio-temporal coordinate system; Step S3: Perform dynamic light compensation according to the environmental brightness, switch the sensor mode, and use the IMU angular velocity data to perform inverse motion compensation on the RGB image; Step S4: Input the DVS event stream into the bionic pulse neural network to simulate the response of retinal ganglion cells, and the output is a spatio-temporal pulse feature map; Step S5: Determine the high-risk area through dynamic saliency analysis, generate a high-resolution detection ROI, perform full-pixel analysis on the focus area, and output a list of obstacles with 3D positions, categories, and danger levels; Step S6: Generate a personalized safe path based on the 3D positions of the obstacles, the motion model of the visually impaired person, and the real-time position through the safe corridor generation algorithm; Step S7: Based on the path generation result, perform multi-channel feedback and emergency response based on the 3D audio navigation engine.

2. The image processing method of an intelligent glasses according to claim 1, characterized in that, In the above step S1, the heterogeneous sensors installed in the smart glasses include a dynamic vision sensor, an RGB-D camera, a millimeter-wave radar, and an inertial navigation unit; the specific process of spatio-temporal synchronization of the heterogeneous sensor data is as follows: Step S11: Use a dual-frequency GNSS receiver to generate a 1PPS signal as the global time reference source; Step S12: Each heterogeneous sensor is built-in with a temperature-controlled crystal oscillator, which is synchronized with the 1PPS through a PLL phase-locked loop to control the clock deviation within ±50ns; Step S13: Based on the 100Hz angular velocity data of the IMU, construct the motion trajectory model of each sensor; Step S14: Deploy a cold atom interferometer to measure the absolute acceleration. When an abnormal motion is detected, trigger multi-sensor data resampling, verify the time series consistency through quantum sensor data, and correct the crystal oscillator drift error; Step S15: Establish a quaternion interpolation model to convert the data of each sensor to the global coordinate in real time.

3. The image processing method of an intelligent glasses according to claim 2, characterized in that, In the above step S14, after correcting the crystal oscillator offset error, drive the user's head to perform an online calibration of a preset action, and calculate the rigid body transformation matrix between the sensors; the specific processing process is as follows: Step S141: Calculate the centroid of the two point sets and perform centering processing; Step S142: Calculate the covariance matrix; Step S143: Singular value decomposition; Step S144: Calculate the rotation matrix; Step S145: Calculate the translation vector.

4. The image processing method of an intelligent glasses according to claim 2, characterized in that In the above step S15, establish a quaternion interpolation model to convert the data of each sensor to the global coordinate in real time. The specific process is as follows: Step S151: Collect the synchronous motion data of the IMU and each sensor through a preset action, and use the Kabsch algorithm to solve the rotation matrix; Step S152: Establish an extrinsic quaternion and convert the rotation matrix into a quaternion form; Step S153: Based on the IMU trigger signal, align the time stamps of each sensor, and use the least squares method to optimize the time offset; Step S154: Perform real-time attitude update calculation and calculate quaternion interpolation; Step S155: Convert the sensor data in quaternion form into the global coordinate system; Step S156: Introduce a velocity vector for the moving target to correct the position offset; Step S157: Use a sliding window to dynamically match time stamps.

5. The image processing method of an intelligent glasses according to claim 1, wherein In the said Step S2, the specific process of constructing the global space-time coordinate system with the IMU as the origin is as follows: Step S21: Customize a dynamic calibration board: Use an active calibration board with an LED array to generate a space-time encoded event stream; Step S22: Collect calibration data: The visually impaired person wears smart glasses and moves linearly according to a preset motion, and record synchronously; Step S23: Solve the external parameters: Optimize the external parameters of the event camera through the spatio-temporal correlation between the LED flashing events and the IMU motion trajectory; Step S24: Radar-IMU calibration: Use the millimeter-wave feature points reflected by the corner points of the calibration board to construct an ICP objective function; Step S25: Extract environmental features: Extract ORB feature points from the RGB image, extract plane features from the radar point cloud, and establish cross-modal feature associations; Step S26: Update based on the graph external parameters: Construct a factor graph to jointly optimize the external parameters and poses; Step S27: Space-time synchronization algorithm: Use FPGA to generate a global clock signal and synchronize all sensor samplings through the hardware trigger line; Step S28: Motion compensation interpolation: When there is a small time deviation in the sensor data, use the IMU motion model for interpolation; Step S29: Dynamic conversion of the head coordinate system: Perform coordinate conversion on each sensor, and trigger online optimization when external parameter errors are detected; Step S10: Verification and accuracy improvement: Establish a polynomial regression model of temperature-external parameter offset.

6. The image processing method of an intelligent glasses according to claim 1, wherein In the said Step S3, the specific process of performing reverse motion compensation on the RGB image using the IMU angular velocity data is as follows: Step S31: Use FPGA to generate a global hardware synchronization pulse to formulate a trigger mechanism; Step S32: Interpolate the IMU data to the attitude at the center moment of the image and correct the time stamp; Step S33: Calibrate the external parameters of each sensor; Step S34: Generate a rotation prior by integrating the angular velocity and perform motion compensation on the event stream; Step S35: Model the non-linear motion trajectory and perform optical flow correction pixel by pixel; Step S36: Generate a motion compensation image through inverse bilinear transformation; Step S37: Perform joint optimization and online calibration through vision and inertia.

7. The image processing method of an intelligent glasses according to claim 1, wherein In the said Step S4, the specific steps of inputting the DVS event stream into the bionic pulse neural network to simulate the response of retinal ganglion cells are as follows: Step S41: Use a biologically inspired difference Gaussian filter to eliminate event noise, split the event stream into positive / negative polarity channels, perform logarithmic polar coordinate transformation, and simulate the resolution distribution of the retinal fovea; Step S42: Use a leaky integrate-and-fire model to simulate the response of ganglion cells; Step S43: Construct a 3D convolution kernel to extract spatio-temporal features; Step S44: Use time-space pooling to generate a feature map and optimize the feature map; Step S45: Construct a retinal eccentricity mapping function and introduce a dopaminergic modulation signal to control the pulse firing pattern.

8. The image processing method of an intelligent glasses according to claim 1, characterized in that, In step S6, the specific process of generating a personalized safe path based on the 3D position of obstacles, the motion model of visually impaired persons, and the real-time position through the safe corridor generation algorithm is as follows: Step S61: Input data of the current pose of the visually impaired person, the target point, and the list of obstacle positions; Step S62: Generate random points in the free space, find the node in the tree closest to the random point, extend a certain step length in the direction of the random point to generate a new node, check whether the path from the new node to the adjacent node is better, and update the parent node to shorten the total path length. Stop when the new node reaches near the target point; Step S63: Output the list of path points; Step S64: Obtain the list of distances from the current point to all nearby obstacles. The closer the obstacle is, the greater its contribution to the risk; Step S65: Generate a polygonal area with a width centered on the path point; Step S66: Calculate the curvature of the corridor center line; Step S67: If the curvature of any point in the path exceeds the preset threshold, trigger deceleration; Step S68: Output the corridor structure composed of multiple polygons. Each path point corresponds to a safety area with a dynamic width for navigation and obstacle avoidance.

9. The image processing method of an intelligent glasses according to claim 1, wherein In step S7, the 3D audio navigation engine uses the head-related transfer function to generate azimuth prompt sounds. When constructing the head-related transfer function, a spherical array microphone is used to collect impulse response data in the anechoic chamber at horizontal azimuths from 0° to 360° and elevation angles from -45° to +45°, covering a distance gradient from 1m to 10m. For each azimuth and distance combination, two signals are recorded: the left-ear HRIR and the right-ear HRIR. The sampling rate is 48 kHz. The original HRIR is converted to the HRTF in the frequency domain and stored as a three-dimensional tensor.

10. An image processing system for smart glasses, characterized in that, It includes a perception layer hardware module, a processing layer software module, and an interaction layer output module; The perception layer hardware module includes a quantum inertial navigation unit, an environmental perception unit, and a physiological signal sensor. The quantum inertial navigation unit includes an interferometer and MEMS-IMU assistance. The interferometer is used for sub-millimeter-level attitude measurement. The MEMS-IMU assistance is used for high-frequency motion capture. The environmental perception unit includes a dynamic vision sensor, an RGB-D camera, and a millimeter-wave radar; The processing layer software module includes an edge computing unit, an obstacle recognition unit, and a text-to-speech conversion unit. The edge computing unit includes time data synchronization and a path planning engine; The interaction layer output module includes an audio navigation unit, a multi-channel tactile feedback unit, and a voice interaction unit.

Images