A rotation-invariant panoramic vision encoding system and method based on spherical harmonic transform

Abstract

This invention discloses a rotation-invariant panoramic visual coding system and method based on spherical harmonic transform, belonging to the fields of computer vision and autonomous navigation technology. The method includes: acquiring panoramic images and mapping them to a spherical coordinate system; performing a fast spherical harmonic transform on the spherical images to calculate all spherical harmonic coefficients up to a preset maximum order L; extracting the complex amplitudes of the coefficients to form a rotation-invariant scene feature vector; and extracting the phase of specific coefficients as a direction coding signal. This invention utilizes the strict mathematical properties of spherical harmonics on a three-dimensional rotation group to achieve perfect decoupling of scene identity and direction information, ensuring strict feature invariance under arbitrary rotation, and compressing high-dimensional panoramic images to 81 dimensions, achieving extreme dimensionality reduction. The system's end-to-end latency on an embedded processor does not exceed 15 milliseconds. It is suitable for autonomous visual navigation in GPS-denied environments such as Mars rovers, deep-sea submersibles, drones, and mining robots.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, autonomous navigation and biomimetic intelligence, and specifically relates to a rotation-invariant panoramic visual coding system and method based on spherical harmonic transformation. Background Technology

[0002] One of the core tasks of autonomous navigation systems is to extract environmental features from sensor data to support functions such as localization, mapping, path planning, and obstacle avoidance. Visual sensors, due to their rich information content, low cost, and moderate power consumption, have become the mainstream perception modality for autonomous navigation. Current visual feature extraction techniques can be divided into three main categories: hand-designed features (such as SIFT, SURF, and ORB), deep learning features (such as NetVLAD, SuperPoint, and Vision Transformer), and methods specifically for panoramic images. However, these methods face many challenges in panoramic image scenarios, especially the overall image translation problem caused by changes in the orientation of the carrier, and the multi-axis visual distortion problem caused by uneven terrain.

[0003] Panoramic images, with their 360-degree omnidirectional field of view, offer unique advantages in autonomous navigation, but also present the following core technical challenges: Challenge 1: Rotation sensitivity. When the vehicle rotates around its vertical axis, the panoramic image undergoes an equal translation, causing fundamental changes to any descriptor based on pixels or local features. Traditional solutions, such as rotation search matching, have high computational complexity, making them unsuitable for real-time navigation. Challenge 2: High-dimensional data processing. Typical panoramic images have high resolution, with dimensions reaching hundreds of thousands or even millions of dimensions when using raw pixels as feature vectors, leading to the "curse of dimensionality." Challenge 3: Multi-axis visual distortion caused by uneven terrain. In rugged terrain, robots experience pitch and roll changes. This multi-axis rotation causes complex nonlinear distortions in panoramic images, which traditional two-dimensional image features cannot handle. Challenge 4: The contradiction between the compactness and discriminative power of scene encoding. Overly compressed features lose discriminative information, while retaining too much detail leads to storage and computational bottlenecks. Spherical harmonics are defined on a unit sphere S. 2 The orthogonal basis function set on the surface. Basri and Jacobs (2003) proved that the illumination image of the Lambertian surface can be represented by spherical harmonic coefficients, and Kazhdan et al. (2003) proposed a three-dimensional shape descriptor based on spherical harmonics. However, none of the above studies have systematically applied spherical harmonics to scene coding in panoramic visual navigation. In particular, they have failed to establish a complete engineering pipeline from panoramic image acquisition to spherical harmonic coding to rotation-invariant feature output, failed to achieve decoupled representation of scene identity and orientation information, and failed to optimize the design for navigation scenarios with uneven terrain.

[0004] This invention draws upon biomimetic inspiration from the study of the visual navigation mechanism of the desert ant (Cataglyphis fortis). Desert ants can perform precise homing navigation solely through vision in extremely barren desert environments. Their visual processing pathways include the optic lobe, mushroom body, and central complex, forming a complete perception-memory-decision loop. Key findings include: ants can perform visual navigation on uneven terrain, with head pitch angles varying by ±75 degrees, suggesting that their visual encoding has inherent robustness to multi-axis rotation; the panoramic field of view of the ant's retina has a spherical topology, naturally fitting the mathematical framework of spherical harmonic transform. These biological findings strongly suggest that applying spherical harmonic transform to panoramic visual encoding may be a key technological path to achieving efficient and robust visual navigation in insect-like organisms.

[0005] The shortcomings of existing technology: Existing technical solution one, Chinese invention patent application with publication number CN118500396A, discloses a purely visual navigation method based on Zernike moments. This method decomposes panoramic images into amplitude and phase using Zernike moments to achieve position navigation and orientation guidance. However, Zernike moments are orthogonal moments defined on a two-dimensional unit disk, and their "rotation invariance" only applies to rotations around the image's central axis in a two-dimensional plane. For three-dimensional rotations introduced by spherical panoramic imaging, especially the nonlinear distortions caused by vehicle pitch and roll, Zernike moments are not invariant, leading to a sharp degradation in scene recognition accuracy on uneven terrain.

[0006] The second existing technical solution, disclosed in Chinese invention patent application CN113238550A, is a visual homing method for mobile robots based on adaptive landmark correction. This method relies on SURF local feature extraction and matching, calculating the homing vector by assigning weights to landmarks of different accuracies. However, its core algorithm, SURF, has limited robustness to rotation and scale changes. When panoramic images undergo large-angle or three-dimensional rotation, the descriptors of feature points change, leading to matching failure or a severe decrease in accuracy, and rotation invariance cannot be mathematically guaranteed.

[0007] The third existing technical solution, Chinese invention patent application CN118967795A, discloses a visual-inertial navigation tightly coupled SLAM method based on a four-camera panoramic system. This method stitches multi-camera images into a panoramic sphere and extracts ORB features for pose estimation and optimization. However, the ORB features used are also designed for two-dimensional images and do not possess strict three-dimensional rotation invariance. Furthermore, the SLAM framework requires maintaining a large number of map points, resulting in high computational and storage overhead, making it difficult to implement lightweight scene memory and real-time homing on resource-constrained embedded navigation platforms.

[0008] In summary, the existing technologies have the following core shortcomings: a lack of a panoramic visual coding method that theoretically guarantees strict three-dimensional rotation invariance; a lack of an efficient dimensionality reduction coding pipeline from panoramic images to compact features; a lack of a coding scheme that decouples scene identity and orientation information; a lack of a coding method suitable for multi-axis visual distortion in uneven terrain; and a lack of an engineering solution that systematically combines spherical harmonic theory with biomimetic navigation architecture. Summary of the Invention

[0009] This invention aims to solve the following key technical problems in autonomous navigation systems: (1) the problem of unstable feature representation of panoramic visual scenes under carrier rotation conditions, and provides a coding method that strictly guarantees rotation invariance in mathematics; (2) the problem of excessively high feature dimension of panoramic images, and achieves an extreme dimensionality reduction ratio of more than 64:1; (3) the problem of coupling between scene identity and direction information, and achieves decoupling representation through the separation of amplitude / phase of spherical harmonics; (4) the problem of multi-axis visual distortion of uneven terrain, and achieves multi-axis robust coding by utilizing the orthogonality and completeness of spherical harmonics on SO(3); (5) the problem of mismatch between the coding scheme and the interface of the subsequent bionic navigation system, and designs a multi-level feature output interface.

[0010] To achieve the above objectives, the overall architecture of the technical solution of this invention consists of the following five functional modules, forming a complete processing pipeline from the original image to multi-level feature output: Module A: Panoramic image acquisition and preprocessing module; Module B: Leaf simulation and spherical mapping module; Module C: Spherical harmonic transform core calculation module; Module D: Rotation-invariant feature separation and direction encoding module; Module E: Multi-level feature output and interface adaptation module. The signal path is: Panoramic camera → Module A → Module B → Module C → Module D → Module E → Sub-level navigation subsystem.

[0011] According to one aspect of the present invention, a rotation-invariant panoramic visual coding method based on spherical harmonic transform is provided, characterized by comprising the following steps: Step (1): Image acquisition, preprocessing, and spherical mapping. Panoramic environmental images are acquired using an omnidirectional vision sensor. After grayscale conversion, intensity inversion, and normalization, the images are downsampled to standard resolution and mapped to a spherical coordinate system to obtain a spherical image. ,in The zenith angle is defined as the angle between the observation direction and the vertical upward axis, and its value range is determined by the vertical field of view of the acquisition system. The azimuth angle is defined as the horizontal rotation angle about the vertical axis, and its value range is... , Pi is the ratio of the circumference of a circle to its diameter.

[0012] Step (2): Spherical harmonic transformation calculation. For the spherical image... Perform spherical harmonic transformation and calculate up to the preset maximum order. All spherical harmonic coefficients The order ,symbol A non-negative integer, representing the macroscopic level of spatial frequency. Smaller values ​​correspond to more global structural features; frequency ,symbol These are integers representing the frequency components of the azimuth direction; the larger the absolute value, the finer the azimuth variation. The total number of coefficients is... One. The spherical harmonic basis function is defined as... ,in, These are normalization coefficients used to ensure that the square integral of the basis functions over the unit sphere is 1; The associated Legendre polynomial is the basis for the spherical harmonic angle component; It is the zenith angle The cosine function; For a complex exponential term, where It is the base of the natural logarithm. It is an imaginary unit, and this complex exponent term is responsible for capturing the frequency information of the azimuth direction.

[0013] Step (3): Rotation-invariant feature extraction. Extract the spherical harmonic coefficients. complex amplitude The amplitude is defined as the modulus of the complex number, that is, the non-negative square root of the sum of the squares of the real and imaginary parts of the complex number. These amplitudes are arranged in order to form... Rotation-invariant scene feature vectors in dimensionality The rotation invariance of this eigenvector is rigorously guaranteed by mathematics: when the spherical image rotates about any central axis of the sphere, the energy of the spherical harmonic coefficients remains constant only at the same rotational speed. Different ranks The coefficients are redistributed among the given coefficients, with the magnitude of each coefficient remaining unchanged.

[0014] Step (4): Direction-encoded signal extraction. Extract the coefficients of specific orders and degrees from the spherical harmonic coefficients, i.e. coefficient phase The phase is defined as the angle between the complex number and the positive real axis in the complex plane, and serves as the direction-encoded signal. This signal encodes the global pointing direction of the main gradient information in the scene.

[0015] Step (5): Feature Output. The rotation-invariant scene feature vector... With direction-encoded signal The data is output to the subsequent navigation subsystem to enable scene memory, recognition, and heading control.

[0016] As a preferred technical solution, the maximum order is set to This balances scene representation capability with computational complexity. The generated feature vector has a dimension of 81. The spatial angular resolution corresponding to this order is approximately... The angle is similar to the interocular distance of an insect's compound eye, which can capture the main spatial structure of the scene while avoiding high-frequency noise interference.

[0017] Furthermore, the specific parameters for panoramic image acquisition in step (1) are as follows: the horizontal range of the field of view is... Arc (360-degree panoramic), extending vertically to Radius (approximately 28.6 degrees), acquisition resolution is Pixels / radians. These parameters together determine the resolution of the rectangular image before mapping, which is approximately... The angular resolution of pixels. To meet the requirements of rapid transformation and standardize the input, the image is further processed, downsampled to... Perform spherical harmonic transformation after the standard resolution of the pixels.

[0018] Furthermore, the spherical harmonic transformation described in step (2) is implemented using a fast algorithm, which utilizes the spherical harmonic basis functions with respect to variables. and The separability property allows the area of ​​a two-dimensional sphere to be decomposed into two independent one-dimensional transformations. The specific process is as follows: First, fix each zenith angle... Sampling points, along the azimuth angle The direction performs a Fast Fourier Transform (FFT) on each row of spherical image data to obtain the orders. Corresponding Fourier coefficients Then, along the zenith angle Direction, for each pair of orders and number of times Using the product of pre-computed and cached associated Legendre polynomials and their normalized coefficients ,right Perform a weighted summation, i.e., a weighted Legendre transformation. The total computational complexity decreases from that of the naive method. Reduced to , where the symbol Represents the number of sampling points in the azimuth direction of the spherical image, i.e., the image width in pixels; symbol Represents the number of sampling points in the zenith angle direction of the spherical image, i.e., the number of pixels in the image height; symbol It represents the asymptotic upper bound of the algorithm's time complexity; symbol Indicates base 2 The logarithm of .

[0019] Furthermore, the direction-encoded signal extracted in step (4) It can be used to implement a computationally constant complexity. A method for estimating heading deviation. This method includes: during the position memory phase, recording the direction-encoded signal of the current position. During the navigation execution phase, the current direction encoding signal is calculated in real time. By calculating the difference between the two, that is... This allows us to directly obtain the heading deviation of the carrier. This deviation value... Under ideal conditions, it is exactly equal to the actual rotation angle of the carrier around the vertical axis, and its calculation only requires one subtraction and one arctangent (atan2) function operation.

[0020] As a further optimization, the method of the present invention also provides a multi-level power spectral feature expansion step to enhance the robustness of the system to illumination changes and different levels of scene description: (1) Calculate the rotationally invariant power spectral descriptor: for each order (in ), calculate the sum of the power of all coefficients of this order. , where the symbol This represents a summation operation. Arrange these power values ​​in order to form a... dimensional power spectrum descriptor This descriptor serves as a more compact scene fingerprint, retaining only the energy distribution of the scene at different spatial frequencies while discarding all azimuth details.

[0021] (2) Calculate the normalized amplitude characteristics to eliminate the influence of global illumination variations: First, extract the zero-order power. ,Right now The total power of each order, which physically represents the global average brightness of the scene. Then, the amplitude of all orders and degrees. Divide by Normalized amplitude characteristics are obtained. , where the symbol Represents zero-order power The square root of the value. This operation makes the feature insensitive to scaling of the overall illumination intensity.

[0022] According to another aspect of the present invention, a rotation-invariant panoramic visual coding system based on spherical harmonic transform is provided, the system being used to implement the above method, characterized in that it comprises: Module A: Panoramic Image Acquisition and Preprocessing Module. This module contains an omnidirectional camera or fisheye lens for acquiring 360-degree panoramic visual input. Its internal processor performs a series of preprocessing operations, including: converting the acquired color image to a grayscale image; performing intensity inversion on the grayscale values ​​by subtracting the current value from the maximum value to simulate the contrast inversion characteristics of insect photoreceptors; normalizing the inverted image values ​​to the [0,1] interval; and finally downsampling the image to a preset fixed resolution through interpolation or pooling operations.

[0023] Module B: Leaf Simulation and Spherical Mapping Module. This module simulates the visual preprocessing function of an insect's leaf. It receives the rectangular panoramic image output from Module A and calculates the zenith angle corresponding to each pixel based on a predefined vertical field of view and resolution. and azimuth This process resamples and maps the rectangular image to a spherical coordinate system, generating a normalized spherical image that can be directly used for spherical harmonic transformation. .

[0024] Module C: Spherical Harmonic Transform Core Calculation Module. This module is the computational core of the system. It internally implements a fast algorithm combining FFT and Legendre recursion. It first performs FFT in parallel along the azimuth direction of the image, then calls an internal dedicated Legendre polynomial pre-computation cache unit, which stores all possible... Combination The module uses these cached values ​​for weighted summation, ultimately outputting all preset maximum orders quickly. Below Complex spherical harmonic coefficients .

[0025] Module D: Rotation-Invariant Feature Separation and Direction Encoding Module. This module receives the complex coefficients output from Module C. It contains an amplitude calculation unit for each coefficient. Calculate the modulus and arrange all modulus values ​​into a rotation-invariant scene feature vector. It also includes a phase extraction unit specifically for extracting specified coefficients. (Right now phase of ) and use it as a direction-encoded signal. Output.

[0026] Module E: Multi-level Feature Output and Interface Adaptation Module. This module is responsible for formatting and packaging the internal feature data generated by Module D according to the interface protocol of the subsequent navigation subsystem. It will output rotation-invariant scene feature vectors. and direction-encoded signal Send them to the corresponding target modules respectively, for example, send The data is sent to the mushroom-shaped memory network for scene learning and recognition. Send it to the heading controller for direction adjustment.

[0027] Furthermore, the Legendre polynomial pre-computation cache unit in module C has the following storage structure: for all orders from up to the maximum order For all degrees under each order, from to And for each discrete zenith angle sampling point in the height direction of the spherical image. (common (Number of sampling points), store the corresponding pre-calculated values. The total storage required for this cache is floating-point numbers, where the sign is _ ... The number of sampling points in the zenith angle direction, i.e., the height pixels of the spherical image; symbol It represents the summation operation; floating-point numbers refer to the data type used in computers to approximate real numbers.

[0028] Furthermore, to ensure real-time performance and low power consumption, the core computation of the entire system is deployed on an embedded processor, specifically an ARM Cortex-A series or RISC-V architecture processor. The software algorithm is optimized to fully utilize the processor's NEON or equivalent SIMD vectorized instruction set, achieving 4-way or 8-way data parallel processing in critical loops, thereby improving throughput. With this optimization, the system's complete encoding pipeline, from image acquisition to feature output, has an end-to-end latency of no more than 15 milliseconds.

[0029] Furthermore, to verify the system's applicability in special environments, the system also includes at least one of the following features: (a) The system is specifically designed for uneven terrain environments. Its anti-distortion capability is theoretically based on the fact that when the carrier undergoes pitch and roll motions, it is equivalent to performing a three-dimensional rotation of the visual image on its spherical surface. A group action on, in which It is the group of all rotations around the origin in three-dimensional space. Since spherical harmonics are... The basis of the irreducible representation space of group action is such that any such rotation only results in the recombination of energies within coefficients of the same order, while the total power of each order and the overall manifold structure of the characteristics remain stable. By utilizing this orthogonal decomposition property, the system can effectively handle multi-axis visual distortion.

[0030] (b) The system's output interface is designed to be directly adaptable to a brain-inspired intelligent navigation architecture. Specifically, its output is an 81-dimensional rotation-invariant scene feature vector. It is configured to be directly used as input and injected into a subsequent visual projection neuron layer (vPN layer) of a biomimetic neural network that mimics the shape of an insect or mushroom. Each dimension of the feature vector corresponds precisely to the firing rate of a vPN neuron, thus achieving a seamless connection from visual encoding to a bio-inspired learning and memory circuit.

[0031] (c) The system, as a general-purpose panoramic visual coding module, has been designed and verified to be applicable to one or a combination of the following key scenarios: autonomous cruise navigation of Mars / lunar rovers in the absence of GPS and in rugged terrain; visual positioning and mothership return of deep-sea autonomous underwater vehicles (AUVs) in visually degraded environments; terrain matching navigation of unmanned aerial vehicles (UAVs) at low altitudes using downward panoramic images; visual scene coding and path memory of underground mine / tunnel inspection robots in low-light and dusty environments; autonomous navigation of biomimetic legged robots in unstructured environments in the wild; and self-localization and environmental perception of disaster relief robots in GPS-denied environments.

[0032] Compared with the prior art, the present invention has the following outstanding advantages: Effect 1: Rotation Robustness: Scene features based on spherical harmonic amplitude remain strictly invariant under arbitrary horizontal rotation of the carrier. This property has been rigorously proven mathematically (see Section 3.4.2 in "Detailed Implementation"), fundamentally eliminating the interference of orientation changes on scene recognition. In 360-degree full-angle testing, the feature matching accuracy remains consistently 100%, far superior to the traditional SIFT method (accuracy drops below 60% after rotation exceeding 30 degrees) and CNN feature methods relying on data augmentation (typical accuracy 85%-95%).

[0033] Effect 2: Extreme Dimensionality Reduction: This invention compresses the original 72×72=5184-dimensional panoramic pixels to 81-dimensional spherical harmonic coefficients, achieving a dimensionality reduction ratio of 64:1. In comparison, the typical dimensionality reduction ratio of the PCA method is 10:1-20:1, and that of the autoencoder is 5:1-15:1. This extreme dimensionality reduction reduces the number of parameters in the subsequent learning network from tens of millions to 250,000, and the training time from hours to seconds, greatly saving storage and computing resources, with a power consumption of only about 0.5W.

[0034] Effect 3: Direction Information Preservation: By independently encoding direction information through phase components, scene identity and orientation are perfectly decoupled. The same feature set can simultaneously serve two navigation tasks: visual homing (using amplitude) and path following (using phase), eliminating the need to design two separate feature systems and simplifying the navigation architecture.

[0035] Effect 4: Terrain Adaptability: Spherical Harmonics in Three-Dimensional Rotation Groups Its orthogonal completeness makes it naturally suitable for handling multi-axis visual distortion caused by uneven terrain. Under the condition of ±40-degree pitch angle variation, the scene recognition accuracy remains above 90%, breaking through the limitation of existing two-dimensional orthogonal moments (such as Zernike moments) or local feature methods that cannot handle three-dimensional rotational distortion.

[0036] Effect 5: Computational efficiency: The complete spherical harmonic coding pipeline (acquisition → preprocessing → SHT → feature extraction) on the ARM Cortex-A72 processor, through preset Legendre value caching and SIMD vectorization acceleration, has an end-to-end latency of only 15 milliseconds, meeting the real-time navigation requirements of >60Hz and is suitable for battery-powered mobile platforms.

[0037] Effect 6: Biological interpretability: The coding scheme strictly corresponds to the information processing function of the visual lobe in the visual pathway of desert ants. The 81-dimensional feature vector is directly input into the visual projection neuron layer of the mushroom body model. Each dimension corresponds to the firing rate of a projection neuron, which has a clear biological correspondence and provides a theoretical model for the subsequent implementation of neuromorphic navigation chips. Attached Figure Description

[0038] Figure 1 The diagram shows the overall architecture of the spherical harmonic panoramic visual coding system, illustrating the complete signal path from the panoramic camera to the multi-level feature output. First, the obtained color panoramic image is converted to grayscale, then Otsu threshold segmentation is used to extract the skyline binary image, and finally the amplitude spectrum is obtained by spherical harmonic transformation and input into the PN layer. Figure 2 This is a visualization of the spherical harmonic basis functions, where m is the order of the spherical harmonic function. It shows the three-dimensional distribution of spherical harmonic functions of orders l=0 to l=3 on a unit sphere, intuitively illustrating the spatial frequency characteristics corresponding to different orders. Figure 3 This is a schematic diagram of spherical mapping of a panoramic image, illustrating the mapping process from a rectangular image to a spherical coordinate system; Figure 4 To verify rotation invariance, panoramic images of the same scene at four orientations (0°, 90°, 180°, and 270°) and their corresponding 81-dimensional amplitude feature vectors are shown, proving that the features are strictly invariant. Figure 5 A schematic diagram of the uneven terrain construction, ant robot and its panoramic image in the Webots simulation environment; Figure 6 An experimental scenario diagram for navigation on uneven terrain shows the trajectories of an ant robot returning to its nest from four different locations; Figure 7 This diagram illustrates the structure and gate set performance of the mushroom-shaped navigation model, showcasing the complete signal flow of the visual pathway OL→MB→CX. It consists of the SH layer, vPNs layer, KCs layer, output layer (MBON), and motion steering layers (CH and VH) sequentially. In the SH layer, the agent scans and acquires panoramic images of the surrounding area, extracts the amplitude spectrum by performing a spherical harmonic transformation on the binary skyline image, and utilizes its strict rotation invariance and pitch approximate invariance to obtain attitude-robust visual features. These features are then input into the vPNs layer, passed through sparse random connections to the KCs layer, and trained using learning rules. The view of the nest environment is stored in synaptic weights, and visual novelty is represented by the firing rate of MBON. In the motion steering layer, the CH and VH modules, based on loop attractor encoding, dynamically adjust the desired heading according to changes in novelty—maintaining the original heading when novelty decreases and triggering steering when ascending. Combined with frame theory, the noise robustness of the heading estimation is enhanced, ultimately generating motor commands to control the agent to achieve visual homing. Detailed Implementation

[0039] The technical solution of the present invention will now be described in detail and completely with reference to the accompanying drawings and specific embodiments. To support the requirement of sufficient disclosure in the specification, all mathematical symbols are defined upon their first appearance.

[0040] Hardware system configuration: The preferred hardware system of the present invention is composed of the following components working together: Omnidirectional camera: Adopting catadioptric omnidirectional imaging or fisheye lens solution, with a minimum requirement of 360° horizontal field of view (FOV) and no less than 115° vertical field of view to ensure complete acquisition of peripheral visual information.

[0041] Embedded processor: As the main processor of the algorithm, it adopts an ARM Cortex-A72 quad-core processor with a main frequency of 1.5GHz, which is responsible for the scheduling and calculation of all core steps.

[0042] DSP Coprocessor: A DSP unit compatible with the CMSIS-DSP software library, specifically designed to accelerate Fast Fourier Transform.

[0043] Memory: At least 512MB of DDR4 memory for image buffering, Legendre polynomial cache table and intermediate variable storage.

[0044] Electronic compass: As a direction reference, the HMC5883L geomagnetic sensor or polarized compass can be selected, with a static heading accuracy of ±1°. It is used to initialize or correct heading drift that may occur during long-term operation, but this system does not rely on it as the primary heading source.

[0045] Communication interface: UART, SPI or I2C serial bus interface with a speed of not less than 1Mbps, used to send the encoded feature vector and direction signal to the host computer or motion control unit.

[0046] 1. Software algorithm implementation process: The software algorithm of this system follows the detailed process below, where the symbol := represents an assignment operation, and the symbol π represents the mathematical constant pi: Algorithm: Spherical Harmonic Panoramic Visual Encoding; Input: Panoramic image I_raw (RGB color image); Output: V_scene (81-dimensional amplitude vector), θ_dir (direction signal in radians); / / --- Step 1: Preprocessing --- I_gray := GrayScale(I_raw) / / Converts a color image to a grayscale image, retaining one intensity value for each pixel; I_inv := 255 - I_gray / / Invert intensity, subtract each pixel value from 255; I_norm := Normalize(I_inv, 0, 1) / / Normalizes pixel values ​​from [0,255] to the [0,1] range; I_72 := DownSample(I_norm, 72, 72) / / Downsample to 72x72 standard resolution; / / --- Step 2: Spherical Mapping --- Define the vertical field of view angle Δθ := 0.5 radians; Define the image width W := 72, and the image height H := 72; Define the minimum zenith angle θ_min := (π / 2) - (Δθ / 2); For column index j from 0 to H-1 (72 points in total): Zenith angle θ[j] := (j / H) × Δθ + θ_min; For row index i from 0 to W-1 (a total of 72 points): Azimuth φ[i] := (i / W) × 2π; The spherical image f(θ[j], φ[i]) := I_72[j][i]; / / --- Step 3: Fast Spherical Harmonic Transform --- Set the maximum order L := 8; / / This step uses the fast algorithm described in the specific technical solution; / / Step 3a: Perform FFT on each row along the φ direction (complexity O(W·H·logW)); For j from 0 to H-1: F_m[j][:]:= FFT( f(θ[j], :) ) / / Gets a complex sequence of length W, corresponding to different values ​​of m; / / Step 3b: Preload Legendre polynomials (this step is completed once during system initialization and stored in the cache); / / According to the formula, the cache size is Σ_{l=0}^{L} (2l+1) × H floating-point numbers; For l from 0 to L: For m from -l to l: For j from 0 to H-1: Calculate the normalization coefficient N_l^m; N_l^m = 4π2l+1(l+m)!(l−m)! Calculate cos_theta := cos(θ[j]); Calculate the associated Legendre polynomial P_l^m(cos_theta) / / using the three-term recursive formula; The Legendre polynomial is calculated using the following recurrence relation: Initial value , ; Store to cache: P_cache[l][m][j] := N_l^m × P_l^m(cos_theta); / / Step 3c: Weighted Legendre Transform (complexity O((L+1)^2 × H)); For l from 0 to L: For m from -l to l: Complex accumulator sum := 0; For j from 0 to H-1: Extract the component F_m[j][m] corresponding to the mth iteration from the FFT result; Retrieve the weight P_cache[l][m][j] from the cache; The integral weight w := sin(θ[j]) × (Δθ / H) / / Δθ / H is the sampling step size of θ (this weight comes from the infinitesimal area of ​​the sphere dΩ = sinθ dθ dφ. After discretization, dθ is approximated by Δθ / H, and the integral in the dφ direction has been completed by FFT, so only the weight in the θ direction remains here). sum := sum + F_m[j][m]× P_cache[l][m][j] × w; a_l^m := sum / / Get the spherical harmonic coefficients; / / After the above steps, the algorithm complexity is O(L^2 × H + W × H × log_2 W); / / --- Step 4: Feature Extraction --- / / Map (l,m) to a one-dimensional index k; Coefficient index k := 0; For l from 0 to L: For m from -l to l: / / Construct according to the V_scene defined above; V_scene[k] := sqrt( real(a_l^m)^2 + imag(a_l^m)^2 ) / / Calculate the complex magnitude |a_l^m|; k := k + 1; / / Extract the phase of the coefficients l=1 and m=1 as the direction signal, and execute according to the definition above; θ_dir := atan2( imag(a_1^1), real(a_1^1) ); / / --- Step 5: Feature Output --- Send V_scene to the mushroom-shaped vPN_Layer module; Send θ_dir to the compass control module; Returns V_scene and θ_dir; 2. Rigorous mathematical proof of rotation invariance: This section provides the core mathematical proof regarding rotation invariance. Let... Let f be a rotation operator about the z-axis (vertical axis) with a rotation angle α, acting on the spherical function f: The harmonic coefficients of the rotated sphere The calculation is as follows: ; in Represents the complex conjugate of the basis functions. Perform variable substitution. By utilizing the invariance of integral translation and the properties of basis functions , can be obtained .therefore: ; ; This derivation rigorously proves the two core mathematical properties upon which this invention relies, forming the theoretical foundation that distinguishes this invention from all approximate or empirical methods.

[0047] 3. Information theory analysis of dimensionality reduction efficiency: To illustrate the rationale for dimensionality reduction, according to the Parseval equation for spherical harmonics, the total energy of the original signal is completely preserved in the transform domain. (Truncation to...) After the step, the proportion of energy retained is For natural outdoor scenes (such as desert landscapes), low-frequency components ( Typically, it contains 85%-95% of the total energy, indicating that the 81-dimensional representation retains most of the information about the scene.

[0048] 4. System workflow and module collaboration: As attached Figure 1 As shown, the aforementioned hardware configuration and software algorithm together construct an end-to-end coding pipeline from the original panoramic image to multi-level navigation features. The signal flow and module collaboration relationships of this pipeline are described in detail below: 1. Image Acquisition and Preprocessing (Module A): The omnidirectional camera uses a horizontal field of view. Radiance (360° panoramic) and vertical field of view Continuous acquisition of panoramic environmental images in radians (approximately 28.6°). Each frame is a color image. The data is immediately fed into the preprocessing pipeline of module A, where grayscale conversion is performed sequentially to obtain... Intensity reversal operation Normalization to Interval and downsampling to Standard resolution, generating a brightness-normalized rectangular panoramic image. .

[0049] 2. Spherical Mapping (Module B): Module B receives the rectangular panoramic image output by Module A. According to the preset vertical field of view radians and minimum zenith angle The following mapping applies to each pixel in the image. Calculate the corresponding spherical coordinates: This reorganizes the rectangular image data into a spherical image defined on a spherical strip. ,Right now The spherical image is cached and passed to module C.

[0050] 3. Spherical Harmonic Transform Calculation (Module C): Module C is the core calculation unit of the system. It first calculates along the azimuth angle... Direction relative to spherical image Perform an FFT on each line to obtain each Fourier coefficients at sampling points Subsequently, module C reads all pre-stored data from its internal Legendre polynomial pre-computation cache unit. The value, the storage structure of this cache is: for all orders to ,frequency to And all Each zenith angle sampling point stores its corresponding pre-calculated value, with a total storage capacity of [number]. A number of floating-point numbers. The Legendre transformation is performed using the following weighted summation: ; Final output Complex spherical harmonic coefficients (when 8), the spherical harmonic basis functions are ,in The normalization coefficient is... This is for Legendre polynomials. All core operations of this module are executed by an embedded processor (ARM Cortex-A series or RISC-V), and fully utilize NEON / SIMD vectorized instructions for acceleration, with a total computational complexity of O(n log n). This ensures that the end-to-end latency of the complete encoding pipeline from image input to coefficient output does not exceed 15 milliseconds.

[0051] 4. Feature Separation and Directional Encoding (Module D): Module D receives all complex coefficients output by Module C. Its internal amplitude calculation unit calculates the modulus of each coefficient. ,generate Rotation-invariant scene feature vectors in dimensionality: ; Meanwhile, its phase extraction unit is specifically designed to extract coefficients. (Right now The phase of ) is used as the direction-encoded signal: ; Amplitude vector With direction signal Information decoupling is completed in module D, and a timestamp is added. Specifically, when the spherical image rotates around the center of the sphere by an angle... At that time, it can be rigorously proven mathematically that the amplitude remains unchanged. The phase produces a linear shift. .

[0052] 5. Multi-level Feature Output and Interface Adaptation (Module E): Module E is responsible for formatting the feature data generated by Module D and distributing it to the subsequent navigation subsystem. Specifically, this includes an 81-dimensional amplitude vector. Encapsulated as standardized feature vector messages, these are sent via a high-speed serial interface to the visual projection neuron layer (vPN layer) of the mushroom-shaped biomimetic memory network, serving as the input firing rate for its 81 neurons. (Directional encoded signal) It is encapsulated as an angle control command and sent in real time to the central complex heading controller to calculate the heading deviation. And steering correction is performed. Under ideal conditions, this heading deviation is precisely equal to the actual rotation angle of the carrier. Simultaneously, the system can optionally use a rotationally invariant power spectral descriptor. ,in and normalized amplitude characteristics Intermediate features can also be output to the topology map module or anomaly detection module through the extended interface of module E to eliminate the impact of global illumination changes.

[0053] Through the collaborative work of the above modules, this system achieves real-time and efficient transformation from raw panoramic visual signals to compact, rotation-invariant scene features with decoupled identity and orientation, providing a standard visual front-end encoding solution for various autonomous navigation platforms.

[0054] Example 1: Bionic ant robot navigates back to its nest on uneven terrain.

[0055] This embodiment is closely integrated with the appendix. Figure 6The following explanation is provided. A 12Bot ant robot model was constructed in the Webots simulation platform to simulate a desert environment. The desert terrain was designed to be uneven, containing continuous slopes of ±15° and randomly distributed bumpy obstacles. The robot's panoramic camera was mounted on its top, 1cm above the ground, strictly simulating the perspective of a desert ant (Cataglyphis fortis).

[0056] 1. Learning Phase: The operator remotely controls the robot to move along a winding "leaving the nest" foraging path. During the movement, the system collects panoramic images of the current scene at regular intervals (e.g., every 20cm) or by manual triggering. A total of 21 panoramic images of key locations are collected along this path. Each image is encoded into an 81-dimensional amplitude feature vector by the system of this invention. The features are then sequentially stored in the mushroom-shaped biomimetic memory network to complete a single "visual path" learning process. The entire learning process, from feature extraction to memory storage, takes no more than one second.

[0057] 2. Homing Phase: After learning is complete, the robot is randomly placed at a starting point at different distances and orientations from the homing nest. Once the homing procedure is initiated, the robot rotates once in place. During this process, the system acquires and encodes panoramic images in real time, calculating the current scene's... and .

[0058] 21. Scene Recognition and Homing Decision: Mushroom-like networks will recognize the current scene and determine the homing decision. The robot performs parallel matching with all 21 path features in memory, calculating their "familiarity" (e.g., the Euclidean distance of the features). The central complex controller generates decisions based on familiarity: when familiarity is above a threshold, the robot is determined to be at or near a point on the memory path, thus generating a "familiar region" signal and driving the steering system to adopt a path-following strategy; when familiarity is below the threshold, the robot is determined to be in an "unfamiliar region," triggering behavior of tending towards the general direction of the nest or random exploration until a familiar region is found again.

[0059] 22. Heading Recovery and Path Following: Once the robot is determined to have returned to a specific key point on the remembered path, the system reads the memory direction signal corresponding to that point from memory. The controller directly calls the heading deviation calculation method protected by this invention, that is, calculates... Each of them All are obtained in real time by the phase extraction method in step (4). The calculated... As an error signal, the PID controller precisely adjusts the speed difference between the robot's left and right wheels to ensure that its current orientation matches the memorized orientation. The robot then moves forward while maintaining this correct orientation, achieving path following.

[0060] 3. Experimental Results: Under all uneven terrain test conditions, the robot successfully returned to its nest along the remembered path, and its homing trajectory was as follows: Figure 6 This is illustrated schematically. The average homing efficiency is as high as 0.78. The average processing time per frame for the spherical harmonic coding module was recorded as 4.1 milliseconds, which fully meets the requirements of real-time navigation.

[0061] Example 2: Visual coding for autonomous navigation of the Mars rover.

[0062] This embodiment illustrates the application of the present invention in deep space exploration. Mars rovers face extreme challenges when performing long-distance autonomous cruising missions on the Martian surface, including large temperature differences between day and night, rugged terrain (slopes up to 30°), and complete unavailability of GPS.

[0063] System configuration: A specially designed omnidirectional fisheye camera with a horizontal field of view of 360° and a vertical field of view of 180° is installed on the top of the probe's mast. Its computing platform is equipped with the spherical harmonic coding system of this invention.

[0064] Application Process: After departing from the lander, the rover automatically triggers a panoramic image acquisition and encoding every time it travels a preset distance. This encodes macroscopic topographic information such as the sky, ridgeline, and rock distribution at the current location into an 81-dimensional rotation-invariant feature vector. They are then timestamped and stored in local solid-state storage to form a chain of "visual road signs".

[0065] 1. Path backtracking: This is the core task of the implementation. When the probe needs to return to the landing point, it begins to travel in reverse along the route it came from. During this process, it matches the scene features that it collects and encodes in real time with the "visual landmarks" in the memory from near to far. Since the spherical harmonic features utilize their orthogonal decomposition characteristics on SO(3), their matching success rate is much higher than that of traditional visual odometry or local feature matching methods, ensuring the reliability of the return path.

[0066] 2. Anomaly Terrain Detection: To address potential areas of quicksand or sharp rocks, the system also calculates a 9-dimensional rotationally invariant power spectrum descriptor. ,in By monitoring the abrupt changes in the current power spectrum and the power spectra of nearby visual landmarks, drastic changes in the texture and structure of the terrain ahead can be identified in advance, thus providing early warning of potential terrain hazards. Specifically, the power spectrum descriptor P for the current location can be calculated. current Compared with the power spectrum P of the most recent visual road sign memory The Euclidean distance is used to trigger an abnormal alarm when the distance exceeds a preset safety threshold (e.g., empirically set to 3 times the average distance).

[0067] Example 3: Visual positioning for deep-sea autonomous underwater vehicles.

[0068] This embodiment illustrates the robustness of the invention in extreme visual degradation environments. Deep-sea AUVs, used in missions such as hydrothermal vents and mineral exploration, can reach depths exceeding 3000 meters in complete darkness, relying solely on their onboard LED arrays for illumination, which suffers from strong scattering and absorption.

[0069] System Configuration and Adjustments: The AUV is equipped with an omnidirectional camera and LED array with a waterproof cover, specifically designed for deep-sea high pressure systems. Given the extremely low signal-to-noise ratio of underwater images, this embodiment adjusts the system operating mode to "low-frequency mode," prioritizing the use of spherical harmonic coefficients. The low-order part (a total of 25 coefficients) is used to construct the scene feature vector.

[0070] Experimental Results: In a water tank and a computer-simulated 3000-meter deep-sea environment, even when visibility dropped to only 3 meters due to turbid water, the low-frequency coding system of this invention still maintained a global scene recognition accuracy of over 75%. In contrast, methods based on SURF local features (Scheme 2) and ORB features (Scheme 3) suffered from reduced accuracy, with scattered light obscuring most corners and edges, making it almost impossible to extract effective repeatable feature points, resulting in an accuracy rate below 30% and essentially losing navigation capability. Furthermore, the system also calculates normalized amplitude features... This effectively eliminates the impact of global illumination changes caused by the attenuation of LED lighting intensity with distance.

[0071] Example 4: Low-altitude terrain matching navigation for unmanned aerial vehicles.

[0072] This embodiment explains the application of the present invention in a highly dynamic flight platform. When a UAV is flying at low altitude (50-200 meters), if it encounters GPS interference, terrain matching navigation (TERCOM) based on downward panoramic images can be used as a backup positioning method.

[0073] System configuration: A downward-looking fisheye camera is mounted on the underside of the drone. During the mission, the drone's built-in computer uses this system to quickly encode the real-time acquired downward panoramic ground images into a set of 81-dimensional features.

[0074] Terrain matching and pose correction: The onboard flight control system rapidly searches and matches real-time features with a pre-generated "feature-position" association database based on 3D maps. The rotational invariance of spherical harmonic features is crucial here, ensuring that the features of the downward-view image remain unchanged when the UAV makes yaw adjustments, making the matching process unaffected by the UAV's nose direction. The end-to-end latency of the entire feature extraction and matching pipeline does not exceed 15 milliseconds, fully meeting the real-time requirements of high-speed UAV flight.

[0075] Example 5: Visual scene encoding for underground mine inspection robots.

[0076] This embodiment describes the application scenario of the present invention in a harsh industrial environment. The mine inspection robot needs to perform periodic inspections along a fixed route under conditions of no GPS, high dust, and low light.

[0077] System configuration: The inspection robot is equipped with an infrared omnidirectional camera to adapt to low light / no light environments, and its core processor is a low-power ARM embedded motherboard.

[0078] Lightweight path memory: Unlike traditional SLAM methods, this embodiment uses a lightweight map scheme. During the robot's first inspection, whenever it reaches a key scene point, it acquires an unobstructed panoramic infrared image of that location and immediately calls the coding system of this invention to generate a low-dimensional scene feature vector of only 81 dimensions.

[0079] Long-distance autonomous inspection: These 81-dimensional features and a small amount of location metadata are stored as a "visual landmark." A single inspection route of several kilometers may only require recording a few hundred such landmarks, and all data can be easily stored in less than 1MB of flash memory. During routine inspections, the robot starts from one landmark, reaches the next expected location, acquires panoramic images and extracts features, and matches them with target landmarks in its memory. In this process, its output is an 81-dimensional rotation-invariant scene feature vector. The visual projection neuron layer (vPN layer) of the insect-mushroom biomimetic neural network is directly input into the subsequent layer. Each dimension of the feature vector corresponds to the firing rate of a vPN neuron, thereby driving the determination of scene familiarity. Thanks to the rotational invariance of spherical harmonic features, the robot can be accurately identified no matter which direction it approaches the target point, triggering subsequent tasks such as instrument readings.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the spirit and principles of the present invention, such as adjusting the order of L to adapt to different computing power platforms, or porting the method to an FPGA for implementation. These improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rotation-invariant panoramic visual coding method based on spherical harmonic transform, characterized in that, Includes the following steps: Step 1: Image Acquisition, Preprocessing, and Spherical Mapping: Panoramic environmental images are acquired using an omnidirectional vision sensor. After grayscale conversion, intensity inversion, and normalization, the images are downsampled to standard resolution and mapped to a spherical coordinate system to obtain a spherical image. ,in Zenith angle, It is the azimuth angle; Step 2: Spherical Harmonic Transform Calculation: Calculate the spherical image... Perform spherical harmonic transformation and calculate up to the preset maximum order. All spherical harmonic coefficients The order ,frequency The total number of coefficients is One; the spherical harmonic basis function is defined as follows: ,in The normalization coefficient is... For associated Legendre polynomials; Step 3: Rotation-invariant feature extraction: Extract the spherical harmonic coefficients. complex amplitude ,composition 3D rotation-invariant scene feature vector ; Step 4: Directional Encoding Signal Extraction: Extract the spherical harmonic coefficients. phase of coefficients As a direction-encoded signal ; Step 5: Feature Output: Output the rotation-invariant scene feature vector. With direction-encoded signal The outputs are then sent to the subsequent navigation subsystems.

2. The method according to claim 1, characterized in that, The horizontal range of the field of view for panoramic image acquisition in step 1 is: Radiance, extending vertically as radians, acquisition resolution is Pixels / radians, downsampled to Perform spherical harmonic transformation after each pixel.

3. The method according to claim 1, characterized in that, In step 2, the spherical harmonic transformation is implemented using a fast algorithm, specifically: first along the azimuth angle... The direction performs a Fast Fourier Transform on each line of image data, and then along the zenith angle. The direction utilizes pre-computed associated Legendre polynomials to perform a weighted Legendre transformation, with a total computational complexity of O(n log n). ,in This represents the number of sampling points in the azimuth direction of the spherical image. This represents the number of sampling points for the spherical image in the zenith angle direction.

4. The method according to claim 1, characterized in that, The maximum order in step 2 The generated feature vector has a dimension of 81.

5. The method according to claim 1, characterized in that, The direction encoding signal extracted in step 4 is used to calculate the angular deviation between the current orientation and the memorized orientation: ,in The phase of the first-order spherical harmonic coefficient at the current moment. The first-order spherical harmonic coefficient phase at the memory time, the angular deviation It is precisely equal to the actual rotation angle of the carrier.

6. The method according to claim 1, characterized in that, Step 2 is followed by a step of calculating the rotationally invariant power spectral descriptor: for each order... Calculate the total power of this order. ,form 3D power spectrum descriptor ; and calculate the normalized amplitude characteristics To eliminate the effects of global illumination changes, among which for Total power of the first stage.

7. A rotation-invariant panoramic visual coding system based on spherical harmonic transform, characterized in that, For performing the method according to any one of claims 1-6, comprising: Module A: Panoramic Image Acquisition and Preprocessing Module: Includes an omnidirectional camera or fisheye lens to acquire 360-degree panoramic visual input and perform grayscale conversion, intensity inversion, normalization, and downsampling preprocessing; Module B: Leaf Simulation and Spherical Mapping Module: Maps the preprocessed rectangular panoramic image to a spherical coordinate system to generate a standardized spherical image; Module C: Core Calculation Module for Spherical Harmonic Transform: Employs a fast algorithm combining FFT and Legendre recursion to calculate up to the preset maximum order. All spherical harmonic coefficients Output There are complex coefficients; among which The preset maximum order for positive integers. For order ( ), For the number of times ( ); Module D: Rotation-Invariant Feature Separation and Orientation Encoding Module: Extracting Coefficient Amplitude Composition of rotation-invariant scene feature vectors ,extract phase of coefficients As a direction-encoded signal ; Module E: Multi-level feature output and interface adaptation module: and The formatted output is sent to the subsequent scene memory network, heading control system, and topology map module.

8. The system according to claim 7, characterized in that, Module C contains a Legendre polynomial pre-computation cache unit, which stores all to , to and the discrete zenith angle sampling points corresponding to Value, cache size is floating-point numbers, of which This represents the number of sampling points in the zenith angle direction, i.e., the number of height pixels in the spherical image.

9. The system according to claim 8, characterized in that, The spherical harmonic transformation core calculation in module C is performed by an embedded processor, which uses an ARM Cortex-A series or RISC-V processor, supports NEON / SIMD vectorized instruction acceleration, and the end-to-end latency of the complete coding pipeline does not exceed 15 milliseconds.

10. The system according to claim 7, characterized in that, It also includes at least one of the following features: a. The system is suitable for uneven terrain environments, utilizing spherical harmonics in a three-dimensional rotating group. The orthogonal decomposition properties of the surface are used to handle multi-axis visual distortion caused by carrier pitch and roll. b. The rotation-invariant scene feature vector The input is fed into the visual projection neuron layer - vPN layer of the subsequent insect mushroom biomimetic neural network, where each dimension of the feature vector corresponds to the firing rate of a vPN neuron; c. The system is applied to one or a combination of the following scenarios: autonomous cruise navigation for Mars / lunar rovers, visual positioning and mother ship return for deep-sea autonomous underwater vehicles, low-altitude terrain matching navigation for unmanned aerial vehicles, visual scene encoding for underground mine / tunnel inspection robots, autonomous navigation in the field for biomimetic legged robots, and GPS-denied environment positioning for disaster relief robots.

Images