A hybrid space positioning method and system based on a dim light environment

Abstract

This application relates to the field of computer technology, and in particular to a hybrid spatial positioning method and system based on low-light environments. The method includes: entering a low-light positioning mode when the ambient light intensity is below a light intensity threshold; acquiring multiple frames of images in the low-light positioning mode and extracting low-light edge features and frequency domain stability features from the images; acquiring IMU motion data and obtaining IMU motion edge features based on the images and IMU motion data; fusing the low-light edge features, frequency domain stability features, and IMU motion edge features to obtain a single-frame multimodal fusion feature; performing temporal accumulation on the continuous frame multimodal fusion features and weak feature enhancement on the temporally accumulated multimodal fusion features; inputting the enhanced multimodal fusion features and IMU motion data into a Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point. This application enables multisensory immersive experience cabins to achieve accurate spatial positioning in low-light environments without relying on additional supplementary lighting.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a hybrid spatial positioning method and system based on low-light environments. Background Technology

[0002] A multi-sensory immersive experience cabin is a closed and controllable experience space constructed by integrating multi-dimensional sensory feedback modules such as sound, light, heat, touch, and smell with a virtual scene rendering system. It allows users to escape external environmental interference and deeply immerse themselves in a pre-set virtual narrative scene. In order to achieve real-time linkage between user physical behavior and virtual scene and multi-sensory feedback, it is necessary to accurately capture the user's three-dimensional spatial coordinates, body posture, and movement trajectory within the cabin in real time, complete the precise mapping between physical space and virtual space, and then trigger multi-sensory feedback at the corresponding location and time sequence to ensure the user's movement safety within the closed space.

[0003] However, the typical operating environment for multi-sensory immersive experience cabins is a dark or low-light environment to effectively shield external visual interference and allow users to focus their visual attention entirely on the virtual content presented by devices such as holographic projections, panoramic displays, or VR headsets within the cabin. However, current spatial positioning in multi-sensory immersive experience cabins largely relies on spatial anchor point-based technologies. These technologies, whether based on visual features, laser scanning, or infrared sensing, all have significant limitations in dark or low-light environments, making it difficult to meet the precise positioning requirements of the experience cabin.

[0004] To mitigate the adverse effects of dim or low-light environments on space anchor-based technical solutions, existing technologies often employ additional lighting solutions for light compensation (such as global floodlighting and precise point-to-point lighting). However, the continuous operation of these lighting devices leads to excessively high power consumption costs. Furthermore, the supplementary lighting is prone to superimposed on the projected light and sensor signals within the cabin, easily causing multiple signal and visual interferences. Moreover, the supplementary lighting is often directional or high-frequency flickering light sources, which can easily irritate the user's eyes and severely impact the user experience.

[0005] Therefore, how to enable multi-sensory immersive experience cabins to achieve accurate spatial positioning in low-light environments without relying on additional lighting is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] This application provides a hybrid spatial positioning method and system based on low-light environments, enabling multi-sensory immersive experience cabins to achieve accurate spatial positioning in low-light environments without relying on additional supplementary lighting.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution: A hybrid spatial localization method based on low-light environments includes the following steps: Step T110: Detect the ambient light intensity inside a multi-sensory immersive experience cabin; when the ambient light intensity is below a light intensity threshold, enter low-light localization mode; Step T120: Acquire multiple frames of images in low-light localization mode, and extract low-light edge features and frequency domain stability features from the images; Step T130: Synchronously acquire IMU motion data corresponding to each frame of image, and obtain IMU motion edge features based on the images and IMU motion data; Step T140: Combine the low-light edge features, frequency domain stability features, and IMU motion edge features... The multimodal fusion features of a single frame are obtained by fusing. The multimodal fusion features of consecutive frames are accumulated temporally, and the multimodal fusion features accumulated temporally are weakly enhanced. In step T150, the enhanced multimodal fusion features and IMU motion data are input into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point. In step T160, when the dark light edge features and frequency domain stability features are completely lost, the system switches to pure inertial prediction mode, inputs the IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point, and continuously detects the ambient light intensity in the multisensory immersive experience cabin.

[0008] In the hybrid spatial positioning method based on low-light environment described above, preferably, step T110 includes the following sub-steps: acquiring images inside the multi-sensory immersive experience cabin, calculating the average brightness of all pixels in the images, and determining the ambient light intensity based on the average brightness and the parameters of the visual camera that acquired the images; When the ambient light intensity is detected to be continuously lower than the light intensity threshold for a preset number of frames, it is determined that the current multi-sensory immersive experience cabin is in a dark environment, and the system switches from the standard visual positioning mode to the dark light positioning mode.

[0009] The hybrid spatial localization method based on low-light environment described above preferably includes the following sub-steps in step T120: adaptively enhancing each frame image and extracting basic low-light features; upsampling the basic low-light features of the higher layers and fusing them with the basic low-light features of the lower layers to obtain basic low-light fusion features; and then fusing and scaling the basic low-light fusion features of each layer to obtain low-light edge features. Each frame of the image is preprocessed and converted to the frequency domain to obtain a frequency domain image. The frequency domain image is then centered, and noise is separated from the centered frequency domain image by low-pass filtering. The centered frequency domain image after low-pass filtering is then inversely centered to recover the original distribution of the frequency domain stable features in the frequency domain. Finally, the original distribution of the frequency domain stable features in the frequency domain is converted back to the spatial domain by inverse discrete Fourier transform to obtain the frequency domain stable features.

[0010] In the hybrid spatial localization method based on low-light environment described above, preferably, step T130 includes the following sub-steps: converting the image to grayscale, denoising the grayscale image, extracting the original edge gradient magnitude based on the denoised grayscale image; converting IMU motion data into displacement of image pixel plane to obtain pixel displacement prediction values, generating a motion weight map with the same size as the image based on the pixel displacement prediction values; multiplying the original edge gradient magnitude with the motion weight map to filter out edge features strongly correlated with IMU motion, and fusing the edge features strongly correlated with IMU motion through a 3-layer edge feature pyramid network to obtain IMU motion edge features.

[0011] In the hybrid spatial localization method based on low-light environment described above, preferably, step T140 includes the following sub-steps: unifying the dimensions and spatial alignment of low-light edge features, frequency domain stable features, and IMU motion edge features; performing initial fusion of the unified dimension and spatial alignment low-light edge features, frequency domain stable features, and IMU motion edge features according to corresponding fusion weights; multiplying the importance weights with the initial fused features to obtain the current frame multimodal fusion features; performing temporal accumulation of the current frame multimodal fusion features and historical frame multimodal fusion features to obtain temporally accumulated multimodal fusion features; and performing weak feature enhancement processing on the temporally accumulated multimodal fusion features to obtain enhanced multimodal fusion features.

[0012] A hybrid spatial positioning system based on low-light environments includes: a visual camera, an inertial measurement unit (IMU), and a processor. The visual camera captures images within a multi-sensory immersive experience cabin. The processor detects ambient light intensity through these images and enters a low-light positioning mode when the ambient light intensity falls below a threshold. In low-light positioning mode, the visual camera acquires multiple frames of images, and the processor extracts low-light edge features and frequency domain stability features from the images. The IMU synchronously acquires IMU motion data corresponding to each frame of image, and the processor obtains IMU motion edge features based on the images and IMU motion data. The processor then processes the low-light edge features... The processor fuses frequency domain stabilization features and IMU motion edge features to obtain single-frame multimodal fusion features. The multimodal fusion features of consecutive frames are accumulated temporally, and the accumulated multimodal fusion features are then weakly enhanced. The processor inputs the enhanced multimodal fusion features and IMU motion data into a Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point. When the processor detects that the dark light edge features and frequency domain stabilization features are completely lost, it switches to a pure inertial prediction mode, inputs the IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point, and continuously monitors the ambient light intensity inside the multisensory immersive experience cabin.

[0013] In the hybrid spatial positioning system based on low-light environment described above, preferably, the processor calculates the average brightness of all pixels in the image, and determines the ambient light intensity based on the average brightness and the parameters of the visual camera that acquired the image; when the processor detects that the ambient light intensity is continuously lower than the light intensity threshold for a preset number of frames, the processor determines that the current multi-sensory immersive experience cabin is in a low-light environment, and then switches from the standard visual positioning mode to the low-light positioning mode.

[0014] In the hybrid spatial positioning system based on low-light environment described above, preferably, the processor adaptively enhances each frame of image and extracts basic low-light features. The basic low-light features of higher layers are upsampled and fused with those of lower layers to obtain basic low-light fused features. The basic low-light fused features of each layer are then fused and scaled to obtain low-light edge features. The processor preprocesses each frame of image, converts the preprocessed image to the frequency domain to obtain a frequency domain image, centers the frequency domain image, and then separates noise from the centered frequency domain image using a low-pass filter. The centered frequency domain image after low-pass filtering is then inversely centered to restore the original distribution of frequency domain stable features in the frequency domain. Finally, the original distribution of frequency domain stable features in the frequency domain is converted back to the spatial domain using an inverse discrete Fourier transform to obtain frequency domain stable features.

[0015] In the hybrid spatial positioning system based on low-light environment described above, preferably, the processor converts the image to grayscale, denoises the grayscale image, and extracts the original edge gradient magnitude based on the denoised grayscale image; the processor converts the IMU motion data into the displacement of the image pixel plane to obtain the pixel displacement prediction value, generates a motion weight map with the same size as the image based on the pixel displacement prediction value, multiplies the original edge gradient magnitude with the motion weight map, filters out the edge features strongly correlated with IMU motion, and fuses the edge features strongly correlated with IMU motion through a 3-layer edge feature pyramid network to obtain the IMU motion edge features.

[0016] In the hybrid spatial positioning system based on low-light environment described above, preferably, the processor performs unified dimensional and spatial alignment of low-light edge features, frequency domain stable features, and IMU motion edge features; performs initial fusion of the unified dimensional and spatial aligned low-light edge features, frequency domain stable features, and IMU motion edge features according to corresponding fusion weights; multiplies the importance weights with the initial fused features to obtain the current frame multimodal fusion features; performs temporal accumulation of the current frame multimodal fusion features with the historical frame multimodal fusion features to obtain the temporally accumulated multimodal fusion features; and performs weak feature enhancement processing on the temporally accumulated multimodal fusion features to obtain enhanced multimodal fusion features.

[0017] Compared to the aforementioned background technologies, the hybrid spatial positioning method and system based on low-light environments provided in this application extracts weak visual features and simultaneously acquires IMU motion data, and then fuses them through Kalman filtering to output accurate three-dimensional coordinates of spatial anchor points. This enables the multi-sensory immersive experience cabin to achieve accurate spatial positioning in low-light environments without additional lighting. Furthermore, it can automatically switch to pure inertial prediction mode when features are lost, ensuring the continuity and stability of positioning. It achieves high-precision and high-reliability positioning in complex indoor and outdoor low-light scenarios, and is applicable to fields such as robot navigation and autonomous driving. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart of a hybrid spatial positioning method based on a low-light environment provided in an embodiment of this application; Figure 2 This is a schematic diagram of a hybrid spatial positioning system based on a low-light environment provided in an embodiment of this application. Detailed Implementation

[0020] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0021] Example 1 like Figure 1 As shown, this application provides a hybrid spatial positioning method based on low-light environments, including the following steps: Step T110: Detect the ambient light intensity inside the multi-sensory immersive experience cabin. When the ambient light intensity is lower than the light intensity threshold, enter the dark light positioning mode. The multi-sensory immersive experience cabin is equipped with a positioning system that integrates a visual camera, an inertial measurement unit (IMU), and a processor (e.g., a mobile device, AR glasses, or robot controller). The visual camera captures images from within the multi-sensory immersive experience cabin, the processor calculates the average brightness of all pixels in the image, and then, based on the average brightness of all pixels in the image and the parameters of the visual camera that captured the image, determines the ambient light intensity within the multi-sensory immersive experience cabin in real time.

[0022] Furthermore, the formula for calculating the average brightness of all pixels in the image is as follows: ; in, This represents the average brightness of all pixels within the image. For the first in the image The brightness value of each pixel; This represents the total number of pixels in the image.

[0023] Furthermore, the formula for calculating the ambient light intensity inside the multi-sensory immersive experience cabin is as follows: ; in, The ambient lighting intensity inside the multi-sensory immersive experience cabin; The ISO coefficient after calibration of the visual camera; The reflectivity calibration constant for the visual camera (usually around 12.5); This refers to the exposure time.

[0024] Preset light intensity threshold (This light intensity threshold) This can be calibrated experimentally, for example, by determining the minimum lighting conditions under which the visual feature extraction algorithm can reliably operate, when the ambient light intensity inside the multi-sensory immersive experience cabin is detected. Continuously below the light intensity threshold When the preset number of frames is reached (e.g., 3-5 frames to avoid accidental triggering due to instantaneous fluctuations), the processor determines that the current multi-sensory immersive experience cabin is in a dark environment, and then switches from the standard visual positioning mode to the dark positioning mode.

[0025] Step T120: Acquire multiple frames of images in low-light positioning mode, and extract low-light edge features and frequency domain stability features from the images; In low-light positioning mode, the visual camera will capture multiple consecutive frames of images inside the multi-sensory immersive experience cabin at a predetermined frame rate (e.g., the frame rate can be appropriately reduced to increase the exposure time of each frame). The processor will adaptively enhance each frame of image and extract basic low-light features. Then, the basic low-light features of the higher layers will be upsampled and fused with the basic low-light features of the lower layers to obtain basic low-light fusion features. The basic low-light fusion features of each layer will then be fused and scaled to obtain low-light edge features.

[0026] Furthermore, the expressions for adaptive enhancement and basic dark light feature extraction are as follows: ; in, For the first The basic characteristics of a layer; The core convolutional neural network is used to extract basic features from images; The input image; This is the low-light enhancement factor; For pixel neighborhood The average gray level; The standard deviation of the gray level in the neighborhood; For the network layer.

[0027] Furthermore, the expression for the fusion of basic features is as follows: ; in, for convolution; for convolution; This is an upsampling operation; These are the basic features of layer 5; For the first The basic fusion characteristics of layers; For the first The basic fusion characteristics of layers; This is the basic fusion feature of layer 5; This forms the basic fusion feature of the second layer; This is the basic fusion feature of layer 1; For the first Layer edge attention weights; The edge attention weights for layer 5; These are the edge attention weights for the second layer.

[0028] Furthermore, the expressions for the fusion and scaling of the basic fusion features are as follows: ; in, Features of dark light edges; For the first Multi-scale weights of layers; To make the first The basic fusion feature of the layer is scaled to a size of Images; for height, for The width.

[0029] In addition, the processor preprocesses each frame of the image (grayscale conversion, noise reduction, contrast enhancement, etc.) to improve feature extractability. Then, the preprocessed image is converted to the frequency domain to obtain a frequency domain image, which is then centered. Since noise in low-light images is mainly distributed in the high-frequency region of the frequency domain, while frequency-stable features are mainly distributed in the low-frequency region, noise is separated from the centered frequency domain image by low-pass filtering. Since the centered frequency domain image after low-pass filtering is still in a centered state, inverse centering is performed on the centered frequency domain image after low-pass filtering to restore the original distribution of frequency-stable features in the frequency domain. Finally, the original distribution of frequency-stable features in the frequency domain is converted back to the spatial domain by inverse discrete Fourier transform to obtain the final retained frequency-stable features.

[0030] Furthermore, the expression for the frequency domain image is as follows: ; in, This is the frequency domain image obtained by performing a Fourier transform on the preprocessed image; Image frequency domain coordinates; For Fourier transform operators; The image is after preprocessing; These are the pixel coordinates in the image spatial domain.

[0031] Furthermore, the expression for the centered frequency domain image is as follows: ; in, This is the centered frequency domain image.

[0032] Furthermore, the expressions for low-pass filtering, inverse centering, and inverse discrete Fourier transform are as follows: ; in, This is the centered frequency domain image after low-pass filtering separation; The low-pass filter in frequency domain coordinates The transfer coefficient at that location; This represents the original distribution of frequency domain stability features in the frequency domain. It exhibits frequency domain stability characteristics; The height of the preprocessed image. The width of the preprocessed image; It is a natural constant; It is the imaginary unit.

[0033] Step T130: Synchronously acquire IMU motion data corresponding to each frame of image, and obtain IMU motion edge features based on the image and IMU motion data; The image captured by the vision camera is converted to grayscale to reduce channel redundancy and focus on brightness information; then the grayscale image is denoised to obtain a denoised grayscale image; finally, the original edge gradient magnitude is extracted based on the denoised grayscale image.

[0034] Furthermore, the calculation formula for denoising a grayscale image is as follows: ; in, This is a denoised grayscale image.

[0035] Furthermore, the algorithm for extracting the original edge gradient magnitude is as follows: ; in, This represents the original edge gradient magnitude. This is an edge detection algorithm; For low threshold, The threshold is high.

[0036] The Inertial Measurement Unit (IMU) acquires IMU motion data (raw angular velocity and raw acceleration) at a much higher frame rate (far exceeding the frame rate of images captured by a visual camera) to ensure the density of IMU motion data; and the processor assigns a unique timestamp T to each frame, recording the IMU's motion data at a higher frame rate. IMU motion data acquired within the time window is interpolated / downsampled to obtain IMU motion data corresponding one-to-one with a single frame image. (To improve fusion accuracy, the time deviation between IMU motion data and images is considered).

[0037] The processor converts IMU motion data into displacements of image pixel planes to obtain pixel displacement predictions. Then, the processor generates a motion weight map with the same size as the image based on the pixel displacement predictions. Next, the processor multiplies the original edge gradient magnitudes with the motion weight map to filter out edge features that are strongly correlated with IMU motion. Finally, the processor fuses the edge features that are strongly correlated with IMU motion through a 3-layer edge feature pyramid network to obtain IMU motion edge features.

[0038] Furthermore, the formula for calculating the pixel displacement prediction value is as follows: ; in, This represents the predicted pixel displacement along the x-axis. This is the predicted pixel displacement value along the y-axis. Let be the normalized focal length of the visual camera along the x-axis. The normalized focal length of the visual camera along the y-axis; acceleration in the x-axis direction exist Horizontal physical displacement over time ; acceleration in the y-axis direction exist Horizontal physical displacement over time ; This is the pitch attitude angle correction factor. For roll attitude angle correction factor, pitch attitude angle and roll attitude angle All data are IMU motion data; The average depth from the visual camera to the target object (e.g., spatial anchor point, heated area).

[0039] Furthermore, the expression for the motion weight graph is as follows: ; in, For motion weighting graphs; The similarity between each pixel displacement and the predicted pixel displacement value. ; It is the natural exponential function; , For pixels The principal point coordinates relative to the x-axis direction of the image plane. Pixel offset; , For pixels The principal point coordinates relative to the y-axis direction of the image plane. Pixel offset; The standard deviation of the Gaussian kernel for similarity.

[0040] Furthermore, the expression for edge features strongly correlated with IMU motion is as follows: ; in, These are edge features that are strongly correlated with IMU motion.

[0041] Furthermore, the expression for the 3-layer edge feature pyramid network is as follows: ; in, This is the 0th layer of the edge feature pyramid; For the edge feature pyramid number layer; For the edge feature pyramid number layer; For Gaussian blur denoising operation; The standard deviation is the Gaussian kernel. For 2x downsampling, This is a 2x upsampling; For the first Features restored by upsampling from the layer edge feature pyramid; This represents the motion edge features of the IMU.

[0042] Step T140: Fuse the dark light edge features, frequency domain stability features and IMU motion edge features to obtain single-frame multimodal fusion features, perform temporal accumulation on the multimodal fusion features of consecutive frames, and perform weak feature enhancement on the temporally accumulated multimodal fusion features; The processor performs unified dimensional and spatial alignment between the dark light edge features and frequency domain stable features obtained in step T120 and the IMU motion edge features obtained in step T130 to ensure the effectiveness of subsequent single-frame multimodal fusion. Then, the unified dimensional and spatially aligned dark light edge features, frequency domain stable features, and IMU motion edge features are initially fused according to their respective fusion weights. The initial fused features are then globally pooled to obtain feature statistics. Next, the feature statistics are processed through a fully connected layer to learn the importance weights of each dimension of features. Finally, the importance weights are multiplied by the initial fused features to obtain the single-frame multimodal fused features.

[0043] Furthermore, the expression for the initial fused features is as follows: ; in, These are the initial fusion features; To unify the dark light edge features after dimensional and spatial alignment; For frequency domain stability features after unifying dimensions and spatial alignment; To unify the dimensions and spatial alignment of IMU motion edge features; The fusion weights are used for the dark light edge features; The fusion weights are used for frequency domain stable features; The fusion weights for IMU motion edge features; .

[0044] Furthermore, the expression for the single-frame multimodal fusion feature is as follows: ; in, This represents a single-frame multimodal fusion feature. For element-wise multiplication operators; The importance weights are learned from the feature statistics after global pooling through a fully connected layer.

[0045] Based on the above, the processor also calculates the entropy values ​​of dark edge features and frequency domain stable features in multiple consecutive frames of images, and normalizes the entropy values ​​of dark edge features and frequency domain stable features to map them to the [0,1] interval. Then, it establishes the correlation between the normalized entropy values ​​of dark edge features and frequency domain stable features and the fusion weights, thereby obtaining the fusion weight correction values ​​of dark edge features and frequency domain stable features. Finally, it updates the fusion weight correction values ​​of dark edge features and frequency domain stable features to the expression of the initial fusion features, thereby achieving adaptive correction of the fusion weights.

[0046] Furthermore, the expressions for the correlation between the entropy value of the dark edge feature after normalization and the fusion weight of the dark edge feature, and the correlation between the entropy value of the frequency domain stability feature after normalization and the fusion weight of the frequency domain temperature feature are as follows: ; in, This is the fusion weight correction value for the dark light edge features; This represents the entropy value of the dark light edge features after normalization. This is the correction value for the fusion weights of the frequency domain stable features; This represents the entropy value of the frequency domain stability feature after normalization.

[0047] In addition, since single-frame multimodal fusion features still suffer from information sparsity and weak anti-interference capabilities, the processor performs temporal accumulation of the current frame multimodal fusion features and historical frame multimodal fusion features to obtain temporally accumulated multimodal fusion features. This is done to improve feature effectiveness by utilizing feature correlation and suppress noise in single-frame multimodal fusion features under low light conditions. The processor then performs weak feature enhancement processing on the temporally accumulated multimodal fusion features to transform weak features into clear and effective features that can be used for subsequent processing.

[0048] Furthermore, the expression for the temporal accumulation of multimodal fusion features across consecutive frames is as follows: ; in, For the first Multimodal fusion features accumulated from frame time sequence; For the first Frame multimodal fusion features; For the first Multimodal fusion features accumulated from frame time sequence; , These are the weighting coefficients. For the first The confidence level of the multimodal fusion features of the frame is determined by the first... The confidence of frame multimodal fusion features can be dynamically adjusted in terms of weights.

[0049] Furthermore, the expression for weak feature enhancement of multimodal fusion features after time-series accumulation is as follows: ; in, To enhance multimodal fusion features; For contrast-limited adaptive histogram equalization, it can avoid overexposure or noise amplification problems that may be caused by ordinary histogram equalization. This is the normalization function; This is the enhancement coefficient.

[0050] The extraction of the aforementioned multi-scale features overcomes the limitation of relying solely on visible light features in low-light conditions, fully utilizes multi-source information, and mines effective features in low-light conditions, providing a foundation for accurate spatial positioning in low-light environments. In addition, the aforementioned accumulation of temporal features compensates for the lack of information in single-frame multimodal fusion features in low-light environments, effectively reduces noise interference, and improves the stability and reliability of the features.

[0051] Step T150: Input the enhanced multimodal fusion features and IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point; The processor takes the enhanced multimodal fusion features (after time-accumulated weak features) and the IMU motion data collected by the Inertial Measurement Unit (IMU) as inputs to the Kalman filter fusion model. The Kalman filter fusion model performs state prediction using the IMU motion data, corrects the predicted state using the enhanced multimodal fusion features, calculates the three-dimensional coordinates of the spatial anchor point, and outputs the three-dimensional coordinates of the spatial anchor point.

[0052] As an example, the three-dimensional coordinates of the input spatial anchor point are (120.5cm, 85.3cm, 150.2cm), and the positioning error is ±1.2cm. The error level of ±1.2cm indicates that the Kalman filter fusion model can effectively suppress the defects of a single sensor and achieve high-precision positioning.

[0053] Step T160: When the dark light edge features and frequency domain stability features are completely lost, switch to pure inertial prediction mode, input the IMU motion data into the Kalman filter fusion model to solve the three-dimensional coordinates of the spatial anchor point, and continuously detect the ambient light intensity in the multi-sensory immersive experience cabin. In extreme cases (e.g., complete darkness, lens obstruction), the processor may be unable to extract any valid visual features such as dark edge features and frequency domain stable features from the image. Therefore, the processor continuously monitors the number of successfully matched visual features such as dark edge features and frequency domain stable features. When the number of visual features in N consecutive (e.g., N=10) frames is zero or below a very small threshold, the processor determines that the visual features such as dark edge features and frequency domain stable features are completely lost. In addition to determining that the visual features in the aforementioned N consecutive frames are completely lost, the processor also verifies the validity of the IMU motion data. Only when the IMU motion data is normal and the visual features are continuously lost, the processor immediately switches to pure inertial prediction mode. This can avoid abnormal mode switching due to IMU malfunction.

[0054] In this mode, the Kalman filter fusion model uses only IMU motion data to infer the 3D coordinates of the spatial anchor point. This significantly increases the error covariance to reflect the rapid accumulation of errors under pure inertial inference and issues a "precision degradation" warning. The pure inertial prediction mode error model employs a coupled random and systematic error model based on IMU device characteristics. This includes gyroscope zero-bias drift (linear drift over time, not exceeding 0.5° per hour), accelerometer zero-bias (not exceeding 10mg under static conditions), and random walk error (gyroscope ≤ 0.01° / √h, accelerometer ≤ 0.1mg / √h). It also introduces a motion-state adaptive error correction term. When the multi-sensory immersive experience cabin is stationary, the drift weight is reduced; when it is moving, the dynamic error compensation coefficient is increased, and the error covariance matrix is ​​updated in real time, thus more accurately depicting the accumulation of errors over time.

[0055] During the pure inertial prediction mode, the visual camera continues to acquire images and detect two indicators (ambient light intensity and number of visual feature matches). When the light intensity recovers to a preset threshold (e.g., ≥5 lux) and effective visual features (dark light edge features and frequency domain stable features) (≥3) are detected in 3 consecutive frames, or when the light intensity does not meet the threshold but stable visual features (dark light edge features and frequency domain stable features) (≥5) are detected in 5 consecutive frames, the processor immediately triggers the recovery process. The Kalman filter fusion model gradually introduces visual features (dark light edge features and frequency domain stable features) to achieve a smooth switch from the pure inertial prediction mode to the dark light positioning mode, avoiding coordinate jumps. During the switch, the "accuracy recovery in progress" prompt is continuously output until the two data are fused and stabilized, at which point the "accuracy degradation" warning is canceled.

[0056] Example 2 like Figure 2 As shown, this application provides a hybrid spatial positioning system 200 based on low-light environment, including: a visual camera 210, an inertial measurement unit (IMU) 220 and a processor 230.

[0057] The visual camera 210 captures images inside the multi-sensory immersive experience cabin, and the processor 230 detects the ambient light intensity through the images. When the ambient light intensity is lower than the light intensity threshold, it enters the dark light positioning mode.

[0058] The multi-sensory immersive experience cabin has a positioning system integrating a visual camera 210, an inertial measurement unit (IMU) 220, and a processor 230 (e.g., a mobile device, AR glasses, or robot controller). The visual camera 210 captures images within the multi-sensory immersive experience cabin, the processor 230 calculates the average brightness of all pixels in the image, and then, based on the average brightness of all pixels in the image and the parameters of the visual camera that captured the image, determines the ambient light intensity within the multi-sensory immersive experience cabin in real time.

[0059] Furthermore, the formula for calculating the average brightness of all pixels in the image is as follows: ; in, This represents the average brightness of all pixels within the image. For the first in the image The brightness value of each pixel; This represents the total number of pixels in the image.

[0060] Furthermore, the formula for calculating the ambient light intensity inside the multi-sensory immersive experience cabin is as follows: ; in, The ambient lighting intensity inside the multi-sensory immersive experience cabin; The ISO coefficient after calibration of the visual camera; The reflectivity calibration constant for the visual camera (usually around 12.5); This refers to the exposure time.

[0061] The processor 230 has a preset light intensity threshold. (This light intensity threshold) This can be calibrated experimentally, for example, by determining the minimum lighting conditions under which the visual feature extraction algorithm can reliably operate, when the processor 230 detects the ambient light intensity inside the multi-sensory immersive experience cabin. Continuously below the light intensity threshold When the preset frame rate is reached (e.g., 3-5 frames to avoid accidental triggering due to instantaneous fluctuations), the processor 230 determines that the current multi-sensory immersive experience cabin is in a dark environment, and then switches from the standard visual positioning mode to the dark positioning mode.

[0062] The visual camera 210 acquires multiple frames of images in low-light positioning mode, and the processor 230 extracts low-light edge features and frequency domain stability features from the images.

[0063] In low-light positioning mode, the visual camera 210 will acquire multiple consecutive frames of images inside the multi-sensory immersive experience cabin at a predetermined frame rate (e.g., the frame rate can be appropriately reduced to increase the exposure time of each frame). The processor 230 will adaptively enhance each frame of image and extract basic low-light features. Then, the processor 230 will upsample the basic low-light features of the higher layers and fuse them with the basic low-light features of the lower layers to obtain basic low-light fusion features. The processor 230 will then fuse and scale the basic low-light fusion features of each layer to obtain low-light edge features.

[0064] Furthermore, the expressions for adaptive enhancement and basic dark light feature extraction are as follows: ; in, For the first The basic characteristics of a layer; The core convolutional neural network is used to extract basic features from images; The input image; This is the low-light enhancement factor; For pixel neighborhood The average gray level; The standard deviation of the gray level in the neighborhood; For the network layer.

[0065] Furthermore, the expression for the fusion of basic features is as follows: ; in, for convolution; for convolution; This is an upsampling operation; These are the basic features of layer 5; For the first The basic fusion characteristics of layers; For the first The basic fusion characteristics of layers; This is the basic fusion feature of layer 5; This forms the basic fusion feature of the second layer; This is the basic fusion feature of layer 1; For the first Layer edge attention weights; The edge attention weights for layer 5; These are the edge attention weights for the second layer.

[0066] Furthermore, the expressions for the fusion and scaling of the basic fusion features are as follows: ; in, Features of dark light edges; For the first Multi-scale weights of layers; To make the first The basic fusion feature of the layer is scaled to a size of Images; for height, for The width.

[0067] In addition, the processor 230 preprocesses each frame of the image (grayscale conversion, noise reduction, contrast enhancement, etc.) to improve feature extractability. Then, the processor 230 converts the preprocessed image to the frequency domain to obtain a frequency domain image and centers the frequency domain image. Since noise in the low-light image is mainly distributed in the high-frequency region of the frequency domain, and the frequency domain stable features are mainly distributed in the low-frequency region, the processor 230 separates the noise from the centered frequency domain image through low-pass filtering. Since the centered frequency domain image after low-pass filtering is still in a centered state, the processor 230 performs inverse centering on the centered frequency domain image after low-pass filtering to restore the original distribution of the frequency domain stable features in the frequency domain. The processor 230 then uses inverse discrete Fourier transform to convert the original distribution of the frequency domain stable features in the frequency domain back to the spatial domain, thereby obtaining the final retained frequency domain stable features.

[0068] Furthermore, the expression for the frequency domain image is as follows: ; in, This is the frequency domain image obtained by performing a Fourier transform on the preprocessed image; Image frequency domain coordinates; For Fourier transform operators; The image is after preprocessing; These are the pixel coordinates in the image spatial domain.

[0069] Furthermore, the expression for the centered frequency domain image is as follows: ; in, This is the centered frequency domain image.

[0070] Furthermore, the expressions for low-pass filtering, inverse centering, and inverse discrete Fourier transform are as follows: ; in, This is the centered frequency domain image after low-pass filtering separation; The low-pass filter in frequency domain coordinates The transfer coefficient at that location; This represents the original distribution of frequency domain stability features in the frequency domain. It exhibits frequency domain stability characteristics; The height of the preprocessed image. The width of the preprocessed image; It is a natural constant; It is the imaginary unit.

[0071] The inertial measurement unit 220 synchronously acquires IMU motion data corresponding to each frame of image, and the processor 230 obtains IMU motion edge features based on the image and IMU motion data.

[0072] The processor 230 converts the image captured by the vision camera 210 to grayscale to reduce channel redundancy and focus on brightness information; then the processor 230 denoises the grayscale image to obtain a denoised grayscale image; finally, the processor 230 extracts the original edge gradient magnitude based on the denoised grayscale image.

[0073] Furthermore, the calculation formula for denoising a grayscale image is as follows: ; in, This is a denoised grayscale image.

[0074] Furthermore, the algorithm for extracting the original edge gradient magnitude is as follows: ; in, This represents the original edge gradient magnitude. This is an edge detection algorithm; For low threshold, The threshold is high.

[0075] The Inertial Measurement Unit (IMU) 220 acquires IMU motion data (raw angular velocity and raw acceleration) at a higher frame rate (far exceeding the frame rate at which the visual camera 210 acquires images) to ensure the density of the IMU motion data; and the processor 230 assigns a unique timestamp T to each frame, storing the IMU 220 in... IMU motion data acquired within the time window is interpolated / downsampled to obtain IMU motion data corresponding one-to-one with a single frame image. (To improve fusion accuracy, the time deviation between IMU motion data and images is considered).

[0076] The processor 230 converts the IMU motion data into the displacement of the image pixel plane to obtain the pixel displacement prediction value; then the processor 230 generates a motion weight map with the same size as the image based on the pixel displacement prediction value; next, the processor 230 multiplies the original edge gradient magnitude with the motion weight map to filter out the edge features that are strongly correlated with the IMU motion; finally, the processor 230 fuses the edge features that are strongly correlated with the IMU motion through a 3-layer edge feature pyramid network to obtain the IMU motion edge features.

[0077] Furthermore, the formula for calculating the pixel displacement prediction value is as follows: ; in, This represents the predicted pixel displacement along the x-axis. This is the predicted pixel displacement value along the y-axis. Let be the normalized focal length of the visual camera along the x-axis. The normalized focal length of the visual camera along the y-axis; acceleration in the x-axis direction exist Horizontal physical displacement over time ; acceleration in the y-axis direction exist Horizontal physical displacement over time ; This is the pitch attitude angle correction factor. For roll attitude angle correction factor, pitch attitude angle and roll attitude angle All data are IMU motion data; The average depth from the visual camera to the target object (e.g., spatial anchor point, heated area).

[0078] Furthermore, the expression for the motion weight graph is as follows: ; in, For motion weighting graphs; The similarity between each pixel displacement and the predicted pixel displacement value. ; It is the natural exponential function; , For pixels The principal point coordinates relative to the x-axis direction of the image plane. Pixel offset; , For pixels The principal point coordinates relative to the y-axis direction of the image plane. Pixel offset; The standard deviation of the Gaussian kernel for similarity.

[0079] Furthermore, the expression for edge features strongly correlated with IMU motion is as follows: ; in, These are edge features that are strongly correlated with IMU motion.

[0080] Furthermore, the expression for the 3-layer edge feature pyramid network is as follows: ; in, This is the 0th layer of the edge feature pyramid; For the edge feature pyramid number layer; For the edge feature pyramid number layer; For Gaussian blur denoising operation; The standard deviation is the Gaussian kernel. For 2x downsampling, This is a 2x upsampling; For the first Features restored by upsampling from the layer edge feature pyramid; This represents the motion edge features of the IMU.

[0081] The processor 230 fuses dark light edge features, frequency domain stability features and IMU motion edge features to obtain single-frame multimodal fusion features, performs temporal accumulation of multimodal fusion features of consecutive frames, and performs weak feature enhancement on the temporally accumulated multimodal fusion features.

[0082] Processor 230 performs unified dimensional and spatial alignment of the dark light edge features and frequency domain stabilization features obtained in step T120 with the IMU motion edge features obtained in step T130 to ensure the effectiveness of subsequent single-frame multimodal fusion. Then, processor 230 performs initial fusion of the unified dimensional and spatial alignment dark light edge features, frequency domain stabilization features, and IMU motion edge features according to their respective fusion weights. Processor 230 then performs global pooling on the initial fused features to obtain feature statistics, and then learns the importance weights of each dimension of features through a fully connected layer. Finally, processor 230 multiplies the importance weights with the initial fused features to obtain the single-frame multimodal fusion features.

[0083] Furthermore, the expression for the initial fused features is as follows: ; in, These are the initial fusion features; To unify the dark light edge features after dimensional and spatial alignment; For frequency domain stability features after unifying dimensions and spatial alignment; To unify the dimensions and spatial alignment of IMU motion edge features; The fusion weights are used for the dark light edge features; The fusion weights are used for frequency domain stable features; The fusion weights for IMU motion edge features; .

[0084] Furthermore, the expression for the single-frame multimodal fusion feature is as follows: ; in, This represents a single-frame multimodal fusion feature. For element-wise multiplication operators; The importance weights are learned from the feature statistics after global pooling through a fully connected layer.

[0085] Based on the above, the processor 230 further calculates the entropy values ​​of dark edge features and frequency domain stable features in consecutive frames of images, and normalizes the entropy values ​​of dark edge features and frequency domain stable features respectively to map them to the [0,1] interval; then, the processor 230 establishes the correlation between the normalized entropy values ​​of dark edge features and frequency domain stable features and the fusion weights, thereby obtaining the fusion weight correction values ​​of dark edge features and frequency domain stable features; the processor 230 then updates the fusion weight correction values ​​of dark edge features and frequency domain stable features to the expression of the initial fusion features, thereby realizing the adaptive correction of the fusion weights.

[0086] Furthermore, the expressions for the correlation between the entropy value of the dark edge feature after normalization and the fusion weight of the dark edge feature, and the correlation between the entropy value of the frequency domain stability feature after normalization and the fusion weight of the frequency domain temperature feature are as follows: ; in, This is the fusion weight correction value for the dark light edge features; This represents the entropy value of the dark light edge features after normalization. This is the correction value for the fusion weights of the frequency domain stable features; This represents the entropy value of the frequency domain stability feature after normalization.

[0087] In addition, since the single-frame multimodal fusion features still have problems such as information sparsity and weak anti-interference ability, the processor 230 performs temporal accumulation of the current frame multimodal fusion features and the historical frame multimodal fusion features to obtain temporally accumulated multimodal fusion features, so as to improve the effectiveness of features by utilizing feature correlation and suppress noise in single-frame multimodal fusion features under low light. The processor 230 performs weak feature enhancement processing on the temporally accumulated multimodal fusion features to transform weak features into clear and effective features that can be used for subsequent processing.

[0088] Furthermore, the expression for the temporal accumulation of multimodal fusion features across consecutive frames is as follows: ; in, For the first Multimodal fusion features accumulated from frame time sequence; For the first Frame multimodal fusion features; For the first Multimodal fusion features accumulated from frame time sequence; , These are the weighting coefficients. For the first The confidence level of the multimodal fusion features of the frame is determined by the first... The confidence of frame multimodal fusion features can be dynamically adjusted in terms of weights.

[0089] Furthermore, the expression for weak feature enhancement of multimodal fusion features after time-series accumulation is as follows: ; in, To enhance multimodal fusion features; For contrast-limited adaptive histogram equalization, it can avoid overexposure or noise amplification problems that may be caused by ordinary histogram equalization. This is the normalization function; This is the enhancement coefficient.

[0090] The extraction of the aforementioned multi-scale features overcomes the limitation of relying solely on visible light features in low-light conditions, fully utilizes multi-source information, and mines effective features in low-light conditions, providing a foundation for accurate spatial positioning in low-light environments. In addition, the aforementioned accumulation of temporal features compensates for the lack of information in single-frame multimodal fusion features in low-light environments, effectively reduces noise interference, and improves the stability and reliability of the features.

[0091] The processor 230 will input the enhanced multimodal fusion features and IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point.

[0092] The processor 230 takes the enhanced multimodal fusion features (after time-accumulated weak features) and the IMU motion data collected by the inertial measurement unit (IMU) as inputs to the Kalman filter fusion model. The Kalman filter fusion model performs state prediction through the IMU motion data, corrects the predicted state through the enhanced multimodal fusion features, calculates the three-dimensional coordinates of the spatial anchor point, and uses the three-dimensional coordinates of the spatial anchor point as the output of the Kalman filter fusion model.

[0093] As an example, the three-dimensional coordinates of the input spatial anchor point are (120.5cm, 85.3cm, 150.2cm), and the positioning error is ±1.2cm. The error level of ±1.2cm indicates that the Kalman filter fusion model can effectively suppress the defects of a single sensor and achieve high-precision positioning.

[0094] When the processor 230 detects that the dark light edge features and frequency domain stability features are completely lost, it switches to pure inertial prediction mode, inputs the IMU motion data into the Kalman filter fusion model to solve the three-dimensional coordinates of the spatial anchor point, and continuously detects the ambient light intensity in the multi-sensory immersive experience cabin.

[0095] In extreme cases (e.g., complete darkness, lens obstruction), the processor 230 may be unable to extract any valid visual features such as dark edge features and frequency domain stable features from the image. Therefore, the processor 230 continuously monitors the number of successfully matched visual features such as dark edge features and frequency domain stable features. When the number of visual features in N consecutive (e.g., N=10) frames is zero or below a very small threshold, the processor 230 determines that the visual features such as dark edge features and frequency domain stable features are completely lost. In addition to determining that the visual features in the aforementioned N consecutive frames are completely lost, the processor 230 also verifies the validity of the IMU motion data. Only when the IMU motion data is normal and the visual features are continuously lost, the processor 230 immediately switches to the pure inertial prediction mode. This can avoid abnormal mode switching due to IMU malfunction.

[0096] In this mode, the Kalman filter fusion model uses only IMU motion data to infer the 3D coordinates of the spatial anchor point. This significantly increases the error covariance to reflect the rapid accumulation of errors under pure inertial inference and issues a "precision degradation" warning. The pure inertial prediction mode error model employs a coupled random and systematic error model based on IMU device characteristics. This includes gyroscope zero-bias drift (linear drift over time, not exceeding 0.5° per hour), accelerometer zero-bias (not exceeding 10mg under static conditions), and random walk error (gyroscope ≤ 0.01° / √h, accelerometer ≤ 0.1mg / √h). It also introduces a motion-state adaptive error correction term. When the multi-sensory immersive experience cabin is stationary, the drift weight is reduced; when it is moving, the dynamic error compensation coefficient is increased, and the error covariance matrix is ​​updated in real time, thus more accurately depicting the accumulation of errors over time.

[0097] During the pure inertial prediction mode, the visual camera 210 continues to acquire images and detect two indicators (ambient light intensity and number of visual feature matches). When the light intensity recovers to a preset threshold (e.g., ≥5 lux) and effective visual features (dark light edge features and frequency domain stable features) (≥3) are detected in 3 consecutive frames, or when the light intensity does not meet the threshold but stable visual features (dark light edge features and frequency domain stable features) (≥5) are detected in 5 consecutive frames, the processor immediately triggers the recovery process. The Kalman filter fusion model gradually introduces visual features (dark light edge features and frequency domain stable features) to achieve a smooth switch from the pure inertial prediction mode to the dark light positioning mode, avoiding coordinate jumps. During the switching process, the "accuracy recovery in progress" prompt is continuously output until the two data are fused and stabilized, at which point the "accuracy degradation" warning is canceled.

[0098] This application extracts weak visual features and simultaneously acquires IMU motion data, then fuses them using Kalman filtering to output precise three-dimensional coordinates of spatial anchor points. This enables the multi-sensory immersive experience cabin to achieve accurate spatial positioning in low-light environments without additional lighting. Furthermore, it can automatically switch to pure inertial prediction mode when features are lost, ensuring the continuity and stability of positioning. This achieves high-precision and high-reliability positioning in complex indoor and outdoor low-light scenarios, making it suitable for fields such as robot navigation and autonomous driving.

[0099] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0100] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A hybrid spatial positioning method based on low-light environments, characterized in that, Includes the following steps: Step T110: Detect the ambient light intensity inside the multi-sensory immersive experience cabin. When the ambient light intensity is lower than the light intensity threshold, enter the dark light positioning mode. Step T120: Acquire multiple frames of images in low-light positioning mode, and extract low-light edge features and frequency domain stability features from the images; Step T130: Synchronously acquire IMU motion data corresponding to each frame of image, and obtain IMU motion edge features based on the image and IMU motion data; Step T140: Fuse the dark light edge features, frequency domain stability features and IMU motion edge features to obtain single-frame multimodal fusion features, perform temporal accumulation on the multimodal fusion features of consecutive frames, and perform weak feature enhancement on the temporally accumulated multimodal fusion features; Step T150: Input the enhanced multimodal fusion features and IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point; Step T160: When the dark light edge features and frequency domain stability features are completely lost, switch to pure inertial prediction mode, input the IMU motion data into the Kalman filter fusion model to solve the three-dimensional coordinates of the spatial anchor point, and continuously detect the ambient light intensity in the multi-sensory immersive experience cabin.

2. The hybrid spatial positioning method based on low-light environments according to claim 1, characterized in that, Step T110 includes the following sub-steps: Images are captured inside the multi-sensory immersive experience cabin. The average brightness of all pixels in the images is calculated. Based on the average brightness and the parameters of the visual camera used to capture the images, the ambient light intensity is determined. When the ambient light intensity is detected to be continuously lower than the light intensity threshold for a preset number of frames, it is determined that the current multi-sensory immersive experience cabin is in a dark environment, and the system switches from the standard visual positioning mode to the dark light positioning mode.

3. The hybrid spatial positioning method based on low-light environments according to claim 1 or 2, characterized in that, Step T120 includes the following sub-steps: Adaptive enhancement is performed on each frame of image and basic dark light features are extracted. The basic dark light features of the higher layers are upsampled and fused with the basic dark light features of the lower layers to obtain basic dark light fused features. Then, the basic dark light fused features of each layer are fused and scaled to obtain dark light edge features. Each frame of the image is preprocessed and converted to the frequency domain to obtain a frequency domain image. The frequency domain image is then centered, and noise is separated from the centered frequency domain image by low-pass filtering. The centered frequency domain image after low-pass filtering is then inversely centered to recover the original distribution of the frequency domain stable features in the frequency domain. Finally, the original distribution of the frequency domain stable features in the frequency domain is converted back to the spatial domain by inverse discrete Fourier transform to obtain the frequency domain stable features.

4. The hybrid spatial positioning method based on low-light environments according to claim 1 or 2, characterized in that, Step T130 includes the following sub-steps: The image is converted to grayscale, the grayscale image is denoised, and the original edge gradient magnitude is extracted based on the denoised grayscale image. IMU motion data is converted into displacement of the image pixel plane to obtain pixel displacement prediction values. A motion weight map with the same size as the image is generated based on the pixel displacement prediction values. The original edge gradient magnitude is multiplied by the motion weight map to filter out edge features that are strongly correlated with IMU motion. These edge features are then fused through a 3-layer edge feature pyramid network to obtain the IMU motion edge features.

5. The hybrid spatial positioning method based on low-light environments according to claim 1 or 2, characterized in that, Step T140 includes the following sub-steps: The dark light edge features, frequency domain stability features and IMU motion edge features are aligned in a unified dimension and space. After unification and spatial alignment, the dark light edge features, frequency domain stability features and IMU motion edge features are initially fused according to the corresponding fusion weights. The importance weights are multiplied by the initial fused features to obtain the multimodal fusion features of the current frame. The multimodal fusion features of the current frame and the multimodal fusion features of historical frames are accumulated in time to obtain the multimodal fusion features after time accumulation. Weak feature enhancement processing is performed on the time-accumulated multimodal fusion features to obtain enhanced multimodal fusion features.

6. A hybrid spatial positioning system based on low-light environments, characterized in that, include: Visual camera, inertial measurement unit, and processor; Among them, the visual camera captures images inside the multi-sensory immersive experience cabin, the processor detects the ambient light intensity through the images, and enters the dark light positioning mode when the ambient light intensity is lower than the light intensity threshold. The visual camera acquires multiple frames of images in low-light positioning mode, and the processor extracts low-light edge features and frequency domain stability features from the images. The inertial measurement unit synchronously acquires IMU motion data corresponding to each frame of image, and the processor obtains IMU motion edge features based on the image and IMU motion data; The processor fuses dark light edge features, frequency domain stability features, and IMU motion edge features to obtain single-frame multimodal fusion features, performs temporal accumulation on the multimodal fusion features of consecutive frames, and performs weak feature enhancement on the temporally accumulated multimodal fusion features. The processor will input enhanced multimodal fusion features and IMU motion data into the Kalman filter fusion model to calculate the three-dimensional coordinates of the spatial anchor point; When the processor detects that the edge features and frequency domain stability features in dark light are completely lost, it switches to pure inertial prediction mode, inputs the IMU motion data into the Kalman filter fusion model to solve the three-dimensional coordinates of the spatial anchor point, and continuously detects the ambient light intensity in the multi-sensory immersive experience cabin.

7. The hybrid spatial positioning system based on low-light environments according to claim 1, characterized in that, The processor calculates the average brightness of all pixels in the image, and determines the ambient light intensity based on the average brightness and the parameters of the visual camera that acquired the image. When the processor detects that the ambient light intensity is continuously lower than the light intensity threshold for a preset number of frames, the processor determines that the current multi-sensory immersive experience cabin is in a dark environment, and then switches from the standard visual positioning mode to the dark light positioning mode.

8. The hybrid spatial positioning system based on low-light environments according to claim 1 or 2, characterized in that, The processor adaptively enhances each frame of image and extracts basic dark light features. It upsamples the basic dark light features of the higher layers and fuses them with the basic dark light features of the lower layers to obtain basic dark light fused features. Then, it fuses and scales the basic dark light fused features of each layer to obtain dark light edge features. The processor preprocesses each frame of image, converts the preprocessed image to the frequency domain to obtain a frequency domain image, centers the frequency domain image, and then separates the noise from the centered frequency domain image through low-pass filtering. The centered frequency domain image after low-pass filtering is inversely centered to recover the original distribution of frequency domain stable features in the frequency domain. Finally, the original distribution of frequency domain stable features in the frequency domain is converted back to the spatial domain through inverse discrete Fourier transform to obtain frequency domain stable features.

9. The hybrid spatial positioning system based on low-light environments according to claim 1 or 2, characterized in that, The processor converts the image to grayscale, denoises the grayscale image, and extracts the original edge gradient magnitude based on the denoised grayscale image. The processor converts IMU motion data into displacements of image pixel planes to obtain pixel displacement predictions. Based on the pixel displacement predictions, a motion weight map with the same size as the image is generated. The original edge gradient magnitude is multiplied by the motion weight map to filter out edge features that are strongly correlated with IMU motion. The edge features that are strongly correlated with IMU motion are fused through a 3-layer edge feature pyramid network to obtain IMU motion edge features.

10. The hybrid spatial positioning system based on low-light environments according to claim 1 or 2, characterized in that, The processor unifies the dimensions and spatial alignment of dark light edge features, frequency domain stable features, and IMU motion edge features. After unifying the dimensions and spatial alignment, the dark light edge features, frequency domain stable features, and IMU motion edge features are initially fused according to their respective fusion weights. The importance weights are multiplied by the initial fused features to obtain the current frame multimodal fusion features. The current frame multimodal fusion features are temporally accumulated with the historical frame multimodal fusion features to obtain the temporally accumulated multimodal fusion features. Weak feature enhancement processing is performed on the temporally accumulated multimodal fusion features to obtain enhanced multimodal fusion features.

Images