Roadway heading machine positioning method, device and system based on vision-inertia multistage depth fusion
By employing a vision-inertial multi-level deep fusion positioning method, which combines visual and inertial information, the problem of unstable positioning in tunnel excavation operations has been solved, achieving high-precision and robust positioning of tunnel excavators and supporting intelligent and unmanned operations.
Patent Information
- Application Number
- CN202511769738.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-01-09
AI Technical Summary
In tunnel excavation operations, a single sensor solution is difficult to guarantee the continuity and robustness of positioning, especially under extreme conditions. Visual positioning is greatly affected by the environment but has a small cumulative error, while inertial positioning has high short-term accuracy but serious cumulative error, making it difficult to achieve high-precision and stable autonomous positioning.
A vision-inertial multi-level deep fusion positioning method is adopted. Images are acquired through industrial cameras, and inertial information is acquired through strapdown inertial navigation. Combined with artificial targets and computing systems, an image robust feature extraction model, an inertial positioning model, and a tunnel boring machine pose regression cyclic model are used. The LSTM network is used for deep fusion to achieve high-precision and robust positioning of the tunnel boring machine.
It achieves high-precision and robust online positioning under extreme working conditions, and can maintain reliable positioning by relying on short-term inertial priors in the event of visual failure, thereby reducing the risk of system failure. It has good cross-working-condition generalization ability and supports intelligent and unmanned operation of tunnel boring machines.
Smart Images

Figure CN121297833A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to a roadway tunneling machine positioning method, in particular to a roadway tunneling machine positioning method based on visual-inertial multi-level deep fusion, a device and a system. BACKGROUND
[0002] Roadway tunneling operation belongs to a typical high-risk working condition, and accidents frequently occur in a closed operation environment, large dust and strong vibration. In order to realize unmanned and few-person operation of the tunneling equipment, self-positioning of the tunneling machine with high precision must be carried out. In an extreme tunneling working condition, a single sensor scheme is difficult to ensure the continuity and robustness of positioning, and therefore, the application develops a visual-inertial deep fusion positioning method. In a visual-inertial combined positioning system of the roadway tunneling equipment, time series images continuously acquired have time-space correlation on effective features and interference features, and reasonable use of the time series visual information plays an important role in improving the accuracy and robustness of the current time visual positioning. In addition, the visual and inertial signals are sampled at different frequencies, the visual signal sampling frequency is low, the positioning solution has no cumulative error, and the positioning accuracy is greatly affected by the roadway environment, the inertial signal sampling frequency is high, there is cumulative error, the short-time positioning accuracy is high, and the positioning accuracy is affected by the dynamic of the tunneling equipment itself. Therefore, the application provides a roadway tunneling machine positioning method based on visual-inertial multi-level deep fusion, so as to give full play to the advantages of the visual and inertial positioning methods and improve the positioning performance. SUMMARY
[0003] The application aims to provide a roadway tunneling machine positioning method based on visual-inertial multi-level deep fusion.
[0004] Technical scheme: The positioning device provided by the application comprises an industrial camera, a strapdown inertial navigation system, an artificial target and a computing system. The industrial camera is used for acquiring artificial target images, the strapdown inertial navigation system is used for acquiring inertial information, the artificial target is arranged in a roadway behind the tunneling machine and is used for providing rich information for full-state visual measurement, and the computing system is integrated with a trained positioning model. During tunneling work of the roadway tunneling machine, the computing system estimates the machine body pose online based on the positioning model. The positioning model uses previous visual-inertial positioning comprehensive features and inter-frame short-time inertial positioning features as joint prior information, combines current time visual sequence fusion features, and estimates the current six-degree-of-freedom machine body pose of the roadway tunneling machine online.
[0005] Optionally, the positioning model comprises an image robust feature extraction model, an inertial positioning model and a roadheader pose regression recurrent model, the image robust feature extraction model is used for extracting robust representation of a current sampling image, and forms an image robust representation time sequence with historical image robust representation, and further uses an attention mechanism to obtain a current time visual fusion feature;The inertial positioning model is used for solving an inertial signal sequence, and mapping it to a feature space through a plurality of neural network layers to obtain inter-frame short-time inertial positioning features;The roadheader pose regression recurrent model is an LSTM network structure, in the current time pose estimation, the state vector output by the last time step LSTM unit is used as the previous frame visual-inertial positioning comprehensive feature, the inter-frame short-time inertial features of the current period are respectively spliced to the state vector output by the last time step LSTM unit to obtain the current time pose estimation joint prior information, the joint prior and the current time visual fusion feature are deeply fused in the LSTM unit to obtain the current frame visual-inertial positioning comprehensive feature, and then the current body pose is estimated through the pose regression layer, and the initial state of the inertial positioning model is updated and corrected.
[0006] The positioning method described in the application comprises the following steps:
[0007] Artificial target image-inertial signal-pose label data are obtained in a simulated tunnel environment in a ground laboratory, and the obtained data are augmented to obtain a training data set;
[0008] A visual-inertial multi-level deep fusion positioning model is built, and the model is trained based on the training data set;Wherein the positioning model comprises: an image robust feature extraction model, an inertial positioning model and a roadheader pose regression recurrent model, the image robust feature extraction model is used for extracting robust representation of a current sampling image, and forms an image robust representation time sequence with historical image robust representation, and uses an attention mechanism to obtain a current time visual fusion feature;The inertial positioning model is used for solving an inertial signal sequence, and mapping it to a feature space through a plurality of neural network layers to obtain inter-frame short-time inertial positioning features;The roadheader pose regression recurrent model is an LSTM network structure, in the current time pose estimation, the state vector output by the last time step LSTM unit is used as the previous frame visual-inertial positioning comprehensive feature, the inter-frame short-time inertial features of the current period are respectively spliced to the state vector output by the last time step LSTM unit to obtain the current time pose estimation joint prior information, the joint prior and the current time visual fusion feature are deeply fused in the LSTM unit to obtain the current frame visual-inertial positioning comprehensive feature, and then the current body pose is estimated through the pose regression layer, and the initial state of the inertial positioning model is updated and corrected;
[0009] In actual engineering deployment, according to the actual obtained artificial target image signal and inertial signal, the trained positioning model is used to estimate the body pose of the heading machine online, and the heading machine positioning is realized.
[0010] Further, the image data augmentation method is: using a controllable pose image generation method based on neural radiation field or 3D Gaussian sputtering, taking each image sample as a reference in turn, a plurality of fixed-length image sequence samples reflecting the random motion trend of the moving body are generated;
[0011] The inertial signal augmentation method is: during the image sequence generation process, according to the pose change between image frames, a certain noise is added to simulate the short-time inertial calculation result.
[0012] Further, the model training adopts a strategy combining fixed-length time window processing and small-batch gradient descent.
[0013] Further, the current time visual fusion feature acquisition method is specifically:
[0014] Based on the robust representation of the current sampling image, and reading A history image feature in memory, a time sequence image robust representation sequence is formed, the current sampling image robust representation is used as a query vector, and the attention mechanism is used to complete the time sequence feature fusion based on the current image representation, and the current time visual fusion feature is obtained.
[0015] The positioning system provided by the application comprises:
[0016] A data set making unit is used for obtaining artificial target image-inertial signal-pose label data in a simulated roadway environment in a ground laboratory, and the obtained data is augmented to obtain a training data set;
[0017] The model building and training unit is configured to build a visual-inertial multi-level deep fusion positioning model and train the model based on a training data set; wherein the positioning model comprises an image robust feature extraction model, an inertial positioning model and a roadheader pose regression loop model, the image robust feature extraction model is configured to extract robust features of a current sampling image, and form an image robust feature time sequence together with historical image robust features, and further use an attention mechanism to obtain a current visual fusion feature; the inertial positioning model is configured to solve an inertial signal sequence and map it to a feature space through a plurality of neural network layers to obtain inter-frame short-time inertial positioning features; the roadheader pose regression loop model is an LSTM network structure, in current pose estimation, a state vector output by a last time step LSTM unit is used as a previous frame visual-inertial positioning comprehensive feature, the current time period inter-frame short-time inertial features are spliced to the state vector output by the last time step LSTM unit respectively to obtain a current time pose estimation joint prior information, the joint prior and the current visual fusion feature are deeply fused in the LSTM unit to obtain a current frame visual-inertial positioning comprehensive feature, and then a current body pose is estimated through a pose regression layer, and an initial state of the inertial positioning model is updated and corrected.
[0018] The real-time positioning unit is configured to use the trained positioning model to online estimate the roadheader body pose according to the actual acquired artificial target image signal and inertial signal, and realize the roadheader positioning in actual engineering deployment.
[0019] The application further provides an electronic device, which comprises:
[0020] a memory storing executable program codes;
[0021] a processor coupled with the memory;
[0022] The processor invokes the executable program codes stored in the memory to execute the method.
[0023] The application further provides a non-volatile storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the method.
[0024] The application further provides a computer program product comprising computer programs / instructions, which are executed by a processor to implement the method.
[0025] Beneficial effects: Compared with the prior art, the significant technical effects of the present application are: the method proposed in the present application fully utilizes the complementary advantages of high short-time accuracy of inertial positioning and no cumulative drift of visual positioning, realizes high-precision and strong-robust online positioning in extreme roadway working conditions by constructing visual sequence feature fusion, short-time inertial feature extraction and LSTM-based time series deep fusion model, and still maintains reliable positioning for a certain period of time relying on short-time inertial prior in the case of visual failure such as complete or partial occlusion of artificial targets, reduces the risk of system failure, and has good cross-condition generalization ability due to the use of image robust feature extraction model and the fact that inertial positioning does not depend on the external environment, can be stably migrated in the simulation environment and the actual roadway environment, and provides key technical support for the intelligent and unmanned operation of the roadway boring machine. BRIEF DESCRIPTION OF DRAWINGS
[0026] Figure 1 Deployment schematic diagram of the visual-inertial combined positioning device of the roadway boring machine;
[0027] Figure 2 Flowchart of the positioning method of the present application;
[0028] Figure 3 Schematic diagram of the visual-inertial multi-level deep fusion positioning model structure. DETAILED DESCRIPTION
[0029] The present application will be described in detail below in combination with the drawings and specific embodiments.
[0030] The visual-inertial combined positioning device of the roadway boring machine shown in the present application comprises an industrial camera, a strapdown inertial navigation system, an artificial target and a computing system, the industrial camera and the strapdown inertial navigation system are installed on the body of the boring machine as shown in Figure 1 The structured artificial target is provided in the roadway behind the boring machine, and the artificial target can be composed of LED arrays and laser light sources and the like to provide structured features for visual measurement. The trained positioning model is integrated in the computing system for online positioning. In actual engineering, in order to realize long-distance continuous positioning of the roadway boring machine, when the boring machine runs beyond the effective visual positioning range, the artificial target needs to be moved forward to the station, and other artificial auxiliary measurement precise positioning methods (such as total station) are used to measure the six-degree-of-freedom pose of the artificial target, and the absolute pose state of the boring machine body in the roadway is obtained through coordinate transformation.
[0031] Inertial positioning is unaffected by the external environment and has high short-term accuracy, but suffers from severe long-term cumulative errors. Visual positioning provides globally consistent pose constraints with no cumulative errors, but is greatly affected by environmental changes. To address the complex working conditions and harsh environments of tunnel boring machines, and to fully leverage the advantages of both visual and inertial positioning, this invention proposes a tunnel boring machine positioning method based on multi-level deep fusion of visual and inertial positioning. The core idea is to use the combined visual-inertial positioning features of the previous frame and the short-term inertial positioning features between frames as joint prior information, combined with the current visual fusion features, to estimate the current six-degree-of-freedom pose of the tunnel boring machine online.
[0032] like Figure 2 As shown, the positioning method of the present invention includes the following steps:
[0033] S1, Data Acquisition;
[0034] For deep learning-based tunnel boring machine (TBM) positioning strategies, deep learning models require a large amount of training data. However, it is difficult to obtain sufficient image signal-inertial signal-pose label sequence data pairs in actual underground tunnel excavation environments. Therefore, this invention obtains training data in a simulated tunnel environment in a laboratory setting: In the simulated tunnel environment, an industrial camera and strapdown inertial navigation system are configured on a mobile machine, and an artificial target is placed behind the machine. A motion capture system is used to calibrate the true six-degree-of-freedom pose of the mobile machine at each visual sampling moment. By randomly running multiple trajectories within a certain range, training data for the proposed vision-inertial multi-level deep fusion positioning model is efficiently obtained.
[0035] In addition, to further improve model performance, a training dataset augmentation strategy was designed: In the acquired tunnel boring machine positioning model training dataset, a controllable pose image generation method based on Neural Radiation Field (NeRF) or 3D Gaussian Sputtering (3DGS) was used to generate multiple fixed-length image sequences reflecting a random motion trend of the moving machine, using each image sample as a reference. During sequence generation, the pose change between image frames was monitored. It is known that a certain amount of noise is added to it to simulate short-time inertial calculation results. In this way, a large number of image signal-inertial signal-pose label sequence data pairs can be effectively generated.
[0036] S2. Select a robust image feature extraction model based on self-supervised learning (e.g., a denoising variational autoencoder), and pre-train the model using artificial target images from the acquired training dataset. The trained robust image feature extraction model (e.g., the encoder network of the denoising variational autoencoder) can extract robust representations of the image.
[0037] S3. Build a vision-inertial multi-level deep fusion positioning model (i.e., a tunnel boring machine positioning model based on vision-inertial multi-level deep fusion) and train the model based on the training dataset.
[0038] like Figure 3 As shown, the vision-inertial multi-level deep fusion localization model includes a trained image robust feature extraction model, an inertial localization model, and a tunneling machine pose regression cyclic model.
[0039] The input to the vision-inertial multi-level deep fusion localization model includes visual signals (time-series images). ,in (for the currently acquired image) and inertial signals (within the current and previous frame visual sampling intervals) Here, we assume that the inertial signal sampling frequency is the same as the visual sampling frequency, and that the inertial signal sampling frequency is [number of] inertial sampled values. First, for the inertial signal, the input inertial positioning module uses conventional inertial navigation calculation methods (such as inertial navigation calculation based on the equivalent rotation vector method), based on the initial pose state (position, attitude, velocity) of the tunneling machine body, and the visual sampling interval between the current and previous frames. The displacement change within the visual sampling interval of the tunnel boring machine is calculated from the sampled inertial signal values. and attitude change Furthermore, by mapping these features to a feature space through several neural network layers, short-time inertial positioning features between frames are obtained, fully leveraging the high short-time positioning accuracy of strapdown inertial navigation. Secondly, for visual signals, considering the spatiotemporal correlation of artificial targets and external interference features in continuously acquired images, a time-series image information fusion strategy is used: a trained robust feature extraction model (e.g., an encoder network of a denoising variational autoencoder) is used to extract robust representations of the currently sampled image. and read from memory Historical image features constitute a robust representation sequence of time-series images. Robust characterization of the current sampled image For the query vector, an attention mechanism is used to perform temporal feature fusion based on the current image representation, thereby obtaining the visual fusion features at the current time step. Finally, based on the LSTM network structure (all time steps share the same LSTM unit), an end-to-end tunneling machine pose regression cyclic model is established. In the pose estimation at the current time step, the state vector output by the LSTM unit of the previous time step is used. It can be used as the comprehensive feature of the previous frame's visual-inertial positioning, and the short-time inertial features between frames in the current time period can be stitched together. and The system obtains the joint prior information for the pose estimation at the current moment. Then, it performs deep fusion of the joint prior information and the visual fusion features at the current moment in the LSTM unit to obtain the comprehensive visual-inertial localization features of the current frame. Then, the current body pose is estimated through a pose regression layer (several layers of fully connected neural networks). The method updates and corrects the initial pose state of the inertial positioning model. Through multi-level deep fusion of visual and inertial heterogeneous temporal signals, the positioning accuracy and reliability of the tunnel boring machine can be effectively improved, and the risk of system failure can be reduced. Even under visual failure conditions such as complete occlusion of effective features, the proposed method can still provide effective positioning within a certain period of time.
[0040] To improve training efficiency and stabilize gradient updates, the model training employs a strategy combining fixed-length time windows with mini-batch gradient descent. Thanks to the use of a visually robust feature extraction method and the fact that inertial positioning is independent of the external environment, the trained positioning model can achieve cross-condition generalization and transfer in scenarios such as simulated tunnels and real-world engineering projects.
[0041] S4. In actual engineering deployment, based on the acquired artificial target image signals and inertial signals, a trained positioning model is used to estimate the tunnel boring machine's pose online, thereby achieving tunnel boring machine positioning. This includes:
[0042] (1) Deploy the trained localization model on the computing system, and according to Figure 1 The diagram shows the initial placement of the positioning device in the tunnel to be excavated;
[0043] (2) Use other precise positioning methods for manual measurement (e.g., total station) to detect the pose parameters of the artificial target in the roadway;
[0044] (3) The tunnel boring machine performs tunneling work and uses the trained visual-inertial fusion positioning model to estimate the machine's pose online;
[0045] (4) Continue to excavate forward until it exceeds the effective working range of visual positioning, then move the manual target forward station.
[0046] (5) Repeat steps (2) to (4).
[0047] Based on the same inventive concept, embodiments of the present invention also provide a vision-inertial fusion-based positioning system for tunnel boring machines, comprising:
[0048] The dataset creation unit is used to acquire artificial target images, inertial signals, and pose label data in a simulated tunnel environment in a ground-based laboratory, and to augment the acquired data to obtain a training dataset.
[0049] The model building and training unit is used to build a vision-inertial multi-level deep fusion localization model and train the model based on the training dataset. The localization model includes: an image robust feature extraction model, an inertial localization model, and a tunnel boring machine pose regression recurrent model. The image robust feature extraction model extracts robust representations of the currently sampled image and combines them with historical image robust representations to form an image robust representation time series. An attention mechanism is then used to obtain the visual fusion features at the current moment. The inertial localization model solves the inertial signal sequence and maps it to the feature space through several neural network layers to obtain short-term inertial localization between frames. Position features; the tunneling machine pose regression recurrent model is an LSTM network structure. In the pose estimation at the current time, the state vector output by the LSTM unit at the previous time step is used as the comprehensive visual-inertial positioning feature of the previous frame. The short-term inertial features between frames in the current time period are respectively concatenated to the state vector output by the LSTM unit at the previous time step to obtain the joint prior information of pose estimation at the current time. In the LSTM unit, the joint prior and the visual fusion features at the current time are deeply fused to obtain the comprehensive visual-inertial positioning features of the current frame. Then, the pose of the current machine is estimated through the pose regression layer, and the initial state of the inertial positioning model is updated and corrected.
[0050] In actual engineering deployments, the real-time positioning unit is used to estimate the orientation of the tunneling machine online using a trained positioning model based on the acquired artificial target image signals and inertial signals, thereby achieving tunneling machine positioning.
[0051] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, the device comprising:
[0052] Memory containing executable program code;
[0053] A processor coupled to the memory;
[0054] The processor calls the executable program code stored in the memory to execute the method.
[0055] Based on the same inventive concept, embodiments of the present invention also provide a non-volatile storage medium for storing a computer program, wherein the computer program implements the method described when executed by a processor.
[0056] Based on the same inventive concept, embodiments of the present invention also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described.
Claims
1. A positioning device for a tunnel boring machine based on vision-inertial multi-level deep fusion, characterized in that, include: The system comprises an industrial camera, a strapdown inertial navigation system (SINS), a human target, and a computing system. The industrial camera acquires images of the human target, the SINS acquires inertial information, and the human target is placed in the tunnel behind the tunnel boring machine (TBM) to provide rich information for full-state visual measurement. The computing system integrates a pre-trained positioning model. During the TBM's tunneling operation, the computing system estimates the machine's pose online based on the positioning model. The positioning model uses the combined visual-inertial positioning features of the previous frame and the short-time inertial positioning features between frames as joint prior information, combined with the visual sequence fusion features at the current moment, to estimate the current six-degree-of-freedom pose of the TBM online.
2. The positioning device according to claim 1, characterized in that, The localization model includes an image robust feature extraction model, an inertial localization model, and a tunnel boring machine pose regression recurrent model. The image robust feature extraction model extracts robust representations of the currently sampled image and combines them with historical image robust representations to form an image robust representation time series. An attention mechanism is then used to obtain the visual fusion features at the current moment. The inertial localization model solves the inertial signal sequence and maps it to the feature space through several neural network layers to obtain short-term inertial localization features between frames. The tunnel boring machine pose regression recurrent model uses an LSTM network structure. In the pose estimation at the current moment, the state vector output by the LSTM unit at the previous time step is used as the comprehensive visual-inertial localization feature of the previous frame. The short-term inertial features between frames in the current time period are concatenated to the state vector output by the LSTM unit at the previous time step to obtain the joint prior information for the pose estimation at the current moment. The joint prior and the visual fusion features at the current moment are deeply fused in the LSTM unit to obtain the comprehensive visual-inertial localization features of the current frame. The pose regression layer then estimates the current machine pose and updates and corrects the initial state of the inertial localization model.
3. A method for locating a tunnel boring machine based on vision-inertial multi-level deep fusion, characterized in that, Includes the following steps: Artificial target images, inertial signals, and pose label data were acquired in a simulated tunnel environment in a ground-based laboratory, and the acquired data was augmented to obtain a training dataset. A vision-inertial multi-level deep fusion localization model was built and trained on a training dataset. The localization model includes an image robust feature extraction model, an inertial localization model, and a tunneling machine pose regression loop model. The image robust feature extraction model is used to extract the robust representation of the current sampled image and form an image robust representation time series with the robust representation of historical images. The visual fusion features at the current moment are obtained using an attention mechanism. The inertial positioning model is used to solve the inertial signal sequence and map it to the feature space through several neural network layers to obtain short-term inertial positioning features between frames. The tunneling machine pose regression recurrent model is an LSTM network structure. In the pose estimation at the current time, the state vector output by the LSTM unit at the previous time step is used as the comprehensive visual-inertial positioning feature of the previous frame. The short-term inertial features between frames in the current time period are concatenated to the state vector output by the LSTM unit at the previous time step to obtain the joint prior information of pose estimation at the current time. In the LSTM unit, the joint prior and the visual fusion features at the current time are deeply fused to obtain the comprehensive visual-inertial positioning features of the current frame. Then, the pose regression layer estimates the current machine pose and updates and corrects the initial state of the inertial positioning model. In actual engineering deployment, the tunnel boring machine's pose is estimated online using a trained positioning model based on the acquired artificial target image signals and inertial signals, thereby achieving tunnel boring machine positioning.
4. The positioning method according to claim 3, characterized in that, The method for image data augmentation is as follows: using a controllable pose image generation method based on neural radiation field or 3D Gaussian sputtering, multiple fixed-length image sequence samples reflecting the random motion trend of the moving body are generated sequentially using each image sample as a reference. The method for inertial signal augmentation is as follows: during the image sequence generation process, a certain amount of noise is added to the image based on the pose change between image frames to simulate the short-time inertial calculation results.
5. The positioning method according to claim 3, characterized in that, The model training employs a strategy that combines fixed-length time windows with mini-batch gradient descent.
6. The positioning method according to claim 3, characterized in that, The specific method for obtaining visual fusion features at the current moment is as follows: Robust representation based on the currently sampled image, and read from memory. The historical image features are used to form a robust representation sequence of time series images. Using the robust representation of the currently sampled image as the query vector, an attention mechanism is used to complete the fusion of time series features based on the current image representation, and obtain the visual fusion features at the current moment.
7. A tunnel boring machine positioning system based on vision-inertial multi-level deep fusion, characterized in that, include: The dataset creation unit is used to acquire artificial target images, inertial signals, and pose label data in a simulated tunnel environment in a ground-based laboratory, and to augment the acquired data to obtain a training dataset. The model building and training unit is used to build a vision-inertial multi-level deep fusion localization model and train the model based on the training dataset. The localization model includes: an image robust feature extraction model, an inertial localization model, and a tunnel boring machine pose regression recurrent model. The image robust feature extraction model is used to extract robust representations of the current sampled image and form an image robust representation time series with the robust representations of historical images. Further attention mechanism is used to obtain the visual fusion features at the current moment. The inertial localization model is used to solve the inertial signal sequence and map it to the feature space through several neural network layers to obtain short-term inertial localization features between frames. The tunnel boring machine pose regression recurrent model is an LSTM network structure. In the pose estimation at the current moment, the state vector output by the LSTM unit of the previous time step is used as the comprehensive visual-inertial localization feature of the previous frame. The short-term inertial features between frames of the current time period are concatenated to the state vector output by the LSTM unit of the previous time step to obtain the joint prior information of the pose estimation at the current moment. The joint prior and the visual fusion features at the current moment are deeply fused in the LSTM unit to obtain the comprehensive visual-inertial localization features of the current frame. Then, the pose regression layer estimates the current machine pose and updates and corrects the initial state of the inertial localization model. In actual engineering deployments, the real-time positioning unit is used to estimate the orientation of the tunneling machine online using a trained positioning model based on the acquired artificial target image signals and inertial signals, thereby achieving tunneling machine positioning.
8. An electronic device, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor invokes the executable program code stored in the memory to execute the method according to any one of claims 3 to 6.
9. A non-volatile storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method as described in any one of claims 3 to 6.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 3 to 6.
Citation Information
Cited By
Intelligent sensing robot system for driving working face
CN121733633A