Robot space positioning and map reconstruction system based on binocular vision and inertial navigation

Abstract

The invention relates to the technical field of robots, and discloses a robot space positioning and map reconstruction system based on binocular vision and inertial navigation, and the system comprises a binocular camera module which is used for obtaining a stereo image of a surrounding environment; and the inertial measurement unit is used for providing acceleration and angular velocity information. According to the robot space positioning and map reconstruction system based on binocular vision fused inertial navigation, through fusion of the binocular camera and the IMU, the positioning precision and robustness of the system are improved, the system adopts a deep learning algorithm and an optimized feature matching algorithm, the accuracy and reliability of feature extraction and matching are improved, and the system is suitable for popularization and application. According to the system, the visual feature points and IMU data are combined, high-precision positioning is achieved, meanwhile, system parameters are adjusted through the reinforcement learning technology, the intelligence and adaptability of the system are improved, remote transmission and sharing of data are achieved through the wireless communication module, and the manageability and maintainability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a robot space positioning and map reconstruction system based on binocular vision fusion inertial navigation. Background Art

[0002] The robot spatial positioning and mapping system, commonly known as the SLAM (Simultaneous Localization And Mapping) system, refers to the technology of a robot that simultaneously determines its own position (positioning) and constructs a map of the surrounding environment (mapping) when moving in an unknown environment. SLAM is a key issue in robotics and is essential for applications such as autonomous navigation and environmental exploration. However, the existing robot spatial positioning and mapping system based on binocular vision and IMU has deficiencies in environmental adaptability, long-term operation stability, energy consumption management, and autonomous learning ability. Under different lighting conditions, the binocular vision system cannot effectively extract feature points, resulting in positioning and map reconstruction failures. Moreover, the drift error of the IMU will gradually accumulate over time, resulting in a decrease in positioning accuracy. Summary of the invention

[0003] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation, the system includes:

[0006] Binocular camera module, used to obtain stereo images of the surrounding environment;

[0007] Inertial measurement unit, which provides acceleration and angular velocity information;

[0008] A data processing unit, used to process data from the binocular camera and IMU;

[0009] A storage unit for storing processed data and constructed maps;

[0010] Power management module for power supply and energy efficiency management;

[0011] Wireless communication module, used to exchange data with other devices or cloud platforms;

[0012] The data processing unit further includes a feature extraction submodule that uses a deep learning algorithm to automatically identify and extract key feature points in the environment;

[0013] The data processing unit also includes a feature matching submodule, which matches feature points under different viewing angles through an optimized algorithm;

[0014] The data processing unit also includes a fusion positioning submodule, which combines visual feature points and IMU data and uses an extended Kalman filter to achieve accurate positioning;

[0015] The data processing unit also includes an adaptive learning submodule, which uses reinforcement learning technology to adjust system parameters to adapt to different environmental conditions.

[0016] As a further solution of the present invention: the data processing unit further comprises:

[0017] The feature extraction submodule uses a deep learning algorithm to automatically identify and extract key feature points in the environment. The specific steps are as follows:

[0018] Use convolutional neural network to detect feature points in images acquired by binocular camera;

[0019] The most significant feature points are selected through the non-maximum suppression algorithm;

[0020] Generate descriptors of feature points using scale-invariant feature transformation algorithm;

[0021] The feature matching submodule matches feature points from different perspectives through an optimized algorithm. The specific steps include:

[0022] Use random sample consistency algorithm to remove mismatched feature points;

[0023] Calculate the correspondence between feature points through the homography matrix;

[0024] The fusion positioning submodule combines visual feature points and IMU data and uses the extended Kalman filter to achieve accurate positioning. The specific steps include:

[0025] Initialize the state vector and covariance matrix;

[0026] Prediction step: predict the state at the next moment based on IMU data;

[0027] Update step: Correct the prediction results according to the visual feature point observations;

[0028] The map construction submodule dynamically updates the environment map based on the positioning results. The specific steps include:

[0029] Convert the positioning results into points in the map coordinate system;

[0030] Use octree structure to store and manage map data;

[0031] Regularly optimize the map to remove redundant points and noise;

[0032] The adaptive learning submodule uses reinforcement learning technology to adjust system parameters to adapt to different environmental conditions. The specific steps include:

[0033] Define a reward function to give rewards or penalties based on positioning accuracy and map quality;

[0034] Update the strategy using the Q-learning algorithm;

[0035] Dynamically adjust parameters for feature extraction and matching.

[0036] As a further solution of the present invention: the system further includes an environmental perception enhancement module, which integrates multiple sensors, including infrared and ultrasonic, to assist the binocular vision system to work in low-light or texture-free areas. The specific steps include:

[0037] Use infrared sensors to detect low-light areas;

[0038] Use ultrasonic sensors to detect texture-free areas;

[0039] The sensor data is integrated with the binocular vision data to improve the detection and matching accuracy of feature points.

[0040] As a further solution of the present invention: the power management module adopts energy recovery technology, including solar panels and motion power generation devices, to extend the endurance of the system. The specific steps include:

[0041] Install solar panels to use solar energy to power the system;

[0042] Install a motion power generation device to convert the mechanical energy of the robot into electrical energy when it moves;

[0043] Dynamically allocate and manage power through the power management system to optimize energy efficiency.

[0044] As a further solution of the present invention: the system also includes a safety mechanism, which can automatically take measures to protect the system when an abnormal situation is detected. The specific steps include:

[0045] Monitor system temperature and start cooling fans or reduce system load when the temperature exceeds a preset threshold;

[0046] Monitor battery power and automatically enter low-power mode or return to the charging station when power falls below a preset threshold;

[0047] In the event of a serious failure, all operations are automatically stopped and an alarm is sent.

[0048] As a further solution of the present invention: the feature extraction submodule uses a convolutional neural network to detect feature points, which is specifically as follows:

[0049] Input image: The left and right images acquired by the binocular camera are denoted as IL and IR respectively;

[0050] Feature extraction: Use a pre-trained convolutional neural network to extract the image feature maps FL and FR;

[0051] Adaptive attention mechanism: Enhance the feature representation of key areas through the AAM module;

[0052] Attention weight calculation: ;

[0053] Among them, W1 and W2 are learnable weight matrices, b1 and b2 are bias terms, and σ is the sigmoid activation function;

[0054] Feature weighting: ;

[0055] Among them, ⊙ represents element-by-element multiplication;

[0056] Feature point detection: Feature point detection algorithm is used to extract feature points on the weighted feature maps FL′ and FR′.

[0057] As a further solution of the present invention: the specific process of selecting the most significant feature points by the non-maximum suppression algorithm is as follows:

[0058] Response value calculation: For each feature point p, calculate its response value R(p);

[0059] Local maximum detection: In a 3×3 or 5×5 neighborhood, if R(p) is a local maximum, the feature point is retained;

[0060] Threshold filtering: Set a threshold T and only retain feature points whose response values ​​are greater than T.

[0061] As a further solution of the present invention: the specific process of generating the descriptor of the feature points by using the scale-invariant feature transformation algorithm is as follows:

[0062] Scale space construction: construct the scale space L(x,y,σ) of the image;

[0063] Key point detection: Detect extreme points in scale space as key points;

[0064] Key point positioning: Accurately locate the position and scale of key points by fitting quadratic functions;

[0065] Direction assignment: Calculate the gradient direction histogram around the key point and assign the main direction to each key point;

[0066] Descriptor generation: A 16×16 region is extracted around the key point and divided into 4×4 sub-regions. The gradient histogram in 8 directions is calculated for each sub-region, and finally a 128-dimensional descriptor is generated.

[0067] As a further solution of the present invention: the fusion positioning submodule combines visual feature points and IMU data and uses an extended Kalman filter to achieve precise positioning. The specific process is as follows:

[0068] State vector: The initial state vector x0 includes position, velocity and attitude;

[0069] Covariance matrix: The initial covariance matrix P0 reflects the uncertainty of the initial state;

[0070] Prediction step: Predict the state at the next moment based on IMU data:

[0071] State transfer equation: ;

[0072] Where uk is the IMU measurement value and wk is the process noise;

[0073] Covariance prediction: ;

[0074] Among them, Fk is the state transfer matrix, Qk is the process noise covariance matrix;

[0075] Update step: Correct the prediction results according to the visual feature point observations;

[0076] Observation equation: ;

[0077] Among them, zk is the visual feature point observation value, vk is the observation noise;

[0078] Kalman gain: ;

[0079] Where Hk is the observation matrix, Rk is the observation noise covariance matrix;

[0080] Status Update: ;

[0081] Covariance update: .

[0082] As a further solution of the present invention: in the adaptive learning submodule, a reward function is defined to give rewards or penalties according to the positioning accuracy and map quality, as follows:

[0083] Reward function: ;

[0084] Among them, w1, w2, w3 are weight coefficients;

[0085] Update the policy using the Q-learning algorithm:

[0086] State representation: The system state is represented as s, including the current positioning accuracy, map quality and computational cost;

[0087] Action representation: define the action set A, including adjusting the parameters for feature extraction and matching;

[0088] Q value update: ;

[0089] Among them, α is the learning rate and γ is the discount factor;

[0090] Dynamically adjust the parameters of feature extraction and matching:

[0091] Parameter selection: select the optimal action a according to the Q value;

[0092] Parameter update: adjust the parameters of feature extraction and matching.

[0093] Compared with the prior art, the present invention has the following beneficial effects:

[0094] 1. The present invention improves the accuracy and robustness of system positioning through the fusion of binocular cameras and IMUs. The system adopts a deep learning algorithm and an optimized feature matching algorithm to improve the accuracy and reliability of feature extraction and matching. It combines visual feature points and IMU data to achieve high-precision positioning. At the same time, it adjusts system parameters through reinforcement learning technology to improve the intelligence and adaptability of the system. It realizes remote transmission and sharing of data through the wireless communication module, improves the manageability and maintainability of the system, and the storage unit provides complete data management and map building functions to improve the reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 This is a system connection diagram of the robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation. DETAILED DESCRIPTION

[0096] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0097] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0098] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0099] See also Figure 1 , is an embodiment of the present invention, which provides a robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation, including:

[0100] Binocular camera module: used to obtain stereo images of the surrounding environment;

[0101] Inertial measurement unit: used to provide acceleration and angular velocity information;

[0102] Data processing unit: used to process data from the binocular camera and IMU;

[0103] Storage unit: used to store processed data and constructed maps;

[0104] Power management module: used for power supply and energy efficiency management;

[0105] Wireless communication module: used to exchange data with other devices or cloud platforms;

[0106] Data processing unit:

[0107] Feature extraction submodule:

[0108] Input image:

[0109] The left and right images acquired by the binocular camera are denoted as IL and IR respectively, assuming that the image size is 640 × 480 pixels;

[0110] Feature extraction:

[0111] Use the pre-trained convolutional neural network to extract the feature maps FL and FR of the image;

[0112] ;

[0113] The size of the feature map is 160×120 pixels and the number of channels is 2048;

[0114] Adaptive Attention Mechanism:

[0115] Calculate the attention weights: ;

[0116] Among them, W1 and W2 are learnable weight matrices with sizes of 2048×512 and 512×1 respectively, b1 and b2 are bias terms, and σ is the sigmoid activation function;

[0117] Feature weighting: ;

[0118] Among them, ⊙⊙ represents element-by-element multiplication;

[0119] Feature point detection:

[0120] Harris corner detection algorithm is used to extract feature points on the weighted feature maps FL′ and FR′. Assume that 1000 feature points are extracted from each image.

[0121] Feature matching submodule:

[0122] Response value calculation:

[0123] For each feature point p, calculate its response value R(p);

[0124] Local maximum detection:

[0125] In a 3×3 neighborhood, if R(p) is a local maximum, the feature point is retained;

[0126] Threshold filtering:

[0127] Set a threshold T=0.01, and only retain feature points with response values ​​greater than TT. Assume that 500 feature points are retained after filtering;

[0128] Feature Matching:

[0129] Use the random sample consistency algorithm to remove the mismatched feature points. Assume that after RANSAC, 300 matching points are retained;

[0130] The correspondence between feature points is calculated through the homography matrix:

[0131] ;

[0132] Among them, pi and qi are matching feature point pairs, n=300;

[0133] Fusion positioning submodule:

[0134] Initialize the state vector and covariance matrix:

[0135] Initial state vector , assuming that the initial position is (0,0,0), the initial velocity is (0,0,0), the initial posture is (0,0,0), and the initial covariance matrix P0P0 is a diagonal matrix with diagonal elements of [1,1,1,0.1,0.1,0.1,0.01,0.01,0.01];

[0136] Prediction steps:

[0137] Predict the state at the next moment based on IMU data:

[0138] ;

[0139] Where uk is the IMU measurement value, assuming that the acceleration is (0.1, 0.1, 0.1), the angular velocity is (0.01, 0.01, 0.01), and the process noise wk is zero-mean Gaussian white noise;

[0140] Covariance prediction: ;

[0141] Where Fk is the state transfer matrix, which is assumed to be the identity matrix; Qk is the process noise covariance matrix, and the diagonal elements are [0.01, 0.01, 0.01, 0.001, 0.001, 0.001, 0.0001, 0.0001, 0.0001];

[0142] Update steps:

[0143] Correct the prediction results based on the visual feature point observations:

[0144] Observation equation: ;

[0145] Among them, zk is the observation value of the visual feature point, and the observation noise vk is zero-mean Gaussian white noise;

[0146] Kalman gain: ;

[0147] Where Hk is the observation matrix, assumed to be the identity matrix, and Rk is the observation noise covariance matrix, with diagonal elements of [0.1, 0.1, 0.1, 0.01, 0.01, 0.01, 0.001, 0.001, 0.001]

[0148] Map building submodule:

[0149] Convert the positioning result into a point in the map coordinate system:

[0150] Convert the robot's position (x, y, z) in the world coordinate system to a point (x′, y′, z′) in the map coordinate system;

[0151] Use the octree structure to store and manage map data:

[0152] Insert the point cloud data into the octree, each node represents a cubic space, assuming that the initial node size is 1×1×1 meter;

[0153] When the number of points in a node exceeds 100, the node is split into 8 child nodes;

[0154] When the number of points in a node is less than 10, merge the adjacent empty nodes;

[0155] Regularly optimize the map to remove redundant points and noise:

[0156] Use statistical filtering or voxel filtering to remove noise points, assuming that the filter window size is 0.1×0.1×0.1 meters;

[0157] Use random sampling or uniform sampling to reduce redundant points, assuming that the sampling interval is 0.5 meters;

[0158] Use the ICP algorithm to optimize the alignment and fusion of point clouds, assuming that the maximum number of iterations is 50;

[0159] Adaptive learning submodule:

[0160] Define the reward function: ;

[0161] Among them, the positioning accuracy is 0.1 meter, the map quality is 0.9, and the calculation cost is 0.05;

[0162] Update the policy using the Q-learning algorithm:

[0163] State representation: The system state is represented as s, including the current positioning accuracy, map quality and computational cost;

[0164] Action representation: define the action set A, including adjusting the parameters for feature extraction and matching;

[0165] Q value update: ;

[0166] Among them, the learning rate α=0.1, the discount factor γ=0.9;

[0167] Dynamically adjust the parameters of feature extraction and matching:

[0168] Parameter selection: select the optimal action a according to the Q value;

[0169] Parameter update: adjust the parameters of feature extraction and matching;

[0170] Environmental perception enhancement module:

[0171] Using infrared sensors to detect low-light areas:

[0172] The infrared sensor detects the light intensity in the environment. When the light intensity is lower than 10lux, the infrared auxiliary mode is enabled.

[0173] Detecting texture-free areas using ultrasonic sensors:

[0174] The ultrasonic sensor detects the texture density in the environment. When the texture density is lower than 0.1, the ultrasonic assistance mode is enabled.

[0175] Fusion of sensor data with binocular vision data:

[0176] Fuse the data from infrared and ultrasonic sensors with binocular vision data to improve the detection and matching accuracy of feature points;

[0177] Power Management Module:

[0178] Install solar panels to use solar energy to power the system:

[0179] The solar panel is installed on the top of the robot, and the solar energy is converted into electrical energy through the solar controller. Assume that the maximum output power of the solar panel is 100W;

[0180] Install a motion power generation device to convert the mechanical energy of the robot into electrical energy when it moves:

[0181] The motion generator is installed on the wheel axle of the robot, and the mechanical energy is converted into electrical energy through the generator. Assume that the maximum output power of the motion generator is 50W;

[0182] Dynamically distribute and manage power through the power management system to optimize energy efficiency:

[0183] The power management system monitors the battery power and system load in real time, dynamically adjusts the power supply strategy, and optimizes energy efficiency. Assuming the battery capacity is 10Ah and the average system power consumption is 20W.

[0184] In summary, high-precision positioning and map construction are achieved through multi-sensor fusion, deep learning and optimization algorithms. The system includes binocular cameras, IMUs, data processing units, storage units, power management modules and wireless communication modules. It can automatically extract and match environmental feature points, combine extended Kalman filters for precise positioning, and optimize system parameters through adaptive learning technology to adapt to different environmental conditions. In addition, the system also integrates environmental perception enhancement modules and energy efficiency management modules to improve robustness and endurance, and is suitable for robot navigation and map reconstruction tasks in complex environments.

[0185] Importantly, it should be noted that the construction and arrangement of the present application shown in a number of different exemplary embodiments are only exemplary. Although only a few embodiments are described in detail in this disclosure, it should be readily understood by those who refer to this disclosure that many modifications are possible (e.g., the size, scale, structure, shape and proportion of various elements, and parameter values ​​(e.g., temperature, pressure, etc.), mounting arrangements, use of materials, color, changes in orientation, etc.) without substantially departing from the novel teachings and advantages of the subject matter described in the application. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of the element may be inverted or otherwise changed, and the nature or number or position of the discrete element may be altered or changed. Therefore, all such modifications are intended to be included within the scope of the present invention. The order or sequence of any process or method steps may be changed or reordered according to an alternative embodiment. In the claims, any "device plus function" clause is intended to cover the structure described herein that performs the function, and is not only structurally equivalent but also equivalent structure. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the invention is not limited to a specific embodiment, but extends to numerous modifications still falling within the scope of the appended claims.

[0186] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment may not be described (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those features that are not relevant to implementing the invention).

[0187] It will be appreciated that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will be a routine task of design, fabrication, and production for those of ordinary skill having the benefit of this disclosure without undue experimentation.

[0188] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation, characterized by: The system includes: Binocular camera module, used to obtain stereo images of the surrounding environment; Inertial measurement unit, which provides acceleration and angular velocity information; A data processing unit, used to process data from the binocular camera and IMU; A storage unit for storing processed data and constructed maps; Power management module for power supply and energy efficiency management; Wireless communication module, used to exchange data with other devices or cloud platforms; The data processing unit further includes a feature extraction submodule that uses a deep learning algorithm to automatically identify and extract key feature points in the environment; The data processing unit also includes a feature matching submodule, which matches feature points under different viewing angles through an optimized algorithm; The data processing unit also includes a fusion positioning submodule, which combines visual feature points and IMU data and uses an extended Kalman filter to achieve accurate positioning; The data processing unit also includes an adaptive learning submodule, which uses reinforcement learning technology to adjust system parameters to adapt to different environmental conditions.

2. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 1 is characterized in that: The specific steps of using the deep learning algorithm to automatically identify and extract key feature points in the environment are as follows: Use convolutional neural network to detect feature points in images acquired by binocular camera; The most significant feature points are selected through the non-maximum suppression algorithm; Generate descriptors of feature points using scale-invariant feature transformation algorithm; The feature matching submodule matches feature points from different perspectives through an optimized algorithm. The specific steps include: Use random sample consistency algorithm to remove mismatched feature points; Calculate the correspondence between feature points through the homography matrix; The fusion positioning submodule combines visual feature points and IMU data and uses the extended Kalman filter to achieve accurate positioning. The specific steps include: Initialize the state vector and covariance matrix; Prediction step: predict the state at the next moment based on IMU data; Update step: Correct the prediction results according to the visual feature point observations; The map construction submodule dynamically updates the environment map based on the positioning results. The specific steps include: Convert the positioning results into points in the map coordinate system; Use octree structure to store and manage map data; Regularly optimize the map to remove redundant points and noise; The adaptive learning submodule uses reinforcement learning technology to adjust system parameters to adapt to different environmental conditions. The specific steps include: Define a reward function to give rewards or penalties based on positioning accuracy and map quality; Update the strategy using the Q-learning algorithm; Dynamically adjust parameters for feature extraction and matching.

3. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 1 is characterized in that: The system also includes an environmental perception enhancement module, which integrates multiple sensors, including infrared and ultrasonic, to assist the binocular vision system in working in low-light or texture-free areas. The specific steps include: Use infrared sensors to detect low-light areas; Use ultrasonic sensors to detect texture-free areas; The sensor data is integrated with the binocular vision data to improve the detection and matching accuracy of feature points.

4. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to any one of claims 1 to 3, characterized in that: The power management module uses energy recovery technology, including solar panels and motion power generation devices, to extend the endurance of the system. The specific steps include: Install solar panels to use solar energy to power the system; Install a motion power generation device to convert the mechanical energy of the robot into electrical energy when it moves; Dynamically allocate and manage power through the power management system to optimize energy efficiency.

5. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to any one of claims 1 to 4, characterized in that: The system also includes a safety mechanism that can automatically take measures to protect the system when an abnormal situation is detected. The specific steps include: Monitor system temperature and start cooling fans or reduce system load when the temperature exceeds a preset threshold; Monitor battery power and automatically enter low-power mode or return to the charging station when power falls below a preset threshold; In the event of a serious failure, all operations are automatically stopped and an alarm is sent.

6. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 2 is characterized in that: The feature extraction submodule uses a convolutional neural network to detect feature points, which is specifically as follows: Input image: The left and right images acquired by the binocular camera are denoted as IL and IR respectively; Feature extraction: Use a pre-trained convolutional neural network to extract the image feature maps FL and FR; Adaptive attention mechanism: Enhance the feature representation of key areas through the AAM module; Attention weight calculation: ; Among them, W1 and W2 are learnable weight matrices, b1 and b2 are bias terms, and σ is the sigmoid activation function; Feature weighting: ; Among them, ⊙ represents element-by-element multiplication; Feature point detection: Feature point detection algorithm is used to extract feature points on the weighted feature maps FL′ and FR′.

7. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 2 is characterized in that: The specific process of selecting the most significant feature points by the non-maximum suppression algorithm is as follows: Response value calculation: For each feature point p, calculate its response value R(p); Local maximum detection: In a 3×3 or 5×5 neighborhood, if R(p) is a local maximum, the feature point is retained; Threshold filtering: Set a threshold T and only retain feature points whose response values ​​are greater than T.

8. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 2 is characterized in that: The specific process of generating the descriptor of feature points using the scale-invariant feature transformation algorithm is as follows: Scale space construction: construct the scale space L(x,y,σ) of the image; Key point detection: Detect extreme points in scale space as key points; Key point positioning: Accurately locate the position and scale of key points by fitting quadratic functions; Direction assignment: Calculate the gradient direction histogram around the key point and assign the main direction to each key point; Descriptor generation: A 16×16 region is extracted around the key point and divided into 4×4 sub-regions. The gradient histogram in 8 directions is calculated for each sub-region, and finally a 128-dimensional descriptor is generated.

9. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 2 is characterized in that: The specific process of the fusion positioning submodule, combining visual feature points and IMU data, and using the extended Kalman filter to achieve precise positioning is as follows: State vector: The initial state vector x0 includes position, velocity and attitude; Covariance matrix: The initial covariance matrix P0 reflects the uncertainty of the initial state; Prediction step: Predict the state at the next moment based on IMU data: State transfer equation: ; Where uk is the IMU measurement value and wk is the process noise; Covariance prediction: ; Among them, Fk is the state transfer matrix, Qk is the process noise covariance matrix; Update step: Correct the prediction results according to the visual feature point observations; Observation equation: ; Among them, zk is the visual feature point observation value, vk is the observation noise; Kalman gain: ; Where Hk is the observation matrix, Rk is the observation noise covariance matrix; Status Update: ; Covariance update: .

10. The robot spatial positioning and map reconstruction system based on binocular vision fusion inertial navigation according to claim 2, characterized in that: In the adaptive learning submodule, a reward function is defined to give rewards or penalties based on positioning accuracy and map quality, as follows: Reward function: ; Among them, w1, w2, w3 are weight coefficients; use Algorithm update strategy: State representation: The system state is represented as s, including the current positioning accuracy, map quality and computational cost; Action representation: define the action set A, including adjusting the parameters for feature extraction and matching; Q value update: ; Among them, α is the learning rate and γ is the discount factor; Dynamically adjust the parameters of feature extraction and matching: Parameter selection: select the optimal action a according to the Q value; Parameter update: adjust the parameters of feature extraction and matching.

Images