A visual-inertial dynamic target tracking slam device, method, computer and storage medium

Abstract

The application discloses a visual-inertial dynamic target tracking SLAM device, method, computer and storage medium, and relates to the field of artificial intelligence. The application solves the problem that the prior art consumes excessive computing resources to obtain dynamic target semantic information and is not suitable for a SLAM system with high real-time requirement. The device comprises an input unit, a tracking thread unit, a mapping thread unit and an output unit. The input unit is used for obtaining image information and IMU data. The tracking thread unit is used for performing pose estimation according to the obtained image information and IMU data, and obtaining camera and dynamic object pose information. The mapping thread unit is used for performing optimization of the camera and dynamic object pose information according to a local and global batch optimization method, and obtaining a local and global map. The output unit is used for outputting the obtained global map. The application is applied to the field of robot autonomous positioning and dynamic target tracking.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence. Background Technology

[0002] To meet the practical application needs of robots in real-world environments, an increasing number of scholars are focusing on dynamic SLAM systems and have proposed numerous algorithms to solve the self-localization problem in dynamic environments. These algorithms can be broadly categorized into three types. The first type aims to filter dynamic information in the environment and utilize static information to achieve system self-localization. An existing technology proposes the RDS-SLAM algorithm. This method uses different semantic segmentation methods to extract dynamic objects and proposes a keyframe selection strategy to obtain the most up-to-date semantic information. DP-SLAM, proposed by Li et al., combines geometric constraints and semantic segmentation results to filter out dynamic feature points in the Bayesian probability estimation framework and repairs the background occlusion caused by detected dynamic objects. These state-of-the-art solutions all achieve robust localization estimation by filtering dynamic information. However, recent research has shown that dynamic information has potential benefits for SLAM system self-localization and plays a crucial role in robot planning and obstacle avoidance tasks.

[0003] The second category of algorithms is object-based SLAM systems. These algorithms typically require special modeling and representation of dynamic objects. Existing technology provides a novel multi-instance dynamic RGBD-SLAM system that uses an object-level octree-based volume representation. It can provide powerful camera tracking in dynamic environments while continuously estimating the motion properties of any object in the scene. Yang et al. used a cuboid-based modeling approach. CubeSLAM proposes a method for single-image 3D cuboid object detection and multi-viewpoint object simultaneous localization and mapping in both static and dynamic environments, and proves that these two parts can improve upon each other. The prerequisite for implementing this type of method is the need for specific modeling of dynamic objects, which inevitably makes these algorithms lack scalability and generality.

[0004] The third type of algorithm integrates target tracking and SLAM systems. These algorithms, as extensions of traditional SLAM methods, fully perceive dynamic scene information. A visual SLAM system for stereo and RGB-D configurations was developed, integrating multi-target tracking functionality. DynaSLAM II utilizes instance semantic segmentation and ORB features to track dynamic objects. Zhang et al. proposed a novel method for jointly optimizing SE(3) motion and optical flow using semantic segmentation and optical flow estimation, namely Robust Ego and Object6-DOF Motion Estimation and Tracking.

[0005] All of these methods consume excessive computational resources to obtain dynamic target semantic information, and are not suitable for SLAM systems with high real-time requirements. Summary of the Invention

[0006] This invention solves the problem that existing technologies consume excessive computing resources to obtain dynamic target semantic information, making them unsuitable for SLAM systems with high real-time requirements.

[0007] This invention provides a visual-inertial dynamic target tracking (SLAM) device, the device comprising:

[0008] Input unit, tracking thread unit, graph building thread unit, and output unit;

[0009] The input unit is used to acquire image information and IMU data;

[0010] The tracking thread unit is used to perform pose estimation based on the acquired image information and IMU data, and to obtain the pose information of the camera and the dynamic object.

[0011] The mapping thread unit is used to optimize the pose information of the camera and dynamic objects according to the local and global batch optimization methods, and to obtain local and global maps.

[0012] The output unit is used to output the acquired global map.

[0013] Furthermore, a preferred embodiment is also provided, wherein the input unit includes: a monocular image input module and an IMU data input module;

[0014] The monocular image input module is used to acquire monocular image information and transmit the monocular image information to the tracking thread unit;

[0015] The IMU data input module is used to acquire IMU data information and transmit the IMU data information to the tracking thread unit.

[0016] Furthermore, a preferred embodiment is also provided, wherein the tracking thread unit includes:

[0017] Information extraction module, camera pose estimation module, and dynamic object pose estimation module;

[0018] The information extraction module is used to extract static structures and dynamic objects in the environment based on monocular image information and IMU data information.

[0019] The camera pose estimation module is used to perform camera pose estimation on the static structure.

[0020] The dynamic object pose estimation module is used to perform six-degree-of-freedom pose estimation on the dynamic object.

[0021] Furthermore, a preferred embodiment is also provided, wherein the extraction of dynamic objects in the environment specifically includes:

[0022] Evaluate the movement probability P(X) of the feature point i |d i ):

[0023]

[0024] Where α is the influence factor of smoothing the instantaneous calculation result, and d i The distance from the feature to the epipolar line;

[0025] Set a dynamic feature point threshold based on the movement probability of the feature points;

[0026] The image is segmented using the Yolact++ real-time segmentation method to obtain dynamic feature points within the segmented target region;

[0027] The dynamic target is determined based on the proportion of dynamic feature points in the segmented target area to the total number of feature points.

[0028] Furthermore, a preferred embodiment is also provided, wherein the camera pose estimation based on the static structure specifically comprises:

[0029]

[0030] in, Let k be the camera pose at time k. Let k be the camera pose at time k-1. This refers to the camera pose between two adjacent frames.

[0031] Furthermore, a preferred embodiment is also provided, wherein the six-degree-of-freedom pose estimation based on the dynamic object specifically includes:

[0032]

[0033] in, o X k (j) represents the coordinates of the j-th feature point in the world coordinate system at time k. Let the j-th feature point be the motion pose at time k. Let the j-th feature point be the motion pose at time k-1. o X k-1 (j) represents the coordinates of the j-th feature point in the world coordinate system at time k-1;

[0034] The motion speed of a dynamic object is:

[0035]

[0036] in, o X k Let k be the coordinates in the world coordinate system at time k. o X k-1 Δt represents the coordinates in the world coordinate system at time k-1, and Δt represents the time interval between two adjacent frames.

[0037] Furthermore, a preferred embodiment is also provided, wherein the optimization of pose information based on the local and global batch optimization method specifically includes:

[0038]

[0039] Where, θ * The result of batch optimization is the final 3D point coordinates, camera pose, and dynamic object motion state output by the system. ρ is the robust kernel function, k is the time step, and r is the time step. k,k+1 For the IMU inertial residual, e( l D k-1,k ) represents the smooth motion error of the dynamic object, j represents the j-th feature point, l represents the l-th dynamic object, and n z Let n be the total number of static structural position factors, and n0 be the total number of IMU inertia factors. m n is the total number of motion factors of a dynamic object. s This represents the total number of smooth motion factors.

[0040] Based on the same concept, the present invention also provides a visual-inertial dynamic target tracking SLAM method, the method comprising:

[0041] Acquire image information and IMU data;

[0042] Pose estimation is performed based on the acquired image information and IMU data to obtain the pose information of the camera and dynamic objects.

[0043] The pose information is optimized using local and global batch optimization methods to obtain local and global maps.

[0044] Based on the same concept, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the visual-inertial dynamic target tracking SLAM method described above.

[0045] Based on the same concept, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the aforementioned visual-inertial dynamic target tracking SLAM method.

[0046] The advantages of this invention are:

[0047] This invention solves the problem that existing technologies consume excessive computing resources to obtain dynamic target semantic information, making them unsuitable for SLAM systems with high real-time requirements.

[0048] The visual-inertial dynamic target tracking SLAM device described in this invention solves the problem that existing technologies rely entirely on semantic information and cannot overcome challenging scenarios. By combining IMU measurement data, visual information, and relevant matching point labels, it can quickly acquire dynamic objects in a scene without relying entirely on semantic information. Furthermore, it can accurately and robustly extract dynamic objects in challenging scenes, such as partially occluded and distant objects.

[0049] The visual-inertial dynamic target tracking SLAM device described in this invention avoids the problem of modeling dynamic objects required in existing technologies. It represents the changes in the pose of a dynamic rigid body only through points on the dynamic object, without prior object modeling. Furthermore, in subsequent object motion estimation, the object's 3D pose does not need to be used as a random variable. This accelerates target acquisition and saves time.

[0050] In a preferred embodiment of this invention, the problem of persistent self-localization accuracy in existing technologies that merely filter out dynamic information from the environment is addressed. A factor graph optimization framework is proposed. This framework is applied to local and global batch optimization components to optimize camera pose and object motion states. By adding the motion states of dynamic objects as constraints to batch optimization, the self-localization accuracy and dynamic target tracking performance of the system can be improved.

[0051] This invention is applied to the fields of robot autonomous localization and dynamic target tracking. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of a visual-inertial dynamic target tracking SLAM device as described in Embodiment 1;

[0053] Figure 2 This is a schematic diagram of the camera pose estimation module and the dynamic object pose estimation module for target tracking as described in Implementation Method Six;

[0054] Figure 3 This is the factor graph for batch optimization as described in Implementation Method Seven. Detailed Implementation

[0055] To make the technical solutions and advantages of the present invention clearer, several embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. However, the embodiments described below are only a few preferred embodiments of the present invention and are not intended to limit the invention.

[0056] Implementation Method 1, see Figure 1 This embodiment describes a visual-inertial dynamic target tracking (SLAM) device, which includes:

[0057] Input unit, tracking thread unit, graph building thread unit, and output unit;

[0058] The input unit is used to acquire image information and IMU data;

[0059] The tracking thread unit is used to perform pose estimation based on the acquired image information and IMU data, and to obtain the pose information of the camera and the dynamic object.

[0060] The mapping thread unit is used to optimize the pose information of the camera and dynamic objects according to the local and global batch optimization methods, and to obtain local and global maps.

[0061] The output unit is used to output the acquired global map.

[0062] The visual-inertial dynamic target tracking SLAM device described in this invention solves the problem that existing technologies rely entirely on semantic information and cannot overcome challenging scenarios. By combining IMU measurement data, visual information, and relevant matching point labels, it can quickly acquire dynamic objects in a scene without relying entirely on semantic information. Furthermore, it can accurately and robustly extract dynamic objects in challenging scenes, such as partially occluded and distant objects.

[0063] The visual-inertial dynamic target tracking SLAM device described in this invention avoids the problem of modeling dynamic objects required in existing technologies. It represents the changes in the pose of a dynamic rigid body only through points on the dynamic object, without prior object modeling. Furthermore, in subsequent object motion estimation, the object's 3D pose does not need to be used as a random variable. This accelerates target acquisition and saves time.

[0064] Implementation Method 2: This implementation method further defines the visual-inertial dynamic target tracking SLAM device described in Implementation Method 1. The input unit includes: a monocular image input module and an IMU data input module.

[0065] The monocular image input module is used to acquire monocular image information and transmit the monocular image information to the tracking thread unit;

[0066] The IMU data input module is used to acquire IMU data information and transmit the IMU data information to the tracking thread unit.

[0067] Implementation Method 3: This implementation method further defines the visual-inertial dynamic target tracking SLAM device described in Implementation Method 1. The tracking thread unit includes:

[0068] Information extraction module, camera pose estimation module, and dynamic object pose estimation module;

[0069] The information extraction module is used to extract static structures and dynamic objects in the environment based on monocular image information and IMU data information.

[0070] The camera pose estimation module is used to perform camera pose estimation on the static structure.

[0071] The dynamic object pose estimation module is used to perform six-degree-of-freedom pose estimation on the dynamic object.

[0072] In practical applications, static structures and dynamic objects in the environment are extracted based on the ORB features of monocular images and the IMU integrals in IMU data.

[0073] Implementation Method Four: This implementation method further defines the visual-inertial dynamic target tracking SLAM device described in Implementation Method Three. The extraction of dynamic objects from the IMU data specifically includes:

[0074] Evaluate the movement probability P(X) of the feature point i |d i ):

[0075]

[0076] Where α is the influence factor of smoothing the instantaneous calculation result, and d i The distance from the feature to the epipolar line;

[0077] Set a dynamic feature point threshold based on the movement probability of the feature points;

[0078] The image is segmented using the Yolact++ real-time segmentation method to obtain dynamic feature points within the segmented target region;

[0079] The dynamic target is determined based on the proportion of dynamic feature points in the segmented target area to the total number of feature points.

[0080] In practical applications, extracting dynamic objects in the environment helps SLAM systems achieve higher-precision self-localization and avoids wasting computational resources on estimating the velocity of stationary objects.

[0081] Specifically, the position, velocity, and rotation information at the current moment can be obtained by integrating all IMU data between two adjacent frames.

[0082] For example: Given the position of the camera in the carrier coordinate system relative to the world coordinate system in the k-th frame. speed and rotation The rotation information is represented by four elements, and the information at time k+1 can be calculated using the following formula:

[0083]

[0084] Among them, a t It is the acceleration of the IMU data in its own coordinate system, g w It is the vector of gravitational acceleration. Multiplication represents four elements. For the rotation at time k+1, The velocity at time k+1 For the translation at time k+1, For a rotation transformation represented by four elements, ω x ω y ω z These are the rotation angle increments for the three axes, Let be the rotation matrix from the world coordinate system at time t.

[0085] The IMU sensor and camera are fixed together, and the rotation matrix and translation vector in the camera coordinate system can be obtained using the following formula:

[0086]

[0087]

[0088] Where R represents rotation within a relative time interval, and t represents rotation and translation within a relative time interval. This represents the transformation matrix from the carrier coordinate system to the camera coordinate system. and These represent the transformation matrices in the carrier coordinate system and the camera coordinate system over a relative time period, respectively.

[0089] The fundamental matrix F can be obtained by using the rotation matrix and translation vector between two adjacent frames, as shown in the following formula:

[0090]

[0091] Where (t×) is the antisymmetric matrix of translation t, and K is the intrinsic parameter matrix of the camera. The fundamental matrix between two frames. Let be the antisymmetric matrix of the translation vector between the two frames. This is the rotation matrix between the two frames.

[0092] The fundamental matrix is ​​a projection mapping that maps feature points to epipolar lines. Therefore, the distance d from the feature match to the epipolar line can be obtained using the following formula:

[0093]

[0094]

[0095] Wherein, the three-dimensional coordinates of the i-th feature point at time k+1, Let x, y, z be the three-dimensional coordinates of the i-th feature point at time k, where x, y, z are the three-axis coordinates.

[0096] The distance *d* can be used to determine whether a matched feature point is a dynamic feature point. The larger the distance *d*, the higher the probability that the point is a dynamic feature point. To quantitatively assess the probability of a dynamic feature point, a binomial logistic regression model is used, as shown in the following equation:

[0097]

[0098] Here, α is the influence factor for smoothing the instantaneous calculation results.

[0099] When P(X) i |d i When P(X) < 0.5, the feature point is defined as a static feature point; when P(X) < 0.5, the feature point is defined as a static feature point. i |d i When the value is greater than 0.5, the feature point is a dynamic feature point.

[0100] Subsequently, the Yolact++ real-time segmentation method was used to segment the target in each frame of the image, obtaining high-quality segmentation results so as to quickly achieve feature point clustering.

[0101] If the proportion of dynamic feature points within a segmented target region exceeds a certain level, specifically above 30% of the total number of feature points, then the target is defined as a dynamic target; otherwise, it is a stationary target. The Yolact++ method can only provide a single image object label; however, we need to track objects and their motion models over time between frames. This implementation uses a point label matching method between associated frames, where each point label is identical to the unique target identifier of the sampled point, to address challenging scenarios such as partial occlusion of dynamic objects and long-range views. Specifically, dynamic feature points not located within the segmented region obtain unique target labels through matching relationships between adjacent frames; simultaneously, as more dynamic targets are detected, the number of labels increases, while static targets and static map points are labeled with label 0.

[0102] Implementation Method 5: This implementation method further defines the visual-inertial dynamic target tracking SLAM device described in Implementation Method 3. Specifically, the camera pose estimation based on the static structure is as follows:

[0103]

[0104] in, Let k be the camera pose at time k. Let k be the camera pose at time k-1. This refers to the camera pose between two adjacent frames.

[0105] Implementation Method Six, see below Figure 2 This embodiment describes a further limitation of the visual-inertial dynamic target tracking SLAM device described in Embodiment 3. Specifically, the six-degree-of-freedom pose estimation based on the dynamic object is as follows:

[0106]

[0107] in, o X k (j) represents the coordinates of the j-th feature point in the world coordinate system at time k. Let j be the motion pose of the j-th feature point at time k. Let the j-th feature point be the motion pose at time k-1. o X k-1 (j) represents the coordinates of the j-th feature point in the world coordinate system at time k-1.

[0108] The motion speed of a dynamic object is:

[0109]

[0110] in, o X k Let k be the coordinates in the world coordinate system at time k. o X k-1 Δt represents the coordinates in the world coordinate system at time k-1, and Δt represents the time interval between two adjacent frames.

[0111] This implementation method will be described in conjunction with Implementation Method Five. For example... Figure 2 As shown, This represents the camera pose at time k. This represents the pose of the dynamic object point j at time k. (Object point j in the object coordinate system) th The coordinates are object point j in camera coordinate system th The coordinates are c X k (j), object point j in the world coordinate system th The coordinates are o X k (j), c X k (j)=[ c xk (j), c y k (j), c z k (j)] T The corresponding pixel coordinates are: U = Π( c X k (j))=[u k (j),v k (j)] T .

[0112] Camera motion is represented as:

[0113]

[0114] The motion of a dynamic object point j is represented as:

[0115]

[0116] The coordinates of dynamic object point j in the world coordinate system at frame k can be represented as:

[0117]

[0118] Assume that all dynamic objects are rigid bodies, meaning that the position of a point on the object will not change over time, as shown in the following formula:

[0119]

[0120] The transformation relationship of a 3D point in a dynamic object over consecutive time frames is as follows:

[0121]

[0122] The above formula is the core of target motion estimation in this embodiment. Because it only represents the rigid body pose change through points of the dynamic object, it does not require prior modeling of the object, nor does it require treating the target's three-dimensional pose as a random variable in subsequent target motion estimation.

[0123] Camera pose is estimated by minimizing reprojection error. To achieve self-localization of the system, its reprojection error cost function can be expressed as:

[0124]

[0125] in, Let j be the camera pose reprojection error cost function, and j be the dynamic object point. Let J be the true pixel coordinates of the j-th feature point at time k. Let be the measured pixel coordinates of the j-th feature point at time k. This is the transformation matrix from the carrier coordinate system to the camera coordinate system. Let be the transformation matrix from the world coordinate system to the vehicle coordinate system at time k. o X k (j) represents the coordinates of the j-th feature point in the world coordinate system at time k.

[0126] Similar to camera pose estimation, target motion estimation can also be achieved by constructing a cost function for reprojection error.

[0127] Based on the transformation relationship of a 3D point in a dynamic object over continuous time frames, the error in projecting a 3D point j on the dynamic object onto the 2D pixel coordinates in the image is:

[0128]

[0129] in, The error in the two-pixel coordinates of a point on a dynamic object projected onto the image. Let k be the camera pose at time k. Let l be the pose of the dynamic object l at time k. Let l be the motion pose of the dynamic object at time k-1. o X k-1 (j) represents the coordinates of the j-th feature point in the world coordinate system at time k-1.

[0130] Estimate the velocity of a dynamic object. Select the center of n matching feature points of the object as the representative point for estimating the velocity:

[0131]

[0132] The motion speed of a dynamic object is:

[0133]

[0134] Where Vt is the time interval between two adjacent frames.

[0135] Implementation Method Seven, see below Figure 3 This embodiment describes a further limitation of the visual-inertial dynamic target tracking SLAM device described in Embodiment 1. Specifically, the optimization of pose information based on local and global batch optimization methods is as follows:

[0136]

[0137] Where, θ * The result of batch optimization is the final 3D point coordinates, camera pose, and dynamic object motion state output by the system. ρ is the robust kernel function, k is the time step, and r is the time step. k,k+1 For IMU inertial residuals, e(l D k-1,k ) represents the smooth motion error of the dynamic object, j represents the j-th feature point, l represents the l-th dynamic object, and n z Let n be the total number of static structural position factors, and n0 be the total number of IMU inertia factors. m n is the total number of motion factors of a dynamic object. s This represents the total number of smooth motion factors.

[0138] Specifically, in the mapping thread unit, a global map is constructed, consisting of camera pose, dynamic target motion, and static structure. To improve efficiency, a local map is constructed, including commonly visible landmarks and dynamic objects within the commonly visible map. Simultaneously, the system constructs a factor graph framework based on both the local and global maps to perform local or global batch optimization of the four types of measurement information. For example... Figure 3 As shown, the batch optimization factor graph framework mainly includes four types of measurement information: static structural 3D points, inertial visual odometry, dynamic object 3D point motion observation, and dynamic target smooth motion.

[0139] In the diagram, Static Map Points represent static map points; Smooth factor represents the smoothing factor; the small circles in the first row represent the static structure visual residuals, the small circles in the second row represent the inertial residuals, the small circles in the third row represent the dynamic object visual residuals, and the small circles in the fourth row represent the smoothing motion factor; the boxes in the first row represent the velocity in the Body coordinate system, the boxes in the second row represent the attitude in the coordinate system in the third row, and the boxes in the third row represent the velocity in the Body coordinate system. and Represents the motion of a dynamic object in the Body coordinate system; the third box contains b. k-1 and b k-1 The fourth box represents the acceleration in the Body coordinate system, and the fourth box represents the motion of the dynamic object between two adjacent frames.

[0140] Given that the IMU inertial residual is:

[0141]

[0142] Where, e(r) k,k+1 ) represents the IMU inertial residual. For IMU rotational residuals, For IMU speed residual, For IMU displacement residuals, Let be the rotation transformation matrix for the two frames. R is the transpose of the rotation matrix at time k. k+1 V is the rotation matrix at time k+1. k+1 Let V be the velocity at time k+1. kLet Δt be the velocity at time k, g be the acceleration due to gravity, and Δt be the velocity at time k. k,k+1 ΔV is the time interval between two frames. k,k+1 ΔP represents the velocity change between two frames. k,k+1 This represents the translation between two frames.

[0143] In batch optimization, a smoothing factor is introduced to address the impact of camera acquisition frequency and prevent abrupt changes in the motion state of dynamic targets. The smoothing factor minimizes the variation in the motion state of dynamic objects between consecutive time frames, and its error model is defined as follows:

[0144]

[0145] Combining the visual reprojection error of static structures, IMU inertial residuals, the visual reprojection error of dynamic objects, and the smoothing motion error of dynamic objects, the final result of the batch optimization problem is as follows:

[0146]

[0147] Where ρHub represents the robust kernel function, which can reduce the impact of false matches; ∑ -1 It is an information matrix; θ represents Figure 3 All nodes in θ. * The final 3D point coordinates, camera pose, and dynamic object motion states will be output to the system as a result of global batch optimization. A globally consistent map containing both static and dynamic structures will also be output.

[0148] Implementation Method Eight: A visual-inertial dynamic target tracking SLAM method according to this implementation method, the method comprising:

[0149] Acquire image information and IMU data;

[0150] Pose estimation is performed based on the acquired image information and IMU data to obtain pose information;

[0151] The pose information is optimized using local and global batch optimization methods to obtain local and global maps.

[0152] Implementation Method Nine: A computer device according to this implementation method includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a visual-inertial dynamic target tracking SLAM method according to Implementation Method Eight.

[0153] Implementation Method 10: A computer-readable storage medium according to this implementation method stores a computer program, which, when executed by a processor, performs a visual-inertial dynamic target tracking SLAM method according to Implementation Method 8.

[0154] The present application has been described in detail above through specific embodiments. However, the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, combinations of embodiments, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A visual-inertial dynamic target tracking SLAM device, characterized in that, The device includes: Input unit, tracking thread unit, graph building thread unit, and output unit; The input unit is used to acquire image information and IMU data; The tracking thread unit is used to perform pose estimation based on the acquired image information and IMU data, and to obtain the pose information of the camera and the dynamic object. The mapping thread unit is used to optimize pose information according to local and global batch optimization methods to obtain local and global maps. The output unit is used to output the acquired global map; The tracking thread unit includes: Information extraction module, camera pose estimation module, and dynamic object pose estimation module; The information extraction module is used to extract static structures and dynamic objects in the environment based on monocular image information and IMU data information. The camera pose estimation module is used to perform camera pose estimation on the static structure. The dynamic object pose estimation module is used to perform six-degree-of-freedom pose estimation on the dynamic object, including: Calculate the coordinates of the dynamic object point j in the world coordinate system at the current moment. The value is equal to the current motion pose of the point, the inverse of the motion pose of the point at the previous moment, and the coordinates of the point in the world coordinate system at the previous moment, which are synthesized based on the pose synthesis rule. The pose estimation represents the rigid body pose change only through the motion of the dynamic object points; The motion speed of the dynamic object is obtained by calculating the ratio of the change in position of the center point of all matching feature points on the object in the world coordinate system at adjacent moments to the time interval between two adjacent frames. The system achieves self-localization by minimizing the reprojection error to estimate the camera pose. The reprojection error is the difference between the true pixel coordinates of the dynamic object point j at time k and the pixel coordinates calculated by projecting the current estimated camera pose, coordinate system transformation matrix, and the current world coordinates of the point through the camera model. Target motion estimation is achieved by constructing and minimizing a reprojection error cost function based on the coordinate transformation relationship; The information extraction module extracts dynamic objects from the environment, specifically including: Evaluate the movement probability of image feature points, which is related to the geometric distance from the feature matching point to the epipolar line; Set a dynamic feature point threshold based on the movement probability of the feature points; The image is segmented using the Yolact++ real-time segmentation method to obtain dynamic feature points within the segmented target region; The dynamic target is determined based on the proportion of dynamic feature points in the segmented target area to the total number of feature points.

2. The visual-inertial dynamic target tracking SLAM device according to claim 1, characterized in that, The input unit includes: a monocular image input module and an IMU data input module; The monocular image input module is used to acquire monocular image information and transmit the monocular image information to the tracking thread unit; The IMU data input module is used to acquire IMU data information and transmit the IMU data information to the tracking thread unit.

3. A visual-inertial dynamic target tracking SLAM method, characterized in that, The method is implemented based on the system described in claim 1, and the method includes: Acquire image information and IMU data; Pose estimation is performed based on the acquired image information and IMU data to obtain pose information; The pose information is optimized using local and global batch optimization methods to obtain a global map.

4. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the visual-inertial dynamic target tracking SLAM method as described in claim 3.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the visual-inertial dynamic target tracking SLAM method of claim 3.

Images