Key frame strategy network optimization method and device combined with SLAM system
By introducing a keyframe policy network optimization method in the SLAM system, combining SLAM and deep learning, the problems of insufficient tracking robustness and data processing synchronization of the monocular visual SLAM system are solved, and the accuracy of real-time positioning and map construction are achieved. Stability, suitable for flexible training in different scenarios.
Patent Information
- Application Number
- CN202411070804.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-12-06
AI Technical Summary
The existing monocular vision SLAM system is not tracking robust enough in outdoor public data sets, resulting in a keyframe insertion strategy showing low robustness and instability in the front-end odometer, and the keyframe strategy The network cannot synchronize data processing and prediction with real-time data acquisition, affecting the accuracy of positioning and map construction.
By introducing a keyframe policy network optimization method in the SLAM system, using dual threads for training, combining SLAM and deep learning, sequencing frames are acquired in real time for feature extraction and pose estimation, and file lock management is carried out through information interaction pool to realize keyframe strategy The network's inference and reverse gradient updates, removes the relocation tracking module to improve robustness, and reintroduces the relocation tracking thread during the inference process to improve stability.
The keyframe policy network data processing and prediction are synchronized with real-time data acquisition, which improves the tracking stability and tracking accuracy of the SLAM system, ensures accurate positioning and map construction, and provides flexible training methods suitable for different scenarios. .
Smart Images

Figure CN119088459A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of computer technology, and in particular relates to a key frame strategy network optimization method and device combined with a SLAM system. Background Art
[0002] The keyframe insertion algorithm in the front-end odometer has a great impact on the back-end optimization, and the quality of the inserted keyframe determines the accuracy of the loop. In the existing monocular vision SLAM algorithms, empirical value parameters are usually used to implement the keyframe insertion strategy. These parameters are widely used in mainstream monocular vision SLAM tracking algorithms. However, when conducting a tracking robustness study on outdoor public datasets, this paper found that the keyframe insertion strategy that relies on empirical values shows low robustness in the front-end odometer tracking process, resulting in poor stability of the tracking results, and sometimes even a large number of keyframes but low trajectory accuracy. Therefore, improving the keyframe insertion strategy is an important method to improve the tracking stability and tracking accuracy of the monocular vision SLAM system.
[0003] The current keyframe strategy network cannot batch process video sequences and make predictions at the same time, and the monocular vision SLAM system needs to obtain data from the camera in real time and perform positioning and map construction at each time step. The real-time nature of the system requires that data processing and prediction be carried out simultaneously with real-time data acquisition to ensure accurate positioning and map construction. Therefore, the keyframe strategy network cannot perform offline processing of individual sequence frames.
[0004] Chinese patent application CN117894068A discloses a motion data processing method, a training method and device for a key frame extraction model, and claim 2 thereof proposes a training method for a key frame extraction model, comprising: obtaining a training data set; wherein the training data set comprises a plurality of training states, the training state comprises a training sequence and an indicator sequence, the training sequence comprises a plurality of training data frames, and the indicator sequence is composed of key frame indicators of the training data frames; inputting the training state into the key frame extraction model, selecting a key frame according to the decision value when each training data frame is used as a key frame, and updating the training state based on the key frame to obtain an updated state; calculating the decision reward of the key frame according to the training state and the updated state; taking the training state, the updated state, the key frame and the decision reward as unit training data, and calculating a loss value according to the unit training data and the decision reward of the key frame, and updating the model parameters of the key frame extraction model according to the loss value, until the trained key frame extraction model is obtained. However, its model training method cannot realize data processing and prediction synchronously with real-time data acquisition. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a key frame strategy network optimization method and device combined with a SLAM system.
[0006] The present invention provides a key frame strategy network optimization method combined with a SLAM system, comprising the following steps:
[0007] Step S1: The SLAM system reads the sequence frame, processes the data, and obtains the real-time status of the system;
[0008] Step S2: The SLAM system process acquires the file lock, writes the real-time status of the system into the information interaction pool, releases the file lock, wakes up the key frame strategy network process, and the SLAM system process enters the waiting state;
[0009] Step S3: The key frame strategy network process obtains the file lock, reads the real-time status of the system from the information interaction pool, uses the read real-time status of the system as the input of the key frame strategy network for network reasoning, writes the reasoning decision result into the information interaction pool, releases the file lock and wakes up the SLAM system process, and the key frame strategy network process enters the waiting state;
[0010] Step S4: The SLAM process obtains the file lock, reads the decision result from the interaction pool, and releases the file lock;
[0011] Step S5: Determine whether the tracking thread is finished, if not, execute step S1; if yes, execute step S6;
[0012] Step S6: Calculate the composite loss and perform reverse gradient update of the key frame strategy network.
[0013] The sequence frames of step S1 are acquired in real time by a camera.
[0014] The data processing of step S1 includes: extracting and matching features of sequence frames, screening common viewpoints through feature matching, estimating camera pose and building local maps according to the imaging relationship of common viewpoints in different images, comparing the current estimated pose with the true value to obtain the pose estimation error, and calculating the key frame insertion rate.
[0015] The real-time status of the system is obtained through data processing, including posture estimation error and key frame insertion rate.
[0016] The decision result is a number between 0 and 1.
[0017] After the decision result is read in step S4, the SLAM system compares the decision result with a preset threshold. If it is greater than the preset threshold, the current frame is inserted as a key frame. If it is less than or equal to the preset threshold, it is not processed.
[0018] The tracking thread uses a constant speed tracking method and a reference key frame tracking method for tracking, and removes a relocation tracking module.
[0019] The present invention also provides an improved SLAM system based on a key frame strategy network. The key frame strategy network adopted by the system is trained using the above-mentioned key frame strategy network optimization method combined with the SLAM system.
[0020] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned key frame strategy network training method combined with a SLAM system is implemented.
[0021] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above-mentioned key frame strategy network training method combined with a SLAM system.
[0022] Compared with the prior art solutions, the present invention has the following beneficial effects:
[0023] (1) The present invention realizes the processing and prediction of key frame strategy network data and real-time data acquisition simultaneously, and can simultaneously optimize the selection of key frames of the SLAM system, ensuring accurate positioning and map construction.
[0024] (2) The training method of the present invention is more flexible. The online training method is applicable to different scenarios, and offline training can also be used to improve performance in specific scenarios.
[0025] (3) Improve the tracking stability and accuracy of the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is a framework diagram of the attached network training method combined with the SLAM system;
[0027] Figure 2 A schematic diagram of two threads accessing the same data area;
[0028] Figure 3 It is a dual-process system flow chart;
[0029] Figure 4 Tracking flowchart for training;
[0030] Figure 5 Flowchart for reasoning tracing. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0032] The present invention proposes a key frame strategy network training method combined with a SLAM system. The method is based on SLAM and deep learning dual-thread training, and combines the real-time status information in SLAM to determine whether the current frame is a key frame. The specific implementation steps include:
[0033] Step S1: The SLAM system reads the sequence frames, performs data processing, and obtains the real-time status of the system, wherein the data processing includes: extracting and matching features from the sequence frames, screening common viewpoints through feature matching, performing camera pose estimation and local mapping based on the imaging relationship of the common viewpoints in different images, comparing the current estimated pose with the true value to obtain the pose estimation error, and calculating the key frame insertion rate. The real-time status of the system is obtained through data processing, including the pose estimation error and the key frame insertion rate.
[0034] Step S2: The SLAM system process acquires the file lock, writes the real-time status of the system into the information interaction pool, releases the file lock, wakes up the key frame strategy network process, and the SLAM system process enters the waiting state, such as Figure 1 and Figure 2 As shown in Figure 1, the SLAM system is embedded in the keyframe strategy network, and the entire network training process involves information interaction between the two processes.
[0035] Step S3: The keyframe strategy network process obtains the file lock, reads the real-time status of the system from the information interaction pool, uses the read real-time status of the system as the input of the keyframe strategy network for network reasoning, writes the reasoning decision result into the information interaction pool, releases the file lock and wakes up the SLAM system process, and the keyframe strategy network process enters the waiting state. Figure 3 As shown, the decision result is a number between 0 and 1 calculated by the key frame strategy network. It is compared with the set threshold. If it is greater than the threshold, it means insertion, and the SLAM program will use the current frame as the key frame.
[0036] Step S4: The SLAM process obtains the file lock, reads the decision result from the interaction pool, and releases the file lock.
[0037] Step S5: Determine whether the tracking thread is finished. If not, execute step S1; if yes, execute step S6, wherein the tracking process is a training tracking process, such as Figure 4 shown.
[0038] Step S6: Calculate the composite loss and perform reverse gradient update of the key frame strategy network.
[0039] The trained model file will be deployed in the SLAM system and use the inference tracking process, such as Figure 5 shown.
[0040] According to the above process, the keyframe strategy network can be connected to the real-time SLAM system to complete the training method that better meets the task requirements.
[0041] In order to further improve the robustness of the proposed algorithm, this embodiment designs two different tracking processes for the network training task and reasoning task, namely, the training tracking process and the reasoning tracking process.
[0042] In the ORB-SLAM2 algorithm, tracking failures are common due to environmental changes or occlusions. In order to improve the robustness and stability of the system, ORB-SLAM2 introduces a relocation tracking thread. The relocation tracking module can improve the tracking stability of the SLAM system. In order to highlight the impact of keyframe selection on system performance, it is necessary to remove the relocation tracking module during the training process, so as to better train the model and improve the robustness of the model in the face of scene mutations or occlusions. Therefore, the present invention removes the relocation tracking module during the keyframe strategy network training process, artificially reducing the tracking robustness of the SLAM system. Specifically, Figure 4 As shown. Under low robustness training conditions, the improved SLAM system based on the key frame strategy of deep learning adopts the constant speed tracking method and the reference key frame tracking method for tracking. Through such a training method, a network model that is more focused on the constant speed tracking method and the reference key frame tracking method can be trained. At the same time, the present invention reintroduces the relocation tracking thread in the network reasoning tracking process, such as Figure 5 As shown, the tracking stability of the SLAM system is greatly improved.
[0043] In addition, it should be noted that the specific implementation examples described in this specification may be named differently, and the above content described in this specification is only an example of the structure of the present invention. All equivalent changes or simple changes made based on the structure, features and principles of the present invention are included in the protection scope of the present invention. Technicians in the technical field to which the present invention belongs can make various modifications or supplements to the specific examples described or adopt similar methods, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all belong to the protection scope of the present invention.
Claims
1. A key frame strategy network optimization method combined with a SLAM system, characterized in that: The following steps are involved: Step S1: The SLAM system reads the sequence frame, processes the data, and obtains the real-time status of the system; Step S2: The SLAM system process acquires the file lock, writes the real-time status of the system into the information interaction pool, releases the file lock, wakes up the key frame strategy network process, and the SLAM system process enters the waiting state; Step S3: The key frame strategy network process obtains the file lock, reads the real-time status of the system from the information interaction pool, uses the read real-time status of the system as the input of the key frame strategy network for network reasoning, writes the reasoning decision result into the information interaction pool, releases the file lock and wakes up the SLAM system process, and the key frame strategy network process enters the waiting state; Step S4: The SLAM process obtains the file lock, reads the decision result from the interaction pool, and releases the file lock; Step S5: Determine whether the tracking thread is finished, if not, execute step S1; If yes, execute step S6; Step S6: Calculate the composite loss and perform reverse gradient update of the key frame strategy network.
2. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: The sequence frames of step S1 are acquired in real time by a camera.
3. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: The data processing of step S1 includes: extracting and matching features of sequence frames, screening common viewpoints through feature matching, estimating camera pose and building local maps according to the imaging relationship of common viewpoints in different images, comparing the current estimated pose with the true value to obtain the pose estimation error, and calculating the key frame insertion rate.
4. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: The real-time status of the system is obtained through data processing, including posture estimation error and key frame insertion rate.
5. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: The decision result is a number between 0 and 1.
6. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: After the decision result is read in step S4, the SLAM system compares the decision result with a preset threshold. If it is greater than the preset threshold, the current frame is inserted as a key frame. If it is less than or equal to the preset threshold, it is not processed.
7. A key frame strategy network optimization method combined with a SLAM system according to claim 1, characterized in that: The tracking thread uses a constant speed tracking method and a reference key frame tracking method for tracking, and removes a relocation tracking module.
8. An improved SLAM system based on a keyframe strategy network, characterized in that: The key frame strategy network adopted by the system is trained using a key frame strategy network optimization method combined with a SLAM system as described in any one of claims 1 to 7.
9. An electronic device, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, a key frame strategy network optimization method combined with a SLAM system is implemented as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement a key frame strategy network optimization method combined with a SLAM system according to any one of claims 1 to 7.
