Target object pose estimation method and system, computer equipment and medium

Abstract

The invention provides a target object pose estimation method and system, computer equipment and a medium, and the method comprises the steps: extracting a key point, a local descriptor and a global descriptor from a current frame image through a feature extraction model, and carrying out the pose estimation prediction of a target object based on the key point and the local descriptor; after pose estimation prediction fails, repositioning is carried out through a bag-of-word model and a global descriptor; or after the pose estimation and prediction are successful, loopback detection and loopback correction are carried out on the target object through a dictionary tree and a local map which are established in advance or in real time. According to the method and the device, the positioning precision of the target object in the illumination and view angle changing scene can be improved, and when the feature extraction model is deployed to the central processing unit, the pose estimation of the target object can be realized without a graphics processor, so that the computing resources can be saved, and the positioning cost of the target object can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, system, computer device and medium for estimating the pose of a target object. Background Art

[0002] Among the many visual SLAM (Simultaneous Localization and Mapping, SLAM for short) algorithms, most of the current SLAMs rely on traditional manual features to build the system. In scenes with dramatic changes in lighting and perspective and interference from dynamic factors, the above manual features are difficult to provide consistent feature detection and association, resulting in the corresponding robot being difficult to accurately estimate the pose. On the other hand, the introduction of SLAM algorithms consumes computing resources, and usually requires the configuration of a GPU (Graphics Processing Unit, GPU for short) to increase computing resources, making robot positioning costly. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a target object pose estimation method, system, computer device and medium to solve the technical problems existing in the prior art.

[0004] To achieve the above objectives and other related objectives, the present application provides a method for estimating a target object's position and pose, comprising the following steps:

[0005] Acquire multiple frames of target images captured in advance or in real time, and select one frame of target image from the multiple frames of target images as the current frame of image, wherein the target image includes all or part of the area containing the target object;

[0006] Extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network;

[0007] Performing pose estimation and prediction on the target object based on the key points and the local descriptors;

[0008] After the pose estimation prediction fails, a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold is retrieved from a key frame database established in advance or in real time through a bag-of-words model established in advance or in real time and the global descriptor, and local features of the current frame image and the relocation candidate frame image are matched, and the pose of the target object in the current frame image is estimated based on the local feature matching result of the current frame image and the relocation candidate frame image;

[0009] Alternatively, after the pose estimation prediction is successful, loop detection is performed on the target object through a dictionary tree and a local map established in advance or in real time, and loop correction is performed on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model.

[0010] In one embodiment of the present application, after the pose estimation prediction is successful, the method further includes:

[0011] Traversing the map points of the multiple target image frames, and recording all key frame images in which the map points can be observed, and using the key frame images in which the map points can be observed as local key frames;

[0012] Based on the local key frame, the reference key frame of the current frame image is updated, and the map points of the reference key frame are added after all map points in the current local map are cleared; wherein the map points of the current local map in the initial state are empty;

[0013] Filtering newly added map points within the field of view in the current local map and projecting them into the current frame image for search and matching;

[0014] Based on the number of map points matched by the projection search in the current frame image, it is determined whether the current local map is tracked successfully, and when the tracking is successful, the key frames in the current local map are added to the key frame database, and the current local map is used to perform loop detection on the target object; wherein, if the number of map points is greater than or equal to the preset map point value, it indicates that the current local map tracking is successful; otherwise, if the number of map points is less than the preset map point value, it indicates that the current local map tracking has failed.

[0015] In one embodiment of the present application, the process of performing loop detection on the target object and performing loop correction on the target object based on the loop detection result includes:

[0016] Calculate the similarity between the current frame image and the key frame images in the key frame database, and obtain the first K key frame images with the highest similarity values; wherein K is a natural number;

[0017] Performing loop detection on the target object, including sequentially calculating the distance between the global features of the current frame image and the global features of the first K key frame images, and recording the key frame image with the smallest distance as a loop candidate frame image as the loop detection result;

[0018] The loop closure candidate frame image and the current frame image are taken as a loop closure pair, and the loop closure candidate frame image is used to perform loop correction on the target object; wherein the postures of the two frame images in the same loop closure pair are the same.

[0019] In one embodiment of the present application, the process of extracting key points, local descriptors and global descriptors from the current frame image by using a feature extraction model generated in advance or in real time includes:

[0020] Extracting multi-scale features from the current frame image by using the feature extraction model, and scoring feature points of the multi-scale features, and determining feature points representing the current frame image based on the feature point scoring results, which are recorded as key points; and

[0021] Extracting local features of each pixel point or each pixel region from the current frame image by using the feature extraction model, and recording them as local descriptors; and

[0022] All local features are aggregated through the fully connected layer or the global average pooling layer in the feature extraction model, and the aggregation result is used as the global feature of the current frame image, which is recorded as a global descriptor.

[0023] In one embodiment of the present application, before extracting key points, local descriptors, and global descriptors from the current frame image by using a feature extraction model generated in advance or in real time, the method further includes:

[0024] A first decoder and a second decoder for extracting local depth features in an image are constructed using a convolutional neural network architecture, and a third decoder for extracting global depth features in an image is constructed;

[0025] Selecting an encoder as a shared encoder, and connecting the output of the shared encoder to the first decoder, the second decoder, and the third decoder respectively;

[0026] Constructing the feature extraction model using the shared encoder, the first decoder, the second decoder and the third decoder;

[0027] The first decoder is used to extract key points, the second decoder is used to extract local descriptors, and the third decoder is used to extract global descriptors.

[0028] In one embodiment of the present application, the process of constructing the dictionary tree based on the bag-of-words model includes:

[0029] Acquire multiple frames of training images, each frame of training image includes all or part of the region containing the target object;

[0030] Extracting key points and local descriptors from each frame of training image through the feature extraction model, screening out the most robust local descriptors according to the detection scores of the key points, and generating the most robust local descriptor data set;

[0031] Starting from the first local descriptor of the most robust local descriptor data set, matching is performed with the local descriptors in the adjacent frame training images using a string matching algorithm to determine whether the two are equal; if they are equal, the successfully matched local descriptors are assigned to the same child node; if they are not equal, the unmatched local descriptors are designated as new child nodes, and local descriptors are reselected from the most robust local descriptor data set for matching until a matching local descriptor is found or the most robust local descriptor data set is traversed;

[0032] All local descriptors extracted are clustered as the first layer through the bag-of-words model to obtain a root node;

[0033] The child nodes under the root node are re-clustered until the child nodes are clustered, and the dictionary tree is constructed based on the root node, all the child nodes and the clustering results.

[0034] In one embodiment of the present application, when constructing the feature extraction model, the method also includes: placing the first decoder and the second decoder at the front end of the third decoder so that each frame of the target image first enters the shared encoder, and then enters the first decoder, the second decoder and the third decoder respectively.

[0035] The present application also provides a target object pose estimation system, the system comprising:

[0036] An image acquisition module is used to acquire multiple frames of target images taken in advance or in real time, and select a frame of target image from the multiple frames of target images as a current frame of image, wherein the target image includes all or part of an area containing a target object;

[0037] A feature extraction module, used to extract key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network;

[0038] A pose estimation prediction module, used to perform pose estimation prediction on the target object according to the key points and the local descriptors;

[0039] A relocation module, for retrieving a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold from a key frame database established in advance or in real time through a bag-of-words model established in advance or in real time and the global descriptor, matching local features of the current frame image with the relocation candidate frame image, and estimating the pose of the target object in the current frame image based on the matching result of the local features of the current frame image with the relocation candidate frame image after the pose estimation prediction fails;

[0040] The loop detection and correction module is used to perform loop detection on the target object through a pre- or real-time established dictionary tree and local map after the pose estimation prediction is successful, and to perform loop correction on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model.

[0041] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned target object pose estimation methods.

[0042] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for estimating the position and pose of a target object.

[0043] As described above, the present application provides a method, system, computer device and medium for estimating the pose of a target object, which has the following beneficial effects: the present application first obtains multiple frames of target images that are taken in advance or in real time and contain all or part of the target object, and then selects a frame of the target image from the multiple frames of the target image as the current frame image; then extracts key points, local descriptors and global descriptors from the current frame image through a feature extraction model that is constructed in advance or in real time based on a convolutional neural network; then performs pose estimation on the target object based on the key points and local descriptors; after the pose estimation prediction fails, the pose estimation is performed through a word extraction model that is constructed in advance or in real time. Bag model and global descriptor, retrieve the relocation candidate frame image whose similarity with the current frame image is greater than or equal to the preset threshold from the key frame database established in advance or in real time, match the local features of the current frame image and the relocation candidate frame image, and estimate the pose of the target object in the current frame image based on the local feature matching results of the current frame image and the relocation candidate frame image; or, after the pose estimation prediction is successful, the target object is looped through the dictionary tree and local map established in advance or in real time, and the target object is looped corrected based on the loop detection result; wherein the dictionary tree is constructed based on the bag of words model. It can be seen from this that the present application can not only improve the positioning accuracy of the target object (such as a robot) in scenes with changing illumination and perspective by extracting the local features and global features of each frame image, and performing pose tracking, local map tracking, relocation and loop detection correction based on the local features and global features of each frame image, but also when the feature extraction model is deployed in the central processing unit, the pose estimation of the target object can be realized without the need for a graphics processor, thereby saving computing resources and reducing the positioning cost of the target object. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1A schematic diagram of a target object pose estimation method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the operating architecture of a target object pose estimation method provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the hardware structure of a target object pose estimation system provided in an embodiment of the present application;

[0047] Figure 4 The figure is a schematic diagram of the hardware structure of a computer device suitable for implementing one or more embodiments of the present application. DETAILED DESCRIPTION

[0048] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0049] It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present application in a schematic manner, and the figures only show the components related to the present application rather than being drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0050] Figure 1 FIG. 1 is a flow chart of a method for estimating the position and posture of a target object provided by an embodiment of the present application. Specifically, in an exemplary embodiment, Figure 1 As shown, this embodiment provides a method for estimating a target object's position and posture, the method comprising the following steps:

[0051] S110, obtaining multiple frames of target images captured in advance or in real time, and selecting a frame of target image from the multiple frames of target images as the current frame image, wherein the target image includes all or part of the area containing the target object. As an example, in this embodiment or other embodiments, the target object includes but is not limited to a robot, and the target image can be obtained by capturing the robot in advance or in real time by one or more image capturing devices.

[0052] S120, extract key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network. As an example, the feature extraction model can be deployed in a CPU (Central Processing Unit, CPU for short), so that the pose estimation of the target object can be realized without a GPU, which not only saves computing resources, but also reduces the positioning cost of the target object. In this embodiment or other embodiments, the process of extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time includes: extracting multi-scale features from the current frame image through a feature extraction model, and scoring feature points of the multi-scale features, determining feature points representing the current frame image based on the feature point scoring results, and recording them as key points; and extracting local features of each pixel point or each pixel area from the current frame image through a feature extraction model, and recording them as local descriptors; and aggregating all local features through a fully connected layer or a global average pooling layer in the feature extraction model, and using the aggregation result as a global feature of the current frame image, and recording them as global descriptors. As an example, in this embodiment or other embodiments, feature points may refer to some special places in the image, such as corners, edges, and blocks in the image; key points may refer to the positions of feature points in the image; local descriptors refer to descriptors extracted from each pixel or small area in the image, which can be represented by local features; global descriptors refer to descriptors of the entire image, which can be represented by global features.

[0053] S130, performing pose estimation prediction on the target object based on the key points and the local descriptors. In this embodiment or other embodiments, if the key points of the current frame image and the local descriptor local map do not match, it indicates that the pose estimation prediction fails. On the contrary, if the key points of the current frame image and the local descriptor local map both match, it indicates that the pose estimation prediction is successful.

[0054] S140-1, after the pose estimation prediction fails, the relocation candidate frame image whose similarity with the current frame image is greater than or equal to the preset threshold is retrieved from the key frame database established in advance or in real time through the bag of words model and the global descriptor established in advance or in real time, and the local features of the current frame image and the relocation candidate frame image are matched, and the pose of the target object in the current frame image is estimated based on the local feature matching results of the current frame image and the relocation candidate frame image. In this embodiment or other embodiments, the bag of words model (Bag of Words, BoW) is a text representation method widely used in natural language processing and information retrieval. The bag of words model does not consider the order, grammar and syntax of words in the text, but simply regards the text as a collection of words, or more specifically, a collection of the number of times a word appears. Therefore, the establishment process of the bag of words model will not be repeated in this embodiment, and reference can be made to the relevant technology.

[0055] S140-2, after the pose estimation prediction is successful, loop detection is performed on the target object through the dictionary tree and local map established in advance or in real time, and loop correction is performed on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model. As an example, in this embodiment or other embodiments, the process of performing loop detection on the target object and performing loop correction on the target object based on the loop detection result includes: calculating the similarity between the current frame image and the key frame image in the key frame database, and obtaining the top K key frame images with the highest similarity value; wherein K is a natural number; performing loop detection on the target object, including sequentially calculating the distance between the global features of the current frame image and the global features of the top K key frame images, and recording the key frame image with the smallest distance as a loop candidate frame image as the loop detection result; taking the loop candidate frame image and the current frame image as a loop pair, and using the loop candidate frame image to perform loop correction on the target object; wherein the poses of the two frames in the same loop pair are the same.

[0056] It can be seen from this that this embodiment extracts local features and global features of each frame image, and performs pose tracking, local map tracking, relocalization and loop detection correction based on the local features and global features of each frame image. This can not only improve the positioning accuracy of the target object (such as a robot) in scenes with changing lighting and perspective, but also when the feature extraction model is deployed in the central processing unit, the pose estimation of the target object can be achieved without the need for a graphics processor, thereby saving computing resources and reducing the positioning cost of the target object.

[0057] According to the above records, in an exemplary embodiment, after the pose estimation prediction is successful, the present embodiment may further include: traversing the map points of multiple frames of target images, and recording all key frame images where the map points can be observed, and using the key frame images where the map points can be observed as local key frames; updating the reference key frame of the current frame image based on the local key frame, and adding the map points of the reference key frame after clearing all map points in the current local map; wherein the map points of the current local map in the initial state are empty; filtering the newly added map points within the field of view in the current local map, and projecting them into the current frame image for search and matching; judging whether the current local map is tracked successfully based on the number of map points in the current frame image that are projected and searched for, and adding the key frames in the current local map to the key frame database when the tracking is successful, and performing loop detection on the target object using the current local map; wherein, if the number of map points is greater than or equal to the preset map point value, it indicates that the current local map tracking is successful; otherwise, if the number of map points is less than the preset map point value, it indicates that the current local map tracking fails.

[0058] According to the above records, in an exemplary embodiment, before extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time, this embodiment may also include: using a convolutional neural network architecture to construct a first decoder and a second decoder for extracting local depth features in the image, and constructing a third decoder for extracting global depth features in the image; and placing the first decoder and the second decoder at the front end of the third decoder so that each frame of the target image first enters the shared encoder, and then enters the first decoder, the second decoder and the third decoder respectively. Select an encoder as a shared encoder, and connect the output of the shared encoder to the first decoder, the second decoder and the third decoder respectively. Use the shared encoder, the first decoder, the second decoder and the third decoder to construct a feature extraction model. Among them, the first decoder is used to extract key points, the second decoder is used to extract local descriptors, and the third decoder is used to extract global descriptors. In this embodiment or other embodiments, the first two decoders of the feature extraction model adopt a local feature convolutional neural network architecture, and the other decoder is a global feature convolutional neural network architecture. This network structure design enables the convolutional neural network to extract global and local features of the image with a single forward inference. The extracted features are not only beneficial to traditional SLAM pose tracking tasks, but also help improve the performance of loop detection and relocalization modules in the SLAM algorithm.

[0059] According to the above records, in an exemplary embodiment, the process of constructing a dictionary tree based on the bag-of-words model includes: obtaining multiple frames of training images, each frame of training image includes all or part of the area containing the target object; extracting key points and local descriptors from each frame of training image through a feature extraction model, and screening out the most robust local descriptors according to the detection scores of the key points, and generating the most robust local descriptor data set; starting from the first local descriptor of the most robust local descriptor data set, matching with the local descriptors in the adjacent frame training images using a string matching algorithm to determine whether the two are equal; if they are equal, the successfully matched local descriptors are attributed to the same child node; if they are not equal, the local descriptors that have not been successfully matched are designated as new child nodes, and local descriptors are reselected from the most robust local descriptor data set for matching until a matching local descriptor is found or the most robust local descriptor data set is traversed; all the extracted local descriptors are clustered as the first layer through the bag-of-words model to obtain a root node; the child nodes under the root node are re-clustered until the child node clustering is completed, and a dictionary tree is constructed based on the root node, all child nodes and the clustering results.

[0060] In another embodiment of the present application, the embodiment further provides a method for estimating the position and posture of a target object. The operating architecture of the method is as follows: Figure 2 Specifically, Figure 2 As shown, the method includes:

[0061] The robot is photographed by one or more image photographing devices to obtain multiple frames of images;

[0062] All images are input into the feature extraction model deployed on the CPU to obtain the key points, local descriptors and global descriptors of each frame. In this embodiment or other embodiments, when the feature extraction model is used to extract features, each image first enters the shared encoder and then enters the three decoders. The three decoders complete the feature point detection scoring, local descriptor extraction and global descriptor extraction tasks respectively. Specifically, the feature point detection scoring process includes: inputting the image to be processed into the model to extract the multi-scale features of the image, and then scoring by correctness: Correctness: For the pose estimation value estimated at a certain moment, the pose true value at that moment is given at the same time, and the correctness evaluation is obtained by calculating the error between the estimated value and the pose true value. The local descriptor extraction process includes: generating a feature map through the convolution layer of CNN; directly treating the feature map as a local descriptor, or further processing the feature map (such as normalization, dimensionality reduction) to extract a more compact descriptor. The process of global descriptor extraction includes: extracting image features through the convolution layer, pooling layer, etc. of CNN. Aggregate local features into global features through a fully connected layer or a global average pooling layer. In this embodiment, the first two decoders of the feature extraction model adopt a local feature convolutional neural network architecture, and the other decoder is a global feature convolutional neural network architecture. Such a network structure design enables the convolutional neural network to extract global and local features of the image with a single forward reasoning. The extracted features are not only beneficial to the traditional SLAM posture tracking task, but also help to improve the performance of the loop detection and relocation module of the SLAM algorithm. In addition, in this embodiment, the process of local feature extraction by the feature extraction model on the CPU includes: convolution operation, the convolution layer slides on the input image through a set of learnable filters, multiplies and sums point by point with the local area of ​​the image, and generates a new feature map. This process can capture local features in the image, such as edges, textures, etc. Activation function, after being processed by the activation function, each pixel of the feature map will be mapped to a new value according to the characteristics of the activation function, thereby enhancing the representation ability of the feature. Pooling operation, the pooling layer downsamples the feature map, reduces the size of the feature map, and retains the most important features in the image. Feature deepening, alternating convolution and pooling, multi-scale features of the image. The process of global feature extraction by the feature extraction model on the CPU includes: The extraction of global features usually relies on the combination of multi-layer convolution and pooling operations, as well as the processing of fully connected layers. Specifically, multi-layer convolution and pooling: By stacking multiple convolution layers and pooling layers, CNN is able to gradually extract and identify higher-level image features. Each convolution layer is able to extract more abstract features from the output of the previous layer and pass them to the next layer. Fully connected layer: Each neuron in the fully connected layer is connected to all neurons in the previous layer, and the features are mapped to the output space through matrix multiplication operations and activation function processing. The global feature extraction process is achieved through the combination of convolution operations, activation functions, pooling operations, and fully connected layers.

[0063] The robot's pose estimation is performed based on the key points and local descriptors; in this embodiment or other embodiments, if the key points of the current frame image and the local descriptor local map do not match, it indicates that the pose estimation prediction fails. On the contrary, if the key points of the current frame image and the local descriptor local map both match, it indicates that the pose estimation prediction is successful.

[0064] When the pose estimation prediction is successful, the local map tracking process is entered, including: updating the local map keyframe and the 3D map points of the local map: traversing the map points of all frame images, recording all keyframe images that can observe the map points, recording the information of the keyframe images, taking the keyframe images that can observe the map points as the local keyframe, and updating the reference keyframe of the current frame image; then clearing all the map points in the current local map, and then adding the map points in the updated local map keyframe. Filter the newly added map points in the local map within the field of view, and project them to the current frame image to search for matches; first exclude the map points corresponding to the current frame image and the map points that are not within the field of view of the current frame image, and then determine the search radius for matching. The more stable the current tracking state, the smaller the search radius; then project the local map points to the current frame image to find projection matches. Estimate the pose of the current frame image; estimate the pose corresponding to the current frame image based on the visual optimization of the previous frame image and the current frame image. Finally, determine whether the current tracking of the local map link is successful based on the number of map points projected on the current frame image. The specific map point number threshold is set according to actual needs. When the local map tracking is successful, the key frame images in the map of this area will be extracted and added to the key frame database.

[0065] A visual dictionary tree is constructed based on the features extracted by the feature extraction model. In order to shorten the dictionary training time and improve the dictionary image representation ability, this embodiment first uses the feature extraction model to extract key points and local descriptors from a continuous image sequence, and selects the most robust local descriptors based on the feature point detection score of the feature extraction model, with high accuracy and good robustness. Then, the string matching algorithm is used to match adjacent images, starting from the first descriptor of the most robust local descriptor data set and comparing it with the descriptors in the adjacent images. If they are equal, continue to compare the subsequent ones; if they are not equal, backtrack and restart the comparison with the adjacent images until a matching descriptor is found or the entire data set is traversed. By traversing and comparing the two data sets, the local descriptors that are successfully matched belong to the same child node, and the features that are not successfully matched are designated as new child nodes. Finally, a dictionary tree is established through the fast bag-of-words model to improve the efficiency of feature matching. The process of establishing the dictionary tree is as follows: cluster all the extracted local descriptors to obtain the root node and complete the first layer of clustering; re-cluster the local descriptors under each root node of the first layer to complete the second layer of clustering; continue to cluster in the above way until clustering to the leaf layer, which is the word layer.

[0066] Loop detection is performed through local maps. The local features of the feature extraction model are quantized into words through the trained incremental feature dictionary, that is, the local features are converted into vector representations. After obtaining the word vectors using the incremental feature dictionary, a scoring mechanism is used to calculate the similarity between the current frame image and the images in the key frame database, and the K candidate key frames with the highest similarity to the current frame image are obtained. The distances between the global features corresponding to the current frame image and the global features corresponding to the K candidate frames are calculated in turn, and the candidate frame with the smallest distance is determined as the loop candidate frame. The loop candidate frame and the current frame image are regarded as a loop pair, and the current posture is considered to be in exactly the same position as before.

[0067] When the pose estimation prediction fails, the relocalization function will be enabled. During the relocalization process, the bag-of-words model is first used to retrieve candidate relocalization frames with high similarity to the current frame image from the key frame database, and then the frame-to-frame matching method is used to match the local features of the current frame image and the relocalization candidate frames to estimate the pose of the current frame image until all candidate frames are traversed or the pose is estimated. This is equivalent to first using the global features extracted by the feature extraction model, then retrieving the relocalization candidate frames in the key frame database, and then grouping the retrieved relocalization candidate frames according to the co-viewing relationship, and finally estimating the pose of the current frame image and each group of candidate frames.

[0068] It can be seen that in the feature extraction part, this embodiment proposes a real-time feature extraction solution that can be deployed on the central processing unit CPU based on the convolutional neural network; in the loop detection part, this embodiment first conducts an in-depth analysis of the features, clusters the extracted local feature descriptors based on the bag-of-words model, and the trained binary dictionary tree can greatly shorten the algorithm initialization time and improve the feature matching efficiency; secondly, based on the trained dictionary tree, the loop detection solution combining global and local features can effectively reduce the cumulative positioning error and achieve globally consistent mapping. At the same time, when the pose estimation prediction fails, this embodiment can reposition based on global features, thereby improving the positioning accuracy of the robot in scenes with changing lighting and perspectives, and can run in real time without the central processing unit GPU and relying only on the image processing unit CPU.

[0069] In summary, the present application provides a method for estimating the pose of a target object, first obtaining multiple frames of target images containing all or part of the target object captured in advance or in real time, and then selecting a frame of target image from the multiple frames of target images as the current frame image; then extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model constructed in advance or in real time based on a convolutional neural network; then performing pose estimation prediction on the target object based on the key points and local descriptors; after the pose estimation prediction fails, a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold is retrieved from a key frame database established in advance or in real time through a bag of words model and a global descriptor established in advance or in real time, and the local features of the current frame image and the relocation candidate frame image are matched, and the pose of the target object in the current frame image is estimated based on the local feature matching results of the current frame image and the relocation candidate frame image; or, after the pose estimation prediction is successful, a loop detection is performed on the target object through a dictionary tree and a local map established in advance or in real time, and loop correction is performed on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag of words model. It can be seen that this method can not only improve the positioning accuracy of target objects (such as robots) in scenes with changing lighting and perspectives by extracting local and global features of each frame of image, and performing pose tracking, local map tracking, relocalization and loop detection correction based on the local and global features of each frame of image, but also when the feature extraction model is deployed in the central processing unit, the pose estimation of the target object can be achieved without the need for a graphics processing unit, thereby saving computing resources and reducing the positioning cost of the target object.

[0070] In another exemplary embodiment of the present application, Figure 3 As shown, this embodiment also provides a target object pose estimation system, including:

[0071] The image acquisition module 310 is used to obtain multiple frames of target images captured in advance or in real time, and select a frame of target image from the multiple frames of target images as the current frame image, and the target image includes all or part of the area containing the target object. As an example, in this embodiment or other embodiments, the target object includes but is not limited to a robot, and the target image can be obtained by one or more image capture devices capturing the robot in advance or in real time.

[0072] The feature extraction module 320 is used to extract key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network as an example, and the feature extraction model can be deployed in a CPU (Central Processing Unit, CPU for short), so that the pose estimation of the target object can be realized without a GPU, which not only saves computing resources, but also reduces the positioning cost of the target object. In this embodiment or other embodiments, the process of extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time includes: extracting multi-scale features from the current frame image through a feature extraction model, and scoring feature points of the multi-scale features, determining feature points representing the current frame image based on the feature point scoring results, and recording them as key points; and extracting local features of each pixel point or each pixel area from the current frame image through a feature extraction model, and recording them as local descriptors; and aggregating all local features through a fully connected layer or a global average pooling layer in the feature extraction model, and using the aggregation result as a global feature of the current frame image, and recording them as global descriptors. As an example, in this embodiment or other embodiments, feature points may refer to some special places in the image, such as corners, edges, and blocks in the image; key points may refer to the positions of feature points in the image; local descriptors refer to descriptors extracted from each pixel or small area in the image, which can be represented by local features; global descriptors refer to descriptors of the entire image, which can be represented by global features.

[0073] The pose estimation prediction module 330 is used to perform pose estimation prediction on the target object based on the key points and local descriptors. In this embodiment or other embodiments, if the key points of the current frame image and the local descriptor local map do not match, it means that the pose estimation prediction fails. On the contrary, if the key points of the current frame image and the local descriptor local map both match, it means that the pose estimation prediction is successful.

[0074] The relocation module 340 is used to retrieve a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold from a key frame database established in advance or in real time through a bag of words model and a global descriptor established in advance or in real time after the pose estimation prediction fails, and match the local features of the current frame image and the relocation candidate frame image, and estimate the pose of the target object in the current frame image based on the local feature matching results of the current frame image and the relocation candidate frame image. In this embodiment or other embodiments, the bag of words model (Bag of Words, BoW) is a text representation method widely used in natural language processing and information retrieval. The bag of words model does not consider the order, grammar and syntax of words in the text, but simply regards the text as a collection of words, or more specifically, a collection of the number of times a word appears. Therefore, the establishment process of the bag of words model will not be repeated in this embodiment, and reference can be made to the relevant technology.

[0075] The loop detection and correction module 350 is used to perform loop detection on the target object through the dictionary tree and local map established in advance or in real time after the pose estimation prediction is successful, and perform loop correction on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model. As an example, in this embodiment or other embodiments, the process of performing loop detection on the target object and performing loop correction on the target object based on the loop detection result includes: calculating the similarity between the current frame image and the key frame image in the key frame database, and obtaining the top K key frame images with the highest similarity value; wherein K is a natural number; performing loop detection on the target object, including sequentially calculating the distance between the global features of the current frame image and the global features of the top K key frame images, and recording the key frame image with the smallest distance as a loop candidate frame image as a loop detection result; taking the loop candidate frame image and the current frame image as a loop pair, and using the loop candidate frame image to perform loop correction on the target object; wherein the poses of the two frames in the same loop pair are the same.

[0076] It can be seen from this that this embodiment extracts local features and global features of each frame image, and performs pose tracking, local map tracking, relocalization and loop detection correction based on the local features and global features of each frame image. This can not only improve the positioning accuracy of the target object (such as a robot) in scenes with changing lighting and perspective, but also when the feature extraction model is deployed in the central processing unit, the pose estimation of the target object can be achieved without the need for a graphics processor, thereby saving computing resources and reducing the positioning cost of the target object.

[0077] According to the above records, in an exemplary embodiment, after the pose estimation prediction is successful, the present embodiment may further include: traversing the map points of multiple frames of target images, and recording all key frame images where the map points can be observed, and using the key frame images where the map points can be observed as local key frames; updating the reference key frame of the current frame image based on the local key frame, and adding the map points of the reference key frame after clearing all map points in the current local map; wherein the map points of the current local map in the initial state are empty; filtering the newly added map points within the field of view in the current local map, and projecting them into the current frame image for search and matching; judging whether the current local map is tracked successfully based on the number of map points in the current frame image that are projected and searched for, and adding the key frames in the current local map to the key frame database when the tracking is successful, and performing loop detection on the target object using the current local map; wherein, if the number of map points is greater than or equal to the preset map point value, it indicates that the current local map tracking is successful; otherwise, if the number of map points is less than the preset map point value, it indicates that the current local map tracking fails.

[0078] According to the above records, in an exemplary embodiment, before extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time, this embodiment may also include: using a convolutional neural network architecture to construct a first decoder and a second decoder for extracting local depth features in the image, and constructing a third decoder for extracting global depth features in the image; and placing the first decoder and the second decoder at the front end of the third decoder so that each frame of the target image first enters the shared encoder, and then enters the first decoder, the second decoder and the third decoder respectively. Select an encoder as a shared encoder, and connect the output of the shared encoder to the first decoder, the second decoder and the third decoder respectively. Use the shared encoder, the first decoder, the second decoder and the third decoder to construct a feature extraction model. Among them, the first decoder is used to extract key points, the second decoder is used to extract local descriptors, and the third decoder is used to extract global descriptors. In this embodiment or other embodiments, the first two decoders of the feature extraction model adopt a local feature convolutional neural network architecture, and the other decoder is a global feature convolutional neural network architecture. This network structure design enables the convolutional neural network to extract global and local features of the image with a single forward inference. The extracted features are not only beneficial to traditional SLAM pose tracking tasks, but also help improve the performance of loop detection and relocalization modules in the SLAM algorithm.

[0079] According to the above records, in an exemplary embodiment, the process of constructing a dictionary tree based on the bag-of-words model includes: obtaining multiple frames of training images, each frame of training image includes all or part of the area containing the target object; extracting key points and local descriptors from each frame of training image through a feature extraction model, and screening out the most robust local descriptors according to the detection scores of the key points, and generating the most robust local descriptor data set; starting from the first local descriptor of the most robust local descriptor data set, matching with the local descriptors in the adjacent frame training images using a string matching algorithm to determine whether the two are equal; if they are equal, the successfully matched local descriptors are attributed to the same child node; if they are not equal, the local descriptors that have not been successfully matched are designated as new child nodes, and local descriptors are reselected from the most robust local descriptor data set for matching until a matching local descriptor is found or the most robust local descriptor data set is traversed; all the extracted local descriptors are clustered as the first layer through the bag-of-words model to obtain a root node; the child nodes under the root node are re-clustered until the child node clustering is completed, and a dictionary tree is constructed based on the root node, all child nodes and the clustering results.

[0080] In summary, the present application provides a target object pose estimation system, which first obtains multiple frames of target images containing all or part of the target object that are taken in advance or in real time, and then selects a frame of target image from the multiple frames of target images as the current frame image; then extracts key points, local descriptors and global descriptors from the current frame image through a feature extraction model that is pre- or real-time constructed based on a convolutional neural network; then performs pose estimation prediction on the target object based on the key points and local descriptors; after the pose estimation prediction fails, a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold is retrieved from a pre- or real-time key frame database through a pre- or real-time established bag of words model and a global descriptor, and the local features of the current frame image and the relocation candidate frame image are matched, and the pose of the target object in the current frame image is estimated based on the local feature matching results of the current frame image and the relocation candidate frame image; or, after the pose estimation prediction is successful, a loop detection is performed on the target object through a pre- or real-time established dictionary tree and a local map, and loop correction is performed on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag of words model. It can be seen that this system can not only improve the positioning accuracy of target objects (such as robots) in scenes with changing lighting and perspective by extracting local and global features of each frame of image, and performing pose tracking, local map tracking, relocalization and loop detection correction based on the local and global features of each frame of image, but also when the feature extraction model is deployed in the central processing unit, the pose estimation of the target object can be achieved without the need for a graphics processor, thereby saving computing resources and reducing the positioning cost of the target object.

[0081] It should be noted that the target object pose estimation system provided in the above embodiment and the target object pose estimation method provided in the above embodiment belong to the same concept, wherein the specific manner in which the target object pose estimation method performs operations has been described in detail in the above method embodiment and will not be repeated here. In actual applications, the target object pose estimation system provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the target object pose estimation system into different functional modules, and then implement all or part of the functions of the corresponding functional modules through the target object pose estimation method described in the above embodiment, and no specific restrictions are made here.

[0082] The present application also provides a computer device, which may include a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program so that the computer device executes Figure 1 The steps of the target object pose estimation method. Figure 4 FIG. 1 is a schematic diagram showing a structure of a computer device 1000. Figure 4 As shown, the computer device 1000 includes: a processor 1010 , a memory 1020 , a power supply 1030 , a display unit 1040 , and an input unit 1060 .

[0083] The processor 1010 is the control center of the computer device 1000. It uses various interfaces and lines to connect various components, and executes various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby monitoring the computer device 1000 as a whole. In the embodiment of the present application, when the processor 1010 calls the computer program stored in the memory 1020, the following is executed: Figure 1 The steps of the method for estimating the pose of the target object. Optionally, the processor 1010 may include one or more processing units; preferably, the processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and applications, etc., and the modem processor mainly processes wireless communications. In some embodiments, the processor, memory, and memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0084] The memory 1020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store instruction data, etc. created according to the use of the computer device 1000. In addition, the memory 1020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices, etc.

[0085] The computer device 1000 also includes a power source 1030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 1010 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0086] The display unit 1040 may be used to display information input by the user or information provided to the user and various menus of the computer device 1000, and in the embodiment of the present application, is mainly used to display the display interface of each application in the computer device 1000 and objects such as text and pictures displayed in the display interface. The display unit 1040 may include a display panel 1050. The display panel 1050 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0087] The input unit 1060 may be used to receive information such as numbers or characters input by the user. The input unit 1060 may include a touch panel 1070 and other input devices 1080. The touch panel 1070, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 1070).

[0088] Specifically, the touch panel 1070 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 1010, and receive and execute the command sent by the processor 1010. In addition, the touch panel 1070 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. Other input devices 1080 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0089] Of course, the touch panel 1070 can cover the display panel 1050. When the touch panel 1070 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of touch event, and then the processor 1010 provides a corresponding visual output on the display panel 1050 according to the type of touch event. Figure 4 In the embodiment, the touch panel 1070 and the display panel 1050 are two independent components to implement the input and output functions of the computer device 1000, but in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to implement the input and output functions of the computer device 1000.

[0090] The computer device 1000 may further include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the computer device 1000 may also include other components such as a camera.

[0091] The embodiment of the present application also provides a computer-readable storage medium, in which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the above-mentioned device can perform the above-mentioned Figure 1 The steps of the target object pose estimation method.

[0092] It can be understood by those skilled in the art that Figure 4 This is only an example of a computer device and does not constitute a limitation on the device. The device may include more or fewer components than shown in the figure, or combine certain components, or different components. For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0093] Those skilled in the art should understand that the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be applied to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] It should be noted that the above embodiments collect, store, use, process, transmit, provide, disclose, delete, etc. related data (such as target images, etc.) with or after obtaining the user's consent. For example, the target image is authorized with the user's knowledge and consent; or the user actively provides it after reading the relevant instructions, or the user actively authorizes / provides / uploads it when using some or all of the functions described in the above embodiments, or other methods / ways are used with or after obtaining the user's consent.

[0095] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for estimating a target object's pose, characterized in that: The method comprises the following steps: Acquire multiple frames of target images captured in advance or in real time, and select one frame of target image from the multiple frames of target images as the current frame of image, wherein the target image includes all or part of the area containing the target object; Extracting key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network; Performing pose estimation and prediction on the target object based on the key points and the local descriptors; After the pose estimation prediction fails, a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold is retrieved from a key frame database established in advance or in real time through a bag-of-words model established in advance or in real time and the global descriptor, and local features of the current frame image and the relocation candidate frame image are matched, and the pose of the target object in the current frame image is estimated based on the local feature matching result of the current frame image and the relocation candidate frame image; Alternatively, after the pose estimation prediction is successful, loop detection is performed on the target object through a dictionary tree and a local map established in advance or in real time, and loop correction is performed on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model.

2. The method for estimating the position and pose of a target object according to claim 1, characterized in that: After the pose estimation prediction is successful, the method further includes: Traversing the map points of the multiple target image frames, and recording all key frame images in which the map points can be observed, and using the key frame images in which the map points can be observed as local key frames; Based on the local key frame, the reference key frame of the current frame image is updated, and the map points of the reference key frame are added after all map points in the current local map are cleared; wherein the map points of the current local map in the initial state are empty; Filtering newly added map points within the field of view in the current local map and projecting them into the current frame image for search and matching; Based on the number of map points matched by the projection search in the current frame image, it is determined whether the current local map is tracked successfully, and when the tracking is successful, the key frames in the current local map are added to the key frame database, and the current local map is used to perform loop detection on the target object; wherein, if the number of map points is greater than or equal to the preset map point value, it indicates that the current local map tracking is successful; otherwise, if the number of map points is less than the preset map point value, it indicates that the current local map tracking has failed.

3. The method for estimating the target object pose according to claim 1 or 2, characterized in that: The process of performing loop detection on the target object and performing loop correction on the target object based on the loop detection result includes: Calculate the similarity between the current frame image and the key frame images in the key frame database, and obtain the first K key frame images with the highest similarity values; wherein K is a natural number; Performing loop detection on the target object, including sequentially calculating the distance between the global features of the current frame image and the global features of the first K key frame images, and recording the key frame image with the smallest distance as a loop candidate frame image as the loop detection result; The loop closure candidate frame image and the current frame image are taken as a loop closure pair, and the loop closure candidate frame image is used to perform loop correction on the target object; wherein the postures of the two frame images in the same loop closure pair are the same.

4. The method for estimating the target object pose according to claim 1 or 2, characterized in that: The process of extracting key points, local descriptors and global descriptors from the current frame image by using a feature extraction model generated in advance or in real time includes: Extracting multi-scale features from the current frame image by using the feature extraction model, and scoring feature points of the multi-scale features, and determining feature points representing the current frame image based on the feature point scoring results, which are recorded as key points; and Extracting local features of each pixel point or each pixel region from the current frame image by using the feature extraction model, and recording them as local descriptors; and All local features are aggregated through the fully connected layer or the global average pooling layer in the feature extraction model, and the aggregation result is used as the global feature of the current frame image, which is recorded as a global descriptor.

5. The method for estimating the position and pose of a target object according to claim 4, characterized in that: Before extracting key points, local descriptors and global descriptors from the current frame image by using a feature extraction model generated in advance or in real time, the method further includes: A first decoder and a second decoder for extracting local depth features in an image are constructed using a convolutional neural network architecture, and a third decoder for extracting global depth features in an image is constructed; Selecting an encoder as a shared encoder, and connecting the output of the shared encoder to the first decoder, the second decoder, and the third decoder respectively; Constructing the feature extraction model using the shared encoder, the first decoder, the second decoder and the third decoder; The first decoder is used to extract key points, the second decoder is used to extract local descriptors, and the third decoder is used to extract global descriptors.

6. The method for estimating the position and pose of a target object according to claim 1 or 5, characterized in that: The process of constructing the dictionary tree based on the bag-of-words model includes: Acquire multiple frames of training images, each frame of training image includes all or part of the region containing the target object; Extracting key points and local descriptors from each frame of training image through the feature extraction model, screening out the most robust local descriptors according to the detection scores of the key points, and generating the most robust local descriptor data set; Starting from the first local descriptor of the most robust local descriptor data set, matching is performed with the local descriptors in the adjacent frame training images using a string matching algorithm to determine whether the two are equal; if they are equal, the successfully matched local descriptors are assigned to the same child node; if they are not equal, the unmatched local descriptors are designated as new child nodes, and local descriptors are reselected from the most robust local descriptor data set for matching until a matching local descriptor is found or the most robust local descriptor data set is traversed; All local descriptors extracted are clustered as the first layer through the bag-of-words model to obtain a root node; The child nodes under the root node are re-clustered until the child nodes are clustered, and the dictionary tree is constructed based on the root node, all the child nodes and the clustering results.

7. The method for estimating the position and pose of a target object according to claim 5, characterized in that: When constructing the feature extraction model, the method also includes: placing the first decoder and the second decoder at the front end of the third decoder so that each frame of the target image first enters the shared encoder and then enters the first decoder, the second decoder and the third decoder respectively.

8. A target object pose estimation system, characterized in that: The system comprises: An image acquisition module is used to acquire multiple frames of target images taken in advance or in real time, and select a frame of target image from the multiple frames of target images as a current frame of image, wherein the target image includes all or part of an area containing a target object; A feature extraction module, used to extract key points, local descriptors and global descriptors from the current frame image through a feature extraction model generated in advance or in real time; wherein the feature extraction model is constructed based on a convolutional neural network; A pose estimation prediction module, used to perform pose estimation prediction on the target object according to the key points and the local descriptors; A relocation module, for retrieving a relocation candidate frame image whose similarity with the current frame image is greater than or equal to a preset threshold from a key frame database established in advance or in real time through a bag-of-words model established in advance or in real time and the global descriptor, matching local features of the current frame image with the relocation candidate frame image, and estimating the pose of the target object in the current frame image based on the matching result of the local features of the current frame image with the relocation candidate frame image after the pose estimation prediction fails; The loop detection and correction module is used to perform loop detection on the target object through a pre- or real-time established dictionary tree and local map after the pose estimation prediction is successful, and to perform loop correction on the target object based on the loop detection result; wherein the dictionary tree is constructed based on the bag-of-words model.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the target object pose estimation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the target object pose estimation method described in any one of claims 1 to 7 are implemented.

Images