Method, apparatus, device and storage medium for pose estimation

Abstract

According to embodiments of the present disclosure, methods, apparatuses, devices and storage media for pose estimation are provided. The method comprises generating, in response to determining a presentation range of an object in an image, first local features for a first image region within the presentation range, the first image region being associated with a first component part of the object; determining a first position of a key point of the first component part in the image by at least one update of a first initial quantized representation for the first component part with the first local features; and determining a pose of the object based at least on the first position. In this way, the accuracy of pose estimation can be improved while reducing complexity.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to methods, apparatus, devices, and computer-readable storage media for attitude estimation. Background Technology

[0002] Multi-person pose estimation (MPPE) aims to locate keypoints for all persons in an input image, a fundamental yet challenging task in computer vision. With the widespread adoption of deep learning techniques, MPPE has made significant progress and plays a crucial role in many other vision tasks, such as activity recognition and pose tracking. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for pose estimation is provided. The method includes generating first local features for a first image region within the target rendering range, the first image region being associated with a first component of the object, in response to determining a target rendering range of an object in an image. The method further includes determining a first location of keypoints of the first component in the image by updating a first initial quantization representation of the first component at least once using the first local features, and determining the pose of the object based at least on the first location.

[0004] In a second aspect of this disclosure, an apparatus for pose estimation is provided. The apparatus includes a generation module configured to generate first local features for a first image region within the target rendering range of an object in an image, the first image region being associated with a first component of the object, in response to determining a target rendering range of an object in an image. The apparatus also includes a first position determination module configured to determine a first position of keypoints of the first component in the image by updating a first initial quantization representation of the first component at least once using the first local features, and a pose determination module configured to determine the pose of the object based at least on the first position.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. A computer program is stored on the medium, which, when executed by a processor, implements the method of the first aspect.

[0007] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0010] Figure 2 A block diagram illustrating an example implementation of attitude estimation according to some embodiments of the present disclosure is shown;

[0011] Figures 3A to 3C A schematic diagram illustrating the attitude estimation process according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A flowchart illustrating an attitude estimation process according to some embodiments of the present disclosure is shown;

[0013] Figure 5 A block diagram of an apparatus for attitude estimation according to some embodiments of the present disclosure is shown; and

[0014] Figure 6 A block diagram of an apparatus capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0017] As mentioned above, the goal of multi-person pose estimation (MPPE) is to locate keypoints of all objects in an input image, a fundamental yet challenging task in computer vision. With the widespread adoption of deep learning techniques, MPPE has made significant progress and plays an important role in many other vision tasks, such as activity recognition and pose tracking.

[0018] Several multi-person pose estimation solutions have been proposed. For example, one approach uses a human detector to predict the rendering extent (e.g., object bounding boxes) of each object in an image, and then leverages a single-person pose estimation model to locate keypoints on each cropped object image. However, predicting the rendering extent of each object in the image and estimating the single-person pose for each object require two separate models, which makes it impossible to implement a multi-person pose estimation solution through an end-to-end pipeline. That is, implementing this solution requires at least two stages of pipeline. Furthermore, the human detector requires additional memory and computational costs.

[0019] In the second approach, keypoints for all objects are first located using a heatmap, and then these keypoints are assigned to each object through grouping. For example, some studies have attempted to utilize dense pixel-wise regression, which can predict center heatmaps and pixel-wise keypoint offsets in parallel. This approach allows for end-to-end optimization and is therefore more efficient than a two-stage pipeline. However, the dense representation of the image (e.g., keypoints or center heatmaps) requires manual post-processing to suppress duplicate predictions or to perform keypoint grouping during inference. This significantly increases the manual cost of this approach.

[0020] According to various embodiments of this disclosure, a pose estimation scheme is proposed. For example, if a rendering range of an object in an image is detected, local features are generated for an image region within the rendering range that is associated with a component of the object. An initial quantization representation corresponding to the component is obtained, and the initial quantization representation is updated at least once using the generated local features. The keypoint locations of the component are determined based on the updated initial quantization representation, and the pose of the object is determined at least based on these locations.

[0021] According to the implementation of this disclosure, by introducing a spatial local embedding generation module and a selective iteration module, the features of objects in the image can be captured better, thus improving the accuracy of pose estimation. Meanwhile, the multi-person pose estimation scheme of this disclosure can be implemented end-to-end, thereby effectively improving system efficiency. Furthermore, by adopting the scheme proposed in this disclosure, the manual processing step can be eliminated, thus simplifying the multi-person pose estimation process.

[0022] It should be understood that the embodiments of this disclosure can be applied to various fields such as simulation, modeling, virtual reality, and augmented reality. Furthermore, the embodiments of this disclosure can also be applied to the medical and health field, such as object motion behavior prediction and object behavior monitoring.

[0023] Example Environment

[0024] First see Figure 1 The illustration schematically shows a sample environment 100 in which exemplary implementations according to this disclosure may be implemented. Figure 1 As shown, example environment 100 may include electronic device 120.

[0025] Electronic device 120 can be used to acquire one or more images. At least one or more objects can be presented in the images. For example, objects may include people. Alternatively, objects may include other living beings besides people. Alternatively, objects may include other objects capable of mimicking the movement behaviors of living beings.

[0026] like Figure 1 As shown, image 110 presents objects 101 and 102. Image 110 can represent the pose of objects 101 and 102 at the time image 110 was captured. It should be understood that, in addition to the objects, image 110 can also represent the environment (i.e., background) in which the objects are currently located.

[0027] The electronic device 120 can perform relevant image processing. For example, the electronic device 120 can perform operations such as cropping, trimming, and scaling of the image to determine the presentation range of an object in the image and thereby extract object-related features from the image. In embodiments of this disclosure, the term "presentation range" may also be referred to as a "person box" or a "bounding box".

[0028] The electronic device 120 can also generate quantized representations (queries) of object features at a specific granularity. For example, this quantized representation could correspond to features of a single object or to local features of an object. For instance, this quantized representation could be represented using vector embedding. The detailed process of generating quantized representations related to object features will be described in detail below and will not be elaborated upon here.

[0029] The electronic device 120 can also determine the location 130 of the object's key points in the image based on the quantization representation related to the object's features, thereby further estimating the object's pose. For example, when the object is a human body, the key points may include specific feature points selected from the human body's head, torso, and limbs, such as skeletal joints, junctions or boundary points between the head, torso, or limbs.

[0030] The electronic device 120 can not only estimate the pose of the object presented in the current image, but also estimate the motion trend of the object within a certain time interval based on multiple consecutive or discontinuous frames of images. This allows the estimated results to be applicable to various application fields, such as simulation, virtual reality, augmented reality, and healthcare.

[0031] It should be understood that the structure and function of environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0032] Attitude estimation process

[0033] Figure 2 A block diagram illustrating the attitude estimation process according to some embodiments of the present disclosure is shown. Figure 2 The process shown can be implemented at electronic device 120. For ease of discussion, reference will be made to... Figure 1 To describe the environment in terms of 100 Figure 2 The process is shown.

[0034] like Figure 2 As shown, electronic device 120 can acquire image 110. Image 110 presents objects 101 and 102. It should be understood that the image may present more or fewer objects. The number of objects presented in image 110 should not be taken as a limitation on the scheme of this disclosure.

[0035] The feature extraction module 210 of the electronic device 120 can extract features from the image 110. For example, the multi-level features P of the image 110. l Features can be extracted, for example, l = 2, 3, 4, 5. For instance, the level of a feature can be associated with the resolution size. P l This indicates a feature with a resolution of 1 / 2l of the original input and 256 channels. For example, feature P2 represents a feature with a resolution of 1 / 4 of the original input, feature P3 represents a feature with a resolution of 1 / 6 of the original input, and so on.

[0036] The extracted multi-level features, such as features P2 to P5, can be input into the presentation range decoder 220. Multiple initially predicted presentation ranges B 222 are provided to the presentation range decoder 220, each presentation range potentially corresponding to a learnable human instance level quantized representation (query). The multiple initially predicted presentation ranges 222 may include, for example,... Figure 3AThe image 110 shown contains rendering ranges 301 and 302. Rendering range 301 contains the image rendering object 101, while rendering range 302 contains the image rendering object 102. The plurality of initially predicted rendering ranges 222 can be represented by the coordinates of the boundary points of each rendering range.

[0037] Based on the extracted multi-level features and multiple initial prediction rendering ranges 222, the rendering range decoder 220 can perform region feature aggregation, for example, through a ROI Align operation 221, i.e., performing truncation or clipping on features P2 to P5, to generate features for objects within each of the multiple initial prediction rendering ranges 222. 225.

[0038] The generated features 225 of the object within the rendering range for each initial prediction can be correlated with each other using a learnable quantized representation at the human instance level. Learnable initial vector representations for human instances can be obtained. 223. For example, in the initial training state, this initial vector representation 223 can be the same for all human instances. By updating the initial vector representation 223 with the features 225 of the object for each initially predicted presentation range, a vector representation of the features of the object for each initially predicted presentation range can be obtained.

[0039] For example, such as Figure 2 As shown, after processing the initial vector representation 223 through a multi-head self-attention layer (MHSA) 224, the features 225 of the objects for each initially predicted presentation range are processed by a dynamic channel multilayer perceptron (DyMLP). channel ) 226 is aggregated into an initial vector representation 223 processed by a multi-head self-attention layer 224. For example, the initial vector representation 223 processed by the multi-head self-attention layer 224 can be encoded using features 225 of the object for each initially predicted rendering range to update the initial vector representation 223. The updated (enhanced) initial quantization representation, i.e., the updated quantization representation 227, can be used to determine an improved offset for the initially predicted rendering range. In other words, the updated quantization representation 227 can be used to adjust the rendering range of the initial prediction.

[0040] Optionally or additionally, the above process implemented in the render range decoder can be performed iteratively. For example, the adjusted render range 228 can be further used as the initial render range for the next render range improvement process. This process can be repeated multiple times to obtain a more accurate render range (bounding box) for the object.

[0041] In some embodiments, the above-described process for improving the presentation range can be represented as:

[0042] (1)

[0043] in P2 represents the features of objects extracted within the rendering range after ROI alignment, P5 represents the multi-level features extracted from the image, and B represents the rendering range. Head represents the learnable initial vector representation for human instances, S represents the number of repetitions for improved presentation range, and Head represents the number of repetitions for improved presentation range. box This represents a regression head that can predict offsets to improve the rendering range based on the updated vector representation.

[0044] The rendering range improvement process described above can significantly improve the accuracy of object rendering range prediction, thus laying the foundation for accurate object pose estimation.

[0045] Continue to refer to Figure 2 Multiple rendering ranges 228, improved by rendering range decoder 220, can be input to keypoint decoder 230. Feature extraction module 210 can input extracted high-resolution features (e.g., P2) to keypoint decoder 230.

[0046] By leveraging multiple improved rendering ranges 228, a ROI alignment (ROIAlign) operation 231 is performed on the extracted high-resolution features to perform region feature aggregation, generating multiple features for objects within the multiple improved rendering ranges 228. For example, for the respective improved rendering extents of objects 101 and 102 in image 110, features 232-1 and 232-2 for the objects within their respective rendering extents can be generated. These features are compared to those generated at the rendering extent decoder 220 for objects within the initial rendering extent. 225. The features 232-1 and 232-2 generated at the keypoint decoder 230 can have higher accuracy, which is more beneficial for subsequent pose estimation.

[0047] To achieve more refined pose estimation, the local features of the object within the rendering range will be further processed. For example, the image within the rendering range can be divided into multiple image regions, with each region corresponding to a local part of the object. For example, as... Figure 3BAs shown, the presentation area 301 can be divided into multiple image regions 311, 312, 313, and 314, which correspond to the head, shoulders, hips, and legs of the object, respectively. Similarly, a similar image region division can be performed for the object within the presentation area 302.

[0048] It is possible to associate local features of objects within an image region with learnable, part-level quantization representations specific to that part of the object. This allows for the estimation of the pose of specific parts. Local-level quantization representation. For example, it can be represented as , where n represents the nth object instance, and M represents the number of local hierarchical quantization representations for each instance.

[0049] It should be understood that the division of image regions within the presentation range is not limited to... Figure 3B The example shown illustrates how different regions can be divided for different parts of an object within the rendering area. The image within the rendering area can be divided into a number of image regions.

[0050] It should also be understood that Figure 3B The image region divisions illustrated in the examples are merely for the purpose of better understanding the scheme of this disclosure. In practice, the division of different image regions can be a virtual division rather than an actual operation performed on the image.

[0051] Based on the generated features An attention map with a specific number of channels is generated. The number of channels corresponds to the number of local areas of the divided object, that is, the number of image regions divided within the rendering range. By setting the sum of attention weights, a corresponding number of spatially sensitive local embeddings can be generated based on the attention map. In embodiments of this disclosure, spatially sensitive local embeddings are also referred to as local feature vector representations. .

[0052] In some embodiments, a convolutional layer can be used to compress features along the channel dimension and generate an M-channel attention map A. Each channel corresponds to a spatial attention map of a specific locality of the object. M represents the number of localities of the object that are divided, which corresponds to the number of local hierarchical quantization representations for each instance. The attention map A is flattened along the spatial dimension, and the sum of the attention weights is set. By manipulating the spatial attention map of a specific locality, M spatially sensitive local feature vector representations can be generated based on the spatial features. This process can be represented as ,in This represents the dot product.

[0053] like Figure 2 As shown, it can be based on the generated features Attention maps 232-1 and 232-2 are generated, respectively. Each attention map A 233-1 and 233-2 can have multiple channels M, each corresponding to a local portion of an object within a defined image region of the rendering area. Still using... Figure 3B Taking the image region division of the presentation range 301 as an example, channel 201 of attention map 233-1 can correspond to a local part of object 101 within image region 311, channel 202 of attention map 233-1 can correspond to a local part of object 101 within image region 312, channel 203 of attention map 233-1 can correspond to a local part of object 101 within image region 313, and channel 204 of attention map 233-1 can correspond to a local part of object 101 within image region 314. By adjusting the attention weights for different local parts, multiple local feature vector representations for different parts of object 101 can be generated, such as local feature vector representation 234-1. Similarly, multiple local feature vector representations for different parts of object 102 can also be generated for attention map 233-2, such as local feature vector representation 234-2.

[0054] The process of generating spatially sensitive local vector representations described above can be performed in the Spatial Local Embedding Generation (SPEGM) 241 of the keypoint decoder 230.

[0055] In the selective iteration module (SIM) 242, multiple local feature vectors generated for different parts of different objects can be used as a basis. To update the quantization representation at the local level This allows us to estimate the location of the object's key points.

[0056] In some embodiments, the quantization representation of the local hierarchy It can be a quantized representation of a local level that has already been updated in the previous stage, i.e. Local eigenvectors represent It can be a representation of the local feature vector generated in the current stage, i.e. The MLP can be performed together with the sigmoid operation to output two weight vectors. and It can be used as a control and The contribution can be integrated using a weighted summation method. and The above process can be represented, for example, as:

[0057] (2)

[0058] For example, such as Figure 2 As shown, the generated local feature vector representations 234-1 and 234-2 can be updated at least once by iterating through 236-1 and 236-2 to the local-level quantization representation 235. The updated local-level quantization representations can be input into MHSA layers 237-1 and 237-2 to explore the spatial relationships between different parts of different objects. Thus, the global structural features for each object are also encoded into the updated local-level quantization representation 238, thereby enabling the regression of the object's keypoint coordinates 130.

[0059] Through the above key point regression process, for example Figure 3C As shown, the positions of the head key points 321 to 325 of the object corresponding to image region 311, the shoulder key points 331 and 332 of the object corresponding to image region 312, the hip key points 333 and 334 of the object corresponding to image region 313, and the leg key points 341 to 344 of the object corresponding to image region 314 can be determined.

[0060] In some embodiments, for keypoint regression, the potential key distribution can be analyzed. Capture is performed, and keypoint regression loss is modeled from the perspective of maximum likelihood estimation. Density gradation. Reflects ROI Pose The probability of annotating a keypoint at position x, where θ and These are the trainable parameters for the regression network and the flow model, respectively. For easier optimization, the flow model F is used. Map the simple distribution to a deformed P ( The regression network output two values, namely... and To move and scale the distribution P ( It can be represented as x= * + . It can represent the predicted value at a given location point, while It can represent the uncertainty score of the key point location.

[0061] In some embodiments, the process of determining the location of the key points can be represented as follows:

[0062] (3)

[0063] in This represents multiple features of objects within the improved rendering range 228. Representation of spatially sensitive local feature vectors, This represents the quantization of the local hierarchy, where S represents the number of repetitions for improving the presentation range. This indicates the key point location 130.

[0064] It should be understood that the vector representation of a human instance It can be serially iterated between the rendering range decoder 220 and the keypoint decoder 230, with quantization representation at the local level. The keypoint decoder 230 can iterate between each other. The rendering range decoder 220 of the next stage uses the determined rendering range of the current stage as the initial rendering range. The keypoint coordinates of each stage are predicted independently.

[0065] According to the implementation of this disclosure, by introducing a spatial local embedding generation module and a selective iteration module, the loss and cost in the pose estimation process can be significantly reduced, further improving the accuracy of pose estimation. Meanwhile, the multi-person pose estimation scheme of this disclosure can be implemented in an end-to-end manner, thereby effectively improving system efficiency and reducing the complexity of the implementation scheme.

[0066] Furthermore, the multi-person pose estimation scheme proposed through the embodiments of this disclosure can effectively avoid the manual processing steps used in traditional schemes, thereby simplifying the pose estimation process and further increasing the accuracy of the estimation.

[0067] Example process

[0068] Figure 4 A flowchart of a process 400 for attitude estimation according to some embodiments of the present disclosure is shown. Process 400 may be implemented at an electronic device 120.

[0069] In box 410, if a target rendering range of an object in an image is determined, in box 420, electronic device 120 generates a first local feature for a first image region within the target rendering range, the first image region being associated with a first component of the object.

[0070] In some embodiments, the electronic device 120 may acquire original feature information of the first image region; generate an attention representation of the first component of the object within the first image region; and generate a first local feature for the first image region by adjusting the attention representation based on the original feature information and a predetermined attention weight for the first component.

[0071] In frame 430, electronic device 120 determines the first location of key points of the first component in the image by updating the first initial quantization representation for the first component at least once using the first local features.

[0072] In some embodiments, the electronic device 120 may determine a feature quantization representation corresponding to the first local feature; encode the first initial quantization representation using the feature quantization representation; and determine the first position based on the encoded initial quantization representation.

[0073] At 440, the electronic device 120 determines the pose of the object based at least on the first position.

[0074] In some embodiments, the electronic device 120 may also generate a second local feature for a second image region within the target rendering range, the second image region being associated with a second component of the object, the second component being different from the first component; and determine a second location of key points of the second component in the image by updating a second initial quantization representation of the second component at least once using the second local feature.

[0075] In some embodiments, the electronic device 120 may also determine the spatial positional relationship between the first component and the second component based on the updated first local feature and the updated second local feature; and determine the pose of the object based on the first position, the second position and the spatial positional relationship.

[0076] In some embodiments, the electronic device 120 may further determine an initial prediction of the rendering range for the object and object features of the object within the rendering range of the initial prediction; obtain an initial quantization representation for the object; adjust the initial prediction of the initial rendering range by updating the initial quantization representation of the object at least once using the object features; and determine the target rendering range based on the adjusted rendering range of the initial prediction.

[0077] Example devices and equipment

[0078] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an apparatus 500 for data processing according to some embodiments of the present disclosure is shown.

[0079] like Figure 5As shown, the apparatus 500 may include a generation module 510 configured to generate a first local feature for a first image region within the target rendering range, the first image region being associated with a first component of the object, in response to determining a target rendering range of an object in an image. The apparatus 500 also includes a first position determination module 520 configured to determine a first position of a key point of the first component in the image by updating a first initial quantization representation of the first component at least once using the first local feature. The apparatus 500 further includes a pose determination module 530 configured to determine the pose of the object based at least on the first position.

[0080] In some embodiments, the generation module 510 may also be configured to acquire original feature information of the first image region; generate an attention representation of the first component of the object within the first image region; and generate a first local feature for the first image region by adjusting the attention representation based on the original feature information and a predetermined attention weight for the first component.

[0081] In some embodiments, the first position determination module 520 may also be configured to determine a feature quantization representation corresponding to the first local feature; encode the first initial quantization representation using the feature quantization representation; and determine the first position based on the encoded initial quantization representation.

[0082] In some embodiments, the apparatus 500 may also be configured to generate a second local feature for a second image region within the target rendering range, the second image region being associated with a second component of the object, the second component being different from the first component; and to determine a second location of key points of the second component in the image by updating a second initial quantization representation of the second component at least once using the second local feature.

[0083] In some embodiments, the attitude determination module 530 may also be configured to determine the spatial positional relationship between the first component and the second component based on the updated first local feature and the updated second local feature; and to determine the attitude of the object based on the first position, the second position and the spatial positional relationship.

[0084] In some embodiments, the apparatus 500 may also be configured to determine an initial prediction of the rendering range for the object and object features of the object within the rendering range of the initial prediction; obtain an initial quantization representation for the object; adjust the initial prediction of the initial rendering range by updating the initial quantization representation of the object at least once using the object features; and determine the target rendering range based on the adjusted rendering range of the initial prediction.

[0085] The units included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 500 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0086] Figure 6 A block diagram of a computing device / server 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 6 The computing device / server 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0087] like Figure 6 As shown, the computing device / server 600 is in the form of a general-purpose computing device. Components of the computing device / server 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage devices 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capabilities of the computing device / server 600.

[0088] The computing device / server 600 typically includes multiple computer storage media. Such media can be any available media accessible to the computing device / server 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data (e.g., training data for training) and accessible within the computing device / server 600.

[0089] The computing device / server 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0090] The communication unit 640 enables communication with other computing devices via a communication medium. Additionally, the functionality of the components of the computing device / server 600 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device / server 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or second network nodes.

[0091] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. The computing device / server 600 can also communicate as needed with one or more external devices (not shown) via communication unit 640. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with the computing device / server 600, or with any device (e.g., network card, modem, etc.) that enables the computing device / server 600 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0092] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein one or more computer instructions are executed by a processor to implement the methods described above.

[0093] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0094] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0095] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0097] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A pose estimation method, comprising: In response to determining the target rendering range of an object in an image, a first local feature is generated for a first image region within the target rendering range, the first image region being associated with a first component of the object, and wherein the first local feature is generated by adjusting the attention representation of the first component based on the original feature information of the first image region and a predetermined attention weight for the first component; The first location of the key point of the first component in the image is determined by updating the first initial quantization representation of the first component at least once using the first local features. as well as The pose of the object is determined at least based on the first position. And the at least one update of the first initial quantization representation includes: Obtain the corresponding weight vectors used to control the contributions of the first local feature and the first initial quantized representation; By employing a weighted summation method, the first initial quantization representation is updated at least once based on the corresponding weight vector and the first local feature.

2. The method of claim 1, wherein determining the first position comprises: Determine the feature quantization representation corresponding to the first local feature; The first initial quantization representation is encoded using the feature quantization representation; as well as The first position is determined based on the encoded initial quantization representation.

3. The method according to claim 1, further comprising: Generate a second local feature for a second image region within the target rendering range, the second image region being associated with a second component of the object, the second component being different from the first component; and The second location of the key points of the second component in the image is determined by updating the second initial quantization representation of the second component at least once using the second local features.

4. The method of claim 3, wherein determining the pose of the object comprises: The spatial relationship between the first component and the second component is determined based on the updated first local feature and the updated second local feature; as well as The pose of the object is determined based on the first position, the second position, and the spatial positional relationship.

5. The method according to claim 1, further comprising: Determine an initial prediction of the rendering range of the object and the object features of the object within the rendering range of the initial prediction; Obtain the initial quantization representation for the object; The initial prediction of the presentation range is adjusted by updating the initial quantization representation of the object at least once using the object features; as well as The target rendering range is determined based on the adjusted rendering range of the initial prediction.

6. An apparatus for attitude estimation, comprising: The generation module is configured to generate a first local feature for a first image region within the target rendering range in response to determining a target rendering range of an object in an image, the first image region being associated with a first component of the object, and wherein the first local feature is generated by adjusting the attention representation of the first component based on original feature information of the first image region and a predetermined attention weight for the first component. A first location determination module is configured to determine a first location of a key point of the first component in the image by updating a first initial quantization representation for the first component at least once using the first local features. as well as The pose determination module is configured to determine the pose of the object based at least on the first position. Furthermore, the first position determination module is also configured to: obtain a corresponding weight vector for controlling the contribution of the first local feature and the first initial quantization representation; and update the first initial quantization representation at least once based on the corresponding weight vector and the first local feature by employing a weighted summation method.

7. The apparatus of claim 6, wherein the first position determining module is further configured to: Determine the feature quantization representation corresponding to the first local feature; The first initial quantization representation is encoded using the feature quantization representation; and The first position is determined based on the encoded initial quantization representation.

8. The apparatus according to claim 6, further configured as follows: Generate a second local feature for a second image region within the target rendering range, the second image region being associated with a second component of the object, the second component being different from the first component; and The second location of the key points of the second component in the image is determined by updating the second initial quantization representation of the second component at least once using the second local features.

9. The apparatus of claim 8, wherein the attitude determination module is further configured to: The spatial relationship between the first component and the second component is determined based on the updated first local features and the updated second local features; and The pose of the object is determined based on the first position, the second position, and the spatial positional relationship.

10. The apparatus according to claim 6, further configured to: Determine an initial prediction of the rendering range of the object and the object features of the object within the rendering range of the initial prediction; Obtain the initial quantization representation for the object; The initial prediction of the presentation range is adjusted by updating the initial quantization representation of the object at least once using the object features; as well as The target rendering range is determined based on the adjusted rendering range of the initial prediction.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images