Edge extraction-based pose estimation method and system

Abstract

The application provides an edge extraction-based pose estimation method and system, which comprises the following steps: collecting an object image, constructing a pose estimation network, wherein the pose estimation network comprises a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer and a pose parameter regression layer; the feature and edge extraction layer receives the object image and performs extraction and mapping to obtain an edge response feature map; the edge consistency fusion layer constructs an edge consistency weight and obtains an edge enhanced feature map through feature fusion; the edge confidence aggregation layer obtains an edge confidence map through splicing mapping, and further constructs an edge guided pose feature; the pose parameter regression layer obtains a predicted pose result through parameter regression and matrix conversion; a pose estimation model is obtained through model training and inference, and a predicted pose result of an object to be estimated is obtained. The method of the application realizes fast and accurate estimation of the pose of the object to be estimated.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a pose estimation method and system based on edge extraction. Background Technology

[0002] Pose estimation describes the 3D rotation and translation of an object relative to the camera coordinate system, and has important applications in fields such as robot grasping, automated assembly, intelligent detection, autonomous navigation, and augmented reality. Existing pose estimation methods are usually based on the object's surface texture, local keypoints, depth information, or learn high-level semantic features through deep neural networks to determine the object's pose. However, for surfaces with missing or weak textures, or for objects lacking stable and repeatable texture keypoints within the image, pose estimation relies more on the object's outer contour, structural edges, and local geometric changes. Furthermore, in marine, underwater, strongly reflective, and complex lighting environments, the acquisition of object images may be affected by underwater scattering, local shadows, specular reflection, background interference, and imaging noise. Moreover, relying solely on high-level semantic features at a single scale can easily lose fine edge positions, while relying solely on shallow features is susceptible to illumination and noise interference.

[0003] In pose estimation, existing multi-scale feature fusion methods typically employ direct addition or concatenation after channel alignment, failing to adaptively control the fusion ratio based on the consistency of edge responses across different scales. Furthermore, setting independent edge extraction parameters for features at different scales increases the number of model parameters and lacks a unified criterion for judging edge responses at different scales. Additionally, directly applying global average pooling to the highest-level feature map results in similar aggregation weights for background regions and textureless areas within the object compared to the object's contour region, hindering the recovery of stable poses from limited edge cues. Summary of the Invention

[0004] To address the problems of inaccurate edge feature extraction, lack of adaptive control during feature fusion, and large number of parameters in existing technologies under complex texture conditions and environments, a pose estimation method and system based on edge extraction is proposed.

[0005] A pose estimation method based on edge extraction includes the following steps:

[0006] An object image is acquired, and a pose estimation network is constructed. The pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The feature and edge extraction layer receives the object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales. The edge confidence aggregation layer is obtained by stitching and mapping edge response feature maps of different scales to obtain an edge confidence map. It is then combined with edge enhancement feature maps of different scales to obtain edge confidence aggregation features. Based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, edge guidance posture features are formed. The posture parameter regression layer performs parametric regression and matrix transformation on the edge-guided posture features to obtain the predicted posture results; The pose estimation network is trained to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

[0007] As one possible implementation, the feature and edge extraction layer includes the following steps: Convolutional structures of different scales are constructed, and feature extraction is performed on object images through convolutional structures of different scales to obtain scale feature maps of different scales and construct corresponding mapping convolutional structures. By mapping convolutional structures to perform channel mapping on scale feature maps of different scales, the feature maps of scale objects of different scales have the same channel dimension, resulting in mapped feature maps of different scales. An edge response operator is constructed, and edge extraction is performed on mapping feature maps of different scales based on the edge response operator. Weight sharing is achieved between mapping feature maps of different scales, resulting in edge response feature maps of different scales.

[0008] As one possible implementation, the edge consistency fusion layer includes the following steps: Obtain edge response feature maps of adjacent scales from edge response feature maps of different scales, adjust the edge response feature maps of adjacent scales to the same spatial resolution, and calculate the difference based on the adjusted edge response feature maps of adjacent scales to obtain the edge response difference map. Edge consistency weights are obtained by channel splicing and mapping activation of the edge response difference map and the adjusted edge response feature maps of adjacent scales. The mapped feature maps of adjacent scales are adjusted to the same spatial resolution. Edge feature enhancement is achieved by fusing features based on the adjusted mapped feature maps of adjacent scales and edge consistency weights, resulting in edge-enhanced feature maps of different scales, as shown below:

[0009] in, Indicates the first Edge enhancement feature maps at various scales Indicates the first Mapping feature maps at various scales, Indicates the first Mapping feature maps at various scales, Indicates the first Spatial resolution adjustment function at each scale Indicates the total number of scales.

[0010] As one possible implementation, the method of stitching and mapping edge response feature maps of different scales to obtain an edge confidence map, and then combining edge enhancement feature maps of different scales to obtain an edge confidence aggregated feature, includes the following steps: A spatial resolution threshold is preset, and the spatial resolution of edge response feature maps at different scales is adjusted to the spatial resolution threshold. Based on the adjusted edge response feature maps, channel stitching and mapping activation are performed to obtain edge confidence maps. The total edge confidence is obtained based on the edge confidence maps. Edge enhancement feature maps of different scales are adjusted and fused to obtain an edge fused feature map. Based on the edge fused feature map, the edge confidence map, and the total edge confidence, feature aggregation is performed to obtain the edge confidence aggregated feature, as shown below:

[0011] in, Represents the coordinate position in the edge confidence plot Marginal confidence values, Represents the coordinate position in the edge fusion feature map Edge fusion feature values, This represents the total marginal confidence level. , This represents the stability constant to prevent the denominator from being zero. This represents the set of coordinates of the edge confidence plot. This represents the marginal confidence aggregation feature. Represents the x-axis, Represents the vertical axis.

[0012] As one possible implementation, the spatial parameters of the edge confidence map include the lateral centroid, the longitudinal centroid, and the second-order spatial moment. The edge confidence aggregation feature is spliced ​​and fused with the spatial parameters of the edge confidence map to form the edge guidance posture feature. As one possible implementation, the method of performing parametric regression and matrix transformation on the edge-guided pose features to obtain the predicted pose result includes the following steps: The edge-guided pose features are input into the pose parameter regression layer to obtain the object's rotation and translation vectors; Normalize the different components in the rotation vector, construct a rotation mapping function, and map the normalized rotation vector through the rotation mapping function to obtain the rotation matrix; The predicted pose of an object is formed based on the rotation matrix and translation vector.

[0013] As one possible implementation, the method of training the pose estimation network to obtain a pose estimation model includes the following steps: Scale adjustment and feature fusion are performed on edge response feature maps and edge enhancement feature maps of different scales to obtain predicted edge maps and predicted edge probabilities of pixels; The true edge labels of the object image are obtained. Combined with the predicted edge probability, predicted pose result, true pose result and edge response feature map of different scales, a joint pose estimation loss function is constructed. The joint pose estimation loss function includes a pose estimation loss function, an edge reconstruction loss function and a cross-scale edge consistency loss function. The attitude estimation network is iteratively optimized using the joint loss function for attitude estimation until the preset model index is met, thus obtaining the attitude estimation model.

[0014] A pose estimation system based on edge extraction, comprising: The model building module acquires object images and constructs a pose estimation network, which includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The edge extraction module receives an object image and performs feature extraction and channel mapping at different scales to obtain mapped feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapped feature maps at different scales to obtain edge response feature maps at different scales. The feature fusion module, wherein the edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales; The edge confidence module consists of an edge confidence aggregation layer that stitches and maps edge response feature maps of different scales to obtain an edge confidence map, combines edge enhancement feature maps of different scales to obtain edge confidence aggregation features, and forms edge guidance posture features based on the spatial parameters of the edge confidence map and the edge confidence aggregation features. The attitude regression module, wherein the attitude parameter regression layer performs parameter regression and matrix transformation on the edge-guided attitude features to obtain the predicted attitude result; The model inference module trains the pose estimation network to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the following ways: An object image is acquired, and a pose estimation network is constructed. The pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The feature and edge extraction layer receives the object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales. The edge confidence aggregation layer is obtained by stitching and mapping edge response feature maps of different scales to obtain an edge confidence map. It is then combined with edge enhancement feature maps of different scales to obtain edge confidence aggregation features. Based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, edge guidance posture features are formed. The posture parameter regression layer performs parametric regression and matrix transformation on the edge-guided posture features to obtain the predicted posture results; The pose estimation network is trained to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

[0016] An edge-extraction-based pose estimation apparatus includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the method described in any one of the following ways: An object image is acquired, and a pose estimation network is constructed. The pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The feature and edge extraction layer receives the object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales. The edge confidence aggregation layer is obtained by stitching and mapping edge response feature maps of different scales to obtain an edge confidence map. It is then combined with edge enhancement feature maps of different scales to obtain edge confidence aggregation features. Based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, edge guidance posture features are formed. The posture parameter regression layer performs parametric regression and matrix transformation on the edge-guided posture features to obtain the predicted posture results; The pose estimation network is trained to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

[0017] This invention, by adopting the above technical solutions, has significant technical effects: This invention first applies the same edge response operator to mapped feature maps at different scales, ensuring uniform edge response rules across all spatial scales. This avoids parameter redundancy caused by setting separate edge extraction networks for different scales, resulting in multi-scale edge response feature maps that pass the same edge judgment criterion, thus improving the comparability between edge response feature maps at different scales. Second, the edge consistency weight does not fuse deep and shallow features in a fixed ratio, but rather adaptively controls the fusion ratio between features based on the edge consistency degree of adjacent scales, suppressing unstable edge responses caused by local illumination, background texture, and noise. Then, edge-guided pose features are formed by aggregating spatial parameters and edge confidence features, enabling the pose parameter regression layer to simultaneously utilize local features of the edge region and the overall geometric information of the edge distribution, reducing dependence on object surface texture. Finally, predicted edge maps are only constructed during model training and removed during inference, preserving edge guidance capabilities while reducing unnecessary computation and improving model inference speed. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the modules of the system of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0023] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] Example 1: A pose estimation method based on edge extraction, such as Figure 1 As shown, it includes the following steps: S100. Acquire object images and construct a pose estimation network, wherein the pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. S200: The feature and edge extraction layer receives an object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. S300, The edge consistency fusion layer constructs edge consistency weights based on the edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales; S400, The edge confidence aggregation layer splices and maps edge response feature maps of different scales to obtain an edge confidence map, and combines edge enhancement feature maps of different scales to obtain edge confidence aggregation features. Based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, edge guidance posture features are formed. S500, The posture parameter regression layer performs parameter regression and matrix transformation on the edge-guided posture features to obtain the predicted posture result; S600. Train the pose estimation network to obtain the pose estimation model. Input the image of the object to be estimated into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

[0025] In this embodiment of the invention, the same edge response operator is reused among features at different spatial scales to obtain multi-scale edge responses that meet a unified edge determination standard. Then, cross-scale edge consistency weights are used to control the fusion ratio between deep and shallow features. Finally, edge pose guidance features are formed using the spatial parameters of the edge confidence map and the edge confidence aggregation features, thereby improving the stability of pose estimation for objects with no or weak texture. Furthermore, by constructing the predicted edge map only during model training and removing it during model inference, the computational load during online inference is reduced.

[0026] S100. Acquire object images and construct a pose estimation network, wherein the pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. In this embodiment of the invention, an image acquisition device is used to acquire an image of an object. The image acquisition device can be a camera installed on a robot or industrial robotic arm. If the image acquisition device is capable of acquiring depth information, only the acquired RGB image can be used, or the depth information can be used as additional information for expansion.

[0027] To estimate the pose of an object, this embodiment of the invention constructs a pose estimation network, which includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The feature and edge extraction layer extracts features at different scales from the object image and performs edge extraction. The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps at different scales, and then fuses features at different scales to obtain edge-enhanced feature maps at different scales. The edge confidence aggregation layer forms edge-guided pose features using the spatial parameters of the edge confidence map and the edge confidence aggregation features. The pose parameter regression layer performs regression based on the edge-guided pose features and transforms them to obtain the predicted pose result.

[0028] S200: The feature and edge extraction layer receives an object image and performs feature extraction and channel mapping at different scales to obtain mapped feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapped feature maps at different scales to obtain edge response feature maps at different scales. This includes the following steps: S2001: Perform feature extraction and channel mapping at different scales on object images to obtain mapped feature maps at different scales; The resolution of the object image in this embodiment of the invention is The ResNet-34 backbone network is used to extract features from object images at different scales, resulting in scale feature maps at different scales. ,in , The total number of scales is indicated by the number of channels in the mapping feature maps in the embodiments of the present invention, which are 64, 64, 128, 256 and 512 respectively. The shallower the spatial scale of the mapping feature map, the higher the spatial resolution, and the deeper the spatial scale of the mapping feature map, the stronger the semantic expression ability.

[0029] Channel mapping is performed on the scale feature maps using mapping convolutional structures corresponding to the scale feature maps, resulting in mapped feature maps of different scales. Channel mapping adjusts the mapped feature maps of different scales to the same channel dimension, which facilitates the subsequent edge response operators to handle inputs of different scales, as shown below:

[0030] in, Indicates the first Mapping feature maps at various scales, Represents a mapping convolution structure. Indicates the first Scale feature map at each scale, Indicates the total number of scales.

[0031] S2002: Construct an edge response operator, which performs edge extraction on the mapping feature maps of different scales to obtain edge response feature maps of different scales; Since the mapped feature maps in this embodiment of the invention have the same number of channels, the edge response operator directly reuses parameters when processing mapped feature maps of different scales. The edge response operator includes a convolution kernel with a size of [missing information]. The depth can separate convolutional layers, normalization layers, nonlinear activation layers, and convolutional kernels with a size of [missing information]. The edge response output layer. Edge response feature maps of different scales are obtained by performing edge extraction on the mapped feature maps at different scales based on the edge response operator, as shown below:

[0032] in, Indicates the first Mapping feature maps at various scales, Indicates the first Edge response feature maps at various scales Represents the edge response operator. The parameters represent the edge response operator.

[0033] S300, The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales, including the following steps: S3001: Construct edge consistency weights based on edge response feature maps of adjacent scales; For adjacent th The first scale and the first The edge response feature map at the nth scale will be used to... Edge response feature map at each scale Resolution adjustment mapping to the first Edge response feature map at each scale At a given resolution, the adjusted edge response feature map is obtained, as shown below:

[0034] The difference between the adjusted edge response feature maps of adjacent scales is calculated to obtain the edge response difference map at different scales. , means as follows:

[0035] Based on the Edge response feature map at each scale , No. Edge response feature map after scale adjustment and edge response difference map Channel concatenation is performed, and edge consistency weights are obtained by processing with a nonlinear mapping function and an activation function, as shown below:

[0036] in, Indicates the first The edge response feature map after scale adjustment Indicates the first Resolution adjustment function at each scale This includes bilinear interpolation, deconvolution, pixel rearrangement, or other differentiable spatial scale adjustments. Indicates the first Edge response difference map at each scale Indicates the first Each scale's edge consistency weight, Represents a nonlinear mapping function. This represents the Sigmoid activation function; Cross-scale edge consistency weights in embodiments of the present invention The value range is 0 to 1. When both shallow and deep layers exhibit stable edge responses at the same location, the cross-scale edge consistency weight is... This is used to improve the fusion ratio of deep structural information to shallow high-resolution features; when the difference between the two is large, the cross-scale edge consistency weight... Used to limit the interference of inconsistent deep responses on shallow features; S3002: The resolution of the mapping feature maps at adjacent scales is adjusted and the feature maps are fused together with the edge consistency weight to obtain edge enhancement feature maps at different scales; Based on the mapping feature maps at different scales, obtain the mapping feature maps at adjacent scales, and then... Mapping feature maps at various scales The resolution was adjusted to the 1000 Hz. Mapping feature maps at various scales The resolution is used to obtain the adjusted mapping feature map, based on the first... Mapping feature maps at various scales The adjusted mapping feature map and edge consistency weights are used for feature fusion to achieve edge feature enhancement, resulting in edge-enhanced feature maps at different scales, as shown below:

[0037] For cases with multiple adjacent scales, the above fusion can be performed sequentially from the deepest layer to the shallowest layer, or multiple edge enhancement features can be formed in parallel before convergence. in, Indicates the first Edge enhancement feature maps at various scales Indicates the first Mapping feature maps at various scales, Indicates the first Mapping feature maps at various scales, Indicates the first Individual scale spatial resolution adjustment function Indicates the total number of scales.

[0038] S400, the edge confidence aggregation layer obtains an edge confidence map by stitching and mapping edge response feature maps of different scales, and obtains edge confidence aggregation features by combining edge enhancement feature maps of different scales. Edge guidance pose features are then formed based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, including the following steps: S4001: Edge confidence maps are obtained by stitching and mapping edge response feature maps of different scales, and edge confidence aggregated features are obtained by combining edge enhancement feature maps of different scales. A preset spatial resolution threshold is used. In this embodiment of the invention, the spatial resolution threshold is set to the resolution of the edge response feature map at the first scale. The resolutions of edge response feature maps at different scales are adjusted to the spatial resolution threshold and then stitched together. An edge confidence map is obtained by mapping using a fusion mapping function and an activation function. , means as follows:

[0039] Edge enhancement feature maps of different scales are adjusted in resolution and fused to form a unified edge enhancement feature map. According to the edge confidence graph Edge enhancement feature map We perform weighted aggregation of feature vectors from different spatial locations to obtain edge confidence aggregated features. , means as follows:

[0040] in, Represents the coordinate position in the edge confidence plot Marginal confidence values, Represents the coordinate position in the edge fusion feature map Edge fusion feature values, This represents the total marginal confidence level. , This represents the stability constant to prevent the denominator from being zero. This represents the set of coordinates of the edge confidence plot. This represents the marginal confidence aggregation feature. Represents the x-axis, Represents the ordinate; S4002: Obtain the spatial parameters of the edge confidence map, and form edge guidance posture features based on the spatial parameters and edge confidence aggregation features; The spatial parameters in this embodiment of the invention include the transverse centroid, the longitudinal centroid, and the second-order spatial moment, as shown below:

[0041]

[0042]

[0043]

[0044]

[0045] Spatial parameters are concatenated with edge confidence aggregation features to form edge-guided pose features. , means as follows:

[0046] in, Indicates the transverse centroid. This represents the total marginal confidence level. Represents the coordinate position in the edge confidence plot Marginal confidence values, This represents the set of coordinates of the edge confidence plot. Represents the x-axis, Represents the ordinate, Indicates the longitudinal centroid. , , Denotes the second-order space moment. Indicates edge-guided pose characteristics; The method of this invention provides that the object contour and stable structural edge regions contribute more to the posture parameter regression than the background region and the textureless region inside the object. At the same time, the spatial parameters provide the posture parameter regression layer with information on the position, scale and orientation of the edge distribution.

[0047] S500, The posture parameter regression layer performs parameter regression and matrix transformation on the edge-guided posture features to obtain the predicted posture result, including the following steps: S5001: Perform parameter regression on the edge-guided posture features and obtain normalized rotation quaternions and three-dimensional translation vectors through normalization processing; Edge-guided pose features are input into a pose parameter regression layer, and rotation quaternions are obtained through parameter regression. and three-dimensional translation vector , Normalize the rotation quaternion to obtain the normalized rotation quaternion. , means as follows:

[0048] S5002: Matrix transformation based on normalized rotation quaternions, combined with three-dimensional translation vectors to form the predicted pose of the object; The normalized rotation quaternion Including components Through rotation mapping function The normalized rotation quaternion is transformed to obtain the rotation matrix. , means as follows:

[0049] Based on the rotation matrix and the three-dimensional translation vector, the predicted 6D pose of the object relative to the coordinate system of the image acquisition device is generated.

[0050] S600. Train the pose estimation network to obtain the pose estimation model. Input the image of the object to be estimated into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated. The edge response feature maps and edge enhancement feature maps at different scales are scaled and fused on a scale-by-scale to output a predicted edge map and the predicted edge probability of pixels with the same spatial resolution as the object image. This step is only used for model training and is not a necessary output for the model inference stage. Obtain the true edge labels of the object image, and combine the predicted edge map, the predicted pose result and the true pose result to construct a joint pose estimation loss function. The joint pose estimation loss function includes a pose estimation loss function, an edge reconstruction loss function and a cross-scale edge consistency loss function. Obtain the set of sampling points of the object's 3D model Based on the predicted pose and the actual pose of the sampled points, a pose estimation loss function is constructed. For symmetrical objects, the nearest points in the predicted transformed sampled points can be matched with the nearest points in the actual transformed point set to construct a pose estimation loss suitable for symmetrical objects. The pose estimation loss function is expressed as follows:

[0051] The edge reconstruction loss function is constructed based on the predicted edge map and the true edge map of the object, as follows:

[0052] A cross-scale edge consistency loss function is constructed based on the edge response feature maps of adjacent scales. This function represents the edge representations that are related to the edge response operator and the pose estimation task across different scales. The cross-scale edge consistency loss function is expressed as follows:

[0053] The joint loss function for attitude estimation is formed based on the attitude estimation loss function, the edge reconstruction loss function, and the cross-scale edge consistency loss function, and is expressed as follows:

[0054] in, This represents the attitude estimation loss function. Indicates the number of sampling points in the set. Represents the first in the set of sampling points One sampling point, Indicates the first sampling points The rotation matrix in the predicted pose result, Indicates the first sampling points The three-dimensional translation vector in the predicted pose result, Indicates the first sampling points The rotation matrix in the true pose result, Indicates the first sampling points The three-dimensional translation vector in the true pose result, This represents the edge reconstruction loss function. Indicates the total number of edge pixels. Indicates the first The true edge label of each pixel, Indicates the first Predicted edge probabilities for each pixel. This represents the cross-scale edge consistency loss function. Indicates the total number of scales. No. Edge response feature maps at various scales Indicates the first Edge response feature maps at various scales Indicates the first Resolution adjustment function at each scale This represents the joint loss function for attitude estimation. , Indicates the loss weight; This invention employs data containing a 3D object model, a true rotation matrix, true translation vectors, and true edge labels to train the pose estimation network. The true edge labels can be generated based on the rendered contour of the 3D model, or they can be generated manually, through depth discontinuity detection, or with the assistance of image edge operators and then corrected. During model training, the edge response features undergo progressive upsampling and feature fusion to generate an edge prediction map and the predicted edge probabilities of pixels within the map. The optimizer uses the Adam optimizer, and the initial learning rate can be set to... The batch size can be set to 32, and the loss weight can be adjusted. and It can be determined based on the validation set results.

[0055] In one example, an edge reconstruction loss function can be used first. and cross-scale edge consistency loss function Pre-train the edge response operator and then use the pose estimation joint loss function. The entire pose estimation network can be trained end-to-end; in another example, joint training can also be performed directly from randomly initialized states.

[0056] The pose estimation network is trained using a joint loss function based on pose estimation until preset model metrics are met. In this embodiment, these preset metrics include model performance metrics such as accuracy, false positive rate, and latency. During the model's inference phase, the predicted edge map and the predicted edge probabilities of pixels in the map are removed. The pose estimation model in this embodiment can be deployed on an embedded computing platform with a graphics processing unit, and employs half-precision floating-point quantization, low-precision integer quantization, convolution and normalization fusion, memory reuse, and a batch size of 1 for inference to reduce latency. The rotation matrix obtained in this embodiment... and three-dimensional translation vector It can be sent to the underwater robot navigation controller, robotic arm controller, or detection result display terminal.

[0057] Example 2: A pose estimation system based on edge extraction, such as Figure 2 As shown, it includes: The model building module 100 acquires object images and builds a pose estimation network, which includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The edge extraction module 200 receives an object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. The feature fusion module 300, wherein the edge consistency fusion layer constructs edge consistency weights based on the edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales; The edge confidence module 400, wherein the edge confidence aggregation layer splices and maps edge response feature maps of different scales to obtain an edge confidence map, and combines edge enhancement feature maps of different scales to obtain edge confidence aggregation features, and forms edge guidance posture features based on the spatial parameters of the edge confidence map and the edge confidence aggregation features; The attitude regression module 500, wherein the attitude parameter regression layer performs parameter regression and matrix transformation on the edge-guided attitude features to obtain the predicted attitude result; The model inference module 600 trains the pose estimation network to obtain a pose estimation model, inputs the image of the object to be estimated into the pose estimation model for inference, and obtains the pose estimation result of the object to be estimated.

[0058] Various changes and modifications made without departing from the spirit and scope of this invention, and all equivalent technical solutions, also fall within the scope of this invention.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0060] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] It should be noted that: The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0065] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0066] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of each module is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0067] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The modules in the system of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0068] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0069] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A pose estimation method based on edge extraction, characterized in that, Includes the following steps: An object image is acquired, and a pose estimation network is constructed. The pose estimation network includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The feature and edge extraction layer receives the object image and performs feature extraction and channel mapping at different scales to obtain mapping feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapping feature maps at different scales to obtain edge response feature maps at different scales. The edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales. The edge confidence aggregation layer is obtained by stitching and mapping edge response feature maps of different scales to obtain an edge confidence map. It is then combined with edge enhancement feature maps of different scales to obtain edge confidence aggregation features. Based on the spatial parameters of the edge confidence map and the edge confidence aggregation features, edge guidance posture features are formed. The posture parameter regression layer performs parametric regression and matrix transformation on the edge-guided posture features to obtain the predicted posture results; The pose estimation network is trained to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

2. The pose estimation method based on edge extraction according to claim 1, characterized in that, The feature and edge extraction layer includes the following steps: Convolutional structures of different scales are constructed, and feature extraction is performed on object images through convolutional structures of different scales to obtain scale feature maps of different scales and construct corresponding mapping convolutional structures. By mapping convolutional structures to perform channel mapping on scale feature maps of different scales, scale feature maps of different scales have the same channel dimension, resulting in mapped feature maps of different scales. An edge response operator is constructed, and edge extraction is performed on mapping feature maps of different scales based on the edge response operator. Weight sharing is achieved between mapping feature maps of different scales, resulting in edge response feature maps of different scales.

3. The pose estimation method based on edge extraction according to claim 1, characterized in that, The edge consistency fusion layer includes the following steps: Obtain edge response feature maps of adjacent scales from edge response feature maps of different scales, adjust the edge response feature maps of adjacent scales to the same spatial resolution, and calculate the difference based on the adjusted edge response feature maps of adjacent scales to obtain the edge response difference map. Edge consistency weights are obtained by channel splicing and mapping activation of the edge response difference map and the adjusted edge response feature maps of adjacent scales. The mapped feature maps of adjacent scales are adjusted to the same spatial resolution. Edge feature enhancement is achieved by fusing features based on the adjusted mapped feature maps of adjacent scales and edge consistency weights, resulting in edge-enhanced feature maps of different scales, as shown below: in, Indicates the first Edge enhancement feature maps at various scales Indicates the first Mapping feature maps at various scales, Indicates the first Mapping feature maps at various scales, Indicates the first Spatial resolution adjustment function at each scale Indicates the total number of scales.

4. The pose estimation method based on edge extraction according to claim 1, characterized in that, The process of stitching and mapping edge response feature maps at different scales to obtain an edge confidence map, and then combining it with edge enhancement feature maps at different scales to obtain an edge confidence aggregated feature, includes the following steps: A spatial resolution threshold is preset, and the spatial resolution of edge response feature maps at different scales is adjusted to the spatial resolution threshold. Based on the adjusted edge response feature maps, channel stitching and mapping activation are performed to obtain edge confidence maps. The total edge confidence is obtained based on the edge confidence maps. Edge enhancement feature maps of different scales are adjusted and fused to obtain an edge fused feature map. Based on the edge fused feature map, the edge confidence map, and the total edge confidence, feature aggregation is performed to obtain the edge confidence aggregated feature, as shown below: in, Represents the coordinate position in the edge confidence plot Marginal confidence values, Represents the coordinate position in the edge fusion feature map Edge fusion feature values, This represents the total marginal confidence level. , This represents the stability constant to prevent the denominator from being zero. This represents the set of coordinates of the edge confidence plot. This represents the marginal confidence aggregation feature. Represents the x-axis, Represents the vertical axis.

5. The pose estimation method based on edge extraction according to claim 1, characterized in that, The spatial parameters of the edge confidence map include the lateral centroid, the longitudinal centroid, and the second-order spatial moment. The edge confidence aggregation feature is spliced ​​and fused with the spatial parameters of the edge confidence map to form the edge guidance posture feature.

6. The pose estimation method based on edge extraction according to claim 1, characterized in that, The process of performing parametric regression and matrix transformation on the edge-guided pose features to obtain the predicted pose results includes the following steps: The edge-guided pose features are input into the pose parameter regression layer to obtain the object's rotation and translation vectors; Normalize the different components in the rotation vector, construct a rotation mapping function, and map the normalized rotation vector through the rotation mapping function to obtain the rotation matrix; The predicted pose of an object is formed based on the rotation matrix and translation vector.

7. The pose estimation method based on edge extraction according to claim 1, characterized in that, The process of training the pose estimation network to obtain the pose estimation model includes the following steps: Scale adjustment and feature fusion are performed on edge response feature maps and edge enhancement feature maps of different scales to obtain predicted edge maps and predicted edge probabilities of pixels; The true edge labels of the object image are obtained. Combined with the predicted edge probability, predicted pose result, true pose result and edge response feature map of different scales, a joint pose estimation loss function is constructed. The joint pose estimation loss function includes a pose estimation loss function, an edge reconstruction loss function and a cross-scale edge consistency loss function. The attitude estimation network is iteratively optimized using the joint loss function for attitude estimation until the preset model index is met, thus obtaining the attitude estimation model.

8. A pose estimation system based on edge extraction, characterized in that, include: The model building module acquires object images and constructs a pose estimation network, which includes a feature and edge extraction layer, an edge consistency fusion layer, an edge confidence aggregation layer, and a pose parameter regression layer. The edge extraction module receives an object image and performs feature extraction and channel mapping at different scales to obtain mapped feature maps at different scales. It then constructs an edge response operator and performs edge extraction on the mapped feature maps at different scales to obtain edge response feature maps at different scales. The feature fusion module, wherein the edge consistency fusion layer constructs edge consistency weights based on edge response feature maps of adjacent scales, adjusts the resolution of the mapping feature maps of adjacent scales respectively, and combines the edge consistency weights to perform feature fusion to obtain edge enhancement feature maps of different scales; The edge confidence module consists of an edge confidence aggregation layer that stitches and maps edge response feature maps of different scales to obtain an edge confidence map, combines edge enhancement feature maps of different scales to obtain edge confidence aggregation features, and forms edge guidance posture features based on the spatial parameters of the edge confidence map and the edge confidence aggregation features. The attitude regression module, wherein the attitude parameter regression layer performs parameter regression and matrix transformation on the edge-guided attitude features to obtain the predicted attitude result; The model inference module trains the pose estimation network to obtain a pose estimation model. The image of the object to be estimated is input into the pose estimation model for inference to obtain the pose estimation result of the object to be estimated.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A pose estimation device based on edge extraction, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images