Method, device, equipment, medium and product for generating three-dimensional structure and pose

Abstract

The application relates to the technical field of computers and discloses a three-dimensional structure and pose generation method, device, equipment, medium and product. The method comprises the following steps: acquiring a first image and image features of the first image, and acquiring a first three-dimensional structure representation and a first transformation token; the image features, the first three-dimensional structure representation and the first transformation token are injected into a feature interaction module of a third model to obtain a first three-dimensional feature, a first two-dimensional feature and a first transformation parameter; and the first three-dimensional feature and the first two-dimensional feature are aligned by using the first transformation parameter to generate a three-dimensional structure and a camera pose. Thus, joint optimization for three-dimensional structure generation and camera pose estimation is realized, which is beneficial to improving the estimation accuracy of the camera pose and the reconstruction quality of the three-dimensional structure, so that the three-dimensional structure can be aligned with the first image at a pixel level when the three-dimensional structure is projected back to the first image of the original visual angle, thereby ensuring the visual representation consistency of the three-dimensional structure and the first image for a first object.

Description

Technical Field

[0001] This relates to the field of computer technology, specifically to methods, devices, equipment, media, and products for generating three-dimensional structures and poses. Background Technology

[0002] With the development of computer vision and Artificial Intelligence Generated Content (AIGC) technologies, the demand for 3D structure reconstruction and camera pose estimation using multi-view images is increasing. Therefore, a joint generation method for 3D structures and camera poses is needed to meet these requirements. Summary of the Invention

[0003] A method, apparatus, device, medium, and product for generating three-dimensional structures and poses, to solve the problem of joint generation of three-dimensional structures and camera poses.

[0004] In a first aspect, a method for generating a three-dimensional structure and pose includes: acquiring a first image and image features of the first image, and acquiring a first three-dimensional structure representation and a first transformation token, wherein the first image corresponds to a first object, and the image features are generated based on a first model; the first three-dimensional structure representation is generated based on a second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure; injecting the image features, the first three-dimensional structure representation, and the first transformation token into a feature interaction module of a third model to obtain a first three-dimensional feature, a first two-dimensional feature, and a first transformation parameter; wherein the first three-dimensional feature corresponds to the first three-dimensional structure representation, the first two-dimensional feature corresponds to the image features, and the first transformation parameter corresponds to the first transformation token; and using the first transformation parameter, aligning the first three-dimensional feature and the first two-dimensional feature to generate a three-dimensional structure and a camera pose, wherein the three-dimensional structure corresponds to the first object, and the camera pose corresponds to the first image.

[0005] In a second aspect, a device for generating a three-dimensional structure and pose includes: an acquisition module for acquiring a first image and image features of the first image, and acquiring a first three-dimensional structure representation and a first transformation token, wherein the first image corresponds to a first object, and the image features are generated based on a first model; the first three-dimensional structure representation is generated based on a second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure; a feature acquisition module for injecting the image features, the first three-dimensional structure representation, and the first transformation token into a feature interaction module of a third model to obtain a first three-dimensional feature, a first two-dimensional feature, and a first transformation parameter; wherein the first three-dimensional feature corresponds to the first three-dimensional structure representation, the first two-dimensional feature corresponds to the image feature, and the first transformation parameter corresponds to the first transformation token; and a joint generation module for aligning the first three-dimensional feature and the first two-dimensional feature using the first transformation parameter to generate a three-dimensional structure and a camera pose, wherein the three-dimensional structure corresponds to the first object, and the camera pose corresponds to the first image.

[0006] Thirdly, an electronic device includes: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method for generating three-dimensional structures and poses according to the first aspect or any corresponding embodiment described above.

[0007] Fourthly, a computer-readable storage medium storing computer instructions for causing a computer to perform the method for generating a three-dimensional structure and pose according to the first aspect or any corresponding embodiment thereof.

[0008] Fifthly, a computer program product includes computer instructions for causing a computer to execute the method for generating three-dimensional structures and poses according to the first aspect or any corresponding embodiment thereof.

[0009] In some cases, methods, apparatuses, devices, media, and products for generating 3D structures and poses introduce a first transformation token for transforming a first image into a 3D structure. This token interacts with image features generated by a first model and a first 3D structure representation generated by a second model. This allows for bidirectional reciprocal flow of 2D and 3D information guided by the first transformation token. This enables 2D information to calibrate the 3D structure, and the 3D structure to constrain 2D predictions from all viewpoints, resulting in joint predictions of first 2D features, first 3D features, and first transformation parameters. The first 2D and 3D features are then spatially aligned using the first transformation parameters to obtain the 3D structure of the first object and the camera pose of the first image. Thus, the first transformation parameters achieve joint optimization of 3D structure generation and camera pose estimation, improving the accuracy of camera pose estimation and the reconstruction quality of the 3D structure. This ensures that when the 3D structure is projected back to the first image from the original viewpoint, it achieves pixel-level alignment with the first image, guaranteeing the consistency of the 3D structure's visual representation of the first object with the first image. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in specific implementation methods or related technologies under certain circumstances, the accompanying drawings used in the description of specific implementation methods or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 These are schematic diagrams based on application scenarios under certain conditions; Figure 2 This is a schematic diagram of the first process for generating three-dimensional structures and poses under certain circumstances; Figure 3 This is a schematic diagram of the second process for generating three-dimensional structures and poses under certain circumstances; Figure 4 This is a block matching diagram based on certain scenarios; Figure 5 It is a schematic diagram based on the characteristics of interaction under certain circumstances; Figure 6 This is a schematic diagram of the third method for generating 3D structures and poses under certain circumstances; Figure 7 This is a specific schematic diagram of the feature interaction model under certain circumstances; Figure 8 This is a schematic diagram illustrating the specific process of generating 3D structures and poses under certain circumstances. Figure 9It is a structural block diagram of a device for generating three-dimensional structures and poses under certain conditions; Figure 10 These are schematic diagrams of the hardware structure of electronic devices under certain circumstances. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages clearer in some cases, the technical solutions in some cases will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments, not all embodiments. Based on the embodiments in some cases, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this solution.

[0013] It is understood that before using the technical solutions disclosed in the various embodiments in certain situations, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in certain situations and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0014] For example, upon receiving a user's proactive request, a prompt message can be sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the operation based on the prompt message.

[0015] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0016] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the specific implementation method. Other methods that comply with relevant laws and regulations may also be applied to this implementation method.

[0017] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0018] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this document, "multiple" means two or more, unless otherwise explicitly specified.

[0019] With the development of computer vision and AIGC technologies, three-dimensional (3D) reconstruction and camera pose estimation from multi-view images are important technologies for 3D asset acquisition and spatial perception. Current 3D reconstruction is mainly divided into two categories: feedforward reconstruction and generative reconstruction.

[0020] Feedforward reconstruction directly regresses pixel-aligned geometric properties (such as depth maps and point clouds) using a transformer trained on large-scale data. This method achieves good pixel-level alignment and captures visible area details well, but it heavily relies on the overlap of the input viewpoints. For occluded regions, it is difficult to perform reasonable geometric completion, and it usually requires prior camera pose or can only predict relative poses, making it difficult to unify to a global coordinate system.

[0021] Generative reconstruction utilizes generative models based on diffusion models or flow matching to model 3D reconstruction as a generative process under image conditions. This approach possesses strong geometric priors, enabling the generation of complete, coarse-to-fine 3D structures, even in invisible areas. However, the lack of an explicit input alignment mechanism results in the generated 3D structures often being "look-alikes" of the input image, frequently exhibiting pixel-level misalignment in geometric dimensions and texture details.

[0022] While some methods attempt to fuse the two reconstruction approaches, current methods are only one-way, such as injecting only reconstructed features into the generative model. This one-way fusion lacks mutual benefit; the feedforward reconstruction model cannot utilize the 3D global prior information of the generative reconstruction model to constrain its local predictions, while the generative reconstruction model lacks precise geometric features from the feedforward reconstruction model to force alignment. This results in either inaccurate camera pose estimation or a mismatch between the generated 3D structure and the original image.

[0023] Based on this, a network architecture integrating a 3D generative reconstruction model and a 2D feedforward reconstruction model was set up. A global transformation token was introduced to perform joint attention calculation with each modal feature, realizing the joint optimization of 3D structure generation and camera pose estimation. This allows the generated 3D structure to accurately match the viewpoint and geometric details of the input image. At the same time, it guides the 2D feedforward reconstruction model to predict the camera pose in a unified 3D space, thus obtaining an accurate camera pose.

[0024] As an optional application scenario, such as Figure 1 As shown, application 101 is installed in electronic device 110, and user 130 can interact with application 101 through electronic device 110 and / or access device of electronic device 110.

[0025] For example, application 101 can be any application that provides services related to image generation. For instance, application 101 could be a 3D model generation application. Figure 1 In the application scenario shown, if application 101 is active, electronic device 110 can display the interface 102 of application 101. Interface 102 may include various pages that application 101 can provide, such as a two-dimensional image input page, a three-dimensional model display page, etc.

[0026] In some embodiments, electronic device 110 is communicatively connected to server 120 to provide services to application 101. Electronic device 110 may be a mobile terminal, fixed terminal, or portable terminal, etc., including but not limited to mobile phones, desktop computers, laptop computers, multimedia tablets, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, electronic device 110 may also support any type of interface, and server 120 may be various types of computing systems or servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0027] It should be noted that, Figure 1 This is merely an example of an application scenario and does not limit the scope of protection of this invention.

[0028] The embodiments of the present invention will now be described with reference to the accompanying drawings. It should be understood that the pages shown in the drawings are merely examples, and various page designs are possible in practice. The various graphic elements on the page may have different arrangements and different visual representations; one or more elements may be omitted or replaced, and one or more other elements may also be present, without any limitation in the embodiments of the present invention. Furthermore, the embodiments described below primarily pertain to electronic device 110. It should be understood that the actions described relative to electronic device 110 can be performed by application 101 on electronic device 110, or can be performed by application 101 in conjunction with its server (e.g., server 120).

[0029] In some cases, an embodiment of a method for generating three-dimensional structures and poses is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0030] A method for generating 3D structures and poses in certain situations can be used for the joint generation of 3D structures and camera poses. Figure 2 This is a flowchart illustrating methods for generating 3D structures and poses in certain situations, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the first image and its image features, as well as the first three-dimensional structure representation and the first transformation token.

[0031] The first image corresponds to the first object, and the image features are generated based on the first model; the first three-dimensional structure representation is generated based on the second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure.

[0032] The first image can be a multi-view image representation of the first object; the first object can be various three-dimensional objects in the actual scene, such as cups, furniture, trees, toys, etc., without specific limitations.

[0033] In some cases, the first image The first image can be obtained by taking pictures of the first object from multiple perspectives, or it can be an image of the first object taken from a local image library from multiple perspectives, or it can be an image of the first object taken from an online public database from multiple perspectives. There is no specific limitation on the method of obtaining the first image here.

[0034] The first model can be a feedforward reconstruction model. Accordingly, the image features of the first image correspond to the first image. Based on the given first image, the feedforward encoder in the first model is used to encode the first image to obtain the corresponding image features.

[0035] The first 3D structural representation is used to characterize the initial geometric structure of the first object in 3D space, that is, the structural representation of the first object in the world coordinate system. The second model can be a generative reconstruction model. Accordingly, based on the given first image, the second model is used to perform preliminary modeling of the geometry and spatial occupancy of the first object in the first image, thereby obtaining the first 3D structural representation of the first object.

[0036] The first transformation token is an initialized learnable parameter token used to characterize the transformation from the first image to three-dimensional space, which may include geometric transformation, viewpoint transformation, modal transformation, etc. Specifically, the first transformation token is pre-configured based on the transformation between the first model and the second model, and is used to enable the model to learn that "the features of the same object remain consistent under different transformations".

[0037] Step S202: Inject the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters.

[0038] Among them, the first three-dimensional feature corresponds to the first three-dimensional structure representation, the first two-dimensional feature corresponds to the first image, and the first transformation parameter corresponds to the first transformation token.

[0039] The first three-dimensional feature is used to generate the three-dimensional structure of the first object, the first two-dimensional feature is used to predict the camera pose of the first image, and the first transformation parameter is used to constrain the first three-dimensional feature and the first two-dimensional feature.

[0040] The third model integrates the feedforward reconstruction capability of the first model and the generative reconstruction capability of the second model. The feature interaction module is used to combine the first transformation token to perform feature interaction between image features and the first three-dimensional structure representation, so as to realize the bidirectional flow of two-dimensional and three-dimensional information.

[0041] Image features, a first 3D structural representation, and a first transformation token are input into the feature interaction module for feature matching. The image features are used to calibrate the first 3D structural representation, and the first 3D structural representation is used to constrain the prediction of 2D features from different viewpoints. Simultaneously, the first transformation token is updated so that it can represent the global similarity transformation between 2D and 3D features. Therefore, given the image features, the first 3D structural representation, and the first transformation token, the feature interaction module performs joint prediction of 3D features, 2D features, and transformed features to obtain the corresponding first 3D features, first 2D features, and first transformation parameters.

[0042] Step S203: Using the first transformation parameters, align the first three-dimensional feature and the first two-dimensional feature to generate the three-dimensional structure and camera pose.

[0043] The three-dimensional structure corresponds to the first object, and the camera pose corresponds to the first image.

[0044] As described above, the first transformation parameters are used to constrain the first 3D features and the first 2D features. Under the geometric constraints established by the first transformation features, the first 3D features and the first 2D features are spatially forced to align. The third model can then generate a 3D structure based on the aligned first 3D features to obtain the 3D structure of the first object. This 3D structure can be a sparse structure represented by a voxel mesh. Furthermore, the camera pose corresponding to the first image and the pixel-aligned local point cloud map are decoded from the aligned first 2D features.

[0045] In some methods for generating 3D structures and poses, a first transformation token is introduced to transform a first image into a 3D structure. This token interacts with the image features generated by a first model and the 3D structure representation generated by a second model. This allows for bidirectional reciprocal flow of 2D and 3D information guided by the first transformation token. This enables 2D information to calibrate the 3D structure, and the 3D structure to constrain 2D predictions across all viewpoints, resulting in joint predictions of first 2D features, first 3D features, and first transformation parameters. The first 2D and 3D features are then spatially aligned using the first transformation parameters to obtain the 3D structure of the first object and the camera pose of the first image. Thus, the first transformation parameters achieve joint optimization of 3D structure generation and camera pose estimation, improving the accuracy of camera pose estimation and the reconstruction quality of the 3D structure. This ensures that when the 3D structure is projected back to the first image from the original viewpoint, it achieves pixel-level alignment with the first image, guaranteeing the consistency of the 3D structure and the first image's visual representation of the first object.

[0046] In some cases, a method for generating 3D structures and poses is provided, which can be used for the joint generation of 3D structures and camera poses. Figure 3 It is a flowchart based on the generation method of 3D structure and pose in some cases, such as Figure 3 As shown, the process includes the following steps: Step S301: Obtain a first image and its image features, as well as a first three-dimensional structure representation and a first transformation token. The first image corresponds to a first object, and its image features are generated based on a first model; the first three-dimensional structure representation is generated based on a second model, and the first transformation token characterizes the transformation from the first image to the three-dimensional structure. For details, please refer to the relevant descriptions of the steps in the embodiments shown above, which will not be repeated here.

[0047] Step S302: Inject the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters.

[0048] Among them, the first three-dimensional feature corresponds to the first three-dimensional structure representation, the first two-dimensional feature corresponds to the image feature, and the first transformation parameter corresponds to the first transformation token.

[0049] Specifically, the feature interaction module of the third model includes: non-feature mixing block, local feature mixing block and global feature mixing block.

[0050] The non-feature blending block includes the first local processing block of the first model. The first local processing block can be a local attention block of the first model, which does not interact with the 3D processing branch, such as... Figure 4 As shown. The first local processing block executes independently and is not mixed with other features.

[0051] The local feature blending block includes a first attention block of the first model and a first transformer block of the second model. The first attention block can be a local attention block of the first model, and the first transformer block can be a transformer block of the second model, such as... Figure 4 As shown, a local feature blending block is used to simultaneously process the first transform token and the corresponding 3D token representing the 3D structure.

[0052] The global feature blending block includes a second attention block of the first model and a second transformer block of the second model. The second attention block can be a global self-attention block of the first model, and the second transformer block can be a transformer block of the second model, such as... Figure 4 As shown, a global feature fusion block is used to simultaneously process the first transform token, image features, and 3D structural representation.

[0053] Specifically, non-feature mixed blocks, local feature mixed blocks, and global feature mixed blocks are obtained by performing block matching between the first model and the second model based on a preset block matching strategy.

[0054] The preset block matching strategy is a pre-defined network block matching strategy. Since the first model and the second model differ in model structure and number of network layers, such as... Figure 4 The first and second models shown are illustrated. The first model includes local attention blocks and global self-attention blocks, with a network layer count of 36; the second model includes transformer blocks, with a network layer count of 24. To fuse the first and second models with different structures and network layer counts, a preset block matching strategy needs to be set to perform mixed processing on different blocks, resulting in corresponding non-feature mixed blocks, local feature mixed blocks, and global feature mixed blocks.

[0055] It should be noted that during the training of the third model, all weights with pre-trained values ​​that are not affected by the feature interaction module will be frozen in order to preserve the capabilities of the pre-trained model as much as possible.

[0056] By setting different hybrid blocks to support the fusion of models with different structures, it is possible to achieve bidirectional flow between two-dimensional and three-dimensional models while retaining the pre-training capabilities of the first and second models.

[0057] Accordingly, step S302 above includes: Step S3021: Inject the image features, the first three-dimensional structure representation, and the first transformation token into the global feature mixing block to obtain the first two-dimensional features, the first three-dimensional features, and the first transformation parameters.

[0058] When a global self-attention block of the first model is detected to match the transformer block of the second model based on a preset block matching strategy, the image features, the first three-dimensional structure representation, and the first transform token are simultaneously injected into the global feature mixing block.

[0059] The global feature mixing block processes image features and the first 3D structural representation using different independent projection matrices to adapt to the feature spaces of different features. Simultaneously, the global feature mixing block jointly processes image features, the first 3D structural representation, and the first transformation token to achieve full exchange of global information between the 2D and 3D feature spaces, such as... Figure 5 As shown in Figure (f).

[0060] By using a global feature mixing block, different modalities can interact with each other, thereby aligning the features of different modalities and outputting the corresponding first two-dimensional features, first three-dimensional features, and first transformation parameters for global similarity transformation of the first two-dimensional features and the first three-dimensional features.

[0061] In some optional cases, step S3021 above includes: Step a1: Using a global feature mixing block, image features, a first three-dimensional structural representation, and a first transformation token are spliced ​​together to obtain spliced ​​features.

[0062] Step a2: Use the splicing features to obtain the joint attention value.

[0063] Step a3: Use joint attention values ​​to constrain the feature transformation of two-dimensional space and three-dimensional space to obtain the first two-dimensional feature, the first three-dimensional feature and the first transformation parameter.

[0064] A global feature fusion block is used to concatenate image features from all viewpoints, the first 3D structural representation, and the first transform token to form a concatenated feature. Joint self-attention is then calculated using this concatenated feature to obtain the corresponding joint attention value. This joint attention value is used to instruct the third model on the feature transformation process in 2D and 3D spaces, outputting the corresponding first 2D feature, first 3D feature, and first transform parameters used for feature transformation.

[0065] By performing joint attention value calculation through global feature mixing blocks, full exchange of global information is achieved, which is conducive to achieving multi-view consistency in two dimensions and ensuring the consistency between the overall structural morphology represented by three-dimensional features and the two-dimensional image.

[0066] Step S3022: Extract the intermediate layer features of the first model.

[0067] The intermediate layer features represent the features output by each layer of the first model. By detecting the training process of the first model and collecting the output of each layer, the intermediate layer features of the first model can be obtained.

[0068] Step S3023: Inject the first three-dimensional structure representation and the first transformation token into the local feature mixing block, inject the image features into the two-dimensional processing branch, and inject the intermediate layer features into the three-dimensional processing branch and the transformation processing branch in the form of key-value (KV) to obtain the first three-dimensional feature, the first two-dimensional feature and the first transformation parameter.

[0069] When a local attention block of the first model matches a transformer block of the second model based on a preset block matching strategy, the first 3D structural representation and the first transform token are injected into the local feature mixing block. The local feature mixing block simultaneously processes the first 3D structural representation and the first transform token, and image features are injected into the 2D processing branch. Intermediate layer features are injected into the 3D processing branch and the transform processing branch of the third model. Figure 5 As shown in Figure (e).

[0070] Therefore, alignment cues between the two-dimensional and three-dimensional feature spaces can be provided using intermediate layer features. The corresponding first two-dimensional features are output through the two-dimensional processing branch, the corresponding first three-dimensional features are output through the three-dimensional processing branch, and the corresponding first transformation parameters are output through the transformation processing branch.

[0071] By injecting intermediate layer features into the 3D processing branch and the transformation processing branch through local feature mixing blocks, 2D local geometric features are injected into the 3D feature generation process, providing alignment clues between the 2D feature space and the 3D feature space.

[0072] Step S3024: Inject image features into the non-feature mixing block to obtain the first two-dimensional features.

[0073] When a local attention block of the first model is detected to be mismatched with any transformer block of the second model based on a preset block matching strategy, it indicates that feature mixing will not occur. In this case, the image features are injected into the non-feature mixing block for two-dimensional processing, and the corresponding first two-dimensional features are output.

[0074] Step S3025: Inject the first three-dimensional structure representation into the three-dimensional processing branch to obtain the first three-dimensional feature.

[0075] Since no feature mixing occurs, the first three-dimensional structural representation is injected into the corresponding three-dimensional processing branch for feature output, and the corresponding first three-dimensional feature is obtained. That is, the first three-dimensional structural representation is used as the first three-dimensional feature.

[0076] Step S3026: Inject the first transformation token into the transformation processing branch to obtain the first transformation parameters.

[0077] Since no feature mixing occurs, the first transformation token is injected into the corresponding transformation processing branch for processing to obtain the corresponding first transformation parameter. This first transformation parameter is used to represent the global similarity transformation parameter between the first two-dimensional feature and the first three-dimensional feature.

[0078] Step S303: Using the first transformation parameters, align the first 3D feature and the first 2D feature to generate a 3D structure and a camera pose. The 3D structure corresponds to the first object, and the camera pose corresponds to the first image. For details, please refer to the relevant descriptions of the steps in the embodiments shown above; they will not be repeated here.

[0079] In some cases, the method for generating three-dimensional structures and poses inputs image features, a first three-dimensional structure representation, and a first transform token feature into the corresponding feature mixing for feature interaction. The transform token is used to indicate a full global exchange between the two-dimensional feature space and the three-dimensional feature space. The image features are used to calibrate the three-dimensional features to ensure that the output three-dimensional features can accurately represent the overall shape of the first object. At the same time, the three-dimensional features are used to constrain the two-dimensional features under different viewpoints to ensure consistency across multiple viewpoints.

[0080] In some cases, a method for generating 3D structures and poses is provided, which can be used for the joint generation of 3D structures and camera poses. Figure 6This is a flowchart illustrating methods for generating 3D structures and poses under certain conditions, such as... Figure 6 As shown, the process includes the following steps: Step S401: Obtain a first image and its image features, as well as a first three-dimensional structure representation and a first transformation token. The first image corresponds to a first object, and its image features are generated based on a first model; the first three-dimensional structure representation is generated based on a second model, and the first transformation token characterizes the transformation from the first image to the three-dimensional structure. For details, please refer to the relevant descriptions of the steps in the embodiments shown above, which will not be repeated here.

[0081] Step S402: The image features, the first 3D structural representation, and the first transformation token are injected into the feature interaction module of the third model to obtain the first 3D features, the first 2D features, and the first transformation parameters. The first 3D features correspond to the first 3D structural representation, the first 2D features correspond to the image features, and the first transformation parameters correspond to the first transformation token. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above; they will not be repeated here.

[0082] Step S403: Using the first transformation parameters, align the first 3D features and the first 2D features to generate a 3D structure and a camera pose. The 3D structure corresponds to the first object, and the camera pose corresponds to the first image.

[0083] Specifically, step S403 includes: Step S4031: Decode the first two-dimensional features to obtain the camera pose and the first point cloud information.

[0084] The camera pose represents the position and orientation of the camera capturing the first object; the first point cloud information represents a set of three-dimensional points in a local area, for example, the first point cloud information can be a local point cloud map. It is represented in the form of.

[0085] Specifically, the two-dimensional processing branch of the third model is used to decode the first two-dimensional features to obtain the camera pose at each viewpoint carried by the first two-dimensional features. And the first point of cloud information from each perspective.

[0086] Step S4032: The camera pose and the first point cloud information are transformed using the first transformation parameters to obtain the second point cloud information, which is located in the three-dimensional feature space.

[0087] Since the first point cloud information is predicted by the first model based on the first two-dimensional features, the first point cloud information can be transformed into the three-dimensional feature space (e.g., voxel space) where the second model is located using the first transformation parameters to obtain the second point cloud information in the three-dimensional feature space.

[0088] Specifically, the transformation process is as follows:

[0089] in, This represents the second point cloud information, namely the transformed globally aligned point cloud; This represents the first point cloud information, i.e., the local point cloud before transformation; Let s represent the first transformation parameter, s represent the scaling parameter, R represent the rotation parameter, and T represent the translation parameter; () indicates the camera pose.

[0090] Step S4033: Align the first three-dimensional feature and the second three-dimensional feature to generate a three-dimensional structure, wherein the second three-dimensional feature corresponds to the second point cloud information.

[0091] Align the second three-dimensional features corresponding to the second point cloud information with the first three-dimensional features in space. Combine the three-dimensional feature alignment results to indicate the three-dimensional structure generation process of the third model and output the corresponding three-dimensional structure.

[0092] In some alternative cases, step S4033 above may include: Step c1: Obtain the first loss information of the first three-dimensional feature, and obtain the second loss information and the third loss information of the second point cloud information. The third loss information is used to characterize the normal consistency.

[0093] Step c2 involves weighted fusion of the first loss information, the second loss information, and the third loss information to obtain the fourth loss information.

[0094] Step c3: Using the fourth loss information, the first three-dimensional features and the second three-dimensional features are aligned to generate a three-dimensional structure.

[0095] The first loss information represents the matching loss of the first 3D feature, used to supervise the generation quality of the 3D structure. The first loss information can be expressed as: .in, This indicates the first loss information. The feature vector representing the first three-dimensional feature; Represents a Gaussian noise vector; This represents the offset vector of the three-dimensional structure.

[0096] The second loss information represents the point cloud alignment loss, which can be obtained by calculating the distance between the point cloud corresponding to the second point cloud information and the real point cloud. The third loss information is used to characterize the normal consistency loss between the point cloud corresponding to the second point cloud information and the real point cloud. The difference between the point cloud corresponding to the second point cloud information and the real point cloud is measured by the second and third loss information.

[0097] The first, second, and third loss information are weighted and fused according to their respective weights to obtain the corresponding fourth loss information. This fourth loss information is then used to instruct the forced alignment between the first and second 3D features, enabling the third model to generate the 3D structure based on the aligned 3D features.

[0098] Multiple loss information is introduced to instruct the second three-dimensional features predicted by the first model and the first three-dimensional features generated by the second model to be spatially aligned, thereby establishing explicit geometric constraints.

[0099] In some alternative cases, step c2 above may include: Step c21: Obtain the first weight, the second weight, and the third weight. The first weight corresponds to the first loss information, the second weight corresponds to the second loss information, and the third weight corresponds to the third loss information.

[0100] Step c22: Dynamically adjust the second and third weights.

[0101] Step c23: Using the first weight, the adjusted second weight, and the adjusted third weight, the first loss information, the second loss information, and the third loss information are weighted and fused to obtain the fourth loss information.

[0102] The first weight can be a fixed value, such as 1. The second and third weights can be dynamically adjusted according to the time steps of the generation process. Specifically, in the early stages of 3D structure generation, the 3D structure is heavily noisy. At this time, the second weight can be reduced and the third weight increased to constrain the overall orientation of the 3D structure. In the later stages of 3D structure generation, the second weight is increased to perform fine position alignment, while the third weight is reduced to avoid over-constraint. Correspondingly, the fourth loss information can be updated according to the dynamic adjustment of the second and third weights.

[0103] In some optional scenarios, the determination of the fourth loss information can be expressed as follows:

[0104] in, This indicates the fourth loss information; This represents the first loss information, with a first weight of 1; Indicates the second weight. This indicates the second loss information; Indicates the third weight. This indicates the third loss information.

[0105] By utilizing the time steps of the generation process to dynamically adjust the weights of the second loss information (i.e., point cloud loss) and the third loss information (i.e., normal loss), the problem of unstable alignment supervision caused by noise interference can be solved, ensuring the feature alignment effect.

[0106] In some cases, the provided method for generating 3D structures and poses obtains the corresponding camera pose by decoding the first 2D features. The first point cloud information parsed from the first 2D features is transformed using first transformation parameters to obtain the corresponding second point cloud information. The second 3D features corresponding to the second point cloud information are then aligned with the first 3D features to generate the 3D structure. Thus, the first transformation parameters can uniformly map and align the geometric information in the independent camera coordinate system predicted by the first model to the 3D feature space of the second model, thereby establishing clear geometric constraints between the first and second 3D features, ensuring pixel-level alignment between the 3D structure and the first image, and improving the generation quality of the 3D structure.

[0107] As a specific application example in some cases, the first model is the Pi3 model, and the second model is the TRELLIS flow matching model. A feature interaction module between the Pi3 model and the TRELLIS flow matching model is constructed according to a block matching strategy to simultaneously process features from three modalities, including: flow matching noise latent variables from the TRELLIS flow matching model; and learnable global transformation tokens (Tokens) used to encode global similarity transformation parameters. .like Figure 7 As shown, the feature interaction module includes: non-mixed blocks, local feature mixing blocks, and global feature mixing blocks.

[0108] like Figure 8 As shown, based on the construction of the complete feature interaction module, input data is acquired, including the input image, the initialized 3D structural representation, and the initialized transformation token. In the encoding stage, a feedforward encoder is used to encode the features of the input image to obtain image features. Feature interaction is performed between the image features, the 3D structural representation, and the transformation token to output the corresponding 3D features, 2D features, and transformation parameters. The 2D processing branch (corresponding to the feedforward reconstruction branch of the first model) outputs the camera pose and local point cloud, the 3D processing branch (the generative reconstruction branch of the TRELLIS flow matching model) outputs the velocity field for flow matching, and the transformation branch outputs the global similarity transformation parameters.

[0109] Subsequently, the camera pose and local point cloud are processed using global similarity transformation parameters to obtain predicted 3D features. The alignment loss between the predicted 3D features and the real 3D features corresponding to the velocity field is calculated. The alignment loss is used to train the generation of 3D structures until the alignment loss meets the conditions, and then the final 3D structure is output.

[0110] In some cases, a three-dimensional structure and pose generation apparatus is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0111] In some cases, a device for generating three-dimensional structures and poses is provided, such as... Figure 9 As shown, it includes: The acquisition module 701 is used to acquire a first image and its image features, as well as a first three-dimensional structure representation and a first transformation token. The first image corresponds to a first object, and the image features are generated based on a first model. The first three-dimensional structure representation is generated based on a second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure.

[0112] The feature acquisition module 702 is used to inject image features, a first three-dimensional structural representation, and a first transformation token into the feature interaction module of the third model to obtain a first three-dimensional feature, a first two-dimensional feature, and a first transformation parameter; the first three-dimensional feature corresponds to the first three-dimensional structural representation, the first two-dimensional feature corresponds to the image feature, and the first transformation parameter corresponds to the first transformation token.

[0113] The joint generation module 703 is used to align the first three-dimensional feature and the first two-dimensional feature using the first transformation parameters to generate a three-dimensional structure and a camera pose, wherein the three-dimensional structure corresponds to the first object and the camera pose corresponds to the first image.

[0114] In some optional cases, the above-mentioned device further includes: The block matching module is used to perform block matching between the first model and the second model based on a preset block matching strategy, resulting in non-feature mixing blocks, local feature mixing blocks, and global feature mixing blocks. The feature interaction module of the third model includes: non-feature mixing blocks, local feature mixing blocks, and global feature mixing blocks; the non-feature mixing blocks include the first local processing block of the first model; the local feature mixing blocks include the first attention block of the first model and the first transformer block of the second model; the global feature mixing blocks include the second attention block of the first model and the second transformer block of the second model.

[0115] In some optional cases, the feature acquisition module 702 includes: The first injection unit is used to inject image features, a first three-dimensional structural representation, and a first transformation token into a global feature mixing block to obtain a first two-dimensional feature, a first three-dimensional feature, and a first transformation parameter.

[0116] In some optional cases, the first injection unit includes: The splicing subunit is used to splice image features, a first three-dimensional structural representation, and a first transformation token using a global feature mixing block to obtain spliced ​​features.

[0117] The joint attention acquisition subunit is used to obtain joint attention values ​​by utilizing splicing features.

[0118] The feature transformation subunit is used to constrain the feature transformation between two-dimensional and three-dimensional spaces using joint attention values ​​to obtain the first two-dimensional feature, the first three-dimensional feature, and the first transformation parameter.

[0119] In some optional cases, the feature acquisition module 702 includes: The extraction unit is used to extract intermediate layer features of the first model.

[0120] The second injection unit is used to inject the first three-dimensional structure representation and the first transformation token into the local feature mixing block, inject the image features into the two-dimensional processing branch, and inject the intermediate layer features into the three-dimensional processing branch and the transformation processing branch to obtain the first three-dimensional features, the first two-dimensional features and the first transformation parameters.

[0121] In some optional cases, the feature acquisition module 702 includes: The third injection unit is used to inject image features into the non-feature mixing block to obtain the first two-dimensional features.

[0122] The fourth injection unit is used to inject the first three-dimensional structure representation into the three-dimensional processing branch to obtain the first three-dimensional feature.

[0123] The fifth injection unit is used to inject the first transformation token into the transformation processing branch to obtain the first transformation parameters.

[0124] In some alternative implementations, the joint generation module 703 includes: The decoding unit is used to decode the first two-dimensional features to obtain the camera pose and the first point cloud information.

[0125] The point cloud transformation unit is used to transform the camera pose and the first point cloud information using the first transformation parameters to obtain the second point cloud information, which is located in the three-dimensional feature space.

[0126] The feature alignment unit is used to align the first three-dimensional feature and the second three-dimensional feature to generate a three-dimensional structure, wherein the second three-dimensional feature corresponds to the second point cloud information.

[0127] In some alternative implementations, the feature alignment unit includes: The loss acquisition subunit is used to acquire first loss information of the first three-dimensional feature, and second and third loss information of the second point cloud information. The third loss information is used to characterize normal consistency.

[0128] The weighted sub-unit is used to weight and fuse the first loss information, the second loss information, and the third loss information to obtain the fourth loss information.

[0129] The structure generation sub-unit is used to generate a three-dimensional structure by using the fourth loss information to instruct the alignment of the first three-dimensional features and the second three-dimensional features.

[0130] In some optional cases, the structure generation sub-unit is specifically used to: obtain a first weight, a second weight, and a third weight, where the first weight corresponds to the first loss information, the second weight corresponds to the second loss information, and the third weight corresponds to the third loss information; dynamically adjust the second weight and the third weight; and use the first weight, the adjusted second weight, and the adjusted third weight to weightedly fuse the first loss information, the second loss information, and the third loss information to obtain the fourth loss information.

[0131] In some cases, the provided three-dimensional structure and pose generation apparatus can execute the three-dimensional structure and pose generation method provided in the above embodiments, and has the corresponding functional modules and beneficial effects of the execution method.

[0132] By introducing a first transformation token for transforming a first image into a 3D structure, the first transformation token interacts with the image features generated by the first model and the first 3D structure representation generated by the second model. This allows for bidirectional reciprocal flow of 2D and 3D information guided by the first transformation token. This enables 2D information to calibrate the 3D structure, and the 3D structure to constrain 2D predictions across all viewpoints, resulting in joint predictions of the first 2D features, first 3D features, and first transformation parameters. The first 2D and first 3D features are then spatially aligned using the first transformation parameters to obtain the 3D structure of the first object and the camera pose of the first image. Thus, the first transformation parameters achieve joint optimization of 3D structure generation and camera pose estimation, improving the accuracy of camera pose estimation and the reconstruction quality of the 3D structure. This ensures that when the 3D structure is projected back to the first image from the original viewpoint, it achieves pixel-level alignment with the first image, guaranteeing the consistency of the 3D structure and the first image's visual representation of the first object.

[0133] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0134] Figure 10 This is a schematic diagram of the structure of an electronic device provided in certain situations.

[0135] The following is a detailed reference. Figure 10 This diagram illustrates a structural schematic suitable for implementing an electronic device in certain situations. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 801, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 802 or a program loaded from memory 808 into random access memory (RAM) 803. RAM 803 also stores various programs and data required for the operation of the electronic device. The processor 801, ROM 802, and RAM 803 are interconnected via bus 804. An input / output interface 805 is also connected to bus 804.

[0136] Typically, the following devices can be connected to the input / output interface 805: input devices 806 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 807 including, for example, a liquid crystal display, speaker, vibrator, etc.; memory devices 808 including, for example, magnetic tape, hard disk, etc.; and communication devices 809. Communication device 809 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0137] In particular, the processes described in the flowchart above can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by a processor 801, it performs the functions defined in the methods for generating three-dimensional structures and poses in some cases.

[0138] Figure 10 The electronic devices shown are merely examples and should not be construed as limiting their functionality or scope of use in any situation.

[0139] In some cases, a computer-readable storage medium is also provided, in which the above-described methods can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the three-dimensional structure and pose generation method shown in the above embodiments.

[0140] Some of the above solutions can be applied as computer program products, such as computer program instructions. When executed by a computer, these instructions, through the operation of the computer, can invoke or provide the aforementioned methods and / or technical solutions. Those skilled in the art should understand that the forms in which computer program instructions exist in computer-readable media include, but are not limited to, source files, executable files, and installation package files. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instruction; the computer compiling the instruction and then executing the corresponding compiled program; the computer reading and executing the instruction; or the computer reading and installing the instruction and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0141] Although embodiments in some cases have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the above description, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for generating a three-dimensional structure and pose, comprising: Acquire a first image and its image features, as well as a first three-dimensional structural representation and a first transformation token, wherein the first image corresponds to a first object, and the image features are generated based on a first model; The first three-dimensional structure representation is generated based on the second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure; The image features, the first three-dimensional structure representation, and the first transformation token are injected into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters; the first three-dimensional features correspond to the first three-dimensional structure representation, the first two-dimensional features correspond to the first image, and the first transformation parameters correspond to the first transformation token. Using the first transformation parameters, the first three-dimensional feature and the first two-dimensional feature are aligned to generate a three-dimensional structure and a camera pose. The three-dimensional structure corresponds to the first object, and the camera pose corresponds to the first image.

2. The method according to claim 1, wherein the feature interaction module of the third model comprises: Non-feature blending blocks, local feature blending blocks, and global feature blending blocks; The non-feature blending block includes the first local processing block of the first model; The local feature blending block includes the first attention block of the first model and the first transformer block of the second model; The global feature mixing block includes the second attention block of the first model and the second transformer block of the second model; The non-feature mixing block, the local feature mixing block, and the global feature mixing block are obtained by performing block matching between the first model and the second model based on a preset block matching strategy.

3. The method according to claim 2, wherein injecting the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters includes: The image features, the first three-dimensional structure representation, and the first transformation token are injected into the global feature mixing block to obtain the first two-dimensional features, the first three-dimensional features, and the first transformation parameters.

4. The method according to claim 3, wherein injecting the image features, the first three-dimensional structure representation, and the first transform token into the global feature mixing block to obtain the first two-dimensional features, the first three-dimensional features, and the first transform parameters comprises: Using the global feature mixing block, the image features, the first three-dimensional structure representation, and the first transform token are concatenated to obtain the concatenated features; Using the splicing features, a joint attention value is obtained; By using the joint attention value to constrain the feature transformation of the two-dimensional space and the three-dimensional space, the first two-dimensional feature, the first three-dimensional feature, and the first transformation parameter are obtained.

5. The method according to claim 2, wherein injecting the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters includes: Extract the intermediate layer features of the first model; The first three-dimensional structure representation and the first transformation token are injected into the local feature mixing block, the image features are injected into the two-dimensional processing branch, and the intermediate layer features are injected into the three-dimensional processing branch and the transformation processing branch to obtain the first three-dimensional feature, the first two-dimensional feature and the first transformation parameter.

6. The method according to claim 2, wherein injecting the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters comprises: The image features are injected into the non-feature mixing block to obtain the first two-dimensional features; The first three-dimensional structure representation is injected into the three-dimensional processing branch to obtain the first three-dimensional feature; The first transformation token is injected into the transformation processing branch to obtain the first transformation parameters.

7. The method according to any one of claims 1 to 6, wherein aligning the first three-dimensional feature and the first two-dimensional feature using the first transformation parameter to generate a three-dimensional structure and camera pose comprises: Decode the first two-dimensional feature to obtain the camera pose and the first point cloud information; The camera pose and the first point cloud information are transformed using the first transformation parameters to obtain the second point cloud information, which is located in a three-dimensional feature space. Align the first three-dimensional feature and the second three-dimensional feature to generate the three-dimensional structure, wherein the second three-dimensional feature corresponds to the second point cloud information.

8. The method according to claim 7, wherein aligning the first three-dimensional feature and the second three-dimensional feature to generate the three-dimensional structure comprises: First loss information of the first three-dimensional feature is obtained, and second and third loss information of the second point cloud information are obtained, wherein the third loss information is used to characterize normal consistency; The first loss information, the second loss information, and the third loss information are weighted and fused to obtain the fourth loss information; Using the fourth loss information, the first three-dimensional feature and the second three-dimensional feature are aligned to generate the three-dimensional structure.

9. The method according to claim 8, wherein the weighted fusion of the first loss information, the second loss information, and the third loss information to obtain the fourth loss information comprises: Obtain a first weight, a second weight, and a third weight, wherein the first weight corresponds to the first loss information, the second weight corresponds to the second loss information, and the third weight corresponds to the third loss information; The second weight and the third weight are dynamically adjusted; The first loss information, the second loss information, and the third loss information are weighted and fused using the first weight, the second loss information, and the third loss information to obtain the fourth loss information.

10. A three-dimensional structure and pose generation device, comprising: The acquisition module is used to acquire a first image and its image features, as well as a first three-dimensional structure representation and a first transformation token. The first image corresponds to a first object, and the image features are generated based on a first model. The first three-dimensional structure representation is generated based on a second model, and the first transformation token is used to characterize the transformation from the first image to the three-dimensional structure. The feature acquisition module is used to inject the image features, the first three-dimensional structure representation, and the first transformation token into the feature interaction module of the third model to obtain the first three-dimensional features, the first two-dimensional features, and the first transformation parameters; the first three-dimensional features correspond to the first three-dimensional structure representation, the first two-dimensional features correspond to the image features, and the first transformation parameters correspond to the first transformation token. The joint generation module is used to align the first three-dimensional feature and the first two-dimensional feature using the first transformation parameters to generate a three-dimensional structure and a camera pose, wherein the three-dimensional structure corresponds to the first object and the camera pose corresponds to the first image.

11. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method for generating three-dimensional structures and poses according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute the method for generating a three-dimensional structure and pose according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the method for generating a three-dimensional structure and pose as described in any one of claims 1 to 9.

Images