A transferable 3D reconstruction method and system based on hierarchical reinforcement learning

Abstract

The present invention discloses a transferable three-dimensional reconstruction method and system based on hierarchical reinforcement learning, which belongs to the field of three-dimensional reconstruction. By simplifying the action space, the accuracy of reconstruction is increased, and at the same time, the trained agents are made more transferable between different categories. Hierarchical reinforcement learning aggregates the actions of the same Loop operation, thereby decomposing the tasks to different agents for processing. The top-level agent can grasp the overall shape globally, select the Loop that most needs to be adjusted, and hand it over to the corresponding sub-agent for processing. A sub-agent based on state space enhancement (ASS‑Sub‑Agent) is used to replace a group of sub-agents in the traditional method. The ASS‑Sub‑Agent can share the training process of the atomic agent and has fewer training parameters, which can accelerate the training process of the sub-agent. The more diverse state space promotes the portability of the ASS‑Sub‑Agent.

Description

Technical Field

[0001] The present invention relates to the fields of computer engineering applications, computer graphics, computer-aided design, and three-dimensional reconstruction technology, and in particular to a transferable three-dimensional reconstruction method and system based on hierarchical reinforcement learning. Background Art

[0002] 3D reconstruction plays an important role in engineering. In recent years, it has achieved remarkable success in applications such as computer-aided design (CAD), 3D printing, and 3D accident reconstruction. The classic approach involves training neural networks to predict 3D shapes directly from images. However, this process lacks the ability to interpret or obtain detailed reconstructions, making it difficult to apply in engineering applications.

[0003] To make the reconstruction process interpretable, Lin et al. proposed a method that uses reinforcement learning to mimic human modelers in 3D reconstruction. They first obtain a coarse-grained 3D shape from an image. Loops are then added to this 3D shape to subdivide it. An agent is trained to drag the opposite corners of the loops to bring the 3D shape closer to the target shape. This method provides an interpretable reconstruction process.

[0004] However, the above methods still have the following two technical problems: (1) The reconstruction accuracy is limited by the complexity of the action space, and the search space grows exponentially with the growth of the action space. It is difficult for the agent to search for the optimal action, resulting in low learning efficiency and accuracy. (2) When training agents for different categories of data, they do not consider the transfer of knowledge from agents of other categories. Summary of the Invention

[0005] The present invention proposes a transferable 3D reconstruction method and system based on hierarchical reinforcement learning, which simplifies the action space to make the reconstruction process easier, improves the reconstruction accuracy, increases the mobility of the agent between different categories of data, and accelerates the convergence speed of the agent.

[0006] To achieve the above objectives, the present invention provides a first aspect of a transferable 3D reconstruction method based on hierarchical reinforcement learning, comprising:

[0007] S1: Obtain a coarse-grained 3D shape based on the target depth image and target shape;

[0008] S2: Add n loops to the obtained coarse-grained three-dimensional shape to obtain the three-dimensional shape to be edited. A loop is a rectangle connected end to end.

[0009] S3: The 3D shape to be edited, the target depth image, and the number of moments using One-Hot encoding are input into the top-level agent, which then selects a loop.

[0010] S4: The features of the selected loop are used as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and the basic action to be performed on the loop is output;

[0011] S5: Apply the output of the basic action performed on the loop to the loop to obtain the updated 3D shape to be edited. The number of moments is increased by 1. The reward value obtained is used to guide the training of the top-level agent and the sub-agent based on state space enhancement, and steps S3 to S5 are repeated until the termination state is reached.

[0012] In one embodiment, step S1 includes:

[0013] Based on the existing 3D dataset ShapeNet, we obtain the target depth image and its corresponding 3D voxel shape as training data;

[0014] The convolutional neural network is trained using the training data to obtain a trained convolutional neural network. The trained convolutional neural network is used to output a coarse-grained three-dimensional shape based on the target depth image and the target shape. The coarse-grained three-dimensional shape is composed of a set of cuboid primitives.

[0015] In one embodiment, the three-dimensional shape to be edited in step S2 is:

[0016]

[0017] Among them, L i is the eigenvalue of the i-th Loop, is the coordinate of the two diagonal corners of the Loop, and n is the number of Loops.

[0018] In one embodiment, step S3 includes:

[0019] The three-dimensional shape to be edited L, the target depth image Ig, and the number of moments Sp using One-Hot encoding are used as states, that is, s = {L, Ig, Sp}. s is input to the top-level agent, and n values are output, corresponding to the predicted value function Q of the operation loops. During the training phase, the loop with the largest value function is greedily selected with a preset probability to obtain the feature value of the loop. The loss function for training the top-level agent is as follows:

[0020]

[0021] where s t represents the state at the current time t, ω represents the loop selected at time t, s t+k represents the state after executing k steps based on the current loop, ω' represents the loop with the largest value function at time t+k, θ Ω is the network parameter of the top-level agent, QΩ Represents the value function that can be obtained by selecting the Loop based on the current parameters and state; R t+k represents the cumulative discounted return from time t to time t+k, and the formula is as follows:

[0022] R t+k =r t+1 +γr t＝2 +...+γ k-1 r t+k

[0023] Among them, γ, γ k-1 is the discount factor, included in [0,1], r t+1 represents the reward obtained at time t+1, the subscript represents the number of moments, r t+k represents the reward obtained at time t+k;

[0024] The intersection over union (IoU) of the edited shape and the target shape is used as the evaluation of reconstruction accuracy, and the difference in IoU before and after the action is executed is used as the reward for executing the action. The action refers to the basic action output by the ASS-Sub-Agent, and it is applied to the loop selected by the top-level agent.

[0025] In one embodiment, step S4 includes:

[0026] The obtained Loop feature is used as the enhancement of the state, that is, s ω ={L,Ig,Sp,L ω}, change s ω Input ASS-Sub-Agent, output the value function Q of the basic action ω ,During the training phase, the basic action with the largest value function is greedily selected with a preset probability;

[0027] DDQN is used to train ASS-Sub-Agent, and the loss function is defined as follows:

[0028]

[0029] where θ and θ' represent the network parameters of the current subagent and the target subagent, respectively, t+1 、s t Represent the state at time t+1 and the current time t respectively, γ is the discount factor, a t is the action executed at the current time t, The definition is as follows:

[0030]

[0031] Use s instead of s in the above formula t+1 , a replaces the above formula get:

[0032]

[0033] Among them, β ω (s) represents the probability of selecting a new Loop in state s;

[0034] The basic action for the loop is defined as dragging the two diagonal corners of the loop in the x, y, and z directions.

[0035] In one embodiment, the terminal state in step S5 is a state after executing the preset steps.

[0036] Based on the same inventive concept, the second aspect of the present invention provides a transferable 3D reconstruction system based on hierarchical reinforcement learning, comprising:

[0037] A coarse-grained three-dimensional shape acquisition module is used to obtain a coarse-grained three-dimensional shape based on the target depth image and the target shape;

[0038] The module for obtaining the 3D shape to be edited is used to add n loops to the obtained coarse-grained 3D shape to obtain the 3D shape to be edited. A loop is a rectangle connected end to end.

[0039] The loop selection module is used to input the 3D shape to be edited, the target depth image, and the number of moments using one-hot encoding into the top-level agent, and the top-level agent selects a loop;

[0040] The basic action output module is used to use the characteristics of the selected loop as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and output the basic action to be performed on the loop;

[0041] The 3D reconstruction module is used to apply the basic actions outputted to the loop to the loop, obtain the updated 3D shape to be edited, increase the number of moments by 1, use the obtained reward value to guide the training of the top agent and the sub-agent based on state space enhancement, and repeat the steps from the loop selection module to the 3D reconstruction module until the termination state is reached.

[0042] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in the first aspect when the program is executed.

[0043] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0044] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0045] The present invention proposes a transferable 3D reconstruction method and system based on hierarchical reinforcement learning, which increases the accuracy of reconstruction by simplifying the action space. At the same time, since the agent is used to solve simpler problems, the number of training parameters is smaller and overfitting is less likely to occur. In addition, state enhancement improves the diversity of training data, so that the trained agent is more transferable between different categories; hierarchical reinforcement learning aggregates the actions of the same loop operation, thereby decomposing the task to different agents for processing; the top-level agent can grasp the overall shape globally, select the loop that most needs adjustment, and hand it over to the corresponding sub-agent for processing; using a sub-agent based on state space enhancement (ASS-Sub-Agent) to replace a group of sub-agents in the traditional method, ASS-Sub-Agent can share the training process of the atomic agent and has fewer training parameters, which can accelerate the training process of the sub-agent, and a more diverse state space promotes the portability of the ASS-Sub-Agent. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 Flowchart of the transferable 3D reconstruction method based on hierarchical reinforcement learning;

[0048] Figure 2 The preprocessing and basic action definition of the transferable 3D reconstruction method based on hierarchical reinforcement learning in an embodiment of the present invention include: (a) adding loops to the coarse-grained 3D shape; (b) defining basic actions;

[0049] Figure 3 Figure 1 is an overview of the overall framework of the transferable 3D reconstruction method based on hierarchical reinforcement learning in an embodiment of the present invention. (a) is the reinforcement learning process, and (b) shows the hierarchical structure.

[0050] Figure 4 is a schematic diagram of a sub-agent based on state space enhancement (ASS-Sub-Agent) in a transferable 3D reconstruction method based on hierarchical reinforcement learning in an embodiment of the present invention;

[0051] Figure 5is the visualization result of the transferable 3D reconstruction method based on hierarchical reinforcement learning and the nearest related method in an embodiment of the present invention;

[0052] Figure 6 The present invention compares the effects of a transferable 3D reconstruction method based on hierarchical reinforcement learning with and without knowledge transfer on three classes.

[0053] Figure 7 3. FIG. 3 is a schematic diagram comparing the transferability of the transferable 3D reconstruction method based on hierarchical reinforcement learning and the nearest related method in three classes according to an embodiment of the present invention;

[0054] Figure 8 is a structural block diagram of a transferable 3D reconstruction system based on hierarchical reinforcement learning in an embodiment of the present invention;

[0055] Figure 9 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present invention;

[0056] Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention.

[0057] The English names appearing in the accompanying drawings have the following meanings:

[0058] Top-Agent: top-level agent; Conv: convolutional network; MLP: fully connected full network; Target Image: target depth image; Edge Loop: shape to be edited. DETAILED DESCRIPTION

[0059] Based on the technical problems existing in the prior art, the present invention proposes a transferable three-dimensional reconstruction method and system based on hierarchical reinforcement learning, which simplifies the action space to make the reconstruction process easier, improves the reconstruction accuracy, and increases the mobility of the agent between different categories of data, thereby accelerating the convergence speed of the agent.

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0061] Example 1

[0062] An embodiment of the present invention provides a transferable 3D reconstruction method based on hierarchical reinforcement learning, comprising:

[0063] S1: Obtain a coarse-grained 3D shape based on the target depth image and target shape;

[0064] S2: Add n loops to the obtained coarse-grained three-dimensional shape to obtain the three-dimensional shape to be edited. A loop is a rectangle connected end to end.

[0065] S3: The 3D shape to be edited, the target depth image, and the number of moments using One-Hot encoding are input into the top-level agent, which then selects a loop.

[0066] S4: The features of the selected loop are used as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and the basic action to be performed on the loop is output;

[0067] S5: Apply the output of the basic action performed on the loop to the loop to obtain the updated 3D shape to be edited. The number of moments is increased by 1. The reward value obtained is used to guide the training of the top-level agent and the sub-agent based on state space enhancement, and steps S3 to S5 are repeated until the termination state is reached.

[0068] This paper proposes a transferable 3D reconstruction method based on hierarchical reinforcement learning. This method aggregates basic actions into larger-scale action groups, each of which is called an option. A top-level agent decides which option to select, and the corresponding subagents then decide which basic action to execute within that option. To simplify subagent training, a single subagent based on state-space reinforcement learning replaces the traditional group of subagents.

[0069] Hierarchy refers to the division into top-level agents and sub-agents. The top-level agent in step S3 is used to select a loop, and then the loop's features are used as an enhancement of the input state of the sub-agent (ASS-Sub-Agent) based on state space enhancement. The ASS-Sub-Agent outputs the basic actions to be performed on the loop.

[0070] In one embodiment, step S1 includes:

[0071] Based on the existing 3D dataset ShapeNet, we obtain the target depth image and its corresponding 3D voxel shape as training data;

[0072] The convolutional neural network is trained using the training data to obtain a trained convolutional neural network. The trained convolutional neural network is used to output a coarse-grained three-dimensional shape based on the target depth image and the target shape. The coarse-grained three-dimensional shape is composed of a set of cuboid primitives.

[0073] In one embodiment, the three-dimensional shape to be edited in step S2 is:

[0074]

[0075] Among them, L i is the eigenvalue of the i-th Loop, is the coordinate of the two diagonal corners of the Loop, and n is the number of Loops.

[0076] In one embodiment, step S3 includes:

[0077] The three-dimensional shape to be edited L, the target depth image Ig, and the number of moments Sp using One-Hot encoding are used as states, that is, s = {L, Ig, Sp}. s is input to the top-level agent, and n values are output, corresponding to the predicted value function Q of the operation loops. During the training phase, the loop with the largest value function is greedily selected with a preset probability to obtain the feature value of the loop. The loss function for training the top-level agent is as follows:

[0078]

[0079] where s t represents the state at the current time t, ω represents the loop selected at time t, s t+k represents the state after executing k steps based on the current loop, ω' represents the loop with the largest value function at time t+k, θ Ω is the network parameter of the top-level agent, Q Ω Represents the value function that can be obtained by selecting the Loop based on the current parameters and state; R t+k represents the cumulative discounted return from time t to time t+k, and the formula is as follows:

[0080] R t+k =r t+1 +γr t＝2 +...+γ k-1 r t+k

[0081] Among them, γ, γ k-1 is the discount factor, included in [0,1], r t+1 represents the reward obtained at time t+1, the subscript represents the number of moments, r t+k represents the reward obtained at time t+k;

[0082] The intersection over union (IoU) of the edited shape and the target shape is used as the evaluation of reconstruction accuracy, and the difference in IoU before and after the action is executed is used as the reward for executing the action. The action refers to the basic action output by the ASS-Sub-Agent, and it is applied to the loop selected by the top-level agent.

[0083] Among them, the preset probability is 0.98.

[0084] In one embodiment, step S4 includes:

[0085] The obtained Loop feature is used as the enhancement of the state, that is, s ω ={L,Ig,Sp,L ω}, change s ω Input ASS-Sub-Agent, output the value function Q of the basic action ω ,During the training phase, the basic action with the largest value function is greedily selected with a preset probability;

[0086] DDQN is used to train ASS-Sub-Agent, and the loss function is defined as follows:

[0087]

[0088] where θ and θ' represent the network parameters of the current subagent and the target subagent, respectively, t+1 、s t Represent the state at time t+1 and the current time t respectively, γ is the discount factor, a t is the action executed at the current time t, The definition is as follows:

[0089]

[0090] Use s instead of s in the above formula t+1 , a replaces the above formula get:

[0091]

[0092] Among them, β ω (s) represents the probability of selecting a new Loop in state s;

[0093] The basic action for the loop is defined as dragging the two diagonal corners of the loop in the x, y, and z directions.

[0094] In the specific implementation process, the preset probability is 0.98. In order to simplify the training, β ω (s) is defined as 0 or 1. To discretize the actions, this embodiment defines the drag amplitude as six discrete values {-3, -2, -1, 1, 2, 3}, so the number of basic actions for a loop is 2*3*6=36 in total.

[0095] In one embodiment, the terminal state in step S5 is a state after executing the preset steps.

[0096] During the specific implementation, the preset steps are 100 steps.

[0097] Please see Figure 1 In order to adjust the coarse-grained three-dimensional shape to obtain more details, it needs to be meshed. This implementation adds n loops, where n is defined as 10. The number of loops added to each primitive is proportional to the volume of the primitive. (b) describes the definition of basic actions. An action is defined as a drag operation on two diagonal corners of a loop, with the drag direction being the x, y, and z directions. To discretize the action, the drag amplitude is defined as {-3, -2, -1, 1, 2, 3}. Therefore, the number of operations for a loop is 2*3*6=36.

[0098] Please see Figure 2 , the overall framework is outlined. Part (a) is the RL process, where the agent maps the observed state to the action. The framework can be represented by a tuple (S, A, T, γ, R), where the agent observes the current state s t ∈S, select an action a t ∈A, after the environment executes the action, it will get the next state s t+1 and return r t+1 , repeating the process until the terminal state. The reward will guide the agent to choose the decision with the highest discounted cumulative reward, where the cumulative reward γ is the discount factor. T represents the probability of transitioning from the current state to a specified state by taking a specific action. (b) describes a hierarchical framework where agents can be decomposed into a top-level agent and a set of sub-agents. The top-level agent selects a loop, and the sub-agents associated with that loop decide which action to execute. To simplify sub-agent training, this paper proposes replacing a set of sub-agents with a single ASS-Sub-Agent.

[0099] Please see Figure 3 , describes the ASS-Sub-Agent. At this stage, the characteristic value of the Loop selected by the top-level agent is used as the enhancement of the ASS-Sub-Agent state. As part of the network input, the ASS-Sub-Agent outputs 36 actions, which determine what basic actions to perform on the Loop.

[0100] This implementation uses cumulative return R and IoU as evaluation metrics, comparing the methods of DDQN, Lin et al., and Seiya et al. The reconstruction accuracy using only DDQN is very low due to the large reconstructed action space. Lin et al. achieved better results by combining DDQN with imitation learning (IL). Seiya et al.'s reconstruction accuracy declined slightly because they focused on viewpoint-independent reconstruction and neglected reconstruction accuracy. Our method achieved the highest cumulative return R and IoU, and even without the assistance of IL, our method can achieve better results.

[0101] Please see Figure 4 , is a schematic diagram of a sub-agent based on state space enhancement (ASS-Sub-Agent) in a transferable 3D reconstruction method based on hierarchical reinforcement learning in an embodiment of the present invention;

[0102] Please see Figure 5 , showing the qualitative results of visualization. The first row is the target depth image, and the second and third rows show the reconstruction results of Lin et al. and ours, respectively. It can be seen that the mesh model reconstructed by the method of the present invention is visually closer to the target depth image.

[0103] Figure 6 The figure shows the impact of transferred knowledge, where the lines marked with triangles (with TL) represent the assistance of transferred knowledge, while the lines marked with circles (without TL) represent the absence of transferred knowledge. As we can see, the agent with transferred knowledge can explore strategies that can obtain higher rewards in the initial stage, and it can converge to a good final strategy faster.

[0104] See Figure 7 Figure 2 shows a comparison of the transferability of a transferable 3D reconstruction method based on hierarchical reinforcement learning in an embodiment of the present invention and a recently related method across three classes. The line labeled with triangles (ours) represents our method, and the line labeled with circles (baseline) represents the method of Lin et al. Because the method of Lin et al. generalizes poorly, which can negatively impact training later on, the method of the present invention not only performs better in the initial stages but also has a positive impact on training, demonstrating its superior transferability.

[0105] Example 2

[0106] Based on the same inventive concept, this embodiment provides a transferable 3D reconstruction system based on hierarchical reinforcement learning, see Figure 8 , the system comprises:

[0107] A coarse-grained three-dimensional shape obtaining module 201 is configured to obtain a coarse-grained three-dimensional shape based on the target depth image and the target shape;

[0108] The module 202 for obtaining the 3D shape to be edited is used to add n loops to the obtained coarse-grained 3D shape to obtain the 3D shape to be edited, where a loop is a rectangle connected end to end.

[0109] Loop selection module 203, used to input the 3D shape to be edited, the target depth image and the number of moments using One-Hot encoding into the top-level agent, and the top-level agent selects a loop;

[0110] The basic action output module 204 is used to use the characteristics of the selected loop as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and output the basic action to be performed on the loop;

[0111] The three-dimensional reconstruction module 205 is used to apply the basic actions outputted to the loop to the loop, obtain an updated three-dimensional shape to be edited, increase the number of moments by 1, use the obtained reward value to guide the training of the top-level agent and the sub-agent based on state space enhancement, and repeat the steps from the loop selection module to the three-dimensional reconstruction module until entering the termination state.

[0112] Since the system described in Example 2 of the present invention is the system used to implement the transferable 3D reconstruction method based on hierarchical reinforcement learning described in Example 1 of the present invention, those skilled in the art will be able to understand the specific structure and variations of this system based on the method described in Example 1 of the present invention, and therefore will not be described in detail here. All systems used in the method described in Example 1 of the present invention are within the scope of protection of the present invention.

[0113] Example 3

[0114] Based on the same inventive concept, see Figure 9 The present invention further provides a computer-readable storage medium 300 on which a computer program 311 is stored. When the program is executed, the method described in the first embodiment is implemented.

[0115] Since the computer-readable storage medium described in Example 3 of the present invention is used to implement the method for transferable 3D reconstruction based on hierarchical reinforcement learning in Example 1 of the present invention, the specific structure and variations of the computer-readable storage medium are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and thus will not be further described here. All computer-readable storage media used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0116] Example 4

[0117] Based on the same inventive concept, the present application also provides a computer device, such as Figure 10 As shown, it includes a memory 401, a processor 402 and a computer program 403 stored in the memory and executable on the processor. When the processor executes the above program, the method in the first embodiment is implemented.

[0118] Since the computer device described in Example 4 of the present invention is used to implement the transferable 3D reconstruction method based on hierarchical reinforcement learning in Example 1 of the present invention, the specific structure and variations of this computer device are readily understood by those skilled in the art based on the method described in Example 1 of the present invention, and therefore will not be further described here. All computer devices used in the method of Example 1 of the present invention fall within the scope of protection of the present invention.

[0119] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0122] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A transferable 3D reconstruction method based on hierarchical reinforcement learning, characterized in that: include: S1: Obtain a coarse-grained 3D shape based on the target depth image and target shape; S2: Add n loops to the obtained coarse-grained three-dimensional shape to obtain the three-dimensional shape to be edited. A loop is a rectangle connected end to end. S3: The 3D shape to be edited, the target depth image, and the number of moments using One-Hot encoding are input into the top-level agent, which then selects a loop. S4: The features of the selected loop are used as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and the basic action to be performed on the loop is output; S5: Apply the output of the basic action performed on the loop to the loop to obtain the updated 3D shape to be edited. The number of moments is increased by 1. The reward value obtained is used to guide the training of the top-level agent and the sub-agent based on state space enhancement, and steps S3 to S5 are repeated until the termination state is reached.

2. The transferable 3D reconstruction method based on hierarchical reinforcement learning according to claim 1, characterized in that: Step S1 includes: Based on the existing 3D dataset ShapeNet, we obtain the target depth image and its corresponding 3D voxel shape as training data; The convolutional neural network is trained using the training data to obtain a trained convolutional neural network. The trained convolutional neural network is used to output a coarse-grained three-dimensional shape based on the target depth image and the target shape. The coarse-grained three-dimensional shape is composed of a set of cuboid primitives.

3. The transferable 3D reconstruction method based on hierarchical reinforcement learning according to claim 1, characterized in that: The three-dimensional shape to be edited in step S2 is: Among them, L i is the eigenvalue of the i-th Loop, is the coordinate of the two diagonal corners of the Loop, and n is the number of Loops.

4. The transferable 3D reconstruction method based on hierarchical reinforcement learning according to claim 1, characterized in that: Step S3 includes: The three-dimensional shape to be edited L, the target depth image Ig, and the number of moments Sp using One-Hot encoding are used as states, that is, s = {L, Ig, Sp}. s is input to the top-level agent, and n values are output, corresponding to the predicted value function Q of the operation loops. During the training phase, the loop with the largest value function is greedily selected with a preset probability to obtain the feature value of the loop. The loss function for training the top-level agent is as follows: where s t represents the state at the current time t, ω represents the loop selected at time t, s t+k represents the state after executing k steps based on the current loop, ω' represents the loop with the largest value function at time t+k, θ Ω is the network parameter of the top-level agent, Q Ω Represents the value function that can be obtained by selecting the Loop based on the current parameters and state; R t+k represents the cumulative discounted return from time t to time t+k, and the formula is as follows: R t+k =r t+1 +γr t+2 +...+γ k-1 r t+k Among them, γ, γ k-1 is the discount factor, included in [0,1], r t+1 represents the reward obtained at time t+1, the subscript represents the number of moments, r t+k represents the reward obtained at time t+k; The intersection over union (IoU) between the edited shape and the target shape is used as the evaluation of reconstruction accuracy, and the difference in IoU before and after the action is executed is used as the reward for executing the action. The action refers to the basic action output by the ASS-Sub-Agent, and it is applied to the loop selected by the top-level agent.

5. The transferable 3D reconstruction method based on hierarchical reinforcement learning according to claim 1, characterized in that: Step S4 includes: The obtained Loop feature is used as the enhancement of the state, that is, s ω ={L,Ig,Sp,L ω }, change s ω Input ASS-Sub-Agent, output the value function Q of the basic action ω ,During the training phase, the basic action with the largest value function is greedily selected with a preset probability; DDQN is used to train ASS-Sub-Agent, and the loss function is defined as follows: where θ and θ' represent the network parameters of the current subagent and the target subagent, respectively, t+1 、s t Represent the state at time t+1 and the current time t respectively, γ is the discount factor, a t is the action executed at the current time t, The definition is as follows: Use s instead of s in the above formula t+1 , a replaces the above formula get: Among them, β ω (s) represents the probability of selecting a new Loop in state s; The basic action for the loop is defined as dragging the two diagonal corners of the loop in the x, y, and z directions.

6. The transferable 3D reconstruction method based on hierarchical reinforcement learning according to claim 1, characterized in that: The final state in step S5 is the state after the preset steps are executed.

7. A transferable 3D reconstruction system based on hierarchical reinforcement learning, characterized in that: include: A coarse-grained three-dimensional shape acquisition module is used to obtain a coarse-grained three-dimensional shape based on the target depth image and the target shape; The module for obtaining the 3D shape to be edited is used to add n loops to the obtained coarse-grained 3D shape to obtain the 3D shape to be edited. A loop is a rectangle connected end to end. The loop selection module is used to input the 3D shape to be edited, the target depth image, and the number of moments using one-hot encoding into the top-level agent, and the top-level agent selects a loop; The basic action output module is used to use the characteristics of the selected loop as the enhancement of the input state of the sub-agent ASS-Sub-Agent based on the state space enhancement, and output the basic action to be performed on the loop; The 3D reconstruction module is used to apply the basic actions outputted to the loop to the loop, obtain the updated 3D shape to be edited, increase the number of moments by 1, use the obtained reward value to guide the training of the top agent and the sub-agent based on state space enhancement, and repeat the steps from the loop selection module to the 3D reconstruction module until the termination state is reached.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 6 is implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images