Three-dimensional reconstruction method and device, equipment, storage medium and program product

Abstract

The invention discloses a three-dimensional reconstruction method, a three-dimensional reconstruction device, three-dimensional reconstruction equipment, a storage medium and a computer program product, and relates to the technical field of three-dimensional reconstruction. In the application, the invention provides a self-matching, self-adjusting and self-adapting three-dimensional reconstruction method for the structured light source and the camera. Firstly, according to a preset reinforcement learning model, the real-time light intensity and the real-time pose of a structured light source and the real-time pose of a camera under the real-time light intensity and the real-time pose of the structured light source are adjusted, and then three-dimensional reconstruction is carried out based on the real-time light intensity and the real-time pose of the structured light source and the real-time pose of the camera. Compared with an existing three-dimensional reconstruction method which is difficult to cope with a complex environment, the method adopts a reinforcement learning model to adjust the real-time light intensity and the real-time pose of the structured light source and the real-time pose of the camera under the real-time light intensity and the real-time pose of the structured light source, and achieves the control of the structured light source and the camera under the complex environment. And the three-dimensional reconstruction precision in a complex environment is improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a three-dimensional reconstruction method, a three-dimensional reconstruction apparatus, a three-dimensional reconstruction device, a storage medium and a computer program product. Background Art

[0002] At present, when performing 3D reconstruction, we often face complex 3D reconstruction environments. In complex environments, how to control the 3D reconstruction light source and camera to improve the 3D reconstruction accuracy is a problem that needs to be solved urgently.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present application is to provide a three-dimensional reconstruction method, a three-dimensional reconstruction device, a three-dimensional reconstruction equipment, a storage medium and a computer program product, aiming to solve the technical problem of low three-dimensional reconstruction accuracy in complex environments.

[0005] To achieve the above objectives, the present application proposes a three-dimensional reconstruction method, which includes:

[0006] According to the preset reinforcement learning model, adjust the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source;

[0007] Three-dimensional reconstruction is performed based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0008] In one embodiment, before the step of adjusting the real-time light intensity and real-time posture of the camera under the real-time posture of the structured light source according to the matching relationship generated by the preset reinforcement learning model, the step includes:

[0009] Acquire training data; wherein the training data includes an action space, a state space, and an activation function value determined based on the light intensity and position of the structured light source and the position of the camera;

[0010] The preset reinforcement learning model is obtained by training with the training data.

[0011] In one embodiment, the step of obtaining training data includes:

[0012] Initialize the light intensity and position of the structured light source and the position of the camera;

[0013] Based on the initial light intensity and initial position of the structured light source and the initial position of the camera, determine the initial action space, initial state space and initial excitation function value;

[0014] Based on proximal strategy optimization, determine the new action space, state space and incentive function value;

[0015] The action space, state space and activation function value are combined into data tuples as training data.

[0016] In one embodiment, the step of determining the initial action space, the initial state space, and the initial activation function value based on the initial light intensity and the initial position of the structured light source and the initial position of the camera includes:

[0017] Determine the initial action space based on the initial light intensity and initial position of the structured light source and the initial position of the camera;

[0018] Performing three-dimensional reconstruction based on the initial action space to determine the initial state space;

[0019] The initial action space and the initial state space are input into a preset activation function to obtain an initial activation function value.

[0020] In one embodiment, the step of determining a new action space, state space, and activation function value based on proximal strategy optimization includes:

[0021] Based on the proximal strategy optimization, the new light intensity and position of the structured light source and the new position of the camera are determined;

[0022] Determine a new action space based on the new light intensity and position of the structured light source and the new position of the camera;

[0023] Perform three-dimensional reconstruction based on the new action space to determine a new state space;

[0024] The new action space and the new state space are input into a preset activation function to obtain a new activation function value.

[0025] In one embodiment, the step of obtaining a preset reinforcement learning model through training with the training data includes:

[0026] The actor network and the critic network are trained by the training data to obtain a preset reinforcement learning model; wherein the actor network uses a lightweight residual network, and the critic network uses a long short-term memory network.

[0027] In addition, to achieve the above-mentioned purpose, the present application also proposes a three-dimensional reconstruction device, which includes:

[0028] An adjustment module, used for adjusting the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source according to a preset reinforcement learning model;

[0029] The reconstruction module is used to perform three-dimensional reconstruction based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a three-dimensional reconstruction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the three-dimensional reconstruction method described above.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the three-dimensional reconstruction method described above are implemented.

[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the three-dimensional reconstruction method described above are implemented.

[0033] One or more technical solutions proposed in this application have at least the following technical effects:

[0034] In this application, a self-matching, self-adjusting, and self-adaptive three-dimensional reconstruction method for a structured light source and a camera is proposed. First, according to a preset reinforcement learning model, the real-time light intensity and real-time posture of the structured light source, as well as the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source, are adjusted. Then, three-dimensional reconstruction is performed based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera. Compared with the current three-dimensional reconstruction method that is difficult to cope with complex environments, a reinforcement learning model is used to adjust the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source, so as to realize the control of the structured light source and the camera in a complex environment and improve the accuracy of three-dimensional reconstruction in a complex environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1A schematic diagram of a process flow provided for the first embodiment of the three-dimensional reconstruction method of the present application;

[0038] Figure 2 This is a schematic diagram of the module structure of the three-dimensional reconstruction device according to an embodiment of the present application;

[0039] Figure 3 Schematic diagram of the device structure of the hardware operating environment involved in the three-dimensional reconstruction method in the embodiment of the present application.

[0040] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0042] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a three-dimensional reconstruction device, etc. The following takes a three-dimensional reconstruction device as an example to illustrate this embodiment and the following embodiments.

[0043] Based on this, the present application embodiment provides a three-dimensional reconstruction method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the 3D reconstruction method of the present application.

[0044] In this embodiment, the three-dimensional reconstruction method includes steps S10 to S20:

[0045] Step S10, adjusting the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source according to the preset reinforcement learning model;

[0046] It should be noted that reinforcement learning (RL), also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent learns strategies to maximize rewards or achieve specific goals during its interaction with the environment. Among them, a common model of reinforcement learning is the standard Markov decision process (MDP).

[0047] Corresponding to this embodiment, the specific goal to be achieved or the problem to be solved by the preset reinforcement learning model is, in a complex three-dimensional reconstruction environment, how to match the light intensity and posture of the structured light source with the posture of the camera to achieve the best three-dimensional reconstruction accuracy or further improve the three-dimensional reconstruction accuracy.

[0048] In addition, it should be noted that structured light is a system structure composed of a projector and a camera. After the projector projects specific light information onto the surface of an object and the background, the camera collects it. According to the changes in the light signal caused by the object, the position and depth of the object are calculated, and then the entire three-dimensional space is restored.

[0049] In a feasible implementation manner, before step S10, the following steps are included:

[0050] Acquire training data; wherein the training data includes an action space, a state space, and an activation function value determined based on the light intensity and position of the structured light source and the position of the camera;

[0051] The preset reinforcement learning model is obtained by training with the training data.

[0052] It should be noted that three-dimensional reconstruction relies on a structured light source whose light intensity and position can be electrically controlled to adjust, and a camera whose position can be electrically controlled to adjust.

[0053] In this embodiment, after obtaining the action space, state space and activation function value determined based on the light intensity and posture of the structured light source and the posture of the camera, they are used as training data to train a preset reinforcement learning model.

[0054] In a feasible implementation manner, the step of obtaining training data includes:

[0055] Initialize the light intensity and position of the structured light source and the position of the camera;

[0056] Based on the initial light intensity and initial position of the structured light source and the initial position of the camera, determine the initial action space, initial state space and initial excitation function value;

[0057] Based on proximal strategy optimization, determine the new action space, state space and incentive function value;

[0058] The action space, state space and activation function value are combined into data tuples as training data.

[0059] It should be noted that the action space includes the light intensity, light position, and camera position information at each moment, A = [a1, a2, a3, ..., a t]; the state space includes the model information scanned each time, including vertex and face information, which can be expressed as S = [s1, s2, s3, ..., s t ]; the reward function is determined based on geometric accuracy and completeness, that is, R(s,a)=R geometry (s,a)+R completeness (s,a), where R geometry (s,a) represents the geometric accuracy, R completeness (s,a) represents completeness, where for geometric accuracy, for example, the emission and reception should be straight lines, and positive excitation is given if they meet the requirements, otherwise negative excitation is given. For completeness, for example, the target object to be reconstructed is known to have a preset number of feature points. If all of them can be reconstructed, the completeness is 100%.

[0060] In a feasible implementation, the step of determining the initial action space, the initial state space, and the initial excitation function value based on the initial light intensity and initial posture of the structured light source and the initial posture of the camera includes:

[0061] Determine the initial action space based on the initial light intensity and initial position of the structured light source and the initial position of the camera;

[0062] Performing three-dimensional reconstruction based on the initial action space to determine the initial state space;

[0063] The initial action space and the initial state space are input into a preset activation function to obtain an initial activation function value.

[0064] In this embodiment, the light intensity and position of the structured light source and the position of the camera are initialized to construct the initial action space a1, and then three-dimensional reconstruction is performed on the basis of the initial action space a1 to obtain the initial state space s1 including vertices and faces. The initial action space a1 and the initial state space s1 are substituted into the reward function to obtain the initial incentive function value R(s1,a1), forming a data tuple<a1,s1,R(s1,a1)> .

[0065] In a feasible implementation manner, the step of determining a new action space, state space, and activation function value based on proximal strategy optimization includes:

[0066] Based on the proximal strategy optimization, the new light intensity and position of the structured light source and the new position of the camera are determined;

[0067] Determine a new action space based on the new light intensity and position of the structured light source and the new position of the camera;

[0068] Perform three-dimensional reconstruction based on the new action space to determine a new state space;

[0069] The new action space and the new state space are input into a preset activation function to obtain a new activation function value.

[0070] In this embodiment, according to the proximal strategy optimization, the new light intensity and posture of the structured light source and the new posture of the camera are determined, and a new action space a2 is constructed. Then, three-dimensional reconstruction is performed on the basis of the new action space a2 to obtain a new state space s2 including vertices and faces. The new action space a2 and the new state space s2 are substituted into the reward function to obtain a new incentive function value R(s2,a2), forming a new data tuple.<a2,s2,R(s2,a2)> Then, by analogy, a data set, namely training data, is constructed.

[0071] Step S20: Perform three-dimensional reconstruction based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0072] In this embodiment, an optimal matching relationship between the light source intensity, posture and camera posture is generated through a self-matching, self-adjusting and adaptive three-dimensional reconstruction method of a structured light source and a camera based on proximal strategy optimization. Thus, the real-time light intensity and real-time posture of the structured light source are adjusted, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source is determined according to the matching relationship. Finally, three-dimensional reconstruction is performed based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0073] In a feasible implementation manner, the step S20 includes:

[0074] The actor network and the critic network are trained by the training data to obtain a preset reinforcement learning model; wherein the actor network uses a lightweight residual network, and the critic network uses a long short-term memory network.

[0075] In this embodiment, an actor-critic module is designed. The actor module uses a lightweight residual network, and the critic module uses a long short-term memory network. In addition, the actor network and the critic network are trained using the above-mentioned data set, i.e., the training data, to achieve the self-matching function of the light source and the camera, thereby improving the accuracy of three-dimensional reconstruction. Among them, the actor-critic method simultaneously trains a strategy (actor) and a value function (critic). The actor selects actions, the critic evaluates the quality of the actions, and provides feedback to improve the strategy.

[0076] In an application scenario of a 3D reconstruction method of the present application, for example, when a bathroom cleaning robot is cleaning a toilet, it is necessary to perform 3D reconstruction of the toilet. However, the cluttered items, small space, and dim light in the bathroom pose many technical challenges to the 3D reconstruction of the toilet. In this regard, in the present application, a 3D reconstruction method of a structured light source and a camera that is self-matching, self-adjusting, and self-adaptive is proposed. A reinforcement learning model is used to adjust the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source, so as to achieve control of the structured light source and the camera in a complex environment and improve the accuracy of 3D reconstruction in a complex environment.

[0077] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the three-dimensional reconstruction method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0078] This application also provides a three-dimensional reconstruction device, please refer to Figure 2 , the three-dimensional reconstruction device comprises:

[0079] An adjustment module 10, used to adjust the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source according to a preset reinforcement learning model;

[0080] The reconstruction module 20 is used to perform three-dimensional reconstruction based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0081] In one embodiment, the three-dimensional reconstruction device further includes a training module, which is used to:

[0082] Before the step of adjusting the real-time light intensity and real-time posture of the camera under the structured light source according to the matching relationship generated by the preset reinforcement learning model,

[0083] Acquire training data; wherein the training data includes an action space, a state space, and an activation function value determined based on the light intensity and position of the structured light source and the position of the camera;

[0084] The preset reinforcement learning model is obtained by training with the training data.

[0085] In one embodiment, the training module is further used to:

[0086] Initialize the light intensity and position of the structured light source and the position of the camera;

[0087] Based on the initial light intensity and initial position of the structured light source and the initial position of the camera, determine the initial action space, initial state space and initial excitation function value;

[0088] Based on proximal strategy optimization, determine the new action space, state space and incentive function value;

[0089] The action space, state space and activation function value are combined into data tuples as training data.

[0090] In one embodiment, the training module is further used to:

[0091] Determine the initial action space based on the initial light intensity and initial position of the structured light source and the initial position of the camera;

[0092] Performing three-dimensional reconstruction based on the initial action space to determine the initial state space;

[0093] The initial action space and the initial state space are input into a preset activation function to obtain an initial activation function value.

[0094] In one embodiment, the training module is further used to:

[0095] Based on the proximal strategy optimization, the new light intensity and position of the structured light source and the new position of the camera are determined;

[0096] Determine a new action space based on the new light intensity and position of the structured light source and the new position of the camera;

[0097] Perform three-dimensional reconstruction based on the new action space to determine a new state space;

[0098] The new action space and the new state space are input into a preset activation function to obtain a new activation function value.

[0099] In one embodiment, the training module is further used to:

[0100] The actor network and the critic network are trained by the training data to obtain a preset reinforcement learning model; wherein the actor network uses a lightweight residual network, and the critic network uses a long short-term memory network.

[0101] The 3D reconstruction device provided by the present application adopts the 3D reconstruction method in the above embodiment, which can solve the technical problem of low 3D reconstruction accuracy in complex environments. Compared with the prior art, the beneficial effects of the 3D reconstruction device provided by the present application are the same as those of the 3D reconstruction method provided by the above embodiment, and other technical features in the 3D reconstruction device are the same as those disclosed in the above embodiment method, which will not be described in detail here.

[0102] The present application provides a three-dimensional reconstruction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional reconstruction method in the above-mentioned embodiment one.

[0103] Reference below Figure 3 , which shows a schematic diagram of the structure of a 3D reconstruction device suitable for implementing the embodiment of the present application. The 3D reconstruction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The three-dimensional reconstruction device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0104] like Figure 3 As shown, the three-dimensional reconstruction device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the three-dimensional reconstruction device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the 3D reconstruction device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a 3D reconstruction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0105] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0106] The 3D reconstruction device provided by the present application adopts the 3D reconstruction method in the above embodiment, which can solve the technical problem of low 3D reconstruction accuracy in complex environments. Compared with the prior art, the beneficial effects of the 3D reconstruction device provided by the present application are the same as the beneficial effects of the 3D reconstruction method provided by the above embodiment, and the other technical features in the 3D reconstruction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0108] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0109] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the three-dimensional reconstruction method in the above-mentioned embodiment.

[0110] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0111] The computer-readable storage medium may be included in the three-dimensional reconstruction device; or may exist independently without being assembled into the three-dimensional reconstruction device.

[0112] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a three-dimensional reconstruction device, the three-dimensional reconstruction device: adjusts the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source according to a preset reinforcement learning model; and performs three-dimensional reconstruction based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

[0113] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0116] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned 3D reconstruction method, and can solve the technical problem of low 3D reconstruction accuracy in complex environments. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the 3D reconstruction method provided in the above-mentioned embodiment, and will not be elaborated here.

[0117] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned three-dimensional reconstruction method when executed by a processor.

[0118] The computer program product provided by this application can solve the technical problem of low accuracy of 3D reconstruction in complex environments. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the 3D reconstruction method provided by the above embodiment, which will not be described in detail here.

[0119] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A three-dimensional reconstruction method, characterized in that: The three-dimensional reconstruction method comprises: According to the preset reinforcement learning model, adjust the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source; Three-dimensional reconstruction is performed based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

2. The three-dimensional reconstruction method according to claim 1, characterized in that: Before the step of adjusting the real-time light intensity and real-time posture of the camera under the real-time posture of the structured light source according to the matching relationship generated by the preset reinforcement learning model, the method includes: Acquire training data; wherein the training data includes an action space, a state space, and an activation function value determined based on the light intensity and position of the structured light source and the position of the camera; The preset reinforcement learning model is obtained by training with the training data.

3. The three-dimensional reconstruction method according to claim 2, characterized in that: The step of obtaining training data includes: Initialize the light intensity and position of the structured light source and the position of the camera; Based on the initial light intensity and initial position of the structured light source and the initial position of the camera, determine the initial action space, initial state space and initial excitation function value; Based on proximal strategy optimization, determine the new action space, state space and incentive function value; The action space, state space and activation function value are combined into data tuples as training data.

4. The three-dimensional reconstruction method according to claim 3, characterized in that: The step of determining the initial action space, the initial state space, and the initial excitation function value based on the initial light intensity and initial position of the structured light source and the initial position of the camera comprises: Determine the initial action space based on the initial light intensity and initial position of the structured light source and the initial position of the camera; Performing three-dimensional reconstruction based on the initial action space to determine the initial state space; The initial action space and the initial state space are input into a preset activation function to obtain an initial activation function value.

5. The three-dimensional reconstruction method according to claim 3, characterized in that: The step of determining a new action space, state space, and activation function value based on proximal strategy optimization includes: Based on the proximal strategy optimization, the new light intensity and position of the structured light source and the new position of the camera are determined; Determine a new action space based on the new light intensity and position of the structured light source and the new position of the camera; Perform three-dimensional reconstruction based on the new action space to determine a new state space; The new action space and the new state space are input into a preset activation function to obtain a new activation function value.

6. The three-dimensional reconstruction method according to claim 2, characterized in that: The step of obtaining a preset reinforcement learning model through training the training data includes: The actor network and the critic network are trained by the training data to obtain a preset reinforcement learning model; wherein the actor network uses a lightweight residual network, and the critic network uses a long short-term memory network.

7. A three-dimensional reconstruction device, characterized in that: The three-dimensional reconstruction device comprises: An adjustment module, used for adjusting the real-time light intensity and real-time posture of the structured light source, and the real-time posture of the camera under the real-time light intensity and real-time posture of the structured light source according to a preset reinforcement learning model; The reconstruction module is used to perform three-dimensional reconstruction based on the real-time light intensity and real-time posture of the structured light source and the real-time posture of the camera.

8. A three-dimensional reconstruction device, characterized in that: The three-dimensional reconstruction device includes: a structured light source whose light intensity and posture can be electrically controlled to be adjusted, a camera whose posture can be electrically controlled to be adjusted, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the three-dimensional reconstruction method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the three-dimensional reconstruction method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the three-dimensional reconstruction method according to any one of claims 1 to 6 are implemented.

Images