A three-dimensional reconstruction method, device and system based on graphics processor

By dividing the distance of object marking points into intervals during the three-dimensional reconstruction process, and using multi-threaded processing by the graphics processor, the problem of inefficiency in three-dimensional reconstruction projects is solved, and a faster three-dimensional reconstruction process is achieved.

CN113724370BActive Publication Date: 2024-11-08SHENZHEN REALIS MULTIMEDIA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110889415.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-03
Publication Date
2024-11-08
Estimated Expiration
2041-08-03

AI Technical Summary

Technical Problem

In three-dimensional reconstruction projects, the prior art has problems such as long time and low efficiency, especially when dealing with marking points identification on rigid bodies.

Method used

Using a graphics processor-based method, the maximum distance between the actual marking points on an object is divided into multiple distance intervals, and the threads of the graphics processor are divided into thread groups, each distance interval corresponds to a thread group, through the operation The auxiliary module collects data and distributes it to the corresponding thread group for processing, thereby realizing three-dimensional reconstruction.

Benefits of technology

It improves data processing efficiency and speed, effectively utilizes the multi-threaded processing capabilities of the graphics processor, and significantly accelerates the three-dimensional reconstruction process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113724370B_ABST
    Figure CN113724370B_ABST
Patent Text Reader

Abstract

The present application discloses a three-dimensional reconstruction method, device and system based on a graphics processor, wherein the method includes: determining objects in an environment; determining each actual marking point of the object; for each actual marking point, determining the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point; dividing the specified distance according to a specified step length to obtain a specified number of distance intervals; dividing each thread of the graphics processor into a number of thread groups; for each distance interval, allocating the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval; performing three-dimensional reconstruction according to the processing results obtained by each thread group. The method in this specification effectively utilizes the data processing capability provided by the graphics processor, which is conducive to improving data processing efficiency. In addition, the multi-threaded processing architecture of the graphics processor is also conducive to speeding up data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of simulation display technology, and in particular to a three-dimensional reconstruction method, device and system based on a graphics processor. Background Art

[0002] The English term for 3D reconstruction is 3D Reconstruction. 3D reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of 3D objects. It is the basis for processing, operating and analyzing its properties in a computer environment. It is also a key technology for establishing virtual reality in the computer to express the objective world. The effect of 3D reconstruction will directly affect the user experience. In related technologies, suppose a rigid body has N R There are N marker points and N 3D points identified. S In the 3D reconstruction project, the rigid body needs to be executed Secondly, there are problems of long time consumption and low efficiency. Summary of the invention

[0003] The embodiments of the present application provide a graphics processor-based 3D reconstruction method, device, and system to at least partially solve the above technical problems.

[0004] The present application embodiment adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a three-dimensional reconstruction method based on a graphics processor, the method comprising:

[0006] Identify objects in the environment;

[0007] Determining actual marking points of the object;

[0008] For each actual marking point, determining the maximum distance between the actual marking point and other actual marking points as the designated distance of the actual marking point;

[0009] Dividing the specified distance according to the specified step length to obtain a specified number of distance intervals;

[0010] Dividing each thread of the graphics processor into a plurality of thread groups, such that each distance interval corresponds to a thread group;

[0011] For each distance interval, the data corresponding to the distance interval collected by the dynamic auxiliary module is allocated to the thread group corresponding to the distance interval, so that the thread group processes the allocated data;

[0012] Three-dimensional reconstruction is performed based on the processing results obtained by each thread group.

[0013] In an optional embodiment of the present specification, the dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object.

[0014] In an optional embodiment of the present specification, the graphics processor includes at least one of the following: a GPU, a multi-core CPU; and / or,

[0015] The object is a rigid body.

[0016] In an optional embodiment of the present specification, allocating the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval includes:

[0017] For each frame of image captured by the dynamic auxiliary module, identify each reference mark point of the object in the frame of image;

[0018] For each reference marking point, determining each available distance between the reference marking point and other reference marking points;

[0019] The available distances corresponding to the distance interval among the available distances are allocated to the thread group corresponding to the distance interval.

[0020] In an optional embodiment of the present specification, allocating the available distances corresponding to the distance interval in the available distances to the thread group corresponding to the distance interval includes:

[0021] For each actual marking point, determining an available distance among the available distances that is smaller than the specified distance of the actual marking point as the available distance corresponding to the actual marking point;

[0022] For each distance interval of the actual marking point, determine an available distance matching the distance interval from the available distances corresponding to the actual marking point as the available distance corresponding to the distance interval;

[0023] The available distance corresponding to the distance interval is allocated to the thread group corresponding to the distance interval.

[0024] In an optional embodiment of the present specification, after performing three-dimensional reconstruction according to the processing results obtained by each thread group, the method further includes:

[0025] The 3D image obtained by three-dimensional reconstruction is displayed to the user.

[0026] In a second aspect, an embodiment of the present application further provides a three-dimensional reconstruction system based on a graphics processor, the system comprising a reconstruction processor and a dynamic auxiliary module;

[0027] The reconstruction processor is configured to: determine an object in the environment; determine each actual marking point of the object; for each actual marking point, determine the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point; divide the specified distance according to a specified step length to obtain a specified number of distance intervals; divide each thread of the graphics processor into a plurality of groups, so that each distance interval corresponds to a thread group; for each distance interval, assign the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval, so that the thread group processes the assigned data; and perform three-dimensional reconstruction according to the processing results obtained by each thread group;

[0028] The dynamic auxiliary module is configured as follows: the dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object.

[0029] In a third aspect, an embodiment of the present application further provides a three-dimensional reconstruction device based on a graphics processor, for implementing any one of the methods in the aforementioned first aspect.

[0030] In a fourth aspect, an embodiment of the present application further provides an electronic device, including:

[0031] Processor; and

[0032] A memory arranged to store computer executable instructions which, when executed, cause the processor to perform any of the methods of the first aspect.

[0033] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple applications, the electronic device executes any one of the methods in the first aspect.

[0034] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the three-dimensional reconstruction process in this specification divides the data processing capability provided by the threads of the graphics processor according to the distance between the actual marking points on the object (i.e., the side length between the actual marking points) during the reconstruction process, and obtains a number of thread groups, so that different thread groups process data corresponding to different distance intervals. Effectively utilizing the data processing capability provided by the graphics processor is conducive to improving data processing efficiency. In addition, the multi-threaded processing architecture of the graphics processor is also conducive to speeding up data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0036] Figure 1 A schematic diagram of a three-dimensional reconstruction process based on a graphics processor provided in an embodiment of this specification;

[0037] Figure 2 A schematic diagram of a holographic display scene provided in an embodiment of this specification;

[0038] Figure 3 A schematic diagram of the structure of a three-dimensional reconstruction device based on a graphics processor provided in an embodiment of this specification;

[0039] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, and this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0041] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0042] The serial numbers of the components in this document, such as "second", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0043] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0044] The 3D reconstruction method based on a graphics processor in this specification may include the following steps:

[0045] S100: Determine objects in the environment.

[0046] In an optional embodiment of the present specification, the process in the present specification can be executed by a 3D reconstruction system of a graphics processor. Specifically, the 3D reconstruction system of a graphics processor can include a reconstruction processor and an auxiliary module, and the process in the present specification can be executed by the reconstruction processor.

[0047] The dynamic auxiliary module in this specification is used to capture images of a specified environment. Different dynamic auxiliary modules have different postures when capturing images of a specified environment. The images captured by the dynamic auxiliary module include several two-dimensional points.

[0048] The three-dimensional reconstruction process in this specification does not impose any specific restrictions on the objects in the environment involved. Optionally, there may be two or more objects in the environment.

[0049] The dynamic auxiliary module in this specification can be a physical camera with a physical structure, or it can be a virtual camera corresponding to the user's viewpoint.

[0050] The designated environment in this specification can be determined according to actual needs. For example, the designated environment can be a room or an area in a room.

[0051] When the 3D reconstruction process based on the graphics processor in this specification is applied to the field of holographic display technology, the specified environment may be the environment where the holographic sandbox (sandbox display device) is located, for example: Figure 2 In this example, the execution subject of the process in this specification may be a control module of the holographic display system to which the holographic sandbox belongs.

[0052] S102: Determine actual marking points of the object.

[0053] The actual marking points can be obtained by manual calibration, and then the calibrated actual marking points are input into the reconstruction processor.

[0054] Since the designated environment is a certain space, the actual marking point in the designated environment can be represented by three-dimensional coordinates. Optionally, in the aforementioned holographic display scene, the origin of the three-dimensional coordinate system used to determine the three-dimensional coordinates is at the center of the holographic sandbox.

[0055] S104: For each actual marking point, determine the maximum distance between the actual marking point and other actual marking points as the designated distance (L Max ).

[0056] Suppose there are N R actual marking points, then an object corresponds to The edges of these rigid bodies must match the N s The edges of the reference points have For each actual marker point, the length of the edge is compared with the reference marker point. If the lengths are the same, the current two reference marker points correspond to the current two actual marker points, and a match can be made. After that, 3D reconstruction can be performed based on the matched reference marker points.

[0057] Since multiple actual marking points are determined for each object, it is necessary to determine the specified distance for each actual marking point.

[0058] S106: Divide the specified distance according to the specified step length to obtain a specified number of distance intervals.

[0059] For example: L Max The value of is 1.0 meters, and the specified step size is 0.1 meters. Then 10 distance intervals are obtained, and the specified number is 10.

[0060] S108: Divide each thread of the graphics processor into a plurality of thread groups, so that each distance interval corresponds to a thread group.

[0061] For example, each thread group corresponds to a bucket, which is divided into 10 buckets, bucket 1 contains edges from 0 to 0.1 meters long, bucket 2 contains edges from 0.1 to 0.2 meters long, and so on. Optionally, each thread can be evenly divided into a number of thread groups. The number of thread groups obtained is greater than or equal to the specified number.

[0062] The side length (i.e., distance) is less than L Max The edges can be assigned to different buckets according to their specific lengths, and this operation can also be accelerated by multi-threading.

[0063] S110: For each distance interval, the data corresponding to the distance interval collected by the dynamic auxiliary module is allocated to the thread group corresponding to the distance interval, so that the thread group processes the allocated data.

[0064] When executing the processing, if the distance between the reference markers is 0.15 meters long, it is only necessary to match the edges of the reference markers in bucket 2, which greatly reduces the amount of calculation.

[0065] Optionally, the graphics processor is a GPU. The graphics processor processes all points in the data at the same time to speed up the calculation. Compared with a central processing unit (CPU), the advantage of a GPU is that a large number of threads can be generated, and when a large number of parallel calculations are performed, the total calculation time can be reduced.

[0066] S112: Execute 3D reconstruction according to the processing results obtained by each thread group.

[0067] Afterwards, the 3D image obtained by three-dimensional reconstruction can be displayed to the user.

[0068] It can be seen that through the three-dimensional reconstruction process based on the graphics processor in this specification, a corresponding relationship can be established between the thread group and each distance interval, so as to achieve a reasonable allocation of the processing resources of the graphics processing among the distance intervals.

[0069] Since the rigid body will move, each frame of the collected image must be processed using the above process. S The threads calculate the side lengths of all 3D points. If the side length is less than L Max , keep the reference points corresponding to these edges. Max The number of edges (denoted by E′ S to indicate) is much smaller than E S . Use multithreading to reduce the length of the edge to less than L Max The edge is collected into another variable, and the memory occupied is only E′ s The length of E S Length memory reduces computation time by reducing the required memory bandwidth when accessing this data.

[0070] In an optional embodiment of the present specification, the dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object. In addition, the graphics processor can be a multi-core CPU, and the object can be a rigid body. Each reference mark point corresponds to an actual mark point on the object.

[0071] When the acquired image packets are multiple frames extending over time, for each frame of image acquired by the dynamic auxiliary module, each reference mark point of the object in the frame of image can be identified; for each reference mark point, each available distance between the reference mark point and other reference mark points can be determined; for each actual mark point, the available distance that is smaller than the specified distance of the actual mark point among the available distances is determined as the available distance corresponding to the actual mark point; for each distance interval of the actual mark point, the available distance matching the distance interval is determined among the available distances corresponding to the actual mark point as the available distance corresponding to the distance interval; the available distance corresponding to the distance interval is allocated to the thread group corresponding to the distance interval.

[0072] Based on the same idea, the present specification also provides a corresponding embodiment Figure 1 A 3D reconstruction device based on a graphics processor for the partial process shown. Figure 3As shown, the three-dimensional reconstruction device based on a graphics processor may include one or more of the following modules:

[0073] The object determination module 300 is configured to: determine objects in the environment.

[0074] The actual marking point determination module 302 is configured to determine each actual marking point of the object.

[0075] The designated distance determination module 304 is configured to: for each actual marking point, determine the maximum distance between the actual marking point and other actual marking points as the designated distance of the actual marking point.

[0076] The distance interval determination module 306 is configured to divide the specified distance according to a specified step length to obtain a specified number of distance intervals.

[0077] The thread group determination module 308 is configured to divide the threads of the graphics processor into a plurality of thread groups, so that each distance interval corresponds to a thread group.

[0078] The allocation module 310 is configured to: for each distance interval, allocate the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval, so that the thread group processes the allocated data.

[0079] The 3D reconstruction module 312 is configured to perform 3D reconstruction according to the processing results obtained by each thread group.

[0080] In an optional embodiment of the present specification, the dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object.

[0081] In an optional embodiment of the present specification, the graphics processor includes at least one of the following: a GPU, a multi-core CPU; and / or the object is a rigid body.

[0082] In an optional embodiment of the present specification, the allocation module is specifically configured to: for each frame of image collected by the dynamic auxiliary module, identify each reference mark point of the object in the frame of image;

[0083] For each reference marking point, determining each available distance between the reference marking point and other reference marking points;

[0084] The available distances corresponding to the distance interval among the available distances are allocated to the thread group corresponding to the distance interval.

[0085] In an optional embodiment of the present specification, the allocation module is specifically configured to: for each actual marking point, determine the available distance among the available distances that is smaller than the specified distance of the actual marking point as the available distance corresponding to the actual marking point;

[0086] For each distance interval of the actual marking point, determine an available distance matching the distance interval from the available distances corresponding to the actual marking point as the available distance corresponding to the distance interval;

[0087] The available distance corresponding to the distance interval is allocated to the thread group corresponding to the distance interval.

[0088] In an optional embodiment of the present specification, the device further comprises a display module. The display module is configured to display the 3D image obtained by the 3D reconstruction to the user.

[0089] It can be understood that the above-mentioned graphics processor-based 3D reconstruction device can implement the various steps of the graphics processor-based 3D reconstruction process performed by the graphics processor-based 3D reconstruction device provided in the aforementioned embodiments, and the relevant explanations about the graphics processor-based 3D reconstruction method are applicable to the graphics processor-based 3D reconstruction device, and will not be repeated here.

[0090] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0091] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0092] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0093] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a 3D reconstruction device based on a graphics processor at the logical level. The processor executes the program stored in the memory and is specifically used to perform any of the aforementioned 3D reconstruction processes based on a graphics processor.

[0094] The above application Figure 1 The method disclosed in the illustrated embodiment for executing a three-dimensional reconstruction device based on a graphics processor can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0095] The electronic device may also perform Figure 1 At least some of the method steps performed by the three-dimensional reconstruction device based on the graphics processor are implemented, and a three-dimensional reconstruction device based on the graphics processor is implemented. The embodiments of the present application are not repeated here.

[0096] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The illustrated embodiment shows a method executed by a graphics processor-based 3D reconstruction device, and is specifically used to execute any of the aforementioned graphics processor-based 3D reconstruction methods.

[0097] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0098] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0103] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0104] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0106] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A three-dimensional reconstruction method based on a graphics processor, wherein: The method comprises: Identify objects in the environment; Determining actual marking points of the object; For each actual marking point, determining the maximum distance between the actual marking point and other actual marking points as the designated distance of the actual marking point; Dividing the specified distance according to the specified step length to obtain a specified number of distance intervals; Dividing each thread of the graphics processor into a plurality of thread groups, such that each distance interval corresponds to a thread group; For each distance interval, the data corresponding to the distance interval collected by the dynamic auxiliary module is allocated to the thread group corresponding to the distance interval, so that the thread group processes the allocated data; Perform three-dimensional reconstruction according to the processing results obtained by each thread group; The method includes: for each frame of image collected by the dynamic auxiliary module, identifying each reference mark point of the object in the frame of image; for each reference mark point, determining each available distance between the reference mark point and other reference mark points; and allocating the available distance corresponding to the distance interval among the available distances to the thread group corresponding to the distance interval; Allocating the available distances corresponding to the distance interval among the available distances to the thread group corresponding to the distance interval, including: for each actual marking point, determining the available distances among the available distances that are smaller than the specified distance of the actual marking point as the available distance corresponding to the actual marking point; for each distance interval of the actual marking point, determining the available distances matching the distance interval among the available distances corresponding to the actual marking point as the available distance corresponding to the distance interval; and allocating the available distances corresponding to the distance interval to the thread group corresponding to the distance interval.

2. The method of claim 1, wherein: The dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object.

3. The method of claim 1, wherein: The graphics processor includes at least one of the following: a GPU, a multi-core CPU; and / or, The object is a rigid body.

4. The method of claim 1, wherein: After performing three-dimensional reconstruction according to the processing results obtained by each thread group, the method further includes: The 3D image obtained by three-dimensional reconstruction is displayed to the user.

5. A three-dimensional reconstruction system based on a graphics processor, the system comprising a reconstruction processor and a dynamic auxiliary module; The reconstruction processor is configured to: determine an object in the environment; determine each actual marking point of the object; for each actual marking point, determine the maximum distance between the actual marking point and other actual marking points as the specified distance of the actual marking point; divide the specified distance according to a specified step length to obtain a specified number of distance intervals; Divide the threads of the graphics processor into several groups, so that each distance interval corresponds to a thread group; for each distance interval, allocate the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval, so that the thread group processes the allocated data; perform three-dimensional reconstruction according to the processing results obtained by each thread group; wherein, Allocating the data corresponding to the distance interval collected by the dynamic auxiliary module to the thread group corresponding to the distance interval, including: for each frame of image collected by the dynamic auxiliary module, identifying each reference mark point of the object in the frame image; for each reference mark point, determining each available distance between the reference mark point and other reference mark points; allocating the available distance corresponding to the distance interval in the available distance to the thread group corresponding to the distance interval; allocating the available distance corresponding to the distance interval in the available distance to the thread group corresponding to the distance interval, including: for each actual mark point, determining the available distance smaller than the specified distance of the actual mark point in each available distance as the available distance corresponding to the actual mark point; for each distance interval of the actual mark point, determining the available distance matching the distance interval in the available distance corresponding to the actual mark point as the available distance corresponding to the distance interval; allocating the available distance corresponding to the distance interval to the thread group corresponding to the distance interval; The dynamic auxiliary module is configured as follows: the dynamic auxiliary module includes a dynamic auxiliary camera, and the dynamic auxiliary camera is used to capture images of the object.

6. A three-dimensional reconstruction device based on a graphics processor, wherein the device is used to implement any of the methods of claims 1 to 4.

7. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to execute any one of the methods of claims 1 to 4.

8. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 4.