Object structure reconstruction method and related equipment

Abstract

The present application provides a method for reconstructing the structure of an object and related equipment. The method includes: acquiring multiple transmission holographic images of the object at different acquisition angles; inputting the transmission holographic images and their corresponding acquisition angles into a pre-trained multi-layer perceptron (MLP); constructing and fusing initial point cloud data through the volume density output of the multi-layer perceptron (MLP); comparing and analyzing the initial fused point cloud data with each of the holographic images to generate multiple visual point cloud data; in response to fusing the multiple visual point cloud data and the fusion result reaching a predetermined density threshold, obtaining a voxel body corresponding to the object; performing an extraction operation on the voxel body to obtain the internal structure corresponding to the object. The present application provides a method for reconstructing the structure of an object and related equipment that can conveniently and effectively obtain the internal structure of an object.

Description

Technical Field

[0001] The present invention relates to the field of digital holography technology, and in particular to a method for reconstructing an object structure and related equipment. Background Art

[0002] Typically, transmission holography is used, where electromagnetic waves propagate through space to form a transmission holographic image that encodes information about the structure of the object being passed through. However, if the object to be restored is a closed structure, a transmission holographic image of the object's interior cannot be obtained. Instead, the internal structure can only be reconstructed using transmission holographic images of the object's exterior at different angles.

[0003] It's possible to obtain external transmission holographic images of an object at all angles by varying the signal source position, and then use these transmission holographic images to recover the object's internal structure. However, this method requires very high data measurement requirements, and if one side of the sealed object is obscured by an obstacle, the external transmission holographic image of the object at all angles cannot be obtained. Therefore, how to obtain the internal structure of an object with incomplete acquisition angles has become a hot and difficult research issue. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method for reconstructing object structure and related equipment.

[0005] Based on the above objectives, the present application provides a method for reconstructing an object structure, comprising:

[0006] Acquire multiple transmission holographic images of the object, each of the transmission holographic images being acquired at a different acquisition angle;

[0007] Inputting the transmission holographic image and its corresponding acquisition angle into a pre-trained multi-layer perceptron (MLP), and outputting a volume density via the MLP;

[0008] constructing initial point cloud data based on the volume density;

[0009] Based on all the initial point cloud data, initial fused point cloud data is obtained by point cloud stitching;

[0010] Comparing and analyzing the initial fused point cloud data with each of the holographic images respectively to generate a plurality of visual point cloud data;

[0011] In response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold, a voxel volume corresponding to the object is obtained;

[0012] An extraction operation is performed on the voxel body to obtain the internal structure corresponding to the object.

[0013] Furthermore, the acquiring of multiple transmission holographic images of the object, each of which is acquired at a different acquisition angle, includes: acquiring multiple electromagnetic wave signals of the object by changing the placement angles of an electromagnetic wave signal transmitter and an electromagnetic wave signal receiver relative to the object, and obtaining the multiple transmission holographic images by calculation based on the multiple electromagnetic wave signals.

[0014] Furthermore, the transmission holographic image includes 3D position information, and the transmission holographic image and its corresponding acquisition angle are input into a pre-trained multi-layer perceptron MLP, including:

[0015] Encoding the 3D position information to obtain encoded information;

[0016] The encoded information and the corresponding acquisition angle are input into a pre-trained multi-layer perceptron (MLP).

[0017] Furthermore, the pre-training of the multi-layer perceptron MLP includes:

[0018] Construct a training set;

[0019] Training the multilayer perceptron (MLP) using a neural radiation field (NeRF) algorithm based on the training set to determine a loss function;

[0020] The pre-training of the multi-layer perceptron MLP is completed by minimizing the loss function.

[0021] Furthermore, constructing the initial point cloud data based on the volume density includes: generating the initial point cloud data by calculation based on the volume density and its corresponding acquisition angle.

[0022] Furthermore, before obtaining the voxel volume corresponding to the object, the method includes: discretizing the multiple fused visualization point clouds to obtain the voxel volume.

[0023] Furthermore, the extracting operation on the voxel body to obtain the internal structure corresponding to the object includes: determining the center of gravity of each voxel contained in the voxel body, and connecting all the centers of gravity to form the internal structure.

[0024] Based on the same inventive concept, the present application also provides an object structure reconstruction device, comprising:

[0025] an acquisition module, configured to acquire a plurality of transmission holographic images of the object, wherein the acquisition angle of each transmission holographic image is different;

[0026] an output module configured to input the transmission holographic image and its corresponding acquisition angle into a pre-trained multilayer perceptron (MLP), and output a volume density via the multilayer perceptron (MLP);

[0027] A construction module configured to construct initial point cloud data based on the volume density;

[0028] A stitching module is configured to obtain initial fused point cloud data by stitching the point clouds based on all the initial point cloud data;

[0029] a visualization point cloud generation module, configured to compare and analyze the initial fused point cloud data with each of the holographic images to generate a plurality of visualization point cloud data;

[0030] a fusion module configured to obtain a voxel volume corresponding to the object in response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold;

[0031] The structure extraction module is configured to perform an extraction operation on the voxel body to obtain the internal structure corresponding to the object.

[0032] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the methods described above is implemented.

[0033] Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the methods described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] Figure 1 A schematic flow chart of a method for reconstructing an object structure according to an embodiment of the present application;

[0036] Figure 2 Schematic diagram of a method for generating initial fused point cloud data according to an embodiment of the present application;

[0037] Figure 3a A schematic diagram of the voxel center of gravity according to an embodiment of the present application;

[0038] Figure 3b A schematic diagram of connecting the centroids of all voxels in an embodiment of the present application into a smooth curve;

[0039] Figure 4 This is a schematic structural diagram of an object structure reconstruction device according to an embodiment of the present application;

[0040] Figure 5 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0042] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which the present application belongs. "Include" or "comprising" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0043] As described in the background art, the currently commonly used method for determining the internal structure of an object is transmission holography.

[0044] In a wireless data transmission system, such as Wi-Fi or Bluetooth, a signal source emits coherent light—electromagnetic waves—with precisely known amplitude and phase. These waves propagate through space, forming a three-dimensional image that encodes all objects through which the waves pass. This 3D image is also called a transmission hologram. The method of using this transmission hologram to recover the object's structure is called transmission holography. By varying the position of the signal source to obtain external transmission holographic images of the object at all angles, it is possible to recover the object's internal structure from these transmission holograms. However, this method places high demands on the data measurement. Under certain conditions, such as when one side of the object is obscured by an obstacle, it is impossible to obtain external transmission holographic images of the object at all angles, making it impossible to recover the object's internal structure.

[0045] In light of this, the object structure reconstruction method proposed in this application is based on obtaining a limited transmission holographic image of the object's exterior using transmission holography. A multi-layer perceptron (MLP) consisting of a multi-layer fully connected network is trained using the Neural Radiance Field (NeRF) algorithm. Point cloud fusion is used to generate voxel volumes, from which the object's structure is extracted. This solves the problem of being unable to capture the object's internal structure when the acquisition angle is incomplete.

[0046] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0047] This application provides a method for reconstructing object structure, referring to Figure 1 , specifically including the following steps:

[0048] Step S101: Acquire multiple transmission holographic images of the object, where each transmission holographic image is acquired at a different acquisition angle.

[0049] Specifically, methods for acquiring multiple transmission holographic images of the object include multi-angle electromagnetic wave radiation technology and monocular camera reconstruction technology. The specific acquisition method can be selected based on actual circumstances. Multi-angle electromagnetic wave radiation technology can acquire three-dimensional scale information of the object, while monocular camera reconstruction technology is easy to implement and has low implementation costs. However, the reconstruction process requires the acquisition of a large amount of data, which is computationally time-consuming, and monocular cameras do not have strong penetrating power. Therefore, in this embodiment, multi-angle electromagnetic wave radiation technology is used to acquire transmission holographic images of the object, and multiple transmission holographic images of the object are acquired by varying different acquisition angles. The object can be a building, and some angles of the exterior of the building are obscured. The acquisition angles can be multiple angles of the exterior of the building.

[0050] Step S102: input the transmission holographic image and its corresponding acquisition angle into a pre-trained multi-layer perceptron MLP, and output the volume density through the multi-layer perceptron MLP.

[0051] Specifically, each transmission holographic image obtained in step S101 and its corresponding acquisition angle are input into a multilayer perceptron MLP. The multilayer perceptron MLP is pre-trained, and the input of the multilayer perceptron MLP is the 3D position information x=(x, y, z) included in the transmission holographic image and the corresponding acquisition angle. The output is volume density σ and color information c = (r, g, b). However, in this embodiment, the transmission holographic image is obtained using multi-angle electromagnetic wave radiation technology and does not contain color information c = (r, g, b). Therefore, in this embodiment, this color information c = (r, g, b) can be ignored, and subsequent calculations are performed only based on volume density σ.

[0052] Step S103: constructing initial point cloud data based on the volume density.

[0053] Specifically, the initial point cloud data is constructed based on the volume density and the corresponding acquisition angle. Different acquisition angles generate different volume densities, and each volume density generates an initial point cloud data.

[0054] In some embodiments, the initial point cloud data is generated by calculation based on the volume density and its corresponding acquisition angle. Specifically, the 3D position information contained in the volume density is mapped to a three-dimensional coordinate system corresponding to the acquisition angle, and the initial point cloud data is generated after the coordinate system is converted.

[0055] Step S104: obtaining initial fused point cloud data by point cloud stitching based on all the initial point cloud data.

[0056] Specifically, all the initial point cloud data are spliced ​​together. The main purpose of this process is to filter out point cloud data with a large error range that occurs during the splicing process. Figure 2 A schematic diagram of a method for generating initial fused point cloud data by stitching multiple initial point cloud data is shown in FIG. Figure 2 The square mark indicates that some of the points in the generated initial point cloud data have exceeded the object's outline. The stitching process can identify these points with large errors and delete them to prevent them from affecting the subsequent reconstruction of the object's internal structure and preventing the final internal structure features from being inconsistent with the actual object. The above stitching process completes the fusion of multiple initial point cloud data to obtain the initial fused point cloud data.

[0057] Step S105: Compare and analyze the initial fused point cloud data with each of the holographic images to generate a plurality of visual point cloud data.

[0058] In related technologies, point cloud data is fused through a nonlinear optimization process, but in this embodiment, the fusion is performed by comparing and analyzing the point cloud data with the transmission holographic image. Because electromagnetic wave signals have strong penetrability, the generated transmission holographic image can more clearly show the internal structure. Therefore, the visualized point cloud data generated after comparing and analyzing the initial fused point cloud data with the transmission holographic image is more accurate than that generated by the nonlinear optimization process. After the comparison and analysis, the visualized point cloud data is generated. It should be noted that the visualized point cloud data represents point cloud data within the error range. Compared with the initial point cloud data and the initial fused point cloud data, the object position coordinates contained in the visualized point cloud data are more accurate and suitable for restoring the internal structure of the object.

[0059] Step S106 : In response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold, a voxel volume corresponding to the object is obtained.

[0060] Specifically, multiple visualization point clouds are fused with each other to reach a predetermined density value. In this embodiment, reaching the predetermined density value means that the characteristic contour formed by the fused multiple visualization point clouds can clearly express the object. After the fusion is completed, a voxel body including multiple voxels is formed, and each voxel includes multiple irregularly distributed points.

[0061] Step S107: performing an extraction operation on the voxel volume to obtain the internal structure corresponding to the object.

[0062] Specifically, the extraction operation includes determining the center of gravity of each voxel contained in the voxel volume, and connecting all the centers of gravity to form the internal structure. Figure 3a shows the determined centers of gravity of the plurality of voxels, Figure 3b The structure of the object obtained by connecting all the centroids into a smooth curve is shown. A voxel is the smallest unit in three-dimensional space, shaped like a cube with multiple irregular points inside. The surface smoothness of an object structure formed by directly connecting voxels is poor. By determining the centroid of each voxel, the surface of the object structure formed by connecting all the centroids will be relatively smooth.

[0063] In some embodiments, the acquiring of multiple transmission holographic images of the object, each of which is acquired at a different acquisition angle, includes: acquiring multiple electromagnetic wave signals of the object by changing the placement angles of an electromagnetic wave signal transmitter and an electromagnetic wave signal receiver relative to the object, and obtaining the multiple transmission holographic images by calculation based on the multiple electromagnetic wave signals.

[0064] Specifically, both the electromagnetic wave signal transmitter and the electromagnetic wave signal receiver are existing commercial narrowband Wi-Fi routers. Multi-angle signal collection is described using a regular building as an example. The specific collection method is to first collect electromagnetic wave signals from the bottom of the building to the top, then from the top to the bottom, and finally from the bottom of one side of the building to the bottom of the other, as well as from the top of one side of the building to the top of the other. If the building is irregularly shaped, the Wi-Fi router can be placed in more collection locations to collect electromagnetic wave signals at different angles. In summary, by changing the placement angle of the Wi-Fi router relative to the building, multiple electromagnetic wave signals from the building can be collected.

[0065] This embodiment obtains the multiple transmission holographic images by normalizing all electromagnetic wave signals at different acquisition angles. The normalization method is specifically as follows:

[0066]

[0067] Among them, r represents the signal, x r 、y r represents the coordinates of the signal r in the coordinate system, i represents the complex factor, Φ represents the angle, f represents the frequency, t represents the propagation time, I(f)e iΦ(f) | x,y Represents the complex amplitude of the signal, I em (f) represents the transmitted complex signal, Represents the propagation of the transmitted complex signal to the x-axis and y-axis, the U function represents the amplitude attenuation and phase delay, and I(t) represents the change of phase over time. Expressed as a time Fourier transform, formula (1) recovers the amplitude and phase of the electromagnetic wave in the Wi-Fi channel to a constant factor U(x r ,y r ,f) -1 A transmission holographic image can be obtained by normalizing the electromagnetic wave signal according to formula (1). The electromagnetic wave signals at different acquisition angles are all normalized to obtain the multiple transmission holographic images.

[0068] In some embodiments, the transmission holographic image includes 3D position information, and the transmission holographic image and its corresponding acquisition angle are input into a pre-trained multi-layer perceptron (MLP), including:

[0069] Encoding the 3D position information to obtain encoded information;

[0070] The encoded information and the corresponding acquisition angle are input into a pre-trained multi-layer perceptron (MLP).

[0071] Specifically, if the 3D position information in the transmission holographic image is directly input into the multi-layer perceptron (MLP), the quality of the generated volume density will be reduced, and the output volume density data will be blurred in the experiment. Therefore, a sine-cosine periodic function is used to position encode the 3D position information. The encoding formula is as follows:

[0072] γ(p)=(sin(2 0 πp, cos(2 0 πp),...,sin(2 L-1 πp),cos(2 L-1 πp)) (2)

[0073] Wherein, p represents 3D position information, L represents the encoding dimension, which can be set according to actual needs. In this embodiment, L is set to 10. After obtaining the encoding information, the encoding information and its corresponding acquisition angle are input into the pre-trained multi-layer perceptron MLP.

[0074] In some embodiments, the pre-training of the multi-layer perceptron (MLP) comprises:

[0075] Construct a training set;

[0076] Training the multilayer perceptron (MLP) using a neural radiation field (NeRF) algorithm based on the training set to determine a loss function;

[0077] The pre-training of the multi-layer perceptron MLP is completed by minimizing the loss function.

[0078] Specifically, the transmission holographic images of different objects and the corresponding acquisition angles are collected as training sets to pre-train the multi-layer perceptron MLP. However, directly inputting the 3D position information in the transmission holographic image into the multi-layer perceptron MLP will reduce the quality of the generated volume density, and the output volume density data obtained in the experiment is fuzzy. Therefore, the 3D position information is encoded using formula (2), and after the encoded information and the corresponding acquisition angle are input into the multi-layer perceptron MLP, the multi-layer perceptron MLP is trained using the Neural Radiation Field (NeRF) algorithm. The loss function used in the training process is specifically:

[0079]

[0080] Where r represents the ray, which consists of the ray origin and the ray angle. The ray angle is related to the acquisition angle. C(r) represents the integral of the ray from the near end to the far end. represents the sum of the discretized integrals, C f(r) represents the integral of the probability density, and R represents the set of rays. After multiple iterative training, if the loss function value drops to a pre-set threshold, training can be stopped. In this embodiment, the pre-set threshold can be 20% of the initial value of the loss function. If, after multiple iterative training, the current loss function value is less than or equal to 20% of the initial value of the loss function, then pre-training of the multilayer perceptron (MLP) is stopped.

[0081] In some embodiments, before obtaining the voxel volume corresponding to the object, the method includes:

[0082] Discretization processing is performed on the multiple visualization point clouds that have been fused to obtain the voxel volume.

[0083] Specifically, discretizing multiple visual point clouds yields a voxel volume, which contains multiple voxels, each of which contains multiple irregular points. Discretization of the visual point cloud can be performed by selecting an existing model. For example, if the object is a regular building, the octree model from the Point Cloud Library (PCL), which is well-suited for modeling regular objects, can be selected for discretization.

[0084] In some embodiments, the multilayer perceptron (MLP) includes 8 fully connected layers.

[0085] Specifically, the multilayer perceptron MLP includes 8 fully connected layers, and there is a certain functional relationship between the associated nodes of each layer, which is the ReLU activation function. The ReLU activation function is specifically

[0086] f(x)=max(0,x) (4)

[0087] Where x is the independent variable. The functional relationship in this embodiment can also be other activation functions, which can be selected according to actual conditions and are not specifically limited here. After multiple experiments, the applicant determined that the volume density output by a multilayer perceptron MLP including 8 fully connected layers is most suitable. Therefore, the multilayer perceptron MLP in this embodiment uses 8 fully connected layers.

[0088] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0089] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0090] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an object structure reconstruction device.

[0091] refer to Figure 4 , an object structure reconstruction device, comprising:

[0092] An acquisition module 401 is configured to acquire a plurality of transmission holographic images of the object, wherein each transmission holographic image is acquired at a different acquisition angle;

[0093] An output module 402 is configured to input the transmission holographic image and its corresponding acquisition angle into a pre-trained multi-layer perceptron MLP, and output volume density via the multi-layer perceptron MLP;

[0094] A construction module 403 is configured to construct initial point cloud data based on the volume density;

[0095] A stitching module 404 is configured to obtain initial fused point cloud data by stitching the point clouds based on all the initial point cloud data;

[0096] The visualization point cloud generation module 405 is configured to compare and analyze the initial fused point cloud data with each of the holographic images to generate a plurality of visualization point cloud data;

[0097] A fusion module 406 is configured to obtain a voxel volume corresponding to the object in response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold;

[0098] The structure extraction module 407 is configured to perform an extraction operation on the voxel volume to obtain the internal structure corresponding to the object.

[0099] In some embodiments, the acquisition module 401 is further configured to collect multiple electromagnetic wave signals of the object by changing the placement angle of the electromagnetic wave signal transmitter and the electromagnetic wave signal receiver relative to the object, and obtain the multiple transmission holographic images by calculation based on the multiple electromagnetic wave signals.

[0100] In some embodiments, the output module 402 is further configured to encode the 3D position information to obtain encoded information;

[0101] The encoded information and the corresponding acquisition angle are input into a pre-trained multi-layer perceptron (MLP).

[0102] In some embodiments, the pre-training of the multi-layer perceptron (MLP) comprises:

[0103] Construct a training set;

[0104] Training the multilayer perceptron (MLP) using a neural radiation field (NeRF) algorithm based on the training set to determine a loss function;

[0105] The pre-training of the multi-layer perceptron MLP is completed by minimizing the loss function.

[0106] In some embodiments, the construction module 403 is further configured to generate the initial point cloud data through calculation based on the volume density and its corresponding acquisition angle.

[0107] In some embodiments, the fusion module 406 is further configured to discretize the fused plurality of visualization point clouds to obtain the voxel volume.

[0108] In some embodiments, the structure extraction module 407 is further configured to determine the center of gravity of each voxel contained in the voxel volume, and connect all the centers of gravity to form the internal structure.

[0109] In some embodiments, the multilayer perceptron (MLP) includes 8 fully connected layers.

[0110] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0111] The apparatus of the above embodiment is used to implement the corresponding object structure reconstruction method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0112] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the object structure reconstruction method described in any of the above embodiments is implemented.

[0113] Figure 510 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0114] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0115] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0116] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0117] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0118] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0119] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0120] The electronic device of the above embodiment is used to implement the corresponding object structure reconstruction method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0121] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the object structure reconstruction method described in any of the above embodiments.

[0122] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0123] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the object structure reconstruction method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0124] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0125] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0126] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0127] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A method for reconstructing an object structure, characterized in that: include: Acquire multiple transmission holographic images of the object, each of the transmission holographic images being acquired at a different acquisition angle; Inputting the transmission holographic image and its corresponding acquisition angle into a pre-trained multi-layer perceptron (MLP), and outputting a volume density via the MLP; constructing initial point cloud data based on the volume density; Based on all the initial point cloud data, initial fused point cloud data is obtained by point cloud stitching; Comparing and analyzing the initial fused point cloud data with each of the holographic images respectively to generate a plurality of visual point cloud data; In response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold, a voxel volume corresponding to the object is obtained; An extraction operation is performed on the voxel body to obtain the internal structure corresponding to the object.

2. The method according to claim 1, characterized in that The acquiring of a plurality of transmission holographic images of the object, wherein each transmission holographic image is acquired at a different acquisition angle, comprises: By changing the placement angles of the electromagnetic wave signal transmitter and the electromagnetic wave signal receiver relative to the object, multiple electromagnetic wave signals of the object are collected, and the multiple transmission holographic images are obtained by calculation based on the multiple electromagnetic wave signals.

3. The method according to claim 1, characterized in that The transmission holographic image includes 3D position information, and the transmission holographic image and its corresponding acquisition angle are input into a pre-trained multi-layer perceptron MLP, including: Encoding the 3D position information to obtain encoded information; The encoded information and the corresponding acquisition angle are input into a pre-trained multi-layer perceptron (MLP).

4. The method according to claim 1, wherein The pre-training of the multi-layer perceptron MLP comprises: Construct a training set; Training the multilayer perceptron (MLP) using a neural radiation field (NeRF) algorithm based on the training set to determine a loss function; The pre-training of the multi-layer perceptron MLP is completed by minimizing the loss function.

5. The method according to claim 1, wherein The constructing of initial point cloud data based on the volume density includes: The initial point cloud data is generated by calculation based on the volume density and its corresponding acquisition angle.

6. The method according to claim 1, characterized in that Before obtaining the voxel volume corresponding to the object, it includes: Discretization processing is performed on the multiple visualization point clouds that have been fused to obtain the voxel volume.

7. The method according to claim 1, characterized in that The extracting operation on the voxel body to obtain the internal structure corresponding to the object includes: The center of gravity of each voxel contained in the voxel volume is determined, and all the centers of gravity are connected to form the internal structure.

8. An object structure reconstruction device, characterized in that: include: an acquisition module, configured to acquire a plurality of transmission holographic images of the object, wherein the acquisition angle of each transmission holographic image is different; an output module configured to input the transmission holographic image and its corresponding acquisition angle into a pre-trained multilayer perceptron (MLP), and output a volume density via the multilayer perceptron (MLP); A construction module configured to construct initial point cloud data based on the volume density; A stitching module is configured to obtain initial fused point cloud data by stitching the point clouds based on all the initial point cloud data; a visualization point cloud generation module, configured to compare and analyze the initial fused point cloud data with each of the holographic images to generate a plurality of visualization point cloud data; a fusion module configured to obtain a voxel volume corresponding to the object in response to fusing the plurality of visualization point cloud data and the fusion result reaching a predetermined density threshold; The structure extraction module is configured to perform an extraction operation on the voxel body to obtain the internal structure corresponding to the object.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause a computer to execute the method according to claim 1 .

Images