Three-dimensional reconstruction method, device and equipment for spatial targets with edge enhancement and multi-exposure

Abstract

The present application relates to a three-dimensional reconstruction method, device and equipment for spatial targets with edge-enhanced multi-exposure. By arbitrarily selecting an optical image from each dataset with different exposure levels as a set of data to be reconstructed, the 3DGS method is used under a randomly initialized three-dimensional Gaussian distribution to reconstruct the spatial target according to this set of data to obtain three-dimensional Gaussian data, and multiple corresponding rendered images are obtained through two-dimensional projection according to different poses of the spatial target in the data to be reconstructed. The Sobel operator is used to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively, and the structure-aware gradient loss function is calculated. Combining with the multi-exposure joint optimization strategy, the three-dimensional Gaussian distribution is optimized, and three-dimensional reconstruction is performed based on the finally optimized three-dimensional Gaussian distribution to overcome the problem of uneven illumination of the spatial target images and obtain an accurate three-dimensional reconstruction model of the spatial target.

Description

Technical Field

[0001] This application relates to the technical field of computer vision three-dimensional reconstruction, and particularly to a three-dimensional reconstruction method, device, and equipment for spatial targets with edge-enhanced multi-exposure. Background Art

[0002] In today's aerospace field, with the rapid progress of technology, numerous spacecraft have been successfully deployed into space orbits, greatly promoting humanity's exploration and utilization of the universe. However, long-term in-orbit operation makes some satellites face the risk of failure and even get out of control and become space debris. These debris seriously threaten the safety of other spacecraft and pose a severe challenge to the sustainability of the space environment. The in-orbit maintenance technology has emerged as the key means to ensure space safety and realize the effective utilization of space resources. In this technical system, accurately describing the current geometric shape of the target spacecraft is the basis and prerequisite for effective maintenance. However, due to the complex and variable cosmic illumination conditions, the images captured by satellite observation platforms often exhibit uneven illumination, which brings great difficulties to the description of the geometric shape of the target spacecraft. To solve this problem, exposure synthesis technology has been widely applied to in-orbit rendezvous observation, and multiple shots with different exposure times are taken to obtain as much comprehensive information of the target spacecraft as possible.

[0003] Although exposure synthesis technology alleviates the problem of uneven illumination to a certain extent, the existing spatial target processing technologies still have many limitations. Traditional three-dimensional reconstruction methods based on the Structure from Motion (SfM) framework rely on feature point extraction (such as SIFT) and camera parameter estimation to complete the reconstruction work. However, in the multi-exposure observation scenario, the photometric changes of the same component in different images make the feature point extraction and matching extremely difficult, and it is difficult to guarantee the reconstruction quality, and even may lead to reconstruction failure. Neural Radiance Fields (NeRF) in deep learning, although it implicitly represents the scene through a neural network and achieves high-quality image rendering, has too long training and rendering times, and can only optimize for a single image each time, prone to overfitting, and is greatly limited in practical applications.

[0004] Although existing research has applied it to the field of spatial targets and verified its feasibility in spatial target reconstruction, no in-depth systematic improvement has been carried out for special imaging scenarios such as multi-exposure images. Summary of the Invention

[0005] Based on this, it is necessary to provide a three-dimensional reconstruction method, device, and equipment for spatial targets with edge-enhanced multi-exposure that can overcome the problem of uneven illumination of spatial target images and improve the reconstruction quality in view of the above technical problems.

[0006] A three-dimensional reconstruction method for spatial targets with edge-enhanced multi-exposure, the method includes:

[0007] Obtain multiple optical images of the same spatial target with different poses at different exposure levels. Construct a dataset from the optical images with the same exposure level to obtain multiple datasets;

[0008] Arbitrarily select one optical image from each of the datasets as a set of data to be reconstructed. Among them, the spatial targets in each optical image of the data to be reconstructed present different poses;

[0009] Use the 3DGS method to reconstruct the spatial target according to a set of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution to obtain three-dimensional Gaussian data;

[0010] Based on the three-dimensional Gaussian data, perform two-dimensional projection according to the different poses of the spatial target in the data to be reconstructed to obtain corresponding multiple rendered images. Use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0011] Use the high-frequency feature images corresponding to the rendered images and the optical images to calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain the finally optimized three-dimensional Gaussian distribution;

[0012] Based on the finally optimized three-dimensional Gaussian distribution, convert it into a point cloud for three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the spatial target.

[0013] In one embodiment, the multiple optical images of the same spatial target include three exposure levels: high, medium, and low.

[0014] In one embodiment, when obtaining multiple two-dimensional rendered images from the three-dimensional Gaussian data:

[0015] Given a corresponding transformation matrix according to the pose of the spatial target in each optical image of the data to be reconstructed;

[0016] Perform two-dimensional projection on the three-dimensional Gaussian data using different transformation matrices to obtain the rendered images corresponding to each optical image of the data to be reconstructed.

[0017] In one embodiment, the structure-aware gradient loss function is expressed as:

[0018] ;

[0019] In the above formula, and respectively represent the rendered image and the optical image, and High-frequency feature extraction in the horizontal and vertical directions respectively denote norm

[0020] In one embodiment, the multi-exposure joint optimization strategy is expressed as:

[0021] ;

[0022] In the above formula, denotes optical images at three different exposure levels, denotes the mean absolute error of each exposure level, denotes the structural similarity loss of each exposure level, denotes the weight coefficient for balancing the two losses.

[0023] This application also provides a three-dimensional reconstruction device for a spatial target with edge enhancement and multi-exposure, and the device includes:

[0024] A multi-exposure level data acquisition module, configured to acquire multiple optical images of the same spatial target at different poses under different exposure levels, form a data set with the optical images at the same exposure level, and obtain multiple data sets;

[0025] A data to be reconstructed selection module, configured to arbitrarily select one optical image from each of the data sets as a group of data to be reconstructed, wherein the spatial targets in the optical images of the data to be reconstructed present different poses;

[0026] A three-dimensional Gaussian data generation module, configured to reconstruct the spatial target according to a group of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution by using the 3DGS method to obtain three-dimensional Gaussian data;

[0027] A high-frequency feature image extraction module, configured to perform two-dimensional projection based on the three-dimensional Gaussian data according to different poses of the spatial target in the data to be reconstructed to obtain corresponding multiple rendered images, and use the Sobel operator to extract high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0028] An optimization iteration module, configured to calculate a structure-aware gradient loss function by using the rendered images and the high-frequency feature images corresponding to the optical images, and optimize the three-dimensional Gaussian distribution in combination with the multi-exposure joint optimization strategy, and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain the finally optimized three-dimensional Gaussian distribution;

[0029] A spatial target three-dimensional reconstruction module, configured to perform three-dimensional reconstruction by converting the finally optimized three-dimensional Gaussian distribution into a point cloud to obtain a three-dimensional reconstruction model of the spatial target.

[0030] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0031] Obtain multiple optical images of the same spatial target at different poses under different exposure levels, and form a dataset with the optical images of the same exposure level to obtain multiple datasets;

[0032] Arbitrarily select one optical image from each of the datasets as a set of data to be reconstructed. Among them, the spatial targets in each optical image of the data to be reconstructed present different poses;

[0033] Use the 3DGS method under a randomly initialized three-dimensional Gaussian distribution to reconstruct the spatial target according to a set of data to be reconstructed, and obtain three-dimensional Gaussian data;

[0034] Based on the three-dimensional Gaussian data, perform two-dimensional projection according to the different poses of the spatial target in the data to be reconstructed to obtain corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0035] Use the high-frequency feature images corresponding to the rendered images and the optical images to calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution, and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain the finally optimized three-dimensional Gaussian distribution;

[0036] Based on the finally optimized three-dimensional Gaussian distribution, convert it into a point cloud for three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the spatial target.

[0037] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0038] Obtain multiple optical images of the same spatial target at different poses under different exposure levels, and form a dataset with the optical images of the same exposure level to obtain multiple datasets;

[0039] Arbitrarily select one optical image from each of the datasets as a set of data to be reconstructed. Among them, the spatial targets in each optical image of the data to be reconstructed present different poses;

[0040] Use the 3DGS method under a randomly initialized three-dimensional Gaussian distribution to reconstruct the spatial target according to a set of data to be reconstructed, and obtain three-dimensional Gaussian data;

[0041] Based on the three-dimensional Gaussian data, perform two-dimensional projection according to different poses of the spatial target in the data to be reconstructed, obtain corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0042] Utilize the high-frequency feature images corresponding to the rendered images and the optical images, calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution, and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain the finally optimized three-dimensional Gaussian distribution;

[0043] Based on the finally optimized three-dimensional Gaussian distribution, convert it into a point cloud for three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the spatial target.

[0044] The above three-dimensional reconstruction method, device and equipment for spatial targets with edge-enhanced multi-exposure respectively select an optical image from each dataset with different exposure levels as a group of data to be reconstructed, adopt the 3DGS method under a randomly initialized three-dimensional Gaussian distribution, reconstruct the spatial target according to a group of data to obtain three-dimensional Gaussian data, and based on the three-dimensional Gaussian data, perform two-dimensional projection according to different poses of the spatial target in the data to be reconstructed, obtain corresponding multiple rendered images, use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively, utilize the high-frequency feature images corresponding to the rendered images and the optical images, calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution, and perform three-dimensional reconstruction based on the finally optimized three-dimensional Gaussian distribution to overcome the problem of uneven illumination of spatial target images in real tasks and obtain an accurate three-dimensional reconstruction model of the spatial target. Description of the Drawings

[0045] Figure 1 It is a schematic flowchart of the three-dimensional reconstruction method for spatial targets with edge-enhanced multi-exposure in one embodiment;

[0046] Figure 2 It is a schematic block diagram of the three-dimensional reconstruction method for spatial targets with edge-enhanced multi-exposure in one embodiment;

[0047] Figure 3 It is a structural block diagram of the three-dimensional reconstruction device for spatial targets with edge-enhanced multi-exposure in one embodiment;

[0048] Figure 4 It is an internal structure diagram of a computer device in one embodiment. Detailed Embodiments

[0049] To make the objectives, technical solutions and advantages of this application more clear and understandable, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain this application and are not used to limit this application.

[0050] Accurate description of the geometric features of unknown spacecraft is crucial for on-orbit service tasks such as on-orbit maintenance and condition monitoring. However, due to the changes in the observation angle and lighting conditions caused by the movement of the observation platform, the images of the target spacecraft have problems of uneven lighting, such as Figure 1 shown. In this application, a three-dimensional reconstruction method for space targets with edge enhancement and multi-exposure is proposed, including the following steps:

[0051] Step S100: Obtain multiple optical images of the same space target in different poses at different exposure levels, and form a dataset with the optical images of the same exposure level to obtain multiple datasets.

[0052] Step S110: Arbitrarily select one optical image from each dataset as a group of data to be reconstructed. Among them, the space targets in each optical image of the data to be reconstructed present different poses.

[0053] Step S120: Use the 3DGS method to reconstruct the space target according to a group of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution to obtain three-dimensional Gaussian data.

[0054] Step S130: Based on the three-dimensional Gaussian data, perform two-dimensional projection according to the different poses of the space target in the data to be reconstructed to obtain corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively.

[0055] Step S140: Use the rendered images and the high-frequency feature images corresponding to the optical images to calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, the finally optimized three-dimensional Gaussian distribution is obtained.

[0056] Step S150: Based on the finally optimized three-dimensional Gaussian distribution, convert it into a point cloud for three-dimensional reconstruction to obtain a three-dimensional reconstruction model of the space target.

[0057] In this application, a Gaussian scene representation method based on multi-exposure joint constraint and edge gradient guidance (ME-GS) is proposed for uneven illumination of space target images. Based on the 3D Gaussian Splatting (3DGS) framework, this method innovatively introduces a low, medium, and high exposure constraint mechanism, improves the quality of scene representation through complementary information of different exposure images, and effectively improves the accuracy of detail rendering. At the same time, the Sobel edge detection operator is introduced to extract the structural features of the spacecraft target, enhancing the representation ability of high-frequency information in the scene. To address the problem of a small space target dataset, a multi-exposure synthesis dataset for space target reconstruction is also constructed.

[0058] In step S100, to solve the problem of a small space target dataset, a means of multiple exposures of the same space target is adopted to enrich the data.

[0059] In this embodiment, multiple optical images of the space target with different poses are obtained by continuously taking multiple photos of the moving space target over a period of time.

[0060] In this embodiment, multiple optical images of the same space target include three exposure levels: high, medium, and low, that is, a high-exposure dataset, a medium-exposure dataset, and a low-exposure dataset, respectively.

[0061] In step S110, when performing three-dimensional reconstruction using 3DGS, multiple optical images with different poses of the same target are required to obtain a three-dimensional model from two-dimensional images. Therefore, a set of data to be reconstructed includes three optical images with different exposure levels and different poses of the space target.

[0062] In step S120, the 3DGS method is used for three-dimensional reconstruction. This method is an emerging neural rendering technology. Different from the previous NeRF method, it uses a large number of anisotropic 3D Gaussian distributions to represent the scene, and each 3D Gaussian follows the 3D Gaussian distribution (formula (1)):

[0063] (1)

[0064] In formula (1), represents the mean of the Gaussian point, represents the covariance matrix.

[0065] In addition, each 3D Gaussian also has an opacity and spherical harmonic parameters ( represents the degree of freedom), which are used to model view-dependent colors.

[0066] Further, when 3DGS is first used for modeling, its 3D Gaussian distribution is randomly initialized. This means that there is still a large difference between the three-dimensional Gaussian data of the spatial target constructed at this time and the real spatial target structure. Therefore, it is necessary to continuously optimize the 3D Gaussian distribution so that the generated three-dimensional Gaussian data, that is, the three-dimensional model of the spatial target, gradually approaches the real structure.

[0067] In step S130, after obtaining the three-dimensional Gaussian data, it is necessary to project the 3D Gaussian sphere onto a 2D plane according to the given perspective transformation matrix to obtain the corresponding rendered image, and guide the optimization of the 3D Gaussian distribution through the difference between the rendered image and the real image.

[0068] Specifically, when obtaining multiple 2D rendered images from the three-dimensional Gaussian data: according to the spatial target poses in each optical image in the data to be reconstructed, the corresponding transformation matrices are given, and different transformation matrices are used for 2D projection according to the three-dimensional Gaussian data to obtain the rendered images corresponding to each optical image in the data to be reconstructed.

[0069] In this embodiment, when projecting the 3D Gaussian sphere onto a 2D plane, the corresponding 2D covariance matrix can be expressed as:

[0070] (2)

[0071] In formula (2), represents the affine approximation Jacobian matrix of the projection transformation, represents the perspective transformation matrix, which projects the 3D Gaussian sphere onto a 2D plane.

[0072] Further, after projecting the 3D Gaussian onto a 2D plane, the color of each pixel in the image is calculated. For each pixel in the image, the color of the pixel is calculated by mixing the ordered 3D Gaussians that overlap with it, using the following formula:

[0073] (3)

[0074] In formula (3), represents the final color value of a pixel in the image, represents the spherical harmonic parameter of the i-th Gaussian distribution, represents the effective opacity of the i-th Gaussian distribution, represents the effective opacity of the j-th Gaussian distribution, represents the original opacity of the i-th Gaussian distribution, represents the mean vector of the i-th Gaussian distribution.

[0075] Further, after rendering the image, 3DGS optimizes and the loss, and makes a pixel-by-pixel comparison between the rendered image and the real image, so as to realize the optimization of the 3D Gaussian distribution. Its loss function is expressed as:

[0076] (4)

[0077] In formula (4), represents a hyperparameter. Preferably, in 3DGS, is set to 0.2. represents the structural similarity index loss function.

[0078] Considering that in the process of optimizing the 3D Gaussian distribution, if only the pixel-level loss function ( and ) is used for optimization, this method mainly focuses on the direct difference of pixel values, and it is difficult to fully capture the geometric structure features of spatial targets, which is likely to cause structural distortion and blurred details at the edges. In order to accurately capture and enhance the geometric structure and surface details of the target object, in this method, a structure-aware gradient loss based on the Sobel operator is introduced. As a classic first-order differential operator, the Sobel operator extracts the high-frequency information of the image by calculating the pixel differences within the neighborhood, and can effectively capture the high-frequency structure features such as the edges and surface contours of the object. The Sobel operator contains two convolution kernels, which are respectively used to calculate the gradients in the horizontal direction ( ) and the vertical direction ( ):

[0079] (5)

[0080] (6)

[0081] In step S140, when optimizing the three-dimensional Gaussian distribution, after using the above-mentioned Sobel operator to extract the high-frequency feature images of multiple rendered images and the corresponding optical images (i.e., real images) respectively, use the norm to define the structure-aware gradient loss, which is expressed as:

[0082] (7)

[0083] In formula (7), and respectively represent the rendered image and the optical image, and are the high-frequency feature extractions in the horizontal direction and the vertical direction respectively, represents Norm.

[0084] Furthermore, since 3DGS follows the convention of NeRF, a single image is randomly selected for optimization in each iteration. However, such a random single optimization strategy cannot fully utilize the detailed information obtained by the multi-exposure imaging mechanism. To solve this problem, a multi-view exposure joint optimization strategy is proposed. In each optimization iteration, three images with different exposure levels are selected simultaneously , where , and represent low exposure, medium exposure, and high exposure respectively. This strategy effectively utilizes the complementary information under different exposure conditions: low-exposure images capture the details of high-exposure components; high-exposure images reveal the structural information of shadow areas; medium-exposure images maintain the overall structural balance.

[0085] In this embodiment, when constructing the multi-exposure joint optimization loss, both the mean absolute error and the D-SSIM loss are considered. Then, the multi-exposure joint optimization strategy is expressed as:

[0086] (8)

[0087] In formula (8), represents the optical images of three different exposure levels, represents the mean absolute error of each exposure level, represents the structural similarity loss of each exposure level. represents the weight coefficient used to balance the two losses. Preferably, is set to 0.2.

[0088] To achieve the high-fidelity reconstruction of spatial targets, in this embodiment, the two key components of structure perception and multi-exposure constraint are unified into an optimization framework, that is, the structure perception gradient loss function is combined with the multi-exposure joint optimization strategy, and the overall optimization objective obtained is defined as:

[0089] (9)

[0090] In formula (9), represents the hyperparameter of the gradient loss weight. Preferably, it is set to 0.2. This joint optimization strategy is specifically designed for the characteristics of spatial target imaging. The structure perception gradient loss maintains the edge and contour features of the spacecraft surface by extracting high-frequency features, while the multi-exposure joint optimization is used to solve the problem of multi-exposure imaging.

[0091] Further, after completing one iteration of optimization, randomly select one optical image from each of the three datasets respectively, perform three-dimensional reconstruction using the optimized three-dimensional Gaussian distribution, then perform two-dimensional projection, calculate the loss function, and then optimize the three-dimensional Gaussian distribution. Repeat steps S120 to S140 until the number of iterations reaches the preset number, then the optimization is completed to obtain the final three-dimensional Gaussian distribution. In step S150, use the optimized final three-dimensional Gaussian distribution to generate a three-dimensional model of the space target, and this three-dimensional model can accurately represent the structure of the space target.

[0092] As Figure 2 shown, it is a schematic flow diagram of this method.

[0093] In the above three-dimensional reconstruction method of a space target with edge-enhanced multi-exposure, aiming at the problem of uneven illumination caused by the optical image of the space target in the cosmic environment, while traditional methods are prone to overfitting a certain image when processing such multi-exposure images and it is difficult to make full use of the benefits brought by the multi-exposure imaging mechanism. In this method, by introducing multi-exposure constraints on the basis of 3DGS, a low, medium, and high three-level exposure constraint mechanism is innovatively designed, making full use of the complementary information of different exposures to improve the scene reconstruction quality. At the same time, the Sobel edge feature extraction technology is integrated to further enhance the ability to represent scene structure details. The effectiveness of this method is also verified by experiments. The experimental results show that, as shown in Tables 1 - 3, this method has achieved relatively significant improvements in three key indicators of PSNR, SSIM, and LPIPS. Among them, this method has achieved a 33.8% improvement in the LPIPS indicator, and the LPIPS value has decreased by 33.8% compared to the baseline. This fully proves that this method can overcome the challenges brought by multi-exposure and achieve better visual reconstruction quality.

[0094] Table 1. Comparison values of PSNR (the larger the better)

[0095]

[0096] Table 2. Comparison values of SSIM (the larger the better)

[0097]

[0098] Table 3. Comparison values of LPIPS (the smaller the better)

[0099]

[0100] It should be understood that although Figure 1The steps in the flowchart are shown in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least a part of the steps in

[0101] In one embodiment, as Figure 3 shown, a three-dimensional reconstruction device for a spatial target with edge-enhanced multi-exposure is provided, including: a multi-exposure level data acquisition module 200, a data to be reconstructed selection module 210, a three-dimensional Gaussian data generation module 220, a high-frequency feature image extraction module 230, an optimization iteration module 240, and a three-dimensional reconstruction module 250 for spatial targets, where:

[0102] The multi-exposure level data acquisition module 200 is configured to obtain multiple optical images of the same spatial target at different poses under different exposure levels, form a data set with the optical images of the same exposure level, and obtain multiple data sets;

[0103] The data to be reconstructed selection module 210 is configured to arbitrarily select one optical image from each of the said data sets as a group of data to be reconstructed, where the spatial targets in each of the optical images in the data to be reconstructed present different poses;

[0104] The three-dimensional Gaussian data generation module 220 is configured to reconstruct the spatial target according to a group of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution by using the 3DGS method to obtain three-dimensional Gaussian data;

[0105] The high-frequency feature image extraction module 230 is configured to perform two-dimensional projection based on the three-dimensional Gaussian data according to different poses of the spatial target in the data to be reconstructed to obtain corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0106] The optimization iteration module 240 is configured to calculate a structure-aware gradient loss function by using the rendered images and the high-frequency feature images corresponding to the optical images, and combine a multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution, and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, a finally optimized three-dimensional Gaussian distribution is obtained;

[0107] A three-dimensional reconstruction module 250 for spatial targets is configured to perform three-dimensional reconstruction by converting the finally optimized three-dimensional Gaussian distribution into a point cloud, so as to obtain a three-dimensional reconstruction model of the spatial target.

[0108] For the specific limitations of the three-dimensional reconstruction device for spatial targets with edge enhancement and multi-exposure, reference can be made to the limitations of the three-dimensional reconstruction method for spatial targets with edge enhancement and multi-exposure in the foregoing text, which will not be elaborated herein. Each module in the above-mentioned three-dimensional reconstruction device for spatial targets with edge enhancement and multi-exposure can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0109] In one embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 4 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a three-dimensional reconstruction method for spatial targets with edge enhancement and multi-exposure. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0110] Those skilled in the art can understand that Figure 4 the structure shown in

[0111] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0112] Obtain multiple optical images of the same spatial target with different poses at different exposure levels, and form a dataset with optical images of the same exposure level to obtain multiple datasets;

[0113] Arbitrarily select one optical image from each of the datasets as a set of data to be reconstructed, where the spatial targets in each optical image in the data to be reconstructed exhibit different poses;

[0114] Use the 3DGS method to reconstruct the spatial target according to a set of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution, obtaining three-dimensional Gaussian data;

[0115] Based on the three-dimensional Gaussian data, perform two-dimensional projection according to the different poses of the spatial target in the data to be reconstructed, obtaining corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0116] Utilize the rendered images and the high-frequency feature images corresponding to the optical images to calculate the structure-aware gradient loss function, and combine the multi-exposure joint optimization strategy to optimize the three-dimensional Gaussian distribution, and complete one optimization iteration. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain the finally optimized three-dimensional Gaussian distribution;

[0117] Based on the finally optimized three-dimensional Gaussian distribution, convert it into a point cloud for three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the spatial target.

[0118] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0119] Obtain multiple optical images of the same spatial target at different exposure levels and different poses, and form the optical images with the same exposure level into a dataset to obtain multiple datasets;

[0120] Arbitrarily select one optical image from each of the datasets as a set of data to be reconstructed, where the spatial targets in each optical image in the data to be reconstructed exhibit different poses;

[0121] Use the 3DGS method to reconstruct the spatial target according to a set of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution, obtaining three-dimensional Gaussian data;

[0122] Based on the three-dimensional Gaussian data, perform two-dimensional projection according to the different poses of the spatial target in the data to be reconstructed, obtaining corresponding multiple rendered images, and use the Sobel operator to extract the high-frequency feature images of the multiple rendered images and the corresponding optical images respectively;

[0123] Using the high-frequency feature images corresponding to the rendered image and the optical image, a structure-aware gradient loss function is calculated, and combined with a multi-exposure joint optimization strategy, the three-dimensional Gaussian distribution is optimized, and one optimization iteration is completed. After performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, the finally optimized three-dimensional Gaussian distribution is obtained;

[0124] Based on the finally optimized three-dimensional Gaussian distribution, it is converted into a point cloud for three-dimensional reconstruction to obtain the three-dimensional reconstruction model of the spatial target.

[0125] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0126] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0127] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the method of the present application should be subject to the appended claims.

Claims

1. A method for three-dimensional reconstruction of space objects using edge-enhanced multi-exposure, characterized in that: The method comprises: Acquire multiple optical images of the same space target at different exposure levels and multiple poses, and form a data set of optical images at the same exposure level to obtain multiple data sets; Selecting an optical image from each of the data sets as a group of data to be reconstructed, wherein the spatial target in each optical image in the data to be reconstructed presents a different posture; The space target is reconstructed according to a set of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution by using a 3DGS method to obtain three-dimensional Gaussian data; Based on the three-dimensional Gaussian data, two-dimensional projection is performed according to different postures of the space target in the data to be reconstructed to obtain a corresponding plurality of rendered images, and a Sobel operator is used to respectively extract high-frequency feature images of the plurality of rendered images and the corresponding optical images; Using the high-frequency feature image corresponding to the rendered image and the optical image, a structure-aware gradient loss function is calculated, and the three-dimensional Gaussian distribution is optimized in combination with a multi-exposure joint optimization strategy, and an optimization iteration is completed. After the three-dimensional Gaussian distribution is iteratively optimized for a preset number of times, a final optimized three-dimensional Gaussian distribution is obtained; The three-dimensional Gaussian distribution after the final optimization is converted into a point cloud for three-dimensional reconstruction to obtain a three-dimensional reconstruction model of the space target.

2. The edge-enhanced multi-exposure space target 3D reconstruction method according to claim 1, characterized in that: Multiple optical images of the same space target include three exposure levels: high, medium and low.

3. The edge-enhanced multi-exposure space target 3D reconstruction method according to claim 2, characterized in that: When obtaining multiple two-dimensional rendered images based on three-dimensional Gaussian data: According to the spatial target posture in each optical image in the data to be reconstructed, a corresponding transformation matrix is ​​given; Different transformation matrices are used to perform two-dimensional projection according to the three-dimensional Gaussian data to obtain a rendering image corresponding to each optical image in the data to be reconstructed.

4. The edge-enhanced multi-exposure space target 3D reconstruction method according to claim 3, characterized in that: The structure-aware gradient loss function is expressed as: In the above formula, and represent the rendered image and the optical image respectively, and They are high-frequency feature extraction in the horizontal and vertical directions respectively. express Norm.

5. The edge-enhanced multi-exposure space target 3D reconstruction method according to claim 4, characterized in that: The multi-exposure joint optimization strategy is expressed as: In the above formula, Respectively represent low exposure, medium exposure and high exposure. represents the mean absolute error at each exposure level, represents the structural similarity loss at each exposure level, Represents the weight coefficient used to balance the two losses.

6. An edge-enhanced multi-exposure space target three-dimensional reconstruction device, characterized in that: The device comprises: A multi-exposure level data acquisition module is used to acquire multiple optical images of the same space target at different exposure levels and multiple different postures, and to form a data set of optical images at the same exposure level to obtain multiple data sets; A module for selecting data to be reconstructed, used for selecting an optical image from each of the data sets as a group of data to be reconstructed, wherein the spatial targets in each optical image in the data to be reconstructed present different postures; A three-dimensional Gaussian data generation module is used to reconstruct the space target according to a set of data to be reconstructed under a randomly initialized three-dimensional Gaussian distribution using a 3DGS method to obtain three-dimensional Gaussian data; A high-frequency feature image extraction module is used to perform two-dimensional projection based on the three-dimensional Gaussian data according to different postures of the space target in the data to be reconstructed, obtain a corresponding plurality of rendered images, and use a Sobel operator to respectively extract high-frequency feature images of the plurality of rendered images and the corresponding optical image; An optimization iteration module, used to calculate a structure-aware gradient loss function using the high-frequency feature images corresponding to the rendered image and the optical image, and optimize the three-dimensional Gaussian distribution in combination with a multi-exposure joint optimization strategy, and complete an optimization iteration, and after performing a preset number of iterative optimizations on the three-dimensional Gaussian distribution, obtain a final optimized three-dimensional Gaussian distribution; The space target three-dimensional reconstruction module is used to perform three-dimensional reconstruction based on the point cloud converted into the final optimized three-dimensional Gaussian distribution to obtain a three-dimensional reconstruction model of the space target.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images