Image rendering method, electronic device, readable medium and program product

Abstract

The present disclosure provides an image rendering method, an electronic device, a readable medium and a program product. The method of the present disclosure comprises: obtaining an initial three-dimensional point cloud processed from an original image; determining a first Gaussian kernel of each point in the initial three-dimensional point cloud; and performing region division on the initial three-dimensional point cloud to obtain at least one three-dimensional region; determining an attribution relationship between the first Gaussian kernel and the three-dimensional region; based on the attribution relationship, performing iterative processing on the first Gaussian kernel attributed to the three-dimensional region by using a densification parameter corresponding to the three-dimensional region to obtain a second Gaussian kernel; and performing image reconstruction by using the second Gaussian kernel to obtain a rendered image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to image rendering methods, electronic devices, readable media, and program products. Background Technology

[0002] With the development of Mixed Reality (MR) technology, it has been widely used in more and more scenarios, such as using devices that support mixed reality technology to view houses.

[0003] To display images of houses using hybrid display technology, 3DGS (3D Gaussian Splatting) reconstruction is required. In existing image reconstruction techniques, the Gaussian kernel densification process is uncontrollable due to the significant differences in texture complexity and geometric features between different areas (e.g., sofas, rugs, etc.). For example, areas with rich texture details (such as carpets) are preferentially densified due to their larger gradients, leading to excessive concentration of computational resources; while key furniture areas of interest to the user (such as coffee tables, desks) have smaller gradients and cannot be sufficiently densified, ultimately resulting in blurry reconstructions of these important objects. Consequently, the overall reconstruction effect is poor. Therefore, a solution is needed that can enhance the reconstruction effect of content of interest to the user. Summary of the Invention

[0004] This disclosure provides image rendering methods, electronic devices, readable media, and program products.

[0005] According to a first aspect of this disclosure, an image rendering method is provided. The method specifically includes: acquiring an initial three-dimensional point cloud obtained by processing an original image; determining a first Gaussian kernel for each point in the initial three-dimensional point cloud; dividing the initial three-dimensional point cloud into regions to obtain at least one three-dimensional region; determining the attribution relationship between the first Gaussian kernel and the three-dimensional region; based on the attribution relationship, performing iterative densification processing on the first Gaussian kernel belonging to the three-dimensional region using a densification parameter corresponding to the three-dimensional region to obtain a second Gaussian kernel; and reconstructing the image using the second Gaussian kernel to obtain a rendered image.

[0006] The above solution solves the problem of uneven density caused by global optimization in existing 3DGS reconstruction by controlling independent density parameters based on three-dimensional regions. This allows for fine-tuning of the areas of interest to the user, improving reconstruction results and rendering image quality.

[0007] According to at least one embodiment of this disclosure, based on the attribution relationship, a first Gaussian kernel belonging to a three-dimensional region is subjected to a densification iteration process using a densification parameter corresponding to the three-dimensional region to obtain a second Gaussian kernel, comprising: calculating the position gradient through backpropagation in response to a reconstruction training request; creating at least one three-dimensional region grouping based on the attribution relationship; wherein each three-dimensional region grouping contains at least one first Gaussian kernel belonging to the three-dimensional region; determining the position gradient corresponding to the first Gaussian kernel in each three-dimensional region grouping; sorting the first Gaussian kernels in the three-dimensional region grouping according to the magnitude of the position gradient; and based on the sorting result, performing a densification iteration process on the first Gaussian kernel belonging to the three-dimensional region using a densification parameter of the three-dimensional region to obtain a second Gaussian kernel.

[0008] According to the above scheme, by independently grouping three-dimensional regions and sorting and densifying within each group, the Gaussian kernel is finely densified, avoiding the imbalance problem caused by global densification; the Gaussian kernel processing of different regions is independent of each other, improving reconstruction efficiency and effect.

[0009] According to at least one embodiment of this disclosure, in response to a reconstruction training request, the position gradient is calculated by backpropagation, including: selecting the current camera pose, projecting a first Gaussian kernel onto a two-dimensional plane and performing rasterization processing to obtain a rendered image under the current camera pose; calculating the error between the rendered image and the original image corresponding to the current camera pose; and calculating the position gradient of each first Gaussian kernel in the view space based on the error by using a backpropagation algorithm.

[0010] According to the above scheme, the impact of each Gaussian kernel on the reconstruction quality is accurately quantified through detailed error calculation and backpropagation mechanism, providing a reliable foundation for independent densification. This solves the problem that insufficient gradient in traditional methods leads to the inability to densify important areas (such as coffee tables), and improves the fidelity and efficiency of the overall rendered image.

[0011] According to at least one embodiment of this disclosure, based on the sorting result, a first Gaussian kernel within a three-dimensional region is densified using a densification parameter corresponding to the three-dimensional region to obtain a second Gaussian kernel. This includes: determining a target Gaussian kernel to be densified from the sorted first Gaussian kernels according to the densification parameter set for the three-dimensional region; splitting the target Gaussian kernel to obtain a second Gaussian kernel when the scale of the target Gaussian kernel is greater than a preset scale threshold; or, copying the target Gaussian kernel and moving it along the position gradient direction when the scale of the target Gaussian kernel is less than or equal to the preset scale threshold to generate a second Gaussian kernel.

[0012] According to the above scheme, precise control of Gaussian kernel density is achieved through parameter selection and scale-adaptive densification methods (such as splitting or copying and moving), which solves the problem of insufficient gradient causing important regions to be unable to be densified in the existing technology, and improves the uniformity of reconstruction and image quality.

[0013] According to at least one embodiment of this disclosure, dividing an initial three-dimensional point cloud into regions to obtain at least one three-dimensional region includes: extracting features from the initial three-dimensional point cloud using a trained three-dimensional model to obtain semantic feature vectors; determining the bounding boxes corresponding to each feature; and constructing a three-dimensional region based on the bounding boxes of the features and the corresponding semantic feature vectors.

[0014] According to the above scheme, by using a trained 3D model to achieve region division, the reliance on manual annotation is reduced, and the efficiency and consistency of 3D region acquisition are improved. At the same time, the semantic feature vectors and bounding boxes output by the 3D model can directly provide a basis for setting the densification parameters of the specified 3D region (for example, the importance can be judged based on semantic features), which further enhances the refinement and intelligence of densification.

[0015] Determining the affiliation relationship between a first Gaussian kernel and a three-dimensional region according to at least one embodiment of this disclosure includes: determining the first three-dimensional position coordinates of each first Gaussian kernel; determining whether the first three-dimensional position coordinates are within the bounding box of any three-dimensional region; if so, determining that the first Gaussian kernel belongs to the corresponding three-dimensional region.

[0016] According to the above scheme, the relationship between the Gaussian kernel and the 3D region is accurately determined by using coordinate relationships. This achieves spatial isolation of the Gaussian kernel. After establishing the relationship, the Gaussian kernels of different 3D regions can be sorted and densed separately within groups, which is beneficial for improving the rendering effect of subsequent images.

[0017] According to at least one embodiment of this disclosure, after calculating the position gradient of each first Gaussian kernel in the view space using a backpropagation algorithm, the method further includes: in iterative training, performing backpropagation calculation to obtain the second three-dimensional position coordinates of each first Gaussian kernel; determining whether the second three-dimensional coordinates and the first three-dimensional coordinates are within the bounding box of the same three-dimensional region; if not, re-determining the attribution relationship between the first Gaussian kernel and the three-dimensional region.

[0018] According to the above scheme, the attribution relationship is dynamically updated based on the changes in the three-dimensional coordinate position of the Gaussian kernel during the iterative training process. This ensures that the grouping of the three-dimensional region always reflects the latest spatial distribution, improves the independence and accuracy of Gaussian kernel densification processing, and avoids the impact of position changes on the final rendering effect due to incorrect attribution.

[0019] According to at least one embodiment of this disclosure, obtaining an initial three-dimensional point cloud obtained by processing the original image and determining a first Gaussian kernel for each point in the initial three-dimensional point cloud includes: acquiring a set of original images captured from multiple perspectives; processing the original images using structural motion recovery techniques to generate an initial three-dimensional point cloud; assigning Gaussian attributes to each point in the initial three-dimensional point cloud to generate a first Gaussian kernel; wherein the Gaussian attributes include at least one of three-dimensional position coordinates, opacity, scale, and color.

[0020] According to the above scheme, an initial 3D point cloud is obtained by processing the original image and assigning Gaussian properties, providing a foundation for subsequent region division and classification. This is beneficial for improving image rendering effects.

[0021] After constructing a three-dimensional region based on a feature-based bounding box and a corresponding semantic feature vector according to at least one embodiment of this disclosure, the method further includes: determining the importance level of the three-dimensional region according to the feature type of the feature; determining the densification parameter of the three-dimensional region according to the importance level; if the importance level of the three-dimensional region is high, then setting the densification parameter of the three-dimensional region to: high densification ratio and / or low densification threshold; if the importance level of the three-dimensional region is low, then setting the densification parameter of the three-dimensional region to: low densification ratio and / or high densification threshold.

[0022] According to the above scheme, semantic feature vectors in 3D regions are used to achieve rapid classification of feature types, and then the importance level of the 3D region can be determined based on the feature type. This allows for differentiated settings of density parameters for different 3D regions. Even in complex scenes with uneven texture distribution, this ensures that important regions receive sufficient detail reconstruction, while secondary regions do not excessively consume resources, significantly improving overall reconstruction balance and rendered image quality.

[0023] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs a first aspect of any embodiment of this disclosure.

[0024] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement a first aspect of any embodiment of this disclosure.

[0025] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a first aspect of any embodiment of this disclosure. Attached Figure Description

[0026] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0027] Figure 1 This is a schematic flowchart of an image rendering method provided in an embodiment of the present disclosure.

[0028] Figure 2 This is a schematic flowchart of the densification process provided in an embodiment of the present disclosure.

[0029] Figure 3 This is a flowchart illustrating the method for calculating position gradients provided in an embodiment of this disclosure.

[0030] Figure 4 This is a flowchart illustrating the method for calculating the second Gaussian kernel provided in an embodiment of this disclosure.

[0031] Figure 5 This is a schematic flowchart of a region division method provided in an embodiment of the present disclosure.

[0032] Figure 6 This is a flowchart illustrating the attribution determination method provided in the embodiments of this disclosure.

[0033] Figure 7 This is a flowchart illustrating the attribution relationship update method provided in an embodiment of this disclosure.

[0034] Figure 8 This is a flowchart illustrating the initial three-dimensional point cloud acquisition method provided in the embodiments of this disclosure.

[0035] Figure 9 This is a flowchart illustrating the method for determining density parameters provided in an embodiment of this disclosure.

[0036] Figure 10 This is a schematic block diagram of an image rendering apparatus according to one embodiment of the present disclosure.

[0037] Figure 11 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0038] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0039] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] Figure 1 This is a schematic flowchart illustrating the image rendering method provided in an embodiment of this disclosure. Figure 1 The method shown includes steps 101 to 105. This method can be executed by the server.

[0041] Specifically, Figure 1 The method shown includes step 101: obtaining an initial 3D point cloud obtained by processing the original image.

[0042] It should be noted that the original images mentioned here refer to a collection of images taken by a camera from multiple perspectives, which are used for subsequent 3D reconstruction processing.

[0043] The initial 3D point cloud mentioned here refers to a sparse set of 3D points generated by processing the original image (such as using structure motion recovery techniques), where each point has 3D spatial coordinates.

[0044] In practical applications, a collection of raw images captured by a camera from multiple perspectives can be obtained. Structure from Motion (SfM) technology is then used to process these raw images, generating an initial 3D point cloud. Next, Gaussian attributes are assigned to each point in the initial 3D point cloud, generating the first Gaussian kernel. It should be noted that when assigning Gaussian attributes, the 3D position coordinates of each point are used as the mean of the Gaussian kernel, and the covariance matrix is ​​initialized based on the average distance from that point to its three nearest neighbors, forming an isotropic Gaussian distribution. Simultaneously, attributes such as opacity, scale (including scaling vectors and rotation quaternions), and color are assigned to each Gaussian kernel, thus constructing an initial Gaussian kernel (i.e., the first Gaussian kernel) that can be further optimized.

[0045] Step 102: Determine the first Gaussian kernel for each point in the initial 3D point cloud; and divide the initial 3D point cloud into regions to obtain at least one 3D region.

[0046] The first Gaussian kernel mentioned here refers to the element formed after each point in the initial 3D point cloud is assigned Gaussian properties. These Gaussian properties include: 3D position coordinates, rotation, opacity, scale, and color, which are used to represent the 3D Gaussian distribution of the scene.

[0047] The three-dimensional region mentioned here is a pre-defined three-dimensional region of interest (3D Polygon), whose geometric representation is a three-dimensional bounding box, used to divide different objects (such as sofas, windows, etc.) or regions (such as master bedroom, living room, secondary bedroom, etc.) in the initial three-dimensional point cloud.

[0048] In practical applications, region segmentation involves isolating different objects or parts of interest in the initial 3D point cloud into independent 3D regions. This provides a fine-grained control basis for subsequent densification iterations, avoiding the densification imbalance problem caused by global optimization. Specifically, by segmenting 3D regions, independent densification parameters (such as thresholds or scales) can be set for each region. This ensures that areas of interest to the user (such as important furniture) receive sufficient densification resources, rather than being dominated by textured areas (such as curtains or carpets), thereby improving the overall reconstruction effect and the detail of the rendered image.

[0049] In practice, the region division can be done manually or through a model. For example, an open-source interactive annotation tool can be used to manually divide the initial 3D point cloud, generating multiple 3D regions, each corresponding to an object such as a coffee table or a desk; alternatively, a deep learning-based 3D detection model can be used, taking the initial 3D point cloud as input and outputting several 3D regions, each with a type label such as "table" or "curtain". Specific implementation methods for region division can be found in subsequent examples.

[0050] Step 103: Determine the relationship between the first Gaussian kernel and the three-dimensional region.

[0051] As mentioned earlier, each 3D region has a corresponding bounding box, and each Gaussian kernel has corresponding 3D position coordinates. Determining the attribution relationship involves judging the relative positional relationship between the 3D position coordinates and the bounding box. If the 3D position coordinates are within the bounding box, the first Gaussian kernel belongs to that 3D region; if the 3D position coordinates are outside the bounding box, the first Gaussian kernel does not belong to that 3D region.

[0052] The attribution relationship can be one-to-many, meaning that a first Gaussian kernel may simultaneously belong to the overlapping boundaries of multiple 3D regions, and during densification, it follows the processing of each assigned 3D region. For example, during reconstruction training, the 3D position coordinates of the first Gaussian kernel are updated, so the attribution relationship needs to be recalculated before each densification. If a Gaussian kernel is located at the overlap of the 3D regions of the table and the curtain, it will participate in the independent densification processing of the two regions respectively. As soon as one of the regions is determined to need densification, the operation is performed.

[0053] Step 104: Based on the attribution relationship, the first Gaussian kernel belonging to the three-dimensional region is subjected to denser iteration processing using the denser parameters corresponding to the three-dimensional region to obtain the second Gaussian kernel.

[0054] The densification parameters mentioned here refer to the parameters used to control the densification process, including the densification threshold (the threshold for filtering gradient values) and the densification ratio (the proportion of Gaussian kernels to be densified to the total number of Gaussian kernels), which are set empirically based on the importance of the 3D region. It should be noted that in this disclosed scheme, the densification parameters are set independently for each 3D region, thus facilitating subsequent independent densification processing based on the importance of the region, rather than adopting global densification processing.

[0055] In practical applications, the first Gaussian kernel needs to undergo multiple iterations of densification to obtain the second Gaussian kernel. It's important to note that densification is not performed in every iteration, but rather once every N iterations (e.g., 100). During reconstruction training, densification is performed at intervals, splitting or copying the first Gaussian kernel that meets the conditions to increase the kernel density and improve reconstruction accuracy.

[0056] For example, during iterative training, the original image corresponding to the current camera pose is selected, the first Gaussian kernel is projected onto a two-dimensional plane and rasterized to obtain the rendered image. Then, the error between the rendered image and the real image (i.e., the original image) can be calculated, and the position gradient of each first Gaussian kernel is calculated through backpropagation. Denseization is typically performed every 100 iterations. This allows regions with rich textures (such as curtains) to have a lower threshold for enhanced denserization, while important but less textured regions (such as coffee tables) can be controlled independently to avoid them being overtaken for denserization resources.

[0057] Step 105: Reconstruct the image using the second Gaussian kernel to obtain the rendered image.

[0058] In practical applications, the viewing angle is defined based on the current camera pose (including parameters such as position, orientation, and field of view). Then, a second Gaussian kernel (which has been denser and has refined properties through iterative densification) is projected onto a two-dimensional plane. Specifically, each second Gaussian kernel is mapped to a two-dimensional pixel space in the camera coordinate system using its three-dimensional position coordinates and rotation properties, forming an intermediate projection representation. Next, rasterization is performed, converting the projected Gaussian kernels into pixel colors. By calculating the transparency, size, and color contribution of each Gaussian kernel, overlapping areas are blended (e.g., using alpha blending to handle transparency), thereby generating a smooth rendered image.

[0059] Based on the above-mentioned publicly available solutions, the problem of uneven density caused by global optimization in existing 3DGS reconstruction is solved by controlling the independent density parameters based on the three-dimensional region. This allows for fine-tuning of areas of interest to the user (such as important furniture), thereby improving the reconstruction effect and the quality of the rendered image.

[0060] In one or more embodiments of this disclosure, such as Figure 2 This is a schematic flowchart of the densification process provided in an embodiment of this disclosure. Figure 2 As shown, step 104, based on the attribution relationship, involves using the densification parameters corresponding to the three-dimensional regions to perform a densification iteration process on the first Gaussian kernel belonging to the three-dimensional region to obtain the second Gaussian kernel. This includes: Step 201: In response to a reconstruction training request, calculating the position gradient through backpropagation. Step 202: Based on the attribution relationship, creating at least one independent three-dimensional region group; wherein each three-dimensional region group contains at least one first Gaussian kernel belonging to the three-dimensional region. Step 203: Determining the position gradient corresponding to the first Gaussian kernel in each three-dimensional region group. Step 204: Sort the first Gaussian kernels in the three-dimensional region groups according to the magnitude of the position gradient. Step 205: Based on the sorting result, using the densification parameters of the three-dimensional region to perform a densification iteration process on the first Gaussian kernel belonging to the three-dimensional region to obtain the second Gaussian kernel.

[0061] In the reconstruction training iteration loop, the current camera pose is selected, the first Gaussian kernel is projected onto a 2D plane and rasterized to obtain the rendered image under that camera pose; the original image corresponding to the current camera pose is obtained as the ground truth image; the error between the rendered image and the ground truth image (e.g., pixel-level difference) is calculated; based on this error, the position gradient of each first Gaussian kernel in the view space is calculated using the backpropagation algorithm. This gradient represents the impact of Gaussian kernel position changes on the error and is used for subsequent densification decisions.

[0062] Based on the previously determined attribution relationships (i.e., the 3D position coordinates of the first Gaussian kernel are specifically within the bounding box of a certain 3D region), Gaussian kernels are assigned to corresponding groups. Each group exists independently, unaffected by other groups, to ensure the isolation of the densification process. There is at least one group, corresponding to at least one 3D region. If a Gaussian kernel is located at the overlapping boundary of multiple 3D regions, it can belong to multiple groups simultaneously and be processed independently within each group. For example, in room reconstruction, one group corresponds to the "table" 3D region, containing the first Gaussian kernels located within that region; another group corresponds to the "curtains," achieving independent densification and preventing curtain textures from competing for table resources.

[0063] For each independent 3D region group, iterate through all first Gaussian kernels contained within it and extract the previously calculated positional gradient specific to that group. This extraction of positional gradients ensures that subsequent sorting operations are limited to data within the group. For example, in the "table" group, only the gradient values ​​of the first Gaussian kernel belonging to that group are extracted, ignoring interference from other regions.

[0064] For each 3D region group, the first Gaussian kernel within the group is independently sorted from largest to smallest based on its positional gradient. After sorting, Gaussian kernels with higher gradients are ranked higher, indicating that they contribute more to the error and should be denser first. This sorting is performed within the group itself. For example, in the "table" group, texture detail Gaussian kernels with larger gradient values ​​are ranked higher, ensuring that important regions are processed first.

[0065] For each group, a target Gaussian kernel is determined from the sorted first Gaussian kernel based on the densification parameters (such as densification ratio or densification threshold) set for that 3D region. Then, densification is performed on the target Gaussian kernel: if the scale is larger than a preset scale threshold, it is split into two new Gaussian kernels; otherwise, it is copied and moved along the position gradient direction to generate a new Gaussian kernel. After multiple iterations, a second set of Gaussian kernels with higher density is obtained. This parameter can be set empirically, such as setting a low threshold for densely textured regions to enhance densification.

[0066] Based on the above-mentioned publicly available scheme, it can be seen that by independently grouping three-dimensional regions and sorting and densifying within each group, the Gaussian kernel is finely densified, avoiding the imbalance problem caused by global densification; the Gaussian kernel processing of different regions is independent of each other, improving reconstruction efficiency and effect.

[0067] In one or more embodiments of this disclosure, such as Figure 3 This is a flowchart illustrating the method for calculating the location gradient provided in an embodiment of this disclosure. Figure 3 As shown, step 201, in response to the reconstruction training request, calculates the position gradient through backpropagation, including: Step 2011: Select the current camera pose, project the first Gaussian kernel onto a two-dimensional plane and perform rasterization processing to obtain a rendered image under the current camera pose. Step 2012: Calculate the error between the rendered image and the original image corresponding to the current camera pose. Step 2013: Based on the error, calculate the position gradient of each first Gaussian kernel in the view space using the backpropagation algorithm.

[0068] It should be noted that the current camera pose mentioned here refers to the camera's position, orientation, and field of view parameters in three-dimensional space, used to simulate observation from a specific perspective and to reconstruct the projection process during training.

[0069] In practical applications, during the reconstruction training loop, a current camera pose is randomly or sequentially selected from the set of camera poses corresponding to the original images from multiple perspectives. Then, using camera position, orientation, and field of view parameters, the 3D position coordinates of each first Gaussian kernel are transformed into the camera's 2D image coordinate system to obtain the corresponding 2D image. A rasterization algorithm is used to traverse the pixels of the 2D image, calculating the contribution of each Gaussian kernel to the pixel (e.g., calculating weights using a Gaussian function based on its rotation, size, and transparency), and colors are blended to generate a complete rendered image. The process of reconstructing the rendered image simulates real-world shooting, using volumetric rendering techniques to ensure that the rendered image closely approximates the actual viewing perspective.

[0070] Furthermore, a loss function (such as mean-square error (MSE)) or structural similarity index (SSIM) is used to compare the differences between the rendered image and the original image pixel by pixel to obtain the total error value. This error value is used as an optimization signal to guide the rendering process to iterate towards greater accuracy. For example, in a room scene, if the color or detail of the desk area in the rendered image does not match the original image, the error will be higher, highlighting the parts that need optimization.

[0071] After obtaining the error, the process further propagates backward along the rendering pipeline, starting from the pixel-level loss. First, the contribution of each Gaussian kernel during rasterization is deduced from the pixel error, then its projected coordinates are deduced, and finally, the position gradient of each Gaussian kernel's position parameters in view space is calculated using the chain rule. This position gradient represents the magnitude of the contribution of fine-tuning the Gaussian kernel position to reducing the overall error, providing a basis for subsequent densification decisions and ensuring that Gaussian kernels that contribute significantly to reconstruction are processed first.

[0072] Based on the above-mentioned publicly available scheme, it is known that by using detailed error calculation and backpropagation mechanism, the impact of each Gaussian kernel on the reconstruction quality is accurately quantified, providing a reliable foundation for independent densification. This solves the problem in traditional methods where insufficient gradients prevent important regions (such as coffee tables) from being densified, thereby improving the fidelity and efficiency of the overall rendered image.

[0073] In one or more embodiments of this disclosure, such as Figure 4 This is a schematic flowchart illustrating the method for calculating the second Gaussian kernel provided in an embodiment of this disclosure. Figure 4As shown, step 205, based on the sorting result, uses the densification parameters corresponding to the three-dimensional region to densify the first Gaussian kernel within the three-dimensional region to obtain the second Gaussian kernel, includes: Step 2051: Based on the densification parameters set for the three-dimensional region, determine the target Gaussian kernel to be densified from the sorted first Gaussian kernels. Step 2052: When the scale of the target Gaussian kernel is greater than a preset scale threshold, split the target Gaussian kernel to obtain the second Gaussian kernel (that is, split it into two new Gaussian kernels). Alternatively, Step 2053: When the scale of the target Gaussian kernel is less than or equal to the preset scale threshold, copy the target Gaussian kernel and move it along the position gradient direction to generate the second Gaussian kernel.

[0074] It should be noted that the scale mentioned here is a parameter in the Gaussian properties, including the scaling vector and rotation quaternion defined by the covariance matrix of the Gaussian kernel.

[0075] Since the three-dimensional region may contain at least one first Gaussian kernel that needs to be densed, and some unimportant Gaussian kernels can be ignored, the more important Gaussian kernels can be selected for densening by sorting and filtering.

[0076] Specifically, the sorting sequence of each independent group is traversed (e.g., sorted from highest to lowest based on the magnitude of the positional gradient value), thereby filtering within a three-dimensional region to ensure that no important Gaussian kernels in each region are ignored. If a densification ratio is used, the top X% (e.g., 10%) of high-gradient kernels are selected as target Gaussian kernels; if a densification threshold is used, Gaussian kernels with gradients greater than the threshold are selected as target Gaussian kernels. The densification parameters can be set empirically based on the importance of the region, such as setting a high ratio or a low threshold for important regions, thereby ensuring that all necessary Gaussian kernels have the opportunity to undergo densification processing.

[0077] Check the size attribute of the target Gaussian kernel. If the scale of the target Gaussian kernel exceeds the threshold (that is, it indicates over-reconstruction, which may cover too large an area and cause blurring), split it into two new Gaussian kernels: the original Gaussian kernel attributes are copied to the two child kernels, and their positions and sizes are adjusted to evenly distribute the original coverage area; the new Gaussian kernels inherit other attributes of the original Gaussian kernel, such as color and transparency.

[0078] If the scale of the target Gaussian kernel does not exceed the threshold (indicating under-reconstruction, which may result in insufficient density and loss of detail), then all its attributes are copied to generate a new Gaussian kernel, and the position (i.e., the 3D position coordinates) of the new Gaussian kernel is slightly moved according to the position gradient vector (direction and magnitude) of the new Gaussian kernel to fill the error region indicated by the gradient; the moving distance can be set as needed (e.g., moving within a small range based on the gradient).

[0079] Based on the above-mentioned publicly available solutions, precise control of Gaussian kernel density is achieved through parameter selection and scale-adaptive densification methods (such as splitting or copying and moving), which solves the problem of insufficient gradient in existing technologies that prevents important regions from being densified, thereby improving the uniformity of reconstruction and image quality.

[0080] In one or more embodiments of this disclosure, such as Figure 5 This is a flowchart illustrating the region division method provided in an embodiment of this disclosure. Figure 5 As shown, step 102, which involves dividing the initial 3D point cloud into regions to obtain at least one 3D region, includes: Step 1021: Extracting features from the initial 3D point cloud using a trained 3D model to obtain semantic feature vectors. Step 1022: Determining the bounding boxes corresponding to each feature. Step 1023: Constructing the 3D region based on the feature bounding boxes and the corresponding semantic feature vectors.

[0081] An initial 3D point cloud is fed into a deep learning-based 3D model. This model extracts semantic feature vectors from the point cloud using convolutional neural networks or other feature extraction layers. These vectors capture the geometric and semantic information (such as shape, location, and category) of different objects in the point cloud. The 3D model then calculates and determines the bounding box (i.e., 3D bounding box) for each feature based on the extracted feature vectors. The bounding box defines the object's extent in 3D space. Finally, the bounding boxes are combined with the corresponding semantic feature vectors to construct multiple 3D regions, each associated with a label representing the feature type (such as "table" or "curtain"). For example, in a real estate reconstruction scenario, the 3D model can automatically identify coffee tables and desks in a room and output corresponding 3D regions to support subsequent independent densification.

[0082] It should be noted that the 3D model mentioned here is pre-trained. The specific training process is as follows: Point cloud object detection network structures, such as VoteNet, 3DETR, or PointRCNN, can be used as the 3D model. The training samples can be labeled point cloud datasets. Each training sample includes an initial 3D point cloud of a scene (generated from multi-view images using SfM technology) and multiple corresponding 3D bounding boxes and semantic labels. The training samples are input into the model to be trained to obtain the 3D model. The hierarchical structure of this 3D model varies depending on the type of model selected by the user.

[0083] Besides using 3D models for region division, regions can also be divided manually. For example, open-source interactive annotation tools (such as LabelMe 3D or similar software) can be used. Operators can manually draw or adjust the boundaries of 3D bounding boxes (such as minimum / maximum x, y, z coordinates) in a graphical visualization of the initial 3D point cloud, and optionally assign labels representing feature types to each bounding box. Upon completion, the tool outputs multiple 3D regions, each corresponding to a manually defined bounding box.

[0084] Based on the above-mentioned publicly available solutions, it is known that by using a trained 3D model to achieve region division, the reliance on manual annotation is reduced, and the efficiency and consistency of 3D region acquisition are improved. At the same time, the semantic feature vectors and bounding boxes output by the 3D model can directly provide a basis for setting the densification parameters of the specified 3D region (for example, the importance can be judged based on semantic features), further enhancing the refinement and intelligence of densification.

[0085] In one or more embodiments of this disclosure, such as Figure 6 This is a flowchart illustrating the attribution determination method provided in an embodiment of this disclosure. Figure 6 As shown, step 103, determining the affiliation between the first Gaussian kernel and the three-dimensional region, includes: Step 1031: Determining the first three-dimensional position coordinates of each first Gaussian kernel. Step 1032: Determining whether the first three-dimensional position coordinates are within the bounding box of any three-dimensional region. Step 1033: If so, determining that the first Gaussian kernel belongs to the corresponding three-dimensional region.

[0086] The 3D position coordinates of the first Gaussian kernel are one of the Gaussian properties. During initialization, they originate from the point positions in the initial 3D point cloud. Subsequently, in each iteration of reconstruction training, the position gradients are updated and optimized through backpropagation. The bounding box is a stereo bounding box; for example, it can be represented by six scalar values ​​(min_x, min_y, min_z, max_x, max_y, max_z). For the coordinates (x, y, z) of a first Gaussian kernel, if the following relationships are simultaneously satisfied: min_x ≤ x ≤ max_x, min_y ≤ y ≤ max_y, min_z ≤ z ≤ max_z, then the coordinates are considered to be inside the bounding box. If any one or more of these relationships are not satisfied, the first Gaussian kernel is considered not to belong to this 3D region. If a first Gaussian kernel falls into multiple overlapping bounding boxes, it can belong to multiple 3D regions simultaneously, and is processed independently according to each region during subsequent densification.

[0087] Based on the publicly available solutions described above, the relationship between the Gaussian kernel and the 3D region can be accurately determined using coordinate relationships. This achieves spatial isolation of the Gaussian kernel. After establishing the relationship, the Gaussian kernels of different 3D regions can be sorted and densed separately within groups, which is beneficial for improving the rendering effect of subsequent images.

[0088] In one or more embodiments of this disclosure, such as Figure 7 This is a flowchart illustrating the attribution relationship update method provided in an embodiment of this disclosure. Figure 7 As shown, after calculating the position gradient of each first Gaussian kernel in the view space using the backpropagation algorithm, the method further includes: Step 701: In iterative training, backpropagation is performed to calculate the second three-dimensional position coordinates of each first Gaussian kernel. Step 702: It is determined whether the second three-dimensional coordinates and the first three-dimensional coordinates are within the bounding box of the same three-dimensional region. Step 703: If not, the affiliation between the first Gaussian kernel and the three-dimensional region is redefined.

[0089] In each reconstruction training iteration, not only is the position gradient in view space calculated, but this gradient is also used to update the parameters of the Gaussian kernel (i.e., the parameters in the Gaussian properties mentioned earlier) through an optimizer (e.g., Adam). The 3D position parameters are adjusted by a certain step size along the negative gradient direction to obtain new second 3D position coordinates. The newly obtained 3D position coordinates directly replace the original 3D position coordinates, becoming the current 3D position attribute of the Gaussian kernel. For example, in the reconstruction of a room in a house, the first Gaussian kernel, initially located at the edge of a coffee table, may shift slightly towards the center of the coffee table due to gradient guidance, and the second 3D position coordinates will change accordingly.

[0090] It should be noted that before each densification operation (usually executed at fixed intervals), the first 3D position coordinates at the time of the previous attribution determination are recorded or cached. Then, the updated second 3D position coordinates are compared with the bounding boxes of all 3D regions for inclusion testing (i.e., checking whether min_x ≤ x ≤ max_x, min_y ≤ y ≤ max_y, min_z ≤ z ≤ max_z). The attribution relationship between the old and new coordinates is compared. If the second 3D position coordinates still fall within the same bounding box, no adjustment is needed; otherwise, the attribution relationship between the Gaussian kernel and the 3D regions is updated.

[0091] Based on the above-mentioned publicly available scheme, the attribution relationship is dynamically updated according to the changes in the three-dimensional coordinate position of the Gaussian kernel during the iterative training process. This ensures that the grouping of the three-dimensional region always reflects the latest spatial distribution, improves the independence and accuracy of Gaussian kernel densification processing, and avoids the impact of positional changes on the final rendering effect due to incorrect attribution.

[0092] In one or more embodiments of this disclosure, such as Figure 8This is a flowchart illustrating the initial 3D point cloud acquisition method provided in an embodiment of this disclosure. Figure 8 As shown, step 101, obtaining the initial 3D point cloud from the original image processing and determining the first Gaussian kernel for each point in the initial 3D point cloud, includes: Step 1011: Obtaining a set of original images captured from multiple perspectives. Step 1012: Processing the original images using Structured Motion Recovery (SfM) technology to generate the initial 3D point cloud. Step 1013: Assigning Gaussian attributes to each point in the initial 3D point cloud to generate the first Gaussian kernel; wherein, the Gaussian attributes include at least one of: 3D position coordinates, rotation, opacity, scale, and color.

[0093] During the actual raw image acquisition phase, a series of photos are taken from multiple angles around the scene using a camera, ensuring sufficient overlap between adjacent images (e.g., greater than 60%) to facilitate feature matching between subsequent adjacent images. This set can contain, for example, dozens to hundreds of raw images, covering the entire scene.

[0094] Feature points (such as SIFT or ORB features) are extracted from each original image; then, corresponding feature point pairs are found between the original images. After feature point matching, the relative camera pose can be estimated using the essential matrix or fundamental matrix, and the 3D point coordinates can be initially recovered through triangulation; finally, bundle adjustment is used to globally optimize all camera poses and 3D point coordinates to obtain a sparse but accurate initial 3D point cloud. This point cloud typically contains tens of thousands to hundreds of thousands of points.

[0095] For each point in the initial 3D point cloud, its 3D coordinates are directly used as the center of the Gaussian kernel. Simultaneously, other Gaussian properties need to be initialized. For example, rotation can be set to a unit quaternion (no rotation), opacity to a relatively high value close to 1, size to a small initial value calculated based on the distance to neighboring points, and color can be obtained by backprojecting to the original image and taking the average pixel value, or initially set to grayscale. After initialization, the first set of Gaussian kernels is obtained.

[0096] Based on the publicly available solutions described above, processing the original image to obtain an initial 3D point cloud and assigning it Gaussian properties provides a foundation for subsequent region segmentation and classification. This is beneficial for improving image rendering effects.

[0097] In one or more embodiments of this disclosure, such as Figure 9 This is a schematic flowchart illustrating the method for determining density parameters provided in an embodiment of this disclosure. Figure 9As shown, after constructing the 3D region based on the feature bounding box and the corresponding semantic feature vector, the process further includes: Step 901: Determining the importance level of the 3D region according to the feature type. Step 902: Determining the densification parameters of the 3D region according to the importance level. Step 903: If the importance level of the 3D region is high, then the densification parameters of the 3D region are set to: high densification ratio and / or low densification threshold. Step 904: If the importance level of the 3D region is low, then the densification parameters of the 3D region are set to: low densification ratio and / or high densification threshold.

[0098] In practical applications, 3D models provide corresponding semantic feature vectors along with bounding boxes. These semantic feature vectors can then be used for classification (using classification models) to determine the corresponding feature types. Next, the importance level is determined based on the feature type. For example, features of furniture type (such as "coffee table," "desk," "bed") that users are interested in are marked as high-level, while features of decorative or texture-assisted types (such as "curtains," "carpet," "towel") are marked as low-level.

[0099] It should be noted that importance is positively correlated with the density ratio (higher importance corresponds to a higher ratio) and negatively correlated with the density threshold (higher importance corresponds to a lower threshold).

[0100] Specifically, for high-level regions, the densification ratio can be set to a higher value (e.g., the top 30%-50% of high-gradient kernels), and the densification threshold can be set to a lower value (e.g., smaller gradients can be selected), thereby enhancing the Gaussian kernel density in that region. It should be noted that in practical applications, the ratio and threshold can be set to be used in combination, or either one can be selected (e.g., selecting only the ratio, or selecting only the threshold).

[0101] For low-level areas, the density ratio is set to a lower value (e.g., the top 10%), and the density threshold is set to a higher value (requiring a larger gradient to be selected), thereby reducing over-densification. It should be noted that in practical applications, the ratio and threshold can be set to be used in combination, or either one can be selected (e.g., selecting only the ratio, or selecting only the threshold).

[0102] Based on the aforementioned publicly available solutions, semantic feature vectors in 3D regions can be used to achieve rapid classification of feature types, thereby determining the importance level of the 3D region based on the feature type. This allows for differentiated settings of density parameters for different 3D regions. Even in complex scenes with uneven texture distribution, this ensures that important regions receive sufficient detail reconstruction, while secondary regions do not excessively consume resources, significantly improving overall reconstruction balance and rendered image quality.

[0103] Based on any of the above embodiments, this disclosure also provides an image rendering apparatus. Figure 10 This is a schematic block diagram of an image rendering apparatus according to one embodiment of the present disclosure. Figure 10 As shown, the image rendering apparatus includes: an acquisition module 1001, used to acquire an initial three-dimensional point cloud obtained by processing the original image; a segmentation module 1002, used to determine a first Gaussian kernel for each point in the initial three-dimensional point cloud; and to segment the initial three-dimensional point cloud into regions to obtain at least one three-dimensional region; a determination module 1003, used to determine the affiliation relationship between the first Gaussian kernel and the three-dimensional region; a processing module 1004, used to perform iterative densification processing on the first Gaussian kernel belonging to the three-dimensional region based on the affiliation relationship and using the densification parameters corresponding to the three-dimensional region to obtain a second Gaussian kernel; and a reconstruction module 1005, used to reconstruct the image using the second Gaussian kernel to obtain a rendered image.

[0104] The processing module 1004 is configured to, in response to a reconstruction training request, calculate the position gradient through backpropagation; create at least one three-dimensional region grouping based on attribution relationships; wherein each three-dimensional region grouping contains at least one first Gaussian kernel belonging to the three-dimensional region; determine the position gradient corresponding to the first Gaussian kernel in each three-dimensional region grouping; sort the first Gaussian kernels in the three-dimensional region grouping according to the magnitude of the position gradient; and, based on the sorting result, perform denser iterative processing on the first Gaussian kernels belonging to the three-dimensional region using the denser parameters of the three-dimensional region to obtain a second Gaussian kernel.

[0105] The processing module 1004 is used to select the current camera pose, project the first Gaussian kernel onto a two-dimensional plane and perform rasterization processing to obtain a rendered image under the current camera pose; calculate the error between the rendered image and the original image corresponding to the current camera pose; and calculate the position gradient of each first Gaussian kernel in the view space based on the error through the backpropagation algorithm.

[0106] The processing module 1004 is used to determine the target Gaussian kernel to be densified from the sorted first Gaussian kernels according to the densification parameters set for the three-dimensional region, for the first Gaussian kernel in each group; when the scale of the target Gaussian kernel is greater than a preset scale threshold, the target Gaussian kernel is split to obtain a second Gaussian kernel; or, when the scale of the target Gaussian kernel is less than or equal to the preset scale threshold, the target Gaussian kernel is copied and moved along the position gradient direction to generate a second Gaussian kernel.

[0107] The partitioning module 1002 is used to extract features from the initial 3D point cloud using the trained 3D model to obtain semantic feature vectors; determine the bounding boxes corresponding to each feature; and construct a 3D region based on the bounding boxes and corresponding semantic feature vectors.

[0108] The determination module 1003 is used to determine the first three-dimensional position coordinates of each first Gaussian kernel; determine whether the first three-dimensional position coordinates are within the bounding box of any three-dimensional region; if so, determine that the first Gaussian kernel belongs to the corresponding three-dimensional region.

[0109] The determination module 1003 is used to perform backpropagation calculations during iterative training to obtain the second three-dimensional position coordinates of each first Gaussian kernel; determine whether the second three-dimensional coordinates and the first three-dimensional coordinates are within the bounding box of the same three-dimensional region; if not, redetermine the attribution relationship between the first Gaussian kernel and the three-dimensional region.

[0110] The acquisition module 1001 is used to acquire a set of original images captured from multiple perspectives; process the original images using structural motion recovery technology to generate an initial three-dimensional point cloud; assign Gaussian attributes to each point in the initial three-dimensional point cloud to generate a first Gaussian kernel; wherein, the Gaussian attributes include at least one of the following: three-dimensional position coordinates, opacity, scale, and color.

[0111] The determination module 1003 is used to determine the importance level of the three-dimensional region based on the feature type; and to determine the densification parameters of the three-dimensional region based on the importance level. If the importance level of the three-dimensional region is high, the densification parameters of the three-dimensional region are set to: high densification ratio and / or low densification threshold. If the importance level of the three-dimensional region is low, the densification parameters of the three-dimensional region are set to: low densification ratio and / or high densification threshold.

[0112] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0113] The execution subject of the image rendering method in the specific embodiments of this disclosure can be an electronic device such as a server (including a local server or a cloud computing platform).

[0114] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the image rendering method of any of the embodiments described above.

[0115] Figure 11 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0116] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0117] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0118] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0119] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0120] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0121] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0127] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. An image rendering method, characterized in that, The method includes: Obtain the initial 3D point cloud obtained by processing the original image; Determine the first Gaussian kernel for each point in the initial three-dimensional point cloud; and divide the initial three-dimensional point cloud into regions to obtain at least one three-dimensional region; Determine the attribution relationship between the first Gaussian kernel and the three-dimensional region; Based on the attribution relationship, the first Gaussian kernel belonging to the three-dimensional region is subjected to a densification iteration process using the densification parameter corresponding to the three-dimensional region to obtain the second Gaussian kernel; The rendered image is obtained by reconstructing the image using the second Gaussian kernel.

2. The image rendering method according to claim 1, characterized in that, Based on the attribution relationship, the first Gaussian kernel belonging to the three-dimensional region is subjected to denser iteration processing using the denser parameters corresponding to the three-dimensional region to obtain the second Gaussian kernel, including: In response to the request to rebuild the training, the position gradient is calculated via backpropagation; Based on the attribution relationship, at least one three-dimensional region group is created; wherein each three-dimensional region group contains at least one first Gaussian kernel belonging to the three-dimensional region; Determine the position gradient corresponding to the first Gaussian kernel in each of the three-dimensional region groups; The first Gaussian kernel in the three-dimensional region grouping is sorted according to the magnitude of the position gradient; Based on the sorting results, the first Gaussian kernel belonging to the three-dimensional region is subjected to a densification iteration process using the densification parameter of the three-dimensional region to obtain the second Gaussian kernel.

3. The image rendering method according to claim 2, characterized in that, The step of calculating the position gradient via backpropagation in response to a reconstruction training request includes: Select the current camera pose, project the first Gaussian kernel onto a two-dimensional plane and perform rasterization to obtain the rendered image under the current camera pose; Calculate the error between the rendered image and the original image corresponding to the current camera pose; Based on the error, the position gradient of each first Gaussian kernel in the view space is calculated using the backpropagation algorithm.

4. The image rendering method according to claim 2, characterized in that, Based on the sorting result, the first Gaussian kernel within the three-dimensional region is densified using the densification parameter corresponding to the three-dimensional region to obtain the second Gaussian kernel, including: Based on the densification parameters set for the three-dimensional region, the target Gaussian kernel to be densified is determined from the sorted first Gaussian kernel; When the scale of the target Gaussian kernel is larger than a preset scale threshold, the target Gaussian kernel is split to obtain the second Gaussian kernel; or, When the scale of the target Gaussian kernel is less than or equal to the preset scale threshold, the target Gaussian kernel is copied and moved along the position gradient direction to generate the second Gaussian kernel.

5. The image rendering method according to claim 2, characterized in that, The process of dividing the initial 3D point cloud into regions to obtain at least one 3D region includes: Features are extracted from the initial 3D point cloud using a trained 3D model to obtain semantic feature vectors; bounding boxes corresponding to each feature are determined; and the 3D region is constructed based on the bounding boxes and the corresponding semantic feature vectors. Optionally, determining the affiliation relationship between the first Gaussian kernel and the three-dimensional region includes: determining the first three-dimensional position coordinates of each first Gaussian kernel; determining whether the first three-dimensional position coordinates are within the bounding box of any of the three-dimensional regions; if so, determining that the first Gaussian kernel belongs to the corresponding three-dimensional region. Optionally, after calculating the position gradient of each first Gaussian kernel in the view space using the backpropagation algorithm, the method further includes: in iterative training, performing backpropagation to calculate the second three-dimensional position coordinates of each first Gaussian kernel; determining whether the second three-dimensional coordinates and the first three-dimensional coordinates are located within the bounding box of the same three-dimensional region; if not, re-determining the affiliation relationship between the first Gaussian kernel and the three-dimensional region.

6. The image rendering method according to claim 1, characterized in that, The initial three-dimensional point cloud is obtained by processing the original image. Determining the first Gaussian kernel for each point in the initial 3D point cloud includes: Acquire a collection of raw images taken from multiple perspectives; The original image is processed using a structural motion recovery algorithm to generate the initial 3D point cloud; Assign Gaussian properties to each point in the initial 3D point cloud to generate the first Gaussian kernel; wherein the Gaussian properties include at least one of: 3D position coordinates, opacity, scale, and color.

7. The image rendering method according to claim 5, characterized in that, After constructing the 3D region based on the bounding box and the corresponding semantic feature vector of the aforementioned features, the method further includes: The importance level of the three-dimensional region is determined based on the feature type of the feature. Based on the importance level, the densification parameters of the three-dimensional region are determined; If the importance level of the three-dimensional region is high, then the density parameter of the three-dimensional region is set as: high density ratio and / or low density threshold; If the importance level of the three-dimensional region is low, then the density parameter of the three-dimensional region is set as: low density ratio and / or high density threshold.

8. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 7.

9. A readable storage medium, characterized in that, The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images