Image rendering method, electronic device, readable medium and program product

Abstract

The present disclosure provides an image rendering method, an electronic device, a readable medium and a program product. The method of the present disclosure comprises: obtaining an initial three-dimensional point cloud processed from an original image; extracting edge points representing geometric contours in the original image from the initial three-dimensional point cloud; performing dense processing on a first Gaussian kernel of corresponding edge points in the initial three-dimensional point cloud by using a preset weighting coefficient to obtain a second Gaussian kernel; and performing image reconstruction by using the second Gaussian kernel to obtain a rendered image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to image rendering methods, electronic devices, readable media, and program products. Background Technology

[0002] With the development of Mixed Reality (MR) technology, it has been widely used in more and more scenarios, such as using devices that support mixed reality technology to view houses.

[0003] To display house images in mixed reality technology, 3D Gaussian Splatting (3DGS) reconstruction is required. During reconstruction training, a densification operation is typically triggered every certain number of iterations. This involves splitting or cloning certain Gaussian kernels based on whether the accumulated gradient exceeds a set threshold to increase model detail. However, existing techniques often rely on global densification, using a uniform global threshold, resulting in poor densification in key areas. For example, geometric edges may appear blurred, and structures may lack clarity. Furthermore, flat areas (such as carpets and walls) are prone to over-densification due to concentrated texture gradients, consuming a large number of Gaussian kernels. Ultimately, this leads to poor overall densification. Therefore, a solution is needed to improve the image rendering quality of 3DGS reconstruction technology. Summary of the Invention

[0004] This disclosure provides image rendering methods, electronic devices, readable media, and program products.

[0005] According to a first aspect of this disclosure, an image rendering method is provided. The method specifically includes: acquiring an initial three-dimensional point cloud obtained by processing an original image; extracting edge points representing geometric contours in the original image from the initial three-dimensional point cloud; densifying a first Gaussian kernel of the corresponding edge points in the initial three-dimensional point cloud using preset weighting coefficients to obtain a second Gaussian kernel; and reconstructing the image using the second Gaussian kernel to obtain a rendered image.

[0006] According to at least one embodiment of the present disclosure, extracting edge points representing geometric contours in an original image from an initial three-dimensional point cloud includes: using a preset algorithm to detect points in the initial three-dimensional point cloud that do not have geometric continuity as edge points; and assigning edge point identifiers to the edge points in the initial three-dimensional point cloud.

[0007] According to at least one embodiment of this disclosure, a first Gaussian kernel corresponding to edge points in an initial 3D point cloud is densified using a preset weighting coefficient to obtain a second Gaussian kernel. This includes: assigning a Gaussian attribute to each point in the initial 3D point cloud to obtain a first Gaussian kernel corresponding to each point; assigning edge point identifiers to the first Gaussian kernels corresponding to edge points; during training iterations, triggering densification processing according to a preset number of iterations, and calculating the cumulative gradient for all first Gaussian kernels; if a first Gaussian kernel is assigned an edge point identifier, enhancing the cumulative gradient of the first Gaussian kernel using the weighting coefficients; and performing a splitting or cloning operation on the first Gaussian kernel based on the enhanced cumulative gradient and a preset gradient threshold to obtain an enhanced second Gaussian kernel.

[0008] According to at least one embodiment of this disclosure, a first Gaussian kernel is split or cloned based on the enhanced cumulative gradient and a preset gradient threshold to obtain an enhanced second Gaussian kernel, including: sorting the enhanced cumulative gradient; selecting a first Gaussian kernel with a gradient greater than the gradient threshold from the sorting results for splitting or cloning to obtain an enhanced second Gaussian kernel; and assigning edge point identifiers to the enhanced second Gaussian kernel.

[0009] According to at least one embodiment of this disclosure, if the first Gaussian kernel is not assigned an edge point identifier, the cumulative gradient is weakened using a preset weakening coefficient; and the first Gaussian kernel is split or cloned based on the comparison result between the cumulative gradient obtained after the weakening process and the gradient threshold to obtain a second Gaussian kernel.

[0010] According to at least one embodiment of this disclosure, a preset algorithm is used to detect points in an initial three-dimensional point cloud that do not have geometric continuity as edge points, including: when the preset algorithm is a normal vector change rate algorithm, calculating the normal vectors of adjacent points in the initial three-dimensional point cloud; calculating the change rate between the normal vectors of adjacent points; if the change rate exceeds a preset change rate threshold, then determining that the adjacent points are edge points that do not have geometric continuity.

[0011] According to at least one embodiment of this disclosure, a preset algorithm is used to detect points in an initial three-dimensional point cloud that do not have geometric continuity as edge points, including: when the preset algorithm is a depth difference algorithm, converting the initial three-dimensional point cloud into a depth map; calculating the depth value difference between pixels of adjacent points; if the depth value difference exceeds a preset difference threshold, then determining that the adjacent points are edge points that do not have geometric continuity.

[0012] According to at least one embodiment of this disclosure, a preset algorithm is used to detect points in an initial three-dimensional point cloud that do not have geometric continuity as edge points, including: when the preset algorithm is a point cloud edge detection method, converting the initial three-dimensional point cloud into a discrete geometric signal; analyzing the spatial distribution and geometric features of adjacent points, and if local geometric discontinuities are detected, determining that the adjacent points are edge points that do not have geometric continuity.

[0013] According to at least one embodiment of this disclosure, image reconstruction using a second Gaussian kernel to obtain a rendered image includes: selecting an arbitrary camera pose and projecting the second Gaussian kernel onto a two-dimensional plane; performing rasterization processing to generate a rendered image from a corresponding viewpoint for use in a mixed reality virtual house viewing scene.

[0014] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs a first aspect of any embodiment of this disclosure.

[0015] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement a first aspect of any embodiment of this disclosure.

[0016] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a first aspect of any embodiment of this disclosure. Attached Figure Description

[0017] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0018] Figure 1 This is a schematic flowchart of an image rendering method provided in an embodiment of the present disclosure.

[0019] Figure 2 This is a flowchart illustrating the edge point extraction method provided in an embodiment of this disclosure.

[0020] Figure 3 This is a schematic flowchart of the edge point densification processing method provided in the embodiments of this disclosure.

[0021] Figure 4 This is a flowchart illustrating the edge point identifier inheritance method provided in an embodiment of the present disclosure.

[0022] Figure 5This is a schematic flowchart of a method for extracting edge points using the rate of change of the normal vector, provided in an embodiment of this disclosure.

[0023] Figure 6 This is a schematic flowchart of a method for extracting edge points using a depth difference algorithm, provided in an embodiment of this disclosure.

[0024] Figure 7 This is a schematic flowchart of a method for extracting edge points using a point cloud edge detection algorithm, provided in an embodiment of this disclosure.

[0025] Figure 8 This is a schematic flowchart illustrating an image rendering method as provided in an embodiment of the present disclosure.

[0026] Figure 9 This is a schematic block diagram of an image rendering apparatus according to one embodiment of the present disclosure.

[0027] Figure 10 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0028] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0029] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] Figure 1 This is a schematic flowchart illustrating the image rendering method provided in an embodiment of this disclosure. Figure 1 The method shown includes steps S101 to S104. This method can be executed by a computer or a server (e.g., a local server, a cloud server, etc.).

[0031] Specifically, Figure 1 The method shown includes step S101: obtaining an initial three-dimensional point cloud obtained by processing the original image.

[0032] It should be noted that the original image mentioned here refers to the input image data used to generate the 3D model, such as a set of 2D images obtained by a camera from multiple perspectives. The initial 3D point cloud mentioned here refers to the discrete set of 3D points obtained after processing the original image. These points contain position coordinates and possible color information, which can be used to represent the geometric contours within the scene.

[0033] In practical applications, for example, the initial 3D point cloud is a sparse 3D point set generated by processing the original image (e.g., using Structure from Motion (SfM) technique). Each point has 3D spatial coordinates and can contain tens of thousands to hundreds of thousands of points.

[0034] Multiple original images can be processed using techniques such as structural motion recovery or Neural Radiance Fields (NeRF) to reconstruct an initial 3D point cloud. This point cloud contains 3D point data of the scene. For example, in a mixed reality (MR) house viewing scene, the point cloud can not only represent the interior of the house, but also the geometric structure of the property, such as wall boundaries and furniture outlines.

[0035] Step S102: Extract edge points representing the geometric contours in the original image from the initial 3D point cloud.

[0036] The edge points of the geometric contours mentioned here refer to the points in the initial 3D point cloud that represent the edges of all contours in the original image, including geometric discontinuities of objects in space such as sofa armrests, cushion edges, doors and windows, tables and chairs, and corners. These points are extracted by detecting geometric discontinuities (such as abrupt changes in normal vectors or depth differences).

[0037] In practical applications, a preset algorithm is used to detect points in the initial 3D point cloud that do not have geometric continuity as edge points. Then, in the subsequent image reconstruction and rendering process, edge points and non-edge points are distinguished and processed to improve the edge clarity of the rendered image.

[0038] Step S103: Use preset weighting coefficients to densify the first Gaussian kernel of the corresponding edge points in the initial 3D point cloud to obtain the second Gaussian kernel.

[0039] The first Gaussian kernel mentioned here refers to the initial Gaussian kernel after converting all points (including edge points and non-edge points) in the initial 3D point cloud into Gaussian representations. Each Gaussian kernel carries attributes such as position, orientation, color, transparency, and scale. The second Gaussian kernel mentioned here refers to the new Gaussian kernel generated by splitting or cloning after multiple rounds of denser processing. Some of these newly generated second Gaussian kernels can inherit the attributes of edge points (that is, they completely inherit the edge point identifiers).

[0040] In practical applications, the initial 3D point cloud is transformed into a first Gaussian kernel, with each point (including edge points and non-edge points) corresponding to a Gaussian kernel. During training iterations, a densification process is triggered every certain number of iterations (e.g., 100 times, which can be set by the user as needed in practical applications), calculating the cumulative gradient for all first Gaussian kernels. To effectively improve edge rendering and display effects, differentiated densification processing can be performed on different points. For example, the cumulative gradient of the first Gaussian kernel corresponding to edge points can be enhanced using a preset weighting coefficient (e.g., set to 2; generally, this weighting coefficient should be greater than 1 so that the weighted cumulative gradient is larger than the original cumulative gradient). In an optional scheme, the cumulative gradient of the first Gaussian kernel corresponding to non-edge points can be weakened. This allows for priority densification of edge points, avoiding excessive resource consumption in flat areas (e.g., walls).

[0041] Step S104: Reconstruct the image using the second Gaussian kernel to obtain the rendered image.

[0042] After multiple iterations and densification, the final Gaussian model is obtained. A second Gaussian kernel set is then used for projection and rasterization based on the camera pose to calculate the rendered image. This step outputs a rendered image with higher resolution in geometrically edge regions.

[0043] Based on the publicly available solutions, prioritizing the densification of edge points improves the clarity of geometric edge reconstruction; optimizes resource allocation without increasing overall computational overhead, reducing redundant densification in non-edge areas; and improves the overall rendering effect, especially in the edges such as doors, windows, tables, chairs, and corners, providing a stronger sense of immersion and space for MR house viewing scenarios.

[0044] In one or more embodiments of this disclosure, such as Figure 2 This is a schematic flowchart illustrating the edge point extraction method provided in an embodiment of this disclosure. Figure 2 As shown, step S102, which involves extracting edge points representing the geometric contours of the original image from the initial 3D point cloud, includes: step S1021, using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points; and step S1022, assigning edge point identifiers to the edge points in the initial 3D point cloud.

[0045] The preset algorithm mentioned here can be any algorithm used to detect geometric discontinuities in the initial 3D point cloud, such as the normal vector change rate algorithm, depth difference algorithm, or point cloud edge detection method. Edge points are selected from the initial 3D point cloud using the threshold corresponding to the algorithm. Geometric continuity can refer to the degree of smooth and continuous change in the normal vector, depth value, or spatial distribution of adjacent points in the point cloud. The lack of geometric continuity indicates that abrupt changes are edge points, such as geometric corners, edges, or object boundaries.

[0046] In practical applications, a suitable preset algorithm is selected to analyze the initial 3D point cloud to identify the edge points of the geometric contours. This step ensures that the contour edges of various objects in the original image are extracted, including sofa armrests, cushion edges, etc. Furthermore, corresponding thresholds can be set for different algorithms to avoid mistakenly extracting gently changing areas such as curtain folds, ensuring accurate transmission of edge point information.

[0047] After extracting edge points, to facilitate continuous optimization of the Gaussian kernel for these points in subsequent iterations, edge point identifiers can be assigned. These identifiers allow newly added Gaussian kernels for edge points to inherit their type attributes (i.e., distinguishing whether a point is an edge by the identifier). For example, boolean variables (e.g., true for edge points, false for non-edge points) or integer labels (e.g., 1 for edge points, 0 for non-edge points) can be stored in the point cloud data structure. This step is performed immediately after extraction to ensure that edge attributes are carried over to subsequent steps (such as adding identifiers when converting the initial 3D point cloud into the first Gaussian kernel), avoiding edge blurring issues caused by global threshold limitations.

[0048] Based on the publicly available scheme, it is known that by accurately detecting points lacking geometric continuity through a pre-set algorithm and assigning edge point labels, the accuracy of geometric edge point extraction is improved. The edge point labels are used to achieve continuity between edge points and the newly added Gaussian kernel throughout the iterative training process.

[0049] In one or more embodiments of this disclosure, such as Figure 3 This is a schematic flowchart illustrating the edge point densification method provided in an embodiment of this disclosure. Figure 3As shown, step S103, which involves densifying the first Gaussian kernel corresponding to the edge points in the initial 3D point cloud using preset weighting coefficients to obtain the second Gaussian kernel, includes: Step S1031: Assigning Gaussian attributes to each point in the initial 3D point cloud to obtain the first Gaussian kernel corresponding to each point, and assigning edge point identifiers to the first Gaussian kernels corresponding to edge points. Step S1032: During training iterations, triggering densification processing according to a preset number of iterations, and calculating the cumulative gradient for all first Gaussian kernels. Step S1033: If the first Gaussian kernel is assigned an edge point identifier, then enhancing the cumulative gradient of the first Gaussian kernel using weighting coefficients. Step S1034: Based on the enhanced cumulative gradient and a preset gradient threshold, performing a splitting or cloning operation on the first Gaussian kernel to obtain the enhanced second Gaussian kernel.

[0050] In practical applications, after extracting and identifying edge points, the initial 3D point cloud is assigned Gaussian attributes. These Gaussian attributes refer to the intrinsic parameters of the Gaussian kernel, including position, orientation, color, transparency, and scale, used to represent scene points. Each point (including edge points and non-edge points) corresponds to a first Gaussian kernel. Furthermore, if it is an edge point, the first Gaussian kernel will be assigned a corresponding edge point identifier, for example, by appending an independent variable (e.g., a Boolean value of `true`) to the first Gaussian kernel corresponding to the edge point. This ensures that edge attributes are passed from the point cloud to the Gaussian kernel. When performing density processing on the Gaussian kernel, it is possible to distinguish which Gaussian kernels need to be processed using weighted coefficients, avoiding insufficient edge coverage caused by the existing global threshold.

[0051] During iterative training (e.g., following the 3DGS process), a densification process is triggered every preset number of iterations (e.g., 100 times). The gradient values ​​of each first Gaussian kernel are accumulated through backpropagation over multiple iterations. If the accumulated gradient is large, it indicates that the Gaussian kernel needs further densification to improve model details, and more resources need to be invested in optimizing it during subsequent iterative training.

[0052] During the densification process, it is necessary to first determine whether the first Gaussian kernel to be processed has been assigned edge point markers. If edge point markers are present, a fixed weighting coefficient (e.g., 2, which users can set according to their needs or experience) is applied to amplify the accumulated gradient, thereby improving the densification effect at the edge points. For example, the original accumulated gradient of 0.08 is enhanced to 0.08. 2 = 0.16. By enhancing weighting, the cumulative gradient of the Gaussian kernel at edge points can be amplified in a targeted manner, making it easier to select for densification. This prioritizes resource allocation to edge regions, effectively addressing the problem of insufficient geometric edge optimization caused by the global threshold. The weighted cumulative gradient will have a higher probability of exceeding the preset gradient threshold, making it easier to select for splitting or cloning, ensuring that edge regions receive more Gaussian kernel support, while non-edge regions remain unchanged or weakened.

[0053] The enhanced cumulative gradients are sorted (e.g., in descending order). If the gradient exceeds a preset threshold (e.g., 0.1), a split (i.e., splitting into two Gaussian kernels to add more detail) or cloning (i.e., copying one Gaussian kernel into two to fill the region) is selected to generate an enhanced second Gaussian kernel that fully inherits the edge point identifiers. The gradient threshold here is used to select a subset of Gaussian kernels for densification, preventing indiscriminate densification of all Gaussian kernels, which could lead to resource waste or an excessively large model.

[0054] Based on the publicly available solutions, by assigning Gaussian attributes and edge point identifiers, the priority densification requirement for edge points is ensured to continue during the densification process, thus improving the clarity of geometric edge reconstruction. Furthermore, by using weighted coefficients to enhance the cumulative gradient, differentiated resource allocation is achieved, reducing redundant densification in non-edge areas. Without increasing overall computational overhead, the rendering quality of edges is optimized; for example, edges such as doors, windows, tables, chairs, and corners are rendered more realistically.

[0055] In one or more embodiments of this disclosure, such as Figure 4 This is a flowchart illustrating the edge point identifier inheritance method provided in an embodiment of this disclosure. Figure 4 As shown, step S1034, which involves splitting or cloning the first Gaussian kernel based on the enhanced cumulative gradient and a preset gradient threshold to obtain an enhanced second Gaussian kernel, includes: Step S401: Sorting the enhanced cumulative gradient. Step S402: Selecting the first Gaussian kernel with a gradient greater than the gradient threshold from the sorted results for splitting or cloning to obtain the enhanced second Gaussian kernel. Step S403: Assigning edge point identifiers to the enhanced second Gaussian kernel to ensure the continuity of edge attributes during the iteration process.

[0056] In practical applications, the enhanced cumulative gradients of all first Gaussian kernels are sorted in descending order, for example, using a sorting algorithm (such as quicksort) to organize the gradient list. Gaussian kernels with high contribution edge points are identified first. From the sorted gradient list, first Gaussian kernels exceeding a preset gradient threshold (e.g., 0.1) are selected, and they are split or cloned to generate enhanced second Gaussian kernels.

[0057] After a split or cloning operation, the edge point identifiers (e.g., boolean type true) of the parent first Gaussian kernel are inherited completely and without discrimination to the newly generated second Gaussian kernel, for example, through data structure copying. In multiple rounds of iterative training, by inheriting the edge point identifiers, the Gaussian kernel can effectively ensure that the Gaussian kernels of edge points are prioritized for optimization, ultimately resulting in clearer and more realistic boundary contours.

[0058] Based on the publicly available solutions described above, targeted densification is achieved by sorting and thresholding the accumulated gradients after enhancement, thus improving the clarity of geometric edges. Edge point identifiers are inherited not only by the corresponding Gaussian kernel but also by newly added Gaussian kernels, ensuring that edge attributes are effectively continued throughout the iterative optimization process. This optimizes resource allocation and reduces redundant densification in non-edge regions, enhancing the overall rendering effect of the model without increasing computational overhead.

[0059] In one or more embodiments of this disclosure, such as Figure 3 As shown, it also includes: Step S1035: If the first Gaussian kernel is not assigned an edge point identifier, then the accumulated gradient is weakened using a preset weakening coefficient. Step S1036: Based on the comparison result between the accumulated gradient obtained after the weakening process and the gradient threshold, the first Gaussian kernel is split or cloned to obtain a second Gaussian kernel, so as to avoid redundant densification of non-edge regions.

[0060] When densifying the first Gaussian kernel, in addition to enhancing the Gaussian kernels with edge point markers, a differentiated weakening strategy is adopted for the first Gaussian kernels without edge point markers to further optimize the allocation of computing resources. Specifically, after triggering the densification process and calculating the cumulative gradient for all first Gaussian kernels in step S1032, it is necessary to determine whether each first Gaussian kernel has been assigned an edge point marker. If the determination result is that the first Gaussian kernel has not been assigned an edge point marker, it means that the point corresponding to the first Gaussian kernel belongs to a non-edge point, such as a relatively flat geometric structure like a wall, ceiling, floor, or large textured area. In this case, the cumulative gradient of the first Gaussian kernel is not enhanced, but weakened using a preset weakening coefficient. The weakening coefficient here refers to a preset constant factor used to reduce the value of the cumulative gradient. The value of the weakening coefficient is less than 1, for example, it can be set to 0.5, 0.3, etc., and the specific value can be determined experimentally according to the actual application scenario. The specific operation of gradient reduction can be to multiply the original cumulative gradient by a reduction factor to obtain the reduced cumulative gradient. For example, if the original cumulative gradient of a non-edge Gaussian kernel is 0.12, and the reduction factor is set to 0.5, then the reduced cumulative gradient becomes 0.06 = 0.5. 0.12. The attenuation process can actively reduce the probability of Gaussian kernels in non-edge regions being selected for splitting or cloning during the densification process, thereby suppressing the proliferation of redundant Gaussian kernels in these regions.

[0061] After obtaining the weakened cumulative gradient, it is compared with a preset gradient threshold. It should be noted that the gradient threshold is a critical value used to select which Gaussian kernels need to undergo splitting or cloning operations. In practical applications, users can set it according to their specific needs; for example, it can be set to 0.1. Based on the comparison result, a decision is made on whether to perform a splitting or cloning operation on the first Gaussian kernel: If the weakened cumulative gradient is greater than the gradient threshold, it means that even after weakening, the contribution error of the first Gaussian kernel to the rendering result is still relatively significant, and it is still necessary to increase the number of Gaussian kernels to improve detail. In this case, a splitting or cloning operation is performed on the first Gaussian kernel to generate a new Gaussian kernel, which is the second Gaussian kernel. If the weakened cumulative gradient is less than or equal to the gradient threshold, it means that the contribution error of the first Gaussian kernel has been suppressed to a level that does not require further refinement. In this case, no splitting or cloning operation is performed, and the first Gaussian kernel is not densified.

[0062] By introducing a weakening coefficient to pre-weaken the cumulative gradient of non-edge Gaussian kernels, the gradient values ​​of non-edge Gaussian kernels in most flat regions are reduced to below the gradient threshold, thus avoiding unnecessary splitting or cloning of these regions. Correspondingly, if the first Gaussian kernel is found to have been assigned an edge point identifier during the judgment process, a corresponding enhancement method is adopted, that is, its cumulative gradient is amplified using a weighting coefficient greater than 1, making it more likely to exceed the gradient threshold and become denser. These two differentiated processing mechanisms are based on the same edge point identifier system, jointly achieving a biased allocation of Gaussian kernel resources.

[0063] It should be noted that existing technologies use a globally uniform gradient threshold to filter all Gaussian kernels during densification, without suppressing non-edge regions. In this case, flat regions, due to the presence of texture information such as wallpaper patterns, carpet fibers, and wood floor textures, may still generate high cumulative gradients under photometric error-driven conditions, despite their simple geometric structures. This leads to frequent splitting or cloning of these regions, generating a large number of Gaussian kernels that are not beneficial to the geometric representation, resulting in a serious waste of computational and storage resources.

[0064] The scheme disclosed herein, by pre-establishing an edge point identification system, can identify which Gaussian kernels are located at geometric edges and which are located in non-edge flat regions. This allows for the application of a weakening coefficient to non-edge Gaussian kernels individually, actively suppressing their tendency to become denser. By introducing a weakening coefficient to reduce the accumulated gradient of non-edge Gaussian kernels, this disclosure fundamentally solves the resource waste problem caused by excessive density in flat regions. It allows limited Gaussian kernel resources to be concentrated more in geometric edge regions, improving the clarity and realism of geometric structures in the rendered image without increasing overall computational overhead, thus enhancing the overall rendering quality.

[0065] Based on the publicly available solutions, by processing the accumulated gradient of non-edge Gaussian kernels with a preset weakening coefficient, the density probability is reduced, redundant density in flat areas is avoided, and resource allocation is optimized. Differential control between edge and non-edge regions is achieved, reducing overall Gaussian kernel consumption without increasing computational overhead, especially avoiding excessive detail in flat areas such as walls and carpets. This effectively reduces waste of computational and storage resources.

[0066] The method for extracting edge points will be explained in detail below through specific embodiments.

[0067] In one or more embodiments of this disclosure, such as Figure 5 This is a schematic flowchart illustrating a method for extracting edge points using changes in normal vectors, provided in an embodiment of this disclosure. Figure 5 As shown, step S1021, which involves using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points, includes: Step S501: When the preset algorithm is a normal vector change rate algorithm, calculate the normal vectors of adjacent points in the initial 3D point cloud. Step S502: Calculate the rate of change between the normal vectors of adjacent points. If the rate of change exceeds a preset rate of change threshold, then the adjacent points are determined to be edge points lacking geometric continuity.

[0068] In practical applications, a normal vector change rate algorithm is selected to perform local neighborhood analysis on the initial 3D point cloud. For example, the k-nearest neighbor (k-NN) method is used to estimate the normal vector for each point, and the surface normal is calculated using principal component analysis (PCA). Furthermore, the change rate of the normal vectors of neighboring points is calculated, for example, by using the dot product of vectors to calculate the cosine similarity (change rate = 1 - cosθ, where θ is the angle between the normal vectors of two neighboring points). If the change rate exceeds a preset threshold (assumed to be 0.5, this threshold can be set according to actual needs or experience), the neighboring points are determined to be edge points without geometric continuity. By setting a change rate threshold, true geometric edges (e.g., sharp edges, corners) and gentle transitions (e.g., curtain folds) can be distinguished, avoiding false extraction.

[0069] Based on the aforementioned publicly available scheme, the algorithm of normal vector change rate is used to accurately detect points that do not have geometric continuity, thereby improving the accuracy of geometric edge extraction.

[0070] In one or more embodiments of this disclosure, such as Figure 6 This is a schematic flowchart illustrating a method for extracting edge points using a depth difference algorithm, provided in an embodiment of this disclosure. Figure 6 As shown, step S1021, which involves using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points, includes: Step S601: When the preset algorithm is a depth difference algorithm, the initial 3D point cloud is converted into a depth map. Step S602: The depth value difference between pixels of adjacent points is calculated. If the depth value difference exceeds a preset difference threshold, the adjacent points are determined to be edge points that lack geometric continuity.

[0071] In practical applications, if a depth difference algorithm is chosen, the disordered point cloud is converted into an ordered depth map based on the camera pose, with each pixel storing its corresponding depth value. This essentially transforms the 3D point cloud data into a 2D representation that is easy to calculate using difference. Further, the depth map is traversed through adjacent pixels, and the depth value difference is calculated. For example, the Sobel operator (a method for calculating image differences that combines Gaussian smoothing and differentiation to detect edge features while reducing noise interference) or simple absolute difference (|di-dj|, where i and j represent two adjacent points) can be used. If the difference exceeds a preset threshold, it is determined to be an edge point lacking geometric continuity.

[0072] Based on the above-mentioned publicly available solutions, it is known that by using a preset algorithm to accurately detect points that do not have geometric continuity in the depth difference algorithm, the accuracy of geometric edge extraction is improved.

[0073] In one or more embodiments of this disclosure, such as Figure 7 This is a schematic flowchart illustrating a method for extracting edge points using a point cloud edge detection algorithm, provided in an embodiment of this disclosure. Figure 7 As shown, step S1021, which involves using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points, includes: Step 701: When the preset algorithm is a point cloud edge detection method, the initial 3D point cloud is converted into a discrete geometric signal. Step 702: The spatial distribution and geometric features of adjacent points are analyzed. If local geometric discontinuities are detected, the adjacent points are determined to be edge points that lack geometric continuity.

[0074] When the preset algorithm is a point cloud edge detection method, the initial 3D point cloud is converted into a discrete geometric signal. Further, the neighborhood corresponding to adjacent points is traversed, and spatial distribution (e.g., density variation) and geometric features (e.g., curvature or principal component anomalies) are calculated. If a local geometric discontinuity is detected (e.g., curvature exceeds a threshold of 0.2), it is determined to be an edge point without geometric continuity.

[0075] Based on the above-mentioned publicly available solutions, it is known that the point cloud edge detection method can accurately detect points that do not have geometric continuity, thereby improving the accuracy of geometric edge extraction.

[0076] It should be noted that the three embodiments above provide three edge point extraction algorithms. In practical applications, users can choose any one of these algorithms as needed, or they can choose other algorithms to extract edge points. This is only for illustrative purposes and does not constitute a limitation on the technical solutions disclosed herein.

[0077] In one or more embodiments of this disclosure, such as Figure 8 This is a schematic flowchart illustrating an image rendering method as provided in an embodiment of the present disclosure. Figure 8 As shown, step S104, which involves reconstructing the image using the second Gaussian kernel to obtain the rendered image, includes: Step S1041: Selecting an arbitrary camera pose and projecting the second Gaussian kernel onto a two-dimensional plane. Step S1042: Performing rasterization processing to generate a rendered image from the corresponding viewpoint, used for a mixed reality virtual house viewing scene.

[0078] In practical applications, arbitrary camera poses (including position and rotation matrices) are selected based on user needs or preset paths, and projection calculations are performed on all second Gaussian kernels. A tile-based rasterization technique is employed, distributing the projections to small tiles within the image grid. The second Gaussian kernels within each tile are sorted by depth, and then alpha blending and color accumulation are performed to calculate the final pixel values, generating the rendered image for the corresponding viewpoint. Alpha blending is a core technology in computer graphics for handling fragment color blending; it belongs to the fragment-by-fragment operation stage of the rendering pipeline and, together with depth testing and stencil testing, constitutes the image processing flow.

[0079] Based on the above-mentioned publicly available solutions, high-quality rendering is achieved through projection and rasterization processing of arbitrary camera poses, which enhances the realism of image reconstruction. Combined with the second Gaussian kernel after edge-priority densification, the clarity of geometric edges is significantly improved, enhancing the overall rendering effect. Without increasing additional computational overhead, it provides a stronger sense of immersion and space for mixed reality virtual house viewing scenarios.

[0080] Based on any of the above embodiments, this disclosure also provides an image rendering apparatus. Figure 9This is a schematic block diagram of an image rendering apparatus according to one embodiment of the present disclosure. Figure 9 As shown, the image rendering apparatus includes: an acquisition module 901, used to acquire an initial 3D point cloud obtained by processing the original image; an extraction module 902, used to extract edge points representing the geometric contours of the original image from the initial 3D point cloud; a densification module 903, used to densify a first Gaussian kernel of the corresponding edge points in the initial 3D point cloud using preset weighting coefficients to obtain a second Gaussian kernel; and a rendering module 904, used to reconstruct the image using the second Gaussian kernel to obtain a rendered image.

[0081] The extraction module 902 is used to detect points in the initial 3D point cloud that do not have geometric continuity as edge points using a preset algorithm; and to assign edge point identifiers to the edge points in the initial 3D point cloud.

[0082] The densification module 903 is used to assign Gaussian attributes to each point in the initial 3D point cloud, obtain the first Gaussian kernel corresponding to each point, and assign edge point identifiers to the first Gaussian kernels corresponding to edge points. During the training iteration, densification processing is triggered according to the preset number of iterations, and the cumulative gradient is calculated for all first Gaussian kernels. If the first Gaussian kernel is assigned an edge point identifier, the cumulative gradient of the first Gaussian kernel is enhanced using weighting coefficients. According to the enhanced cumulative gradient and the preset gradient threshold, the first Gaussian kernel is split or cloned to obtain the enhanced second Gaussian kernel.

[0083] The densification module 903 is used to sort the accumulated gradients after enhancement; select the first Gaussian kernel with a gradient threshold from the sorting results and split or clone it to obtain the enhanced second Gaussian kernel; and assign edge point identifiers to the enhanced second Gaussian kernel.

[0084] The densification module 903 is used to weaken the cumulative gradient using a preset weakening coefficient if the first Gaussian kernel is not assigned an edge point identifier; and to perform a splitting or cloning operation on the first Gaussian kernel based on the comparison result of the cumulative gradient obtained after weakening and the gradient threshold, so as to obtain a weakened second Gaussian kernel.

[0085] The extraction module 902 is used to calculate the normal vectors of adjacent points in the initial 3D point cloud when the preset algorithm is the normal vector change rate algorithm; calculate the change rate between the normal vectors of adjacent points; if the change rate exceeds the preset change rate threshold, then the adjacent points are determined to be edge points without geometric continuity.

[0086] The extraction module 902 is used to convert the initial 3D point cloud into a depth map when the preset algorithm is a depth difference algorithm; calculate the depth value difference between pixels of adjacent points; if the depth value difference exceeds the preset difference threshold, then the adjacent points are determined to be edge points without geometric continuity.

[0087] The extraction module 902 is used to convert the initial three-dimensional point cloud into a discrete geometric signal when the preset algorithm is a point cloud edge detection method; analyze the spatial distribution and geometric features of adjacent points; and if local geometric discontinuities are detected, determine that the adjacent points are edge points without geometric continuity.

[0088] The rendering module 904 is used to select any camera pose, project the second Gaussian kernel onto a two-dimensional plane, perform rasterization processing, and generate a rendered image from the corresponding viewpoint for use in mixed reality virtual house viewing scenes.

[0089] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0090] The execution subject of the image rendering method in the specific embodiments of this disclosure can be an electronic device such as a server (including a local server or a cloud computing platform).

[0091] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the image rendering method of any of the embodiments described above.

[0092] Figure 10 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0093] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0094] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, this diagram uses only one connection line, but this does not imply that there is only one bus or one type of bus.

[0095] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0096] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0097] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0098] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. An image rendering method, characterized in that, The method includes: Obtain the initial 3D point cloud obtained by processing the original image; Extract edge points that characterize the geometric contours in the original image from the initial 3D point cloud; The first Gaussian kernel corresponding to the edge points in the initial three-dimensional point cloud is densified using preset weighting coefficients to obtain the second Gaussian kernel; The rendered image is obtained by reconstructing the image using the second Gaussian kernel.

2. The image rendering method according to claim 1, characterized in that, The step of extracting edge points representing the geometric contours in the original image from the initial 3D point cloud includes: Points lacking geometric continuity in the initial 3D point cloud are detected using a preset algorithm and designated as edge points; and, Assign edge point identifiers to the edge points in the initial 3D point cloud.

3. The image rendering method according to claim 2, characterized in that, The process of densifying the first Gaussian kernel of the corresponding edge points in the initial 3D point cloud using preset weighting coefficients to obtain the second Gaussian kernel includes: Assign Gaussian attributes to each point in the initial 3D point cloud to obtain the first Gaussian kernel corresponding to each point, and assign the edge point identifier to the first Gaussian kernel corresponding to the edge point; During the training iteration, a densification process is triggered according to a preset number of iterations, and the cumulative gradient is calculated for all the first Gaussian kernels. If the first Gaussian kernel is assigned the edge point identifier, the cumulative gradient of the first Gaussian kernel is enhanced using the weighting coefficients; Based on the accumulated gradient after enhancement and the preset gradient threshold, the first Gaussian kernel is split or cloned to obtain the enhanced second Gaussian kernel.

4. The image rendering method according to claim 3, characterized in that, The step of splitting or cloning the first Gaussian kernel based on the enhanced accumulated gradient and a preset gradient threshold to obtain an enhanced second Gaussian kernel includes: The accumulated gradients after enhancement are sorted; Select the first Gaussian kernel with a gradient threshold from the sorting results and perform splitting or cloning to obtain the enhanced second Gaussian kernel; The edge point identifier is assigned to the enhanced second Gaussian kernel.

5. The image rendering method according to claim 3, characterized in that, Also includes: If the first Gaussian kernel is not assigned the edge point identifier, the accumulated gradient is weakened using a preset weakening coefficient. Based on the comparison between the accumulated gradient obtained after the attenuation process and the gradient threshold, the first Gaussian kernel is split or cloned to obtain the second Gaussian kernel.

6. The image rendering method according to claim 2, characterized in that, The step of using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points includes: When the preset algorithm is the normal vector change rate algorithm, the normal vectors of adjacent points in the initial three-dimensional point cloud are calculated. Calculate the rate of change between the normal vectors of the adjacent points. If the rate of change exceeds a preset rate of change threshold, then the adjacent points are determined to be edge points that do not have geometric continuity.

7. The image rendering method according to claim 2, characterized in that, The step of using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points includes: When the preset algorithm is a depth difference algorithm, the initial 3D point cloud is converted into a depth map; Calculate the depth value difference between adjacent pixels. If the depth value difference exceeds a preset difference threshold, then the adjacent point is determined to be an edge point that does not have geometric continuity.

8. The image rendering method according to claim 2, characterized in that, The step of using a preset algorithm to detect points in the initial 3D point cloud that lack geometric continuity as edge points includes: When the preset algorithm is a point cloud edge detection method, the initial three-dimensional point cloud is converted into a discrete geometric signal; Analyze the spatial distribution and geometric features of adjacent points. If a local geometric discontinuity is detected, then the adjacent point is determined to be an edge point that does not have geometric continuity.

9. The image rendering method according to claim 1, characterized in that, The process of reconstructing the image using the second Gaussian kernel to obtain the rendered image includes: Choose any camera pose and project the second Gaussian kernel onto a two-dimensional plane; Rasterization is performed to generate a rendered image from the corresponding viewpoint.

10. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 9.

Images