A training and application method, device, and medium for an image reconstruction model.

Abstract

This application relates to the field of image representation technology, and discloses a training and application method, device, and medium for an image reconstruction model. The method determines candidate augmentation scores for pixels based on reconstruction errors, selects a set of high-error pixels based on these scores, divides these pixels into regions to identify spatially continuous error regions as candidate augmentation regions, determines Gaussian parameters based on the candidate augmentation scores of pixels within these regions, and adds a Gaussian parameter to each candidate augmentation region. This method of aggregating high-error pixels into coherent candidate augmentation regions and adding a new Gaussian parameter significantly reduces the computational overhead caused by point-by-point operations. Simultaneously, adding a new Gaussian parameter based on pixel information in local regions makes the new Gaussian parameter more closely match the local structural distribution of the image, thereby avoiding fragmentation of the Gaussian distribution and enabling the Gaussian model using the added Gaussian parameter to obtain a high-quality reconstructed image.

Description

Technical Field

[0001] This application relates to the field of image representation technology, and in particular to a method, device, and medium for training and applying an image reconstruction model. Background Technology

[0002] In current image representation techniques, traditional methods typically describe images using fixed grids or transform domain coefficients. While these methods are mature, their adaptability to the complexity of image content is limited. In recent years, implicit neural representations have established continuous spatial mappings through neural networks, enabling better characterization of image details; however, they usually require large network parameter sizes and high computational resources. Two-dimensional Gaussian image representation offers a more efficient alternative between these two methods. This type of method models image content using a set of explicit two-dimensional Gaussian models and reconstructs it using differentiable rasterization, thus avoiding the high computational overhead of pixel-by-pixel neural network forward inference.

[0003] However, existing 2D Gaussian image representation methods typically perform point-by-point densification of under-reconstructed regions based on pixel error magnitude during the optimization stage, meaning that Gaussian sigmas are added individually at pixels with high errors. This point-by-point densification not only incurs high computational costs during Gaussian optimization but also ignores the spatial continuity of image structure and error distribution. For example, in complex regions such as edges and textures, errors often do not appear as isolated points but rather exhibit locally coherent regional patterns. If only pixel-level densification is used, the added Gaussian sigmas tend to have a fragmented distribution, failing to effectively match the local structural features of the image. Consequently, high-quality reconstructed images cannot be obtained when using the optimized Gaussian sigmas for image reconstruction. Summary of the Invention

[0004] This application provides a training and application method, device, and medium for an image reconstruction model to solve the problem that existing two-dimensional Gaussian image representation methods, during the optimization stage, introduce high computational overhead by progressively densifying under-reconstructed regions based on pixel error magnitude, while also neglecting the spatial continuity of image structure and error distribution.

[0005] The first aspect of this application provides a training method for an image reconstruction model. The method includes: acquiring an original image and using the original image to iteratively train an initial Gaussian model until convergence, using the converged Gaussian model as the image reconstruction model. The initial Gaussian model takes the original image as input and the reconstructed image corresponding to the original image as output. The method further includes the following steps: adding a new Gaussian model to the Gaussian model obtained after a first preset number of iterations to obtain the initial Gaussian model required for the next iteration: calculating the reconstruction error of pixels in the original image based on the original image and the reconstructed image; determining candidate densification scores for pixels based on the reconstruction error, and determining a candidate densification set consisting of a preset number of pixels based on the candidate densification scores; dividing the pixels in the candidate densification set into regions to obtain candidate densification regions consisting of multiple pixels; determining Gaussian parameters based on the candidate densification scores of pixels in the candidate densification regions, and adding a new Gaussian model with Gaussian parameters to the Gaussian model obtained after the first preset number of iterations to obtain the initial Gaussian model required for the next iteration.

[0006] In some embodiments of this application, determining a candidate densification set consisting of a preset number of pixels based on the reconstruction error includes: using the reconstruction error as a candidate densification score for the pixel; comparing the candidate densification score with a preset first score threshold, and when the candidate densification score is greater than or equal to a second score threshold, using the pixel as a candidate densification point; and constructing a candidate densification set from the candidate densification points.

[0007] In some embodiments of this application, determining a candidate densification set consisting of a preset number of pixels based on the reconstruction error includes: performing gradient analysis on the reconstruction error to obtain the distortion gradient direction of the pixel; calculating the local distortion consistency of the pixel based on the distortion gradient direction; calculating the candidate densification score of the pixel based on the reconstruction error and the local distortion consistency; comparing the candidate densification score with a preset second score threshold, and when the candidate densification score is greater than or equal to the second score threshold, designating the pixel as a candidate densification point; and constructing a candidate densification set from the candidate densification points.

[0008] In some embodiments of this application, the local distortion consistency of pixels is calculated based on the distortion gradient direction in the following manner:

[0009] in, Represents pixels The consistency of local distortion Indicated by A local window centered on [the central element]. For pixels The direction of the distortion gradient, This represents the side length of the local window.

[0010] In some embodiments of this application, the candidate pixel densification score is calculated based on the consistency between reconstruction error and local distortion as follows:

[0011] in, Represents pixels Candidate densification scores, Represents pixels Reconstruction error.

[0012] In some embodiments of this application, the pixels in the candidate densification set are divided into regions to obtain a candidate densification region composed of multiple pixels, including: marking the candidate densification points as foreground pixels and marking other pixels outside the candidate densification region as background pixels to obtain a binarized image; using a neighborhood connection strategy to traverse the foreground pixels in the binarized image and dividing the spatially connected foreground pixels into a candidate densification region.

[0013] In some embodiments of this application, the Gaussian parameters include: position parameters and covariance parameters. The Gaussian parameters are determined based on the candidate densification scores of pixels in the candidate densification region. A new Gaussian model with Gaussian parameters is added to the Gaussian model obtained after training for a first preset number of iterations. This includes: calculating the candidate densification score weight of a pixel based on the candidate densification scores of pixels in the candidate densification region and the candidate densification scores of all pixels in the candidate densification region; calculating the center position parameter of the pixel based on the candidate densification score weight and the pixel's position coordinates, and using the center position parameter as the position parameter of the new Gaussian; and calculating the covariance parameter of the new Gaussian based on the candidate densification score weight, position coordinates, and position parameters.

[0014] The second aspect of this application provides an application method for an image reconstruction model. The method of this application includes: using the image reconstruction model trained by any of the image reconstruction models in the first aspect, the reconstructed image corresponding to the original image is calculated based on the acquired original image.

[0015] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in either the first or second aspect of the above embodiments.

[0016] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in either the first or second aspect of the above embodiments.

[0017] This application offers the following advantages: It proposes a training method for an image reconstruction model. This method first determines candidate augmentation scores for pixels based on reconstruction errors, then filters out a set of high-error pixels based on these scores. Next, it divides these pixels into regions to identify spatially continuous error regions (such as edges or textures) as candidate augmentation regions. Finally, it determines Gaussian parameters based on the candidate augmentation scores of pixels within these regions and adds a Gaussian parameter to each candidate augmentation region. This method, which aggregates high-error pixels into coherent candidate augmentation regions and determines Gaussian parameters based on the information of these regions, significantly reduces the computational overhead caused by point-by-point operations. Furthermore, adding a new Gaussian parameter based on local pixel information makes the new Gaussian parameter more closely match the local structural distribution of the image, thus avoiding fragmentation of the Gaussian distribution and enabling the Gaussian model using the newly added Gaussian parameter to obtain a high-quality reconstructed image. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the image reconstruction model construction method provided in this application; Figure 2 This is a visual structural diagram of the first embodiment of the image reconstruction model construction method provided in this application; Figure 3 This is a flowchart illustrating the second embodiment of the image reconstruction model construction method provided in this application; Figure 4 This is a visual structural diagram of the second embodiment of the image reconstruction model construction method provided in this application; Figure 5 This is a flowchart illustrating the third embodiment of the image reconstruction model construction method provided in this application; Figure 6 This is a visual structural diagram of the third embodiment of the image reconstruction model construction method provided in this application; Figure 7 This is a schematic diagram of the framework of an embodiment of the electronic device provided in this application; Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0021] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] I. Under-reconstructed areas appear in the reconstructed image during the optimization process. As described in the background section, existing two-dimensional Gaussian image representation methods typically perform point-by-point densification of under-reconstructed regions during the optimization phase, based on pixel error magnitude; that is, Gaussian sigmas are added individually at pixels with high errors. This point-by-point densification not only incurs high computational overhead during Gaussian optimization but also ignores the spatial continuity of image structure and error distribution. Specifically, the inventors have found that while existing two-dimensional Gaussian image representation methods can explicitly represent images based on two-dimensional Gaussians and possess fast rendering and low decoding overhead, they do not adequately utilize local structural information during training and optimization. For example, pixel-level independent densification is typically used for under-reconstructed regions, resulting in fragmented Gaussian addition that cannot effectively match local continuous structures.

[0023] To address the aforementioned issues, this application proposes a novel Gaussian optimization method. This method is based on a two-dimensional Gaussian image representation model and introduces a locality-oriented density control mechanism during the training process of Gaussian parameters. Specifically, it achieves dynamic optimization of the number and distribution of Gaussians through "region-level Gaussian densification".

[0024] Before introducing specific embodiments of this application, a brief introduction to the basic definitions of relevant terms in this application will be given.

[0025] Under-reconstructed regions are areas where, during the explicit representation and rendering optimization of an image using a two-dimensional Gaussian function, the insufficient number or location of Gaussian distributions leads to the inability to fully express local image structures (such as continuous features like edges and textures) in the rendering results, resulting in areas with missing details or blurred areas.

[0026] In the process of explicitly representing and optimizing the rendering of an image using a two-dimensional Gaussian function, if the number of Gaussian distributions is too large, their positions are too dense, or their attributes are set improperly, the local image content may be over-expressed beyond what is actually needed, resulting in redundant, blurred, or artifact-prone areas in the rendering result.

[0027] This application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0028] According to one embodiment of this application, a training method for an image reconstruction model is proposed. The method includes: acquiring an original image and using the original image to iteratively train an initial Gaussian model until convergence, using the converged Gaussian model as the image reconstruction model. The initial Gaussian model takes the original image as input and the reconstructed image corresponding to the original image as output, and... Figure 1 and Figure 2 As shown, the method of this application performs the following steps S1-S4 to add a new Gaussian to the Gaussian model obtained after training for a first preset number of iterations, so as to obtain the initial Gaussian model required for the next iteration training: S1. Calculate the reconstruction error of the pixels in the original image based on the original image and the reconstructed image; S2. Determine the candidate densification score of the pixel based on the reconstruction error, and determine a candidate densification set consisting of a preset number of pixels according to the candidate densification score; S3. Divide the pixels in the candidate densification set into regions to obtain a candidate densification region consisting of multiple pixels; S4. Divide the pixels in the candidate densification set into regions to obtain a candidate densification region consisting of multiple pixels; Based on the candidate densification score of the pixels in the candidate densification region, determine the Gaussian parameter, and add a new Gaussian with Gaussian parameter to the Gaussian model obtained after training for a first preset number of iterations, so as to obtain the initial Gaussian model required for the next iteration training.

[0029] As described above, the method of adding new Gaussian sigmas to candidate augmentation regions using pixel-based candidate augmentation regions as the basic unit significantly reduces the computational overhead caused by point-by-point operations. Simultaneously, adding new Gaussian sigmas based on the pixel information of the candidate augmentation regions makes the new Gaussian sigmas more closely match the local structural distribution of the image, thereby avoiding the fragmentation of the Gaussian distribution and enabling the Gaussian model using the added Gaussian sigmas to obtain a high-quality reconstructed image.

[0030] In this embodiment, the original input image is obtained. Original image It can be a single 2D color image or a 2D grayscale image. (Regarding the original image) An initial two-dimensional Gaussian set is established as the initial Gaussian model for image representation:

[0031] Where N represents the number of Gaussians in the initial Gaussian model, and each Gaussian... At least the following parameters should be included: two-dimensional spatial position parameters Covariance parameter and color parameters The position parameter characterizes the center position of the Gaussian sigmoid in the image plane, the covariance parameter characterizes the shape, scale, and orientation distribution of the Gaussian sigmoid, and the color parameter characterizes the contribution of the Gaussian sigmoid to the image color information. The initial two-dimensional Gaussian set can be obtained through rule-based initialization, random initialization, heuristic initialization based on image content, or any initialization method found in existing two-dimensional Gaussian representations. This application does not limit the specific initialization method.

[0032] In this embodiment, the original image is processed using an initial Gaussian model. Reconstruction is performed to obtain the reconstructed image. For any pixel in the image Its rendering color It can be represented as:

[0033] in:

[0034] in, Represents the original image The number of pixels in the middle, Represents pixels With Gauss central position The displacement vector between them. for transpose, Gauss The inverse matrix of the covariance parameters.

[0035] In this embodiment, based on the reconstructed image With the original image Based on the differences between them, construct the reconstruction loss function:

[0036] The Gaussian parameters in the initial Gaussian model are optimized and updated based on the reconstruction loss function so that the reconstructed image gradually approximates the original image.

[0037] As described above, the embodiments of this application improve the accuracy and robustness of image reconstruction by introducing Mahalanobis distance to measure the similarity between pixels and Gaussian centers and generating rendering colors using a weighted summation method. At the same time, through a parameter optimization process driven by a loss function, adaptive adjustment of the Gaussian model is achieved, enhancing the model's ability to fit complex image structures.

[0038] To enhance attribute consistency among spatially adjacent Gaussians, this application introduces color consistency constraints during training. In this embodiment, during each training iteration, a color consistency loss function is determined based on the color parameters of spatially adjacent Gaussians in the initial Gaussian model after adding Gaussians. A total loss function is then determined based on the reconstruction loss function and the color consistency loss function, and the Gaussian parameters of the initial Gaussian model after adding Gaussians are updated according to the total loss function. In this embodiment, let Q be the Gaussian parameter after adding Gaussians. The color consistency loss between spatially adjacent Gaussians can be expressed as:

[0039] Based on color consistency loss and reconstruction loss, construct the total loss function:

[0040] in, These are the preset weighting coefficients. Based on this, the parameters of each Gaussian in the Gaussian model after the addition of Gaussians are jointly optimized and updated, which improves the image reconstruction quality while making the color attributes of locally adjacent Gaussians more consistent.

[0041] As described above, the embodiments of this application effectively enhance the attribute consistency between spatially adjacent Gaussian components by introducing a color consistency constraint during training. Specifically, this constraint constructs a loss term by calculating the sum of squares of the color differences between adjacent Gaussian components, and together with the reconstruction loss, forms the total loss function. Therefore, when optimizing the Gaussian model parameters, it not only focuses on the overall reconstruction accuracy of the image but also forces locally adjacent Gaussian components to tend towards consistency in color attributes. This joint optimization mechanism significantly improves the smoothness and visual naturalness of image reconstruction, reduces local color abrupt changes or artifacts caused by independent optimization of Gaussian components, and thus improves the overall quality and structural consistency of the reconstructed image.

[0042] To better understand how this application adds a new Gaussian to the Gaussian model obtained after training for a first preset number of iterations, steps S1-S4 are explained in detail below.

[0043] (a) Step S1 In this embodiment, as Figure 2 As shown, in each iteration of training, based on the reconstructed image With the original image The pixel differences are used to construct the reconstruction error. For any pixel... The formula for calculating reconstruction error is:

[0044] in, Represents pixels Reconstruction error, Represents the original image Pixels , Represents the reconstructed image Pixels It should be noted that the reconstruction error of all pixels in the original image is calculated according to the above formula. The reconstruction error image of the original image is constructed by calculating the reconstruction error of all pixels.

[0045] (ii) Step S2 In this embodiment, step S2 includes: S21, using the reconstruction error as a candidate pixel densification score, i.e. S22. Compare the candidate densification scores with the preset first score threshold, and when the candidate densification score is greater than or equal to the first score threshold, use the pixel as a candidate densification point; S23. Form a candidate densification set from the candidate densification points.

[0046] As described above, the embodiments of this application achieve accurate localization and filtering of poorly reconstructed areas in an image by directly using the reconstruction error as a candidate pixel densification score and setting a scoring threshold to filter candidate densification points. This error-driven densification strategy can adaptively identify areas that need Gaussian component supplementation, thereby selectively increasing the Gaussian quantity in subsequent steps, effectively improving the reconstruction accuracy of image details, while avoiding redundant densification in well-reconstructed areas, optimizing the allocation of computational resources, and improving the efficiency of model optimization and reconstruction quality.

[0047] In another embodiment, such as Figure 2 As shown, step S2 includes: S24, performing gradient analysis on the reconstruction error to obtain the distortion gradient direction of the pixel; S25, calculating the local distortion consistency of the pixel based on the distortion gradient direction; S26, calculating the candidate densification score of the pixel based on the reconstruction error and the local distortion consistency; S27, comparing the candidate densification score with a preset second scoring threshold, and selecting the pixel as a candidate densification point when the candidate densification score is greater than or equal to the second scoring threshold; S28, constructing a candidate densification set from the candidate densification points. Arrows in the same direction indicate that the pixel distortion gradient direction is the same.

[0048] During model training, to identify whether under-reconstructed regions possess local continuous structures, gradient analysis is performed on the reconstruction error map to obtain the distortion gradient direction of each pixel. For example, in step S24, for pixel... Its distortion gradient direction It can be represented as:

[0049] in, Represents pixels The direction of the distortion gradient, Represents pixels In a two-dimensional coordinate system gradient on the axis, Represents pixels In a two-dimensional coordinate system gradient on the axis, This is a preset non-zero stable value used to prevent stable terms with a denominator of zero.

[0050] In step S25, based on the distortion gradient direction, the local distortion consistency of the pixels is calculated in the following manner:

[0051] in, Represents pixels The consistency of local distortion Indicated by A local window centered on [the central element]. For pixels The direction of the distortion gradient, This represents the side length of the local window. Specifically, when the local window size is 7x7, 7. Local distortion consistency This is used to measure whether the trend of error change is consistent within a local window. When the distortion gradient directions of multiple pixels within a local window are highly consistent, that is, the distortion gradient directions are the same, it indicates that the local region corresponding to this local window is more likely to correspond to the edge, texture, or structural under-reconstructed region in the image.

[0052] In step S26, based on the consistency between reconstruction error and local distortion, the candidate pixel densification score is calculated as follows: .

[0053] As described above, the embodiments of this application, by introducing distortion gradient analysis, consider not only the magnitude of the reconstruction error but also the local distortion consistency for comprehensive evaluation when identifying candidate densification points. This strategy can effectively identify under-reconstructed regions with local continuous structures (such as edges or textured regions), avoiding noise misjudgment or isolated point densification that may occur when densifying based solely on a single pixel error value. By calculating the distortion gradient direction and local consistency, the truly necessary structured regions for densification can be selected more accurately, thereby improving the detail quality of image reconstruction while ensuring the effectiveness and specificity of the densification operation, further optimizing the model's convergence efficiency and reconstruction effect.

[0054] (III) Step S3 Furthermore, candidate augmentation points that are spatially adjacent to each other are subjected to region clustering, connected component partitioning, or neighborhood grouping to form one or more candidate augmentation regions. In this embodiment, step S3 includes: S31, marking candidate augmentation points as foreground pixels and other pixels outside the candidate augmentation regions as background pixels to obtain a binarized image; S32, traversing the foreground pixels in the binarized image using a neighborhood connection strategy, and dividing spatially connected foreground pixels into a candidate augmentation region. The neighborhood connection strategy is 8-neighborhood connected component labeling.

[0055] As can be seen from the above description, the above embodiments of this application can more accurately locate the areas in the image that need to be densed by combining reconstruction error and local distortion consistency to calculate pixel candidate density scores. This allows the newly added Gaussian pixels to be preferentially configured to areas with continuous under-reconstruction, rather than being added point by point to individual high-error pixels, thereby improving the structural adaptability of Gaussian density and reducing Gaussian distribution fragmentation.

[0056] (iv) Step S4 In this embodiment, for each candidate denser region, at least one new two-dimensional Gaussian is added, and the added Gaussian is initialized at the center position, weighted center position, or representative position of the corresponding candidate denser region.

[0057] In this embodiment, the Gaussian parameters include: position parameters and covariance parameters. Step S4 includes: S41, calculating the candidate density enhancement score weight of the pixel based on the candidate density enhancement score of the pixel and the candidate density enhancement scores of all pixels within the candidate density enhancement region; S42, calculating the center position parameter of the pixel based on the candidate density enhancement score weight and the pixel's position coordinates, and using the center position parameter as the new Gaussian position parameter; S43, calculating the new Gaussian covariance parameter based on the candidate density enhancement score weight, position coordinates, and position parameter.

[0058] In step S41, based on the candidate densification score of the pixel and the candidate densification scores of all pixels within the candidate densification region, the candidate densification score weight of the pixel is calculated in the following manner:

[0059] in, The set of pixels comprising all pixels within the candidate denser region. Represents pixels Candidate densification scores, Represents pixels The candidate density enhancement score weights.

[0060] As described above, the embodiments of this application calculate the candidate densification score weight based on the candidate densification score of each pixel, and use the weight to adaptively determine the position parameter and covariance parameter of the Gaussian. This allows for dynamic adjustment of the shape and position of the Gaussian according to the densification requirements of local areas of the image, thereby achieving precise densification of image details, improving the visual quality and naturalness of the densified image, and avoiding the problems of over-smoothing or insufficient densification caused by traditional fixed parameter methods.

[0061] In step S42, the center position parameter of the pixel is calculated based on the candidate densification score weights and the pixel's position coordinates as follows:

[0062] in, Represents pixels The center position parameter on the x-axis of the two-dimensional coordinate system. Represents pixels Position parameters on the x-axis of a two-dimensional coordinate system. Represents pixels The center position parameter on the y-axis of the two-dimensional coordinate system. Represents pixels Position parameters on the y-axis of a two-dimensional coordinate system.

[0063] In step S43, based on the candidate densification score weights, location coordinates, and location parameters, the new Gaussian covariance parameters are calculated as follows:

[0064]

[0065]

[0066] in, This represents the covariance parameter of the new Gaussian along the x-axis in a two-dimensional coordinate system. This represents the covariance parameter of the new Gaussian along the y-axis in a two-dimensional coordinate system. This represents the covariance parameter of the new Gaussian in the x-axis and y-axis directions of the two-dimensional coordinate system.

[0067] In this embodiment, the above-described embodiments of this application calculate the center position parameter of the pixel using a weighted average method, and combine the weight, position coordinates, and center position parameter to accurately derive a new Gaussian covariance parameter. This method can dynamically adjust the distribution characteristics of Gaussians in two-dimensional space according to the candidate densification score weight of the pixel, thereby better meeting the densification needs of local areas of the image, making the densified image better in terms of detail and smoothness of transition, and effectively improving the adaptability of image densification and the final visual effect.

[0068] In this embodiment, the new Gaussian color parameters can be initialized based on the original image color statistics, local residual color, or the color of neighboring Gaussians within the corresponding region.

[0069] As described above, the embodiments of this application can ensure that the Gaussian components generated by the densification are highly consistent with the surrounding image content in terms of color by flexibly utilizing the original image color statistics, local residual color, or the color of neighboring Gaussians to initialize the new Gaussian color parameters. This effectively avoids visual artifacts or color differences caused by improper color initialization, thereby significantly improving the visual naturalness and color reproduction accuracy of the densified image.

[0070] Based on the above analysis, this application addresses the problem of unreasonable Gaussian capacity allocation in existing image representation methods based on 2D Gaussian sputtering during the optimization process. It constructs an image reconstruction model by simultaneously utilizing the local reconstruction error continuity in the image space and the local similarity in the Gaussian space. This model can perform more reasonable Gaussian denserization in under-reconstructed regions. Specifically, this application first constructs a reconstruction error map based on the difference between the current reconstructed image and the original image. Then, it further analyzes the gradient changes of the error map in the local neighborhood to obtain the local continuity information of the error. By using both the error magnitude and local continuity as evaluation criteria, this application can identify the truly under-reconstructed regions that require focused enhancement. Unlike existing image representation methods based on 2D Gaussian sputtering that only denser based on single-pixel high errors, this application preferentially adds new Gaussians based on the pixel information of local regions, making the added Gaussians more closely match the local structural distribution, thereby improving the effectiveness of Gaussian supplementation.

[0071] To verify the effectiveness of the above embodiments, the inventors conducted a series of experiments and obtained the experimental results shown in Table 1. Among them, PSNR (Peak Signal-to-Noise Ratio, used to objectively evaluate image quality) and MS-SSIM (Multi-Scale Structural Similarity, used to compare image similarity) are two core indicators used in the fields of image processing and computer vision to objectively evaluate image quality or compare image similarity.

[0072] Table 1: Experimental results of image reconstruction model training using the Kodak dataset

[0073] The above experimental results demonstrate that experiments conducted on the Kodak dataset show that this application is based on... Regional-level densification, based on The image reconstruction model obtained by the region-level densification method has higher image quality than the pixel-level densification method of existing image representation methods. It can more effectively improve the preservation of image details and overall quality, and is the best performing solution.

[0074] II. Over-reconstructed areas appeared in the reconstructed image during the optimization process. Furthermore, the inventors' research revealed that during the optimization phase of existing 2D Gaussian image representation methods, multiple Gaussians may gradually converge to very similar colors and spatial positions, even repeatedly covering the same local content, forming redundant representations. This phenomenon is called over-reconstruction regions. Without an effective redundancy elimination mechanism, these redundant Gaussians will continuously consume limited resources during subsequent optimization and application, wasting storage and computing resources and reducing the chances of obtaining more Gaussian support for other complex regions. Specifically, the inventors found that while existing 2D Gaussian image representation methods can explicitly represent images based on 2D Gaussians and have relatively fast rendering and low decoding overhead, they do not adequately utilize local structural information during training and optimization. For example, spatially adjacent Gaussians with similar properties are often redundantly retained after optimization, resulting in representation redundancy.

[0075] However, existing Gaussian sparsity methods from the field of 3D Gaussian scene reconstruction typically reduce the number of Gaussians through direct pruning or deletion. However, in 2D image representation tasks, small-scale Gaussians in detail regions often play a crucial role in structural representation, and directly deleting these Gaussians can easily lead to a significant degradation in reconstruction quality. Therefore, a more robust mechanism, distinct from simple deletion, is needed to address the Gaussian redundancy problem in 2D image representation.

[0076] To address the aforementioned problems, this application proposes a training method for an image reconstruction model. The method includes: acquiring the original image and using the original image to iteratively train an initial Gaussian model until convergence, using the converged Gaussian model as the image reconstruction model, and as follows... Figure 3 and Figure 4 As shown, the method of this application performs the following steps S5-S7 to fuse the Gaussians in the Gaussian model obtained after a second preset number of iterations of training, so as to obtain the initial Gaussian model required for the next iteration of training: S5, select a first preset number of candidate fused Gaussians from the initial Gaussian model; S6, combine the first preset number of candidate fused Gaussians to obtain a second preset number of candidate fused Gaussian pairs; S7, fuse the candidate fused Gaussian pairs into a fused Gaussian, and use the fused Gaussian and other Gaussians in the Gaussian model obtained after a second preset number of iterations of training that did not participate in the fusion to form the initial Gaussian model required for the next iteration of training.

[0077] As described above, this application selects Gaussian sigmas during the training of the image reconstruction model, combines the selected Gaussian sigmas into pairs, and merges them into a new Gaussian sigma to replace the original two Gaussian sigmas. This proactive fusion mechanism effectively eliminates redundant representations in the model, freeing up storage and computational resources occupied by redundant Gaussian sigmas. This allows the limited model capacity to be more concentrated on the complex regions in the image that truly require detailed modeling, thereby improving resource utilization and optimizing the overall image reconstruction quality. In other words, redundant Gaussian sigmas are merged in over-reconstruction regions, thereby improving Gaussian capacity utilization efficiency, enhancing image representation quality, and achieving better reconstruction results with the same Gaussian budget.

[0078] In this embodiment, acquiring the original image and using the original image to perform multiple iterations of training on the initial Gaussian model until convergence includes: updating the Gaussian parameters of the initial Gaussian model using a preset total loss function during each iteration of training.

[0079] In this embodiment, a total loss function is constructed based on the color consistency loss function and the reconstruction loss function:

[0080] in, is the weighting coefficient for color consistency loss.

[0081] To enhance the property consistency among spatially adjacent Gaussians and provide a more stable similarity basis for Gaussian merging, such as... Figure 4 As shown, this application introduces a local color consistency constraint during the training process. In this embodiment, let P be the set of Gaussian pairs in the local spatial neighborhood, then the color consistency loss function is:

[0082] Based on this, the parameters of each Gaussian in the two-dimensional Gaussian set are jointly optimized and updated, which improves the image reconstruction quality and makes the color attributes of local adjacent Gaussians more coordinated, creating conditions for subsequent Gaussian similarity discrimination and merging operations.

[0083] As described above, the embodiments of this application introduce local color consistency loss during model training to actively constrain the color attributes of adjacent Gaussians during the training phase. By forcing Gaussians in the local space to tend towards color consistency, the color differences between adjacent Gaussians are directly reduced, thereby enhancing their attribute consistency from the source. This consistency ensures that the similarity judgment of Gaussians is no longer distorted due to color fluctuations when Gaussian merging is performed subsequently, providing a more stable similarity basis for merging. At the same time, it can also reduce redundant representations and improve reconstruction quality.

[0084] To better understand how this application integrates the Gaussian model obtained after training for a second preset number of iterations, steps S5-S7 will be explained in detail below.

[0085] (V) Step S5 During the Gaussian parameter update process or during a preset number of training iterations, redundant Gaussian detection is performed on the current two-dimensional Gaussian set (i.e., the current image reconstruction model) to identify mergeable Gaussian pairs.

[0086] In this embodiment, step S5 includes: S51, calculating the cumulative gradient of Gaussian based on the preset total loss function and the Gaussian parameters updated after training for a second preset number of iterations; S52, comparing the cumulative gradient with the preset stability threshold, and when the cumulative gradient is less than the stability threshold, using Gaussian as a candidate fused Gaussian.

[0087] As can be seen from the above description, the embodiments of this application use cumulative gradient as a metric, which can automatically and accurately identify redundant Gaussians that have become stable during the optimization process, thereby selecting the most suitable candidate for subsequent fusion operations and effectively improving the targeting and efficiency of model optimization.

[0088] In this embodiment, step S51 includes: taking the partial derivative of the total loss function with respect to the position parameter to obtain the Gaussian position gradient; taking the partial derivative of the total loss function with respect to the covariance parameter to obtain the Gaussian covariance gradient; taking the partial derivative of the total loss function with respect to the color parameter to obtain the Gaussian color gradient; and averaging the position gradient, covariance gradient, and color gradient to obtain the cumulative gradient. The calculation methods for the position gradient, covariance gradient, color gradient, and cumulative gradient are as follows: Position gradient = Covariance gradient = Color gradient =

[0089]

[0090] In this embodiment, step S52 includes: normalizing the accumulated gradient, and then comparing the normalized result with a preset stability threshold. If the accumulated gradient is less than the stability threshold, a Gaussian is selected as a candidate for fusion. That is, the normalization process is as follows: ,in This represents the maximum cumulative gradient in the image reconstruction model.

[0091]

[0092] in, The stability threshold is preferably 0.5.

[0093] As described above, the embodiments of this application calculate the average value of the Gaussian's position, covariance, and color gradient as the cumulative gradient, and then normalize it. This accurately quantifies the stability of each Gaussian during training. Comparing the normalized cumulative gradient with a preset stability threshold effectively filters out candidate fused Gaussians that have reached a stable state. This ensures reconstruction quality while avoiding unnecessary merging operations on non-converged Gaussians, thus improving the efficiency of model optimization and the accuracy of reconstruction results.

[0094] (vi) Step S6 Furthermore, for Gaussians that satisfy the stability condition, candidate Gaussian pairs are further screened based on spatial proximity and color similarity.

[0095] In this embodiment, step S6 includes: S61, calculating the Mahalanobis distance between the candidate fused Gaussian and other candidate fused Gaussian pairs based on the position parameters of the candidate fused Gaussian; S62, comparing the Mahalanobis distance with a preset spatial proximity threshold, and calculating the difference in color parameters between the two candidate fused Gaussian pairs when the Mahalanobis distance is less than the spatial proximity threshold; S63, comparing the difference with a preset color similarity threshold, and selecting the Gaussian pair as a candidate fused Gaussian pair when the difference is less than the color similarity threshold.

[0096] In step S61, based on the position parameters of the candidate fused Gaussian, the Mahalanobis distance between the Gaussian pairs formed by the candidate fused Gaussian and other candidate fused Gaussian is calculated in the following manner: Spatial proximity can be determined using Mahalanobis distance. For Gaussian... and Gauss If the following conditions are met:

[0097] in, This represents the Mahalanobis distance between the Gaussian pair consisting of Gaussian i and Gaussian j. This represents the position parameter of Gaussian i. This represents the position parameter of Gaussian j. Let represent the covariance parameter of Gaussian i. Then, the two are considered to be spatially adjacent, where This is a preset spatial proximity threshold.

[0098] Color similarity can be determined by the following conditions:

[0099] in, This represents the color parameter of Gaussian i. This represents the color parameter of Gaussian j. The preset color similarity threshold is used. Gaussian pairs that satisfy the stability condition, spatial proximity condition, and color similarity condition are identified as candidate fusion Gaussian pairs.

[0100] As described above, the embodiments of this application, by further combining spatial proximity and color similarity for dual screening of Gaussian pairs that meet stability conditions, can accurately identify candidate Gaussian pairs that are both spatially adjacent and highly consistent in color attributes. This screening mechanism effectively avoids erroneous merging caused by relying solely on spatial distance (such as mistakenly merging Gaussian pairs with large color differences), thereby ensuring the accuracy of the geometric structure of the image reconstruction model while improving the naturalness of color transitions and overall visual quality, enhancing the robustness of the model fusion process and the precision of the reconstruction results.

[0101] (vi) Step S6 Since the same candidate fused Gaussian may satisfy the merging condition with multiple adjacent candidate fused Gaussians at the same time, it is necessary to resolve the conflict between candidate fused Gaussian pairs to avoid a candidate fused Gaussian being merged repeatedly in the same round of processing.

[0102] In this embodiment, step S6 includes: calculating the position difference of the position parameters of the two candidate fused Gaussians in the candidate fused Gaussian pair; calculating the color difference of the color parameters of the two candidate fused Gaussians in the candidate fused Gaussian pair; calculating the shape difference of the shape parameters of the two candidate fused Gaussians in the candidate fused Gaussian pair; and summing the position difference, color difference, and shape difference to obtain the difference score of the candidate fused Gaussian pair.

[0103] Therefore, the following difference scoring function can be constructed to calculate the difference score of candidate fused Gaussian pairs:

[0104] in, Gauss The vectorized representation of the lower triangular part of the covariance matrix is ​​used to reflect the shape characteristics of the Gaussian i. Gauss The vectorized representation of the lower triangular part of the covariance matrix is ​​used to reflect the shape characteristics of the Gaussian j. Because , To decompose the covariance parameter matrix of Gaussian i into a lower triangular matrix, this value can be directly taken. Similarly, The calculation of is the same as that of Gaussian i, and will not be repeated here.

[0105] In this embodiment, candidate fused Gaussian pairs are sorted based on a difference scoring function. If a certain Gaussian appears in multiple candidate fused Gaussian pairs, the candidate fused Gaussian pair with the smallest difference score is retained as the target fused Gaussian pair, and the remaining candidate fused Gaussian pairs containing that Gaussian are discarded. This step prioritizes retaining Gaussian pairs that are closer in position, color, and shape, thereby improving the rationality and stability of the merging operation.

[0106] (vii) Step S7 In this embodiment, step S7 includes: S71, obtaining the area of ​​the two candidate fused Gaussians in the candidate fused Gaussian pair; S72, calculating the area weights of the two candidate fused Gaussians in the candidate fused Gaussian pair based on their areas; S73, determining the position parameters, color parameters, and covariance parameters of the fused Gaussian based on the area weights of the two candidate fused Gaussians, the position parameters, color parameters, and covariance parameters of the two candidate fused Gaussians before fusion.

[0107] For each pair of Gaussian pairs to be merged, let the two Gaussian pairs be... and Their effective support areas are respectively and In step S72, based on the area of ​​the two candidate fused Gaussians, the area weights of the two candidate fused Gaussians in the candidate fused Gaussian pair are calculated as follows:

[0108] The parameters of the Gaussians to be merged are fused using area weighting to obtain the merged Gaussians.

[0109] In step S73, the position parameters of the merged Gaussian can be expressed as:

[0110] In step S73, the color parameters of the merged Gaussian can be expressed as:

[0111] To preserve the spatial coverage characteristics of the original two Gaussians, the second-order moment matching method is used to calculate the covariance parameter after merging. Let the merged second-order moment be:

[0112] In step S73, the covariance parameter of the merged Gaussian is:

[0113] Preferably, to prevent abnormal expansion of the Gaussian coverage area after merging, an area constraint condition can also be set:

[0114] in, These are the preset constraint coefficients. After parameter fusion, the two Gaussians to be merged are replaced with the merged Gaussians, thereby reducing the number of Gaussians while preserving the original representational capabilities as much as possible, thus compressing redundant Gaussians.

[0115] As described above, the embodiments of this application perform weighted fusion of the position, color, and covariance parameters of candidate Gaussians based on area weights, and calculate the covariance after fusion using a second-order moment matching method. This effectively maintains the spatial coverage characteristics and geometry of the original Gaussians during the fusion process. Simultaneously, by introducing area constraints to prevent abnormal expansion of the coverage area after fusion, the number of Gaussians is significantly reduced while ensuring that the reconstruction quality is not degraded, achieving efficient compression and model optimization of redundant Gaussians.

[0116] Based on the above analysis, this application addresses the issue of over-reconstruction regions by filtering spatially adjacent Gaussian pairs with similar properties and fusing suitable Gaussian pairs to reduce local redundancy. To avoid information loss from simple deletion, this application uses a weighted update method to generate new merged Gaussian pairs, ensuring that they inherit the representative content of the original Gaussian pairs in terms of position, color, and shape parameters as much as possible. In this way, multiple redundant Gaussian pairs can be compressed into a more compact representation while maintaining relatively stable image reconstruction quality.

[0117] To further improve the reliability of Gaussian merging, this application also introduces a local color consistency constraint during the training phase. This constraint penalizes excessive color differences between neighboring Gaussians, making spatially adjacent and functionally similar Gaussians more consistent in color attributes, thus providing a more stable basis for subsequent similarity judgment and Gaussian merging.

[0118] To verify the effectiveness of the above embodiments, the inventors conducted a series of experiments and obtained the experimental results shown in Table 2. Among them, Scheme 1: is based on... Pixel-level densification; Scheme 2: Based on Scheme 1: Regional-level densification; Scheme 2: Pixel-level densification based on E(p) + Gaussian pair fusion; Scheme 3: Scheme 4: Scheme 3 + local color consistency constraint; Scheme 5: Regional-level densification based on E(p) + D(p) + Gaussian pair fusion + local color consistency constraint.

[0119] Table 2: Experimental results of image reconstruction model training using Kodak, DIV2K×2, and CLIC datasets.

[0120] Furthermore, such as Figure 5 and 6 As shown, during the multiple iterations of training the image reconstruction model, steps S1-S4 and S5-S7 can be executed alternately to dynamically adjust the distribution density of Gaussians in different regions of the image. Preferably, in the early stage of training, the focus is on performing reconstruction-driven Gaussian parameter optimization and steps S1-S4 (corresponding to region-level Gaussian densification) to enhance the representation ability of complex structures and high-distortion regions; in the middle stage of training, the focus is on performing steps S5-S7 (i.e., candidate fusion Gaussian pair selection and Gaussian merging) to compress redundant Gaussians in smoothed regions or over-reconstructed regions; in the later stage of training, the total loss function is used to further refine and optimize the parameters of the updated image reconstruction model, and Gaussians and / or fused Gaussians are selectively added to the image reconstruction model again according to the remaining budget or error distribution. Training ends when a preset stopping condition is met. The stopping condition may include at least one of the following: reaching the maximum number of training rounds, the decrease in reconstruction loss being lower than a preset loss threshold, or the number of Gaussians reaching a preset budget upper limit.

[0121] After the stopping condition is met, the optimized image reconstruction model, optimized by alternating steps S1-S4 and S5-S7, is output as the final efficient Gaussian representation of the original image. Under Gaussian budget constraints, the final image reconstruction model allocates Gaussian resources more rationally than existing methods, enabling more complete representation of complex image regions while reducing redundant Gaussian occupancy in smooth regions, thereby improving overall image reconstruction quality and representation efficiency.

[0122] As described above, the embodiments of this application jointly model the locality information of image space and Gaussian space, and dynamically perform region-level densification and similar Gaussian merging during Gaussian training, thereby improving the efficiency of Gaussian capacity utilization, enhancing the representation ability of complex structural regions of the image, reducing redundant Gaussian occupancy in smooth or over-represented regions of the image, and ultimately achieving more efficient Gaussian image representation.

[0123] In summary, this application does not simply allocate Gaussians before training begins through initialization, nor does it only perform local patching on high-error pixels during training. Instead, it establishes a localized perceptual density control mechanism (corresponding to the scheme described in steps S1-S7 of the figure) that runs throughout the training process, enabling the Gaussian budget to be continuously and dynamically adjusted based on image content and optimization status. This mechanism utilizes the continuity of local errors in image space to perform more reasonable region-level densification in under-reconstructed areas, and utilizes local similarity in Gaussian space to merge redundant Gaussians in over-reconstructed areas, thus forming a unified framework that coordinates densification and compression. Compared with existing methods, this application introduces the local continuous error structure into the Gaussian densification decision, avoiding the problem that most existing densification methods rely on single-pixel errors, which easily leads to a scattered distribution of newly added Gaussians. Furthermore, by introducing a distortion gradient field and a local distortion consistency metric, this application enables the densification operation to more accurately correspond to regional complex structures such as edges and textures in the image, thereby improving the effectiveness of Gaussian addition. Furthermore, this application proposes a similarity-driven Gaussian merging mechanism for 2D image representation. Existing Gaussian pruning schemes typically employ direct pruning, but in 2D image representation, simple deletion can destroy the expressive power of detailed regions. This application compresses redundant Gaussians through merging rather than deletion, preserving original local information during updates to position, color, and covariance parameters. Therefore, it can reduce the number of Gaussians while maintaining reconstruction quality as much as possible. Further, this application introduces a local color consistency constraint during the training phase, making locally adjacent Gaussians more similar in color attributes, providing a more stable similarity basis for subsequent Gaussian merging. This constraint further enhances the reliability and robustness of the merging strategy. Therefore, the above embodiments of this application revolve around the core factor of "locality" to address the problem of insufficient utilization of local structural information during the training Gaussian optimization process in existing 2D Gaussian image representation methods. Simultaneously, a density control mechanism is established at both the image space and Gaussian space levels, thereby improving Gaussian budget utilization efficiency and image reconstruction results without altering the original 2D Gaussian representation and decoding process, and exhibiting good versatility and scalability.

[0124] Based on the inventive concept of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in the above embodiments. The following is in conjunction with... Figure 7 Please provide a detailed explanation.

[0125] like Figure 7 As shown, it illustrates the electronic device 100 of this application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0126] Processor 110 is used to control the operation of electronic devices. Processor 110 may also be referred to as a CPU (Central Processing Unit). Processor 110 may be an integrated circuit chip with signal processing capabilities. Processor 110 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor may be a microprocessor, or processor 110 may be any conventional processor.

[0127] The memory 120 is used to store computer programs and may be RAM, ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage terminals or flash memory terminals. In some embodiments, the non-transitory computer-readable storage media in the memory 120 is used to store at least one line of program code.

[0128] The processor 110 is used to execute computer programs stored in the memory 120 to implement the methods described in the various method embodiments of this application.

[0129] In some embodiments, the electronic device may further include a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, memory 120, and peripheral terminal interface 130 may be connected via a bus or signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, signal line, or circuit board. Specifically, the peripheral terminal includes at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160, and a power supply 170.

[0130] The peripheral terminal interface 130 can be used to connect at least one I / O (Input / Output) related peripheral terminal to the processor 110 and the memory 120. In some embodiments, the processor 110, memory 120 and peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 110, memory 120 and peripheral terminal interface 130 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0131] The radio frequency (RF) circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 140 communicates with communication networks and other IoT devices via electromagnetic signals; it is the communication circuit of the electronic device. The RF circuit 140 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identification module card, etc. The RF circuit 140 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 140 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0132] Display screen 150 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 150 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 110 for processing. In this case, display screen 150 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 150, located on the front panel of the electronic device; in other embodiments, there may be at least two display screens, located on different surfaces of the electronic device or in a folded design; in still other embodiments, display screen 150 may be a flexible display screen, located on a curved or folded surface of the electronic device. Furthermore, display screen 150 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 150 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0133] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, converting the sound waves into electrical signals that are input to the processor 110 for processing, or input to the radio frequency circuit 140 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned in a different part of the electronic device. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker may be a conventional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0134] Power supply 170 is used to supply power to various components in an electronic device. Power supply 170 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0135] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiments of this application, please refer to the descriptions in the above-described method embodiments of this application, which will not be repeated here.

[0136] In the embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the embodiments of the electronic devices described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some data may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0138] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0139] Based on the inventive concept of the above embodiments, this application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the method described in any of the above embodiments. The following is in conjunction with... Figure 8 This describes the execution process of the above embodiments on a computer-readable storage medium.

[0140] like Figure 8 As shown, it illustrates the computer-readable storage medium of this application. The integrated units described above, if implemented as software functional units and sold or used as independent products, can be stored in the computer-readable storage medium 200. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions / computer programs to cause an Internet of Things device (which may be a personal computer, server, or network terminal, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as electronic terminals such as computers, mobile phones, laptops, tablets, and cameras that have the aforementioned storage media.

[0141] The execution process of program data in a computer-readable storage medium can be described with reference to the above-described method embodiments of this application, and will not be repeated here.

[0142] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0143] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

Claims

1. A method for training an image reconstruction model, characterized in that, The method includes: acquiring an original image and using the original image to iteratively train an initial Gaussian model until convergence, so as to use the converged Gaussian model as an image reconstruction model, wherein the initial Gaussian model takes the original image as input and the reconstructed image corresponding to the original image as output, and the method performs the following step: adding a new Gaussian to the Gaussian model obtained after a first preset number of iterations of training, so as to obtain the initial Gaussian model required for the next iteration of training: Based on the original image and the reconstructed image, the reconstruction error of the pixels in the original image is calculated; Based on the reconstruction error, a candidate densification score for the pixel is determined, and based on the candidate densification score, a candidate densification set consisting of a preset number of pixels is determined. The pixels in the candidate densification set are divided into regions to obtain a candidate densification region composed of multiple pixels; Based on the candidate densification score of the pixels in the candidate densification region, Gaussian parameters are determined, and a new Gaussian model with the parameters of the Gaussian parameters is added to the Gaussian model obtained after training for a first preset number of iterations, so as to obtain the initial Gaussian model required for the next iteration training. 2.The method of claim 1, wherein, The step of determining a candidate densification set consisting of a preset number of pixels based on the reconstruction error includes: The reconstruction error is used as a candidate density enhancement score for the pixel; The candidate density enhancement score is compared with a preset first score threshold, and the pixel is selected as a candidate density enhancement point when the candidate density enhancement score is greater than or equal to the first score threshold. The candidate densification points are used to form the candidate densification set. 3.The method of claim 1, wherein, The step of determining a candidate densification set consisting of a preset number of pixels based on the reconstruction error includes: Gradient analysis is performed on the reconstruction error to obtain the distortion gradient direction of the pixel; Based on the distortion gradient direction, the local distortion consistency of the pixel is calculated; Based on the reconstruction error and the consistency of local distortion, the candidate densification score of the pixel is calculated. The candidate density enhancement score is compared with a preset second score threshold, and the pixel is selected as a candidate density enhancement point when the candidate density enhancement score is greater than or equal to the second score threshold. The candidate densification points are used to form the candidate densification set.

4. The method of training an image reconstruction model of claim 3, wherein, The local distortion consistency of the pixel is calculated based on the distortion gradient direction in the following manner: wherein, denotes a pixel of local distortion consistency, denotes a local window centered at a distortion gradient direction of the pixel , and a side length of the local window.

5. The method of training an image reconstruction model of claim 4, wherein, Based on the reconstruction error and the consistency of local distortion, the candidate enhancement score of the pixel is calculated in the following manner: wherein, denotes the candidate densification score of a pixel denotes the reconstruction error of a pixel denotes the reconstruction error of a pixel​ 6. The training method for the image reconstruction model according to claim 2, characterized in that, The step of dividing the pixels in the candidate enrichment set into regions to obtain candidate enrichment regions composed of multiple pixels includes: The candidate intensification points are marked as foreground pixels, and other pixels outside the candidate intensification regions are marked as background pixels to obtain a binarized image; The foreground pixels in the binarized image are traversed using a neighborhood connection strategy, and the foreground pixels that are spatially connected are divided into a candidate densification region.

7. The training method for the image reconstruction model according to claim 6, characterized in that, The Gaussian parameters include: position parameters and covariance parameters. The step of determining the Gaussian parameters based on the candidate densification scores of the pixels in the candidate densification region, and adding a new Gaussian model with the parameters of the aforementioned Gaussian parameters to the Gaussian model obtained after training for a first preset number of iterations, includes: The candidate density enhancement score weight of the pixel is calculated based on the candidate density enhancement score of the pixel in the candidate density enhancement region and the candidate density enhancement scores of all pixels in the candidate density enhancement region. Based on the candidate densification score weights and the pixel's position coordinates, the center position parameter of the pixel is calculated, and the center position parameter is used as the new Gaussian position parameter; The covariance parameter of the new Gaussian is calculated based on the candidate densification score weights, the location coordinates, and the location parameters.

8. An application method for an image reconstruction model, characterized in that, The method includes: using the image reconstruction model trained by the training method of the image reconstruction model as described in any one of claims 1-7, to calculate the reconstructed image corresponding to the original image based on the acquired original image.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the training method as described in any one of claims 1-7 or the application method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the training method according to any one of claims 1-7 or the application method according to claim 8.

Images