Image processing method and system, terminal and storage medium
By using image decomposition and Gaussian particle fitting, the image is decomposed into structure and texture maps, which are then processed using Gaussian particles. This solves the storage space problem of pixel-level representation and achieves efficient image compression and quality preservation.
Patent Information
- Application Number
- CN202511770977.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-02-24
AI Technical Summary
Existing technologies rely on pixel-level image representation, resulting in large image space requirements and difficulty in meeting the needs of efficient storage and transmission. In particular, when processing images with complex textures and details, they are prone to causing edge blurring or loss of details.
The image is decomposed into a structure map and a texture map using an image decomposition algorithm. The first and second types of Gaussian particles are used for fitting, respectively. The first type of Gaussian particles are used for the structure map, and the second type of Gaussian particles are used for the texture map. The parameter set is used to reconstruct the image.
It effectively reduces image space usage, improves image compression efficiency and quality, adapts to fitting complex textures and details, and overcomes the bottlenecks of traditional methods.
Smart Images
Figure CN121564013A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to an image processing method, system, terminal and storage medium. Background Technology
[0002] Image representation and compression techniques are crucial in the fields of computer vision and image data processing. With the advancement of science and technology, the demands on image processing are constantly increasing.
[0003] Currently, traditional image representation methods rely on pixel-level representation, which directly stores and processes the pixel values of the image. The problem with existing technologies is that this pixel-level representation requires significant storage space, hindering efforts to reduce the image's footprint.
[0004] Therefore, the relevant technologies still need to be improved and developed. Summary of the Invention
[0005] The main objective of this application is to provide an image processing method, system, terminal, and storage medium, aiming to solve the technical problem in related technologies that image representation methods relying on pixel level are not conducive to reducing image space usage.
[0006] To achieve the above objectives, a first aspect of this application provides an image processing method, wherein the method includes: Obtain the image to be processed; The image to be processed is decomposed using a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed. The structure diagram above is fitted with the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the structure diagram. The parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency and non-negative color parameters. The texture image is fitted with the second type of Gaussian particles to obtain the set of parameters of the second type of Gaussian particles corresponding to the texture image. The parameters of the second type of Gaussian particles include center position, covariance, rotation angle, transparency and negative color parameter. Based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set, the particle parameter set corresponding to the image to be processed is obtained.
[0007] Optionally, the aforementioned preset image decomposition algorithm includes at least one of total variation denoising algorithm, wavelet transform and Fourier decomposition.
[0008] Optionally, the above-mentioned fitting of the texture map based on the second type of Gaussian particles to obtain the second type of Gaussian particle parameter set corresponding to the texture map includes: The texture map described above is used as the image to be fitted. The above image to be fitted is divided into image blocks to obtain the image blocks corresponding to the above image to be fitted. Based on the image patch corresponding to the image to be fitted, determine the initial parameters of the particles in the current round; Based on the initial parameters of the particles mentioned above, the image to be fitted is fitted using the second type of Gaussian particles to obtain the set of Gaussian particle parameters for the current round. Based on the image to be fitted and the texture reconstruction image determined by the Gaussian particle parameter set of the current round, the image to be fitted is updated, and the above steps of image segmentation of the image to be fitted are returned and executed until the preset processing round termination condition is met, and the second type of Gaussian particle parameter set corresponding to the texture map is obtained according to the Gaussian particle parameter set of all rounds.
[0009] Optionally, in each round of processing, the above-mentioned image to be fitted is fitted based on a preset number of second-type Gaussian particles in a single round; The termination conditions for the above processing rounds include reaching a preset round threshold.
[0010] Optionally, the above-mentioned image segmentation of the image to be fitted to obtain the image blocks corresponding to the image to be fitted includes: Use the above image to be fitted as the initial image patch; Calculate the variance of each image patch corresponding to the above image to be fitted, and segment the image patch with the largest variance; The image blocks corresponding to the image to be fitted are updated based on the image blocks obtained from the segmentation, and the process of calculating the variance of each image block corresponding to the image to be fitted is returned until the preset image block termination condition is met. The aforementioned image segmentation termination condition includes: the number of image blocks corresponding to the image to be fitted reaches the aforementioned preset number for a single round.
[0011] Optionally, determining the initial particle parameters for the current round based on the image patch corresponding to the image to be fitted includes: The image blocks corresponding to the above-mentioned images to be fitted are sorted in ascending order according to variance, and the above-mentioned single-round preset number of image blocks are selected as target image blocks in turn. For each target image block, the initial center position of the target particle is determined based on the center coordinates of the target image block, the initial covariance of the target particle is determined based on the length and width of the target image block, and the initial negative color value of the target particle is determined based on the average pixel value of the target image block. The target particle is a second type of Gaussian particle corresponding to the target image block, and the target image block corresponds one-to-one with the second type of Gaussian particle.
[0012] Optionally, the above method further includes: Based on the particle parameter set corresponding to the above-mentioned image to be processed, the image is reconstructed to obtain the overall reconstructed image corresponding to the above-mentioned image to be processed.
[0013] A second aspect of this application provides an image processing system, wherein the system comprises: The data acquisition module is used to acquire the image to be processed; The image decomposition module is used to decompose the image to be processed using a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed. The first fitting module is used to fit the above structure diagram based on the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the above structure diagram. The parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency and non-negative color parameters. The second fitting module is used to fit the texture map based on the second type of Gaussian particles to obtain the set of parameters of the second type of Gaussian particles corresponding to the texture map. The parameters of the second type of Gaussian particles include center position, covariance, rotation angle, transparency and negative color parameters. The particle parameter set determination module is used to obtain the particle parameter set corresponding to the image to be processed based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set.
[0014] A third aspect of this application provides a terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of any of the above-described image processing methods.
[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described image processing methods.
[0016] As can be seen from the above, in this application, the following steps are taken: First, an image to be processed is acquired. Then, an image decomposition algorithm is used to decompose the image to obtain a structure map and a texture map corresponding to the image. Next, the structure map is fitted with a first type of Gaussian particle to obtain a first type of Gaussian particle parameter set corresponding to the structure map. The parameters of the first type of Gaussian particle include center position, covariance, rotation angle, transparency, and non-negative color parameters. Finally, the texture map is fitted with a second type of Gaussian particle to obtain a second type of Gaussian particle parameter set corresponding to the texture map. The parameters of the second type of Gaussian particle include center position, covariance, rotation angle, transparency, and non-negative color parameters. Based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set, a particle parameter set corresponding to the image to be processed is obtained.
[0017] Therefore, compared with existing technologies, this application does not rely on pixel-level image representation. Instead, for the image to be processed, image decomposition is first performed to obtain a structure map and a texture map, and then the structure map and texture map are processed separately. Specifically, for the structure map, a first type of Gaussian particle is used for fitting, and for the texture map, a second type of Gaussian particle is used for fitting, and finally the particle parameter set corresponding to the image to be processed is determined. Thus, using a Gaussian particle-based image representation to characterize the image to be processed helps to reduce the image's storage space. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the morphology of the first type of Gaussian particle swarm provided in an embodiment of this application; Figure 3 This is a schematic diagram of the morphology of the second type of Gaussian particle swarm provided in an embodiment of this application; Figure 4 This is a schematic diagram of the constituent modules of an image processing system provided in an embodiment of this application; Figure 5 This is a block diagram illustrating the internal structure of a terminal provided in an embodiment of this application. Detailed Implementation
[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.
[0021] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0022] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0023] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0024] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to classification." Similarly, the phrases "if determined" or "if classified to [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once classified to [the described condition or event]," or "in response to classification to [the described condition or event]."
[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0026] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.
[0027] Traditional image representation methods primarily rely on pixel-level representation, directly storing and processing the pixel values of the image. However, this method has significant limitations when storing, transmitting, and processing large-scale image data.
[0028] To reduce image file size, image compression techniques can be used. In some applications, lossless compression can be employed, preserving image information completely, but with a relatively low compression ratio. In other applications, lossy compression can be used, sacrificing some details imperceptible to the human eye to achieve a higher compression ratio. However, when processing images with complex textures and details, it can easily lead to blurred edges or loss of detail, thus affecting image quality. Furthermore, as image resolution and detail increase, the storage and transmission burdens on these compression methods further increase.
[0029] Traditional pixel-level image representations or representations based on simple features struggle to effectively fit complex textures and edge structures in images, especially in high-detail scenes. Improving compression ratios often requires sacrificing image quality; while lossless compression maintains quality, it suffers from lower compression ratios and space utilization, failing to meet the demands of efficient storage. With the continuous increase in image resolution, existing technologies face immense pressure in terms of storage requirements and computational overhead.
[0030] In some application scenarios, image representation can also be achieved based on particle representation methods. However, ordinary particle representations are mostly fixed on a single shape or parameter configuration, which makes them less adaptable to diverse image features and unable to efficiently process various complex textures and image details.
[0031] To address at least one of the aforementioned technical problems, this application proposes an image processing method. The method involves: acquiring an image to be processed; performing image decomposition on the image to be processed using a preset image decomposition algorithm to obtain a structure map and a texture map corresponding to the image to be processed; fitting the structure map based on first-type Gaussian particles to obtain a set of first-type Gaussian particle parameters corresponding to the structure map, wherein the parameters of the first-type Gaussian particles include center position, covariance, rotation angle, transparency, and non-negative color parameters; fitting the texture map based on second-type Gaussian particles to obtain a set of second-type Gaussian particle parameters corresponding to the texture map, wherein the parameters of the second-type Gaussian particles include center position, covariance, rotation angle, transparency, and non-negative color parameters; and obtaining a set of particle parameters corresponding to the image to be processed based on the first-type Gaussian particle parameter set and the second-type Gaussian particle parameter set.
[0032] Therefore, compared with existing technologies, this application does not rely on pixel-level image representation. Instead, for the image to be processed, image decomposition is first performed to obtain a structure map and a texture map, and then the structure map and texture map are processed separately. Specifically, for the structure map, a first type of Gaussian particle is used for fitting, and for the texture map, a second type of Gaussian particle is used for fitting, and finally the particle parameter set corresponding to the image to be processed is determined. Thus, using a Gaussian particle-based image representation to characterize the image to be processed helps to reduce the image's storage space.
[0033] Specifically, the proposed solution enables image representation and compression using negative Gaussian particles based on two-stream processing and quadtree segmentation initialization. Specifically, the original image to be processed is first decomposed into a structure map and a texture map using a total variational decomposition algorithm, and then these two parts are processed separately.
[0034] The parameters of a regular Gaussian particle (i.e., a type I Gaussian particle) include center position, covariance, rotation angle, color, and transparency, with the color parameter requiring a positive value. Regular Gaussian particles excel at fitting low-frequency component-rich image parts such as structure maps, which typically contain smooth regions and large-scale geometry. Therefore, using regular Gaussian particles directly to represent structure maps simplifies processing.
[0035] However, texture maps contain detailed parts of an image, such as edges, textures, and noise. Ordinary Gaussian particles require a large number of particles to fit these details, resulting in significant computational overhead and long training times. Furthermore, ordinary Gaussian particle representation methods rely on randomly initializing Gaussian point groups, which significantly increases the iterative fitting time. To address these issues, this application proposes an improved Gaussian particle model (i.e., the model corresponding to the second type of Gaussian particle), where the particle color parameter can be negative. Negative Gaussian particles have stronger expressive power and can better fit complex texture structures. Introducing negative Gaussian particles also allows for the correction of errors introduced during rasterization through novel Gaussian rendering, thereby improving the robustness of the fitting process.
[0036] To further improve image fitting processing efficiency, this embodiment uses a quadtree segmentation algorithm to segment the image into blocks before performing Gaussian fitting on the texture map. This segmentation method initializes the parameters of the Gaussian particles based on the center position and size of the segmented image blocks, replacing the random initialization step. This improvement significantly reduces the number of iterations in the fitting process, thereby improving overall efficiency.
[0037] Furthermore, the proposed solution can utilize quadtree segmentation for initialization and individual optimization to reduce residuals, as well as simulated annealing algorithm for overall particle parameter optimization, to further improve fitting accuracy. After fitting a batch of Gaussian particles, the residual map is used to fit the next batch of particles, thereby achieving a more accurate image representation.
[0038] Through the above-described scheme, this application significantly reduces the number of particles required while maintaining high fitting accuracy, effectively reducing memory consumption and computational resource demands. Simultaneously, this application's scheme combines high edge and detail fitting capabilities with excellent compression performance, providing a novel solution for the efficient representation and compression of complex images, overcoming the bottlenecks faced by existing technologies in the efficient storage, transmission, and processing of complex images.
[0039] like Figure 1 As shown in the figure, this application provides an image processing method, which specifically includes the following steps: Step S100: Obtain the image to be processed.
[0040] The images to be processed mentioned above are stored using a pixel-level image representation method. It should be noted that the images to be processed can be medical images, satellite images, or game images. Based on the solution of this application, high-fidelity compression and storage of medical images, representation and transmission of complex terrain in satellite images, and optimization of the expression of fine textures in game graphics rendering can be achieved, but this is not intended as a specific limitation.
[0041] Step S200: The image to be processed is decomposed using a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed.
[0042] The aforementioned preset image decomposition algorithm includes at least one of total variation denoising algorithm, wavelet transform and Fourier decomposition.
[0043] In this embodiment, the pre-defined image decomposition algorithm, Total Variation Denoising, is used as an example for specific explanation, but this is not intended to limit the scope. The Total Variation Denoising algorithm decomposes the original image to be processed into a structure map (low-frequency information) and a texture map (high-frequency information). The low-frequency structure map preserves the global contours and smooth regions of the image, while the high-frequency texture map extracts details and complex textures. This decomposition method effectively reduces the complexity of the image, making subsequent processing of the structure map and texture map more accurate and efficient. It also preserves important edge and texture details, making it suitable for detailed fitting of complex images.
[0044] Step S300: Fit the above structure diagram based on the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the above structure diagram. The parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency, and non-negative color parameters.
[0045] Step S400: Fit the texture map based on the second type of Gaussian particles to obtain the second type of Gaussian particle parameter set corresponding to the texture map. The parameters corresponding to the second type of Gaussian particles include center position, covariance, rotation angle, transparency, and negative color parameter.
[0046] In this application, the first type of Gaussian particle is a regular Gaussian particle, while the second type of Gaussian particle is an improved Gaussian particle. The difference between the two is that the color value of the first type of Gaussian particle is non-negative, while the color value of the second type of Gaussian particle can be negative.
[0047] For structure maps, a Gaussian image method is used directly based on Gaussian particles of type I. Gaussian particles of type I excel at fitting low-frequency component-rich parts of images like structure maps, which typically contain smooth regions and large-scale geometry. Using Gaussian particles of type I allows for adaptation to the smoothness characteristics of structure maps, achieving a better fit.
[0048] Specifically, when fitting the structure map using the GaussianImage method, the initial positions of the Gaussian particles can be generated using a stochastic algorithm (such as a uniform random distribution or a quasi-random sequence). The initial covariance matrix is typically set to the identity matrix or scaled according to the image resolution. Color values are initialized to the RGB values of the corresponding pixels. The parameter update process employs gradient descent-based optimization algorithms, such as Adaptive Moment Estimation (Adam) or Stochastic Gradient Descent (SGD), to minimize reconstruction errors, such as minimizing Mean Squared Error (MSE). The GaussianImage method optimizes this process by using an Accumulated Summation rendering method to improve fitting efficiency and combining it with Vector Quantization to reduce storage requirements while maintaining high fitting accuracy. In each iteration, the Gaussian particles generate a fitted image through rendering, which is then subtracted from the original image to calculate the loss function. Gradients are calculated via backpropagation and used to dynamically update the parameters of the Gaussian particles, making the fitting results more accurate and efficient. Based on the smoothing properties of the Gaussian distribution, the structure diagram can be fitted using particles without complex processing. Gaussian particles perform excellently when handling large smooth regions, accurately and efficiently representing low-frequency information. The use of a cumulative weighted sum rendering method reduces computational resource consumption.
[0049] For texture maps, to handle the rich high-frequency information, an improved type II Gaussian particle is used. This Gaussian particle supports adjustment of the elliptical shape and allows the color parameter to take negative values. Specifically, a two-dimensional Gaussian particle is a planar ellipse projected onto the xoy plane from a two-dimensional Gaussian function volume. The center position of the particle refers to the center coordinate of the planar ellipse, and the covariance of the planar ellipse in the x and y directions determines the flatness of the Gaussian particle in the x and y directions.
[0050] Although the projection of a 2D Gaussian function onto the xoy plane can cover the entire xoy plane, when projecting 2D Gaussian elliptical particles, the three sigma rule is generally used to truncate the Gaussian to limit the boundaries of a Gaussian elliptical particle. The Gaussian color value is essentially the amplitude A. Gaussian particles in the standard GaussianImage method are formed by projecting this standard Gaussian function onto a 2D plane, with at most some rotation angles to allow for a slight tilt in the projected elliptical particles. However, the GaussianImage method restricts the amplitude A of the Gaussian to be positive. This means that each Gaussian particle used to fit and reconstruct the image can only contribute positively to a particular pixel region. When the Gaussian particle swarm has already positively overfitted a pixel region of the image, newly added Gaussian particles, while potentially reducing the error loss in other pixel regions, will increase the error loss in those regions once they cover the already overfitted areas. In other words, ordinary 2D Gaussian particles lack error correction capabilities when fitting pixel images. In this application, a second type of Gaussian particle that supports negative color parameters is used, that is, the Gaussian function is set to... This change gives this new type of Gaussian particle the ability to correct errors when fitting images, and this Gaussian particle performs significantly better than ordinary Gaussian particles when fitting texture maps.
[0051] The contribution of traditional first-type Gaussian particles to image reconstruction can be represented as a function of the particle's position, covariance, tilt angle, transparency, and color value (the color parameter is a three-dimensional vector representing the color values of the RGB channels). The traditional GaussianImage method optimizes and iterates through a large model training process, optimizing a swarm of Gaussian particles whose parameters are pre-initialized (each particle has its own initial parameters as the starting point for training). After training, the parameters of each Gaussian particle are determined, resulting in the particle reconstruction result of an image. In this embodiment, the color value of the second-type Gaussian particles can be negative, and the parameter initialization and training processes differ from the GaussianImage method. These modifications collectively improve the training efficiency and the fitting effect on the high-frequency components of the image in this application's solution.
[0052] Figure 2This is a schematic diagram of the morphology of the first type of Gaussian particle swarm provided in the embodiments of this application. Each particle can change its size, tilt angle, transparency and color value through parameter adjustment during the training process. However, since the color value parameter can only be positive, this means that Gaussian particles that overlap each other can only enhance each other in terms of pixel value. This constraint makes ordinary Gaussian particles only able to fit images with more low-frequency components and less detailed texture well, while the effect is poor for images with more high-frequency components and more obvious differences in light intensity contrast.
[0053] Figure 3 This is a schematic diagram of the morphology of the second type of Gaussian particle swarm provided in the embodiments of this application. It should be noted that, in order to make the display effect more intuitive, Figure 3 It uses extended particles with single-channel color for display. For example... Figure 3 As shown, allowing negative color values during training allows the Gaussian particle swarm to exhibit more complex shapes, including crescent and funnel shapes, instead of just elliptical shapes. Some shapes even appear to resemble a circle with several corners cut off. This is a direct manifestation of how this extended Gaussian particle uses negative color values to cover the previous overfitting. It is precisely because of the error correction ability given to Gaussian particles by negative color values that the second type of Gaussian particle can achieve better results when fitting images with rich details and textures and high-frequency components with significant differences in light intensity contrast.
[0054] Thus, in this embodiment of the application, based on the two-stream decomposition operation, the original image to be processed is decomposed into a structure map and a texture map. Different processing methods are used for these two maps respectively, which is beneficial to improve the image processing effect, obtain better image compression effect, and reduce the image space occupied.
[0055] Specifically, the above-mentioned texture map is fitted based on the second type of Gaussian particles to obtain the set of second type Gaussian particle parameters corresponding to the texture map, including: The texture map described above is used as the image to be fitted. The above image to be fitted is divided into image blocks to obtain the image blocks corresponding to the above image to be fitted. Based on the image patch corresponding to the image to be fitted, determine the initial parameters of the particles in the current round; Based on the initial parameters of the particles mentioned above, the image to be fitted is fitted using the second type of Gaussian particles to obtain the set of Gaussian particle parameters for the current round. Based on the image to be fitted and the texture reconstruction image determined by the Gaussian particle parameter set of the current round, the image to be fitted is updated, and the above steps of image segmentation of the image to be fitted are returned and executed until the preset processing round termination condition is met, and the second type of Gaussian particle parameter set corresponding to the texture map is obtained according to the Gaussian particle parameter set of all rounds.
[0056] In each round of processing, the image to be fitted is fitted based on a preset number of second-type Gaussian particles in a single round. The termination conditions for the above processing rounds include reaching a preset round threshold.
[0057] Specifically, the above-mentioned image segmentation of the image to be fitted to obtain the image blocks corresponding to the image to be fitted includes: Use the above image to be fitted as the initial image patch; Calculate the variance of each image patch corresponding to the above image to be fitted, and segment the image patch with the largest variance; The image blocks corresponding to the image to be fitted are updated based on the image blocks obtained from the segmentation, and the process of calculating the variance of each image block corresponding to the image to be fitted is returned until the preset image block termination condition is met. The aforementioned image segmentation termination condition includes: the number of image blocks corresponding to the image to be fitted reaches the aforementioned preset number for a single round.
[0058] It should be noted that the above image segmentation termination condition may also include: the number of usable image blocks in the image blocks corresponding to the image to be fitted reaches the preset number for a single round, wherein the usable image blocks are image blocks whose variance is less than a preset variance threshold (the specific value can be set and adjusted according to actual needs). The image segmentation condition can also be set and adjusted according to actual needs, and is not specifically limited here.
[0059] Furthermore, the determination of the initial particle parameters for the current round based on the image patch corresponding to the image to be fitted includes: The image blocks corresponding to the above-mentioned images to be fitted are sorted in ascending order according to variance, and the above-mentioned single-round preset number of image blocks are selected as target image blocks in turn. For each target image block, the initial center position of the target particle is determined based on the center coordinates of the target image block, the initial covariance of the target particle is determined based on the length and width of the target image block, and the initial negative color value of the target particle is determined based on the average pixel value of the target image block. The target particle is a second type of Gaussian particle corresponding to the target image block, and the target image block corresponds one-to-one with the second type of Gaussian particle.
[0060] When using Gaussian particles for image fitting and reconstruction, it is necessary to initialize the parameters of the Gaussian particles and then adjust the parameter values through the training process. In this embodiment, for the second type of Gaussian particles, the parameters corresponding to each batch of particles are determined through different processing rounds. During processing, the number of particles processed in each round (i.e., the preset number per round) and the total number of processing rounds (i.e., the round threshold) can be determined according to the number of particles to be used.
[0061] In this embodiment of the application, the initial parameter values are not based on random selection, but instead a quadtree partitioning algorithm is used to determine the initial parameter values of each Gaussian initialization.
[0062] The quadtree segmentation algorithm works by calculating the variance of each image patch in the input image and then performing a quadtree segmentation on the image patch with the largest variance. For example, before the first segmentation, an image might have only one image patch—the image itself—which is a large image patch of 800*600 pixels. Since there is only one image patch, its variance is naturally the largest. Therefore, the first segmentation will cut it twice from the middle, creating four image patch regions of the same size, each 200*150 pixels, located at the four corners of the image. During the second segmentation, the variance of these four image patch regions will be calculated, and the patch with the largest variance will be segmented again.
[0063] Meanwhile, the training process of this application is an iterative process involving loops. Suppose we decide to use 5000 Gaussian particles of type II to reconstruct the texture map. In addition to inputting the parameter 5000, we also need to decide how many particles to sprinkle in each loop. Suppose it is 1000 particles, 5000 / 1000=5, then we will train five loops, and each loop will sprinkle only 1000 particles. That is, before each loop starts, we need to determine the initial parameter value of 1000 Gaussian particles. So, in this example, the quadtree segmentation algorithm will continue to segment until the number of image block regions is greater than or equal to 1000. After the quadtree segmentation algorithm is completed, the top 1000 image block regions with the highest variance values in ascending order are used as a reference to determine the initial parameter values of the 1000 Gaussian particles in this round of iteration. This is because the smaller the variance, the smaller the difference between each pixel in the image block region. Compared with image blocks with large variance, Gaussian particles are better at fitting image blocks with small variance and more low-frequency components. Therefore, image blocks with small variance are selected first, which can guide the initialized Gaussian particles to fit these regions of the image first.
[0064] Using this method, each Gaussian particle corresponds to an image block region. The initial position of the Gaussian particle is taken as the center coordinate of the corresponding pixel region. The initial covariance value of the Gaussian particle is calculated based on the length and width of the corresponding pixel region. The initial color value parameter of the Gaussian particle is taken as the average value of the pixels in the corresponding pixel region. The initial transparency of each Gaussian particle is uniformly set to the default value of 1, and the tilt angle is uniformly set to the default value of 0. In this way, the initial parameter values corresponding to the second type of Gaussian particle in each round of the loop are confirmed.
[0065] After initialization, the parameters of the 1000 particles in the current processing round are trained and iteratively optimized. Unlike the traditional GaussianImage method, which uses a strategy of scattering all Gaussian particles at once and then optimizing them together, this embodiment uses a strategy of scattering particles one by one and optimizing them one by one.
[0066] Specifically, each of the 1000 particles is taken out one by one (the initial shape, position, and color parameters of each particle are already determined), and they are scattered onto the image one by one. Then, the parameters of these particles are optimized one by one using the gradient descent algorithm, optimizing one particle at a time, until all 1000 particles in this round are scattered. After all 1000 particles in one round are scattered, all the parameters of the 1000 particles in this round are treated as a whole and optimized using the simulated annealing algorithm. At this point, the final position of the 1000 particles in the first round is completely determined, becoming part of the reconstructed image, and the set of Gaussian particle parameters corresponding to these 1000 particles in the round is also determined.
[0067] For the next round, subtract the current version of the texture reconstruction image (i.e., the image obtained by reconstructing the Gaussian particle parameter set finally determined in the current round) from the image to be fitted in the current round (the initial round is the texture image; except for the initial round, the image to be fitted in a round is the image used at the beginning of this round, i.e., the image that the particle needs to fit) to obtain a residual image. This residual image is the target of our fitting in the second round (i.e., the 1001st to 2000th particles), which is also the image to be fitted in the next round.
[0068] The quadtree segmentation algorithm is applied again to the residual image until the number of pixel blocks (i.e., image blocks in pixel form) on the residual image is greater than or equal to 1000. Then, the first 1000 pixel block regions are selected in ascending order of the variance of the pixel blocks. The initialization parameters of the 1001st to 2000th particles are initialized with the attributes of these pixel block regions. Then, these particles are sprinkled onto the image one by one and the parameters are optimized one by one. Finally, the overall optimization of the 1000 particles is performed.
[0069] This process continues until all 5000 particles have been dispersed, resulting in a fifth residual image. Subtracting this last residual image from the original texture image yields the reconstructed texture image obtained from the 5000 Gaussian particles. This reconstruction result is the reconstructed image corresponding to the texture image, which is obtained by superimposing 5000 Gaussian particles rendered from the parameter set of 5000 trained, negatively capable Gaussian particles. Furthermore, after the above operations are completed, the second type of Gaussian particle parameter set corresponding to the texture image is obtained based on the parameter sets of Gaussian particles from all rounds.
[0070] It should be noted that, in this embodiment, the example given is that the image segmentation termination condition is that the number of segmented pixel blocks reaches a preset number for a single round. In practical applications, a preset variance threshold can also be set, and the image segmentation termination condition can be set to the number of usable image blocks in the image blocks corresponding to the image to be fitted reaching the preset number for a single round. Here, the usable image blocks are image blocks whose variance is less than the preset variance threshold (the specific value can be set and adjusted according to actual needs), and no specific limitation is made here.
[0071] Based on the first and second types of Gaussian particle parameter sets mentioned above, the particle parameter set corresponding to the image to be processed is obtained. Thus, the parameters corresponding to all particles used for reconstructing the initial image to be processed are obtained. Saving this particle parameter set allows for image reconstruction based on this set to fit the image to be processed. Furthermore, the particle parameter set occupies relatively little storage space; therefore, using a Gaussian particle-based image representation to characterize the image to be processed helps reduce image storage space.
[0072] Furthermore, the above method also includes: reconstructing the image based on the set of particle parameters corresponding to the image to be processed, to obtain the overall reconstructed image corresponding to the image to be processed.
[0073] In this embodiment, the second type of Gaussian particles used have scalable particle shapes, enabling them to fit more complex textures, such as crescent and funnel shapes, thus enhancing expressive power. This improves the particle's ability to fit complex edges and texture details, overcoming the limitations of traditional Gaussian particles. Negative color parameters and diverse shapes reduce overfitting and improve the robustness of the fitting process.
[0074] Specifically, a quadtree segmentation algorithm is used to segment the texture map. A variance threshold is set, and blocks with variance exceeding the threshold are continuously segmented into quadtree nodes. Based on the position and size of the quadtree nodes, the position, size, and color of the first batch of Gaussian particles are initialized. The segmentation algorithm significantly improves the accuracy of the initial particle distribution, enabling each particle to fit the target region more efficiently. Furthermore, it can optimize the distribution of segmented particles, reducing the number of unnecessary particles.
[0075] During particle parameter optimization, the parameters (variance, transparency, color, and rotation angle) of each particle are adjusted individually. The optimal parameter values are determined by minimizing the residual between the fitted image and the original texture image. Minimizing the residual improves the fitting accuracy, making the fitted texture image closer to the real image. Step-by-step optimization avoids the computational complexity that may occur in global optimization.
[0076] Furthermore, for each batch of Type II Gaussian particles, simulated annealing was performed to optimize their parameters. The impact of parameter perturbation on the fitting quality was evaluated, and a decision on whether to accept the perturbation was made based on the results. Simulated annealing optimization enhanced the overall consistency of particle fitting, further reduced residuals, and improved the global optimization capability of the fitting process.
[0077] The difference between the texture map and the fitted map is used to generate a residual map. This new residual map is then used as the target to repeat the quadtree segmentation, particle initialization, and simulated annealing optimization process. Gaussian particle spraying and optimization are performed iteratively until the number of particles or the residual reaches a set threshold. This multi-iterative process gradually reduces the residual, allowing for a more accurate representation of the high-frequency information of the texture, achieving a good balance between particle number and fitting accuracy.
[0078] Finally, image reconstruction can be performed based on the particle parameter set corresponding to the image to be processed, obtaining the overall reconstructed image corresponding to the image to be processed. Specifically, the structure map and texture map are merged and fitted to generate the final image file, achieving image compression and reducing memory usage. The compressed image occupies less storage space while maintaining high-quality representation, making it easier to transmit and store.
[0079] Therefore, compared with existing technologies, this application does not rely on pixel-level image representation. Instead, for the image to be processed, image decomposition is first performed to obtain a structure map and a texture map, and then the structure map and texture map are processed separately. Specifically, for the structure map, a first type of Gaussian particle is used for fitting, and for the texture map, a second type of Gaussian particle is used for fitting, and finally the particle parameter set corresponding to the image to be processed is determined. Thus, using a Gaussian particle-based image representation to characterize the image to be processed helps to reduce the image's storage space.
[0080] We use an improved version of the second type of Gaussian particles (with negative color values and deterministic initial particle parameters) to fit the texture map. This approach significantly reduces the number of iterations required for the GaussianImage model to converge during texture map fitting, resulting in shorter training time and higher training efficiency. Furthermore, the error between the fitted texture map and the original texture map is lower. In addition, this application allows negative color parameter values for the Gaussian particles participating in training. This means that image patches overfitted by previous Gaussian particle swarms have the opportunity to be corrected by newly arriving Gaussian particles using negative color values, which greatly improves the robustness of this application when using a Gaussian particle ensemble to fit images.
[0081] In general, in this embodiment, the original image to be processed is divided into a structure map and a texture map. The first type of Gaussian particles continue to fit the structure map, which contains rich low-frequency components, and is well-suited for fitting this type. For the texture map fitting, a second type of Gaussian particle is used, allowing the color parameters of the Gaussian particles to be negative. These two improvements enable this application to achieve better image fitting results (reduced loss) with fewer training iterations (reduced training time), meaning that higher quality image effects can be rendered using fewer Gaussian particles. The combination of two-stream decomposition and custom particles significantly reduces the number of particles required for fitting, reducing storage and computational resource consumption. The innovative design for high-frequency textures enhances detail fitting capabilities and achieves higher representational accuracy.
[0082] Specifically, a dual-stream processing framework decomposes the original image into a structure map (low-frequency information) and a texture map (high-frequency information), fully utilizing the characteristics of both for targeted optimization. By optimizing low-frequency and smooth regions (structure map) and complex high-frequency regions (texture map) respectively, the computational complexity of fitting is reduced and the representation accuracy is improved.
[0083] Using second-type Gaussian particles, the particle expression capability surpasses that of traditional Gaussian particles. It supports complex shape fitting (such as crescent and funnel shapes) and corrects overfitting through negative color values, significantly improving edge and detail expression capabilities while increasing the robustness of the fitting process.
[0084] The texture map is segmented using a quadtree segmentation algorithm. The position, size, and color of Gaussian particles are initialized based on the image patch variance, thereby optimizing the particle distribution. Accurate segmentation of high-frequency information regions allows the initial particles to more effectively cover key areas, improving fitting efficiency.
[0085] After particle fitting is complete, simulated annealing is used for overall optimization, combined with a perturbation strategy to reduce residuals and improve the global fitting effect. This ensures the synergy of overall particle parameters and improves the fitting accuracy of the final image. By generating and successively optimizing the residual map, high-frequency details are iteratively fitted, and the loop continues until the accuracy or particle number limit is met. An optimal balance is achieved between the number of particles and fitting accuracy, reducing the total number of particles required for fitting while retaining the compression effect.
[0086] It should be noted that in some application scenarios, the particle model can be further extended to irregular polygons or other shapes to adapt to specific scenario requirements. The optimization algorithm can use genetic algorithms or particle swarm optimization algorithms instead of simulated annealing to further improve efficiency. K-means clustering or graph cutting algorithms can also be used instead of quadtree segmentation, segmenting based on feature distribution to obtain a more detailed region distribution.
[0087] like Figure 4 As shown, corresponding to the above image processing method, this application embodiment also provides an image processing system, which includes: Data acquisition module 410 is used to acquire the image to be processed; The image decomposition module 420 is used to decompose the image to be processed by a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed. The first fitting module 430 is used to fit the above structure diagram based on the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the above structure diagram. The parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency and non-negative color parameters. The second fitting module 440 is used to fit the texture map based on the second type of Gaussian particles to obtain the second type of Gaussian particle parameter set corresponding to the texture map. The parameters corresponding to the second type of Gaussian particles include center position, covariance, rotation angle, transparency and negative color parameter. The particle parameter set determination module 450 is used to obtain the particle parameter set corresponding to the image to be processed based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set.
[0088] Therefore, this application does not rely on pixel-level image representation. Instead, for the image to be processed, image decomposition is first performed to obtain a structure map and a texture map, and then the structure map and texture map are processed separately. Specifically, for the structure map, a first type of Gaussian particle is used for fitting, and for the texture map, a second type of Gaussian particle is used for fitting, and finally the particle parameter set corresponding to the image to be processed is determined. Thus, representing the image to be processed based on Gaussian particle image representation helps to reduce the image's space footprint.
[0089] It should be noted that the specific structure and implementation of the above image processing system and its various modules or units can be referred to the corresponding descriptions in the above method embodiments, and will not be repeated here.
[0090] It should be noted that the division of the modules in the above image processing system is not unique and is not intended as a specific limitation.
[0091] Based on the above embodiments, this application also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown. The terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps of any of the above-described image processing methods. The display screen can be a liquid crystal display (LCD) or an electronic ink display.
[0092] Those skilled in the art will understand that Figure 5 The block diagram shown is only a partial structural diagram related to the solution of this application and does not constitute a limitation on the terminal on which the solution of this application is applied. The specific terminal may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.
[0093] In one embodiment, a terminal is provided, the terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the image processing methods provided in the embodiments of this application.
[0094] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the image processing methods provided in this application.
[0095] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0097] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0099] In the embodiments provided in this application, it should be understood that the disclosed systems / terminal devices and methods can be implemented in other ways. For example, the system / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0100] If the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
[0101] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions are not in essence a departure from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. An image processing method, characterized in that, The method includes: Obtain the image to be processed; The image to be processed is decomposed using a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed. The structure diagram is fitted with the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the structure diagram. The parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency and non-negative color parameters. The texture map is fitted based on the second type of Gaussian particles to obtain the second type of Gaussian particle parameter set corresponding to the texture map. The parameters corresponding to the second type of Gaussian particles include center position, covariance, rotation angle, transparency and negative color parameter. Based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set, the particle parameter set corresponding to the image to be processed is obtained.
2. The image processing method according to claim 1, characterized in that, The preset image decomposition algorithm includes at least one of total variation denoising algorithm, wavelet transform and Fourier decomposition.
3. The image processing method according to claim 1, characterized in that, The process of fitting the texture map based on second-type Gaussian particles to obtain the second-type Gaussian particle parameter set corresponding to the texture map includes: The texture map is used as the image to be fitted. The image to be fitted is divided into image blocks to obtain the image blocks corresponding to the image to be fitted. Based on the image patch corresponding to the image to be fitted, determine the initial parameters of the particles in the current round; Based on the initial parameters of the particles, the image to be fitted is fitted using the second type of Gaussian particles to obtain the set of Gaussian particle parameters for the current round. Based on the image to be fitted and the texture reconstruction image determined based on the Gaussian particle parameter set of the current round, the image to be fitted is updated, and the step of image segmentation of the image to be fitted is returned and executed until the preset processing round termination condition is met, and the second type of Gaussian particle parameter set corresponding to the texture map is obtained based on the Gaussian particle parameter set of all rounds.
4. The image processing method according to claim 3, characterized in that, In each round of processing, the image to be fitted is fitted based on a preset number of second-type Gaussian particles in a single round; The termination condition for the processing round includes the processing round reaching a preset round threshold.
5. The image processing method according to claim 4, characterized in that, The step of dividing the image to be fitted into image blocks to obtain the image blocks corresponding to the image to be fitted includes: Use the image to be fitted as the initial image block; Calculate the variance of each image block corresponding to the image to be fitted, and segment the image block with the largest variance; The image blocks corresponding to the image to be fitted are updated based on the image blocks obtained from the segmentation, and the step of calculating the variance of each image block corresponding to the image to be fitted is returned until the preset image block termination condition is met. The image segmentation termination condition includes: the number of image blocks corresponding to the image to be fitted reaches the preset number for a single round.
6. The image processing method according to claim 5, characterized in that, The step of determining the initial particle parameters for the current round based on the image patch corresponding to the image to be fitted includes: The image blocks corresponding to the image to be fitted are sorted in ascending order according to variance, and a preset number of image blocks in a single round are selected as target image blocks in turn. For each target image block, the initial center position of the target particle is determined based on the center coordinates of the target image block, the initial covariance of the target particle is determined based on the length and width of the target image block, and the initial negative color value of the target particle is determined based on the average pixel value of the target image block. Here, the target particle is a second type of Gaussian particle corresponding to the target image block, and the target image block corresponds one-to-one with the second type of Gaussian particle.
7. The image processing method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the set of particle parameters corresponding to the image to be processed, the image is reconstructed to obtain the overall reconstructed image corresponding to the image to be processed.
8. An image processing system, characterized in that, The system includes: The data acquisition module is used to acquire the image to be processed; The image decomposition module is used to decompose the image to be processed using a preset image decomposition algorithm to obtain the structure map and texture map corresponding to the image to be processed. The first fitting module is used to fit the structure diagram based on the first type of Gaussian particles to obtain the first type of Gaussian particle parameter set corresponding to the structure diagram, wherein the parameters corresponding to the first type of Gaussian particles include center position, covariance, rotation angle, transparency and non-negative color parameters. The second fitting module is used to fit the texture map based on the second type of Gaussian particles to obtain the second type of Gaussian particle parameter set corresponding to the texture map, wherein the parameters corresponding to the second type of Gaussian particles include center position, covariance, rotation angle, transparency and negative color parameter. The particle parameter set determination module is used to obtain the particle parameter set corresponding to the image to be processed based on the first type of Gaussian particle parameter set and the second type of Gaussian particle parameter set.
9. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the image processing method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image processing method as described in any one of claims 1 to 7.