Three-dimensional model style conversion method and device

Abstract

The embodiment of the invention discloses a style conversion method and device for a three-dimensional model. The method comprises the steps of obtaining a Gaussian model corresponding to a three-dimensional scene and color distribution of a target color style in a first color space; performing color decoding on the Gaussian model to obtain first color distribution of each Gaussian kernel in a first color space; transforming the first color distribution of each Gaussian kernel in the first color space by using the color distribution of the target color style in the first color space to obtain the second color distribution of each Gaussian kernel in the first color space; and encoding the second color distribution of each Gaussian kernel in the first color space to obtain a Gaussian model corresponding to the target color style. The color distribution of the Gaussian kernel is transformed by using the target color style, and an automatic color style conversion method for the three-dimensional scene is provided.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for style conversion of a three-dimensional model. Background Art

[0002] As a comprehensive technology that combines computer vision, graphics, photogrammetry, machine learning and other multi-field technologies, 3D real-scene reconstruction plays an important role in urban planning, architectural design, game development, virtual reality and other fields. Compared with traditional 3D reconstruction methods, the 3D Gaussian model can achieve real-time rendering based on GPU (Graphics Processing Unit) or CUDA (Compute Unified Device Architecture), while showing excellent rendering quality, and can be used as an implementation solution for on-end rendering.

[0003] In recent years, image scene conversion technology has made significant progress, but it is still challenging to directly apply these technologies to 3D models. Traditionally, the style adjustment and conversion of 3D models rely on manual operations by professional 3D modelers, which is not only time-consuming and labor-intensive, but also difficult to batch process and automate. How to automatically achieve color style conversion of 3D scenes is still a technical problem to be explored. Summary of the invention

[0004] The present application provides a method and device for style conversion of a three-dimensional model, so as to automatically realize color style conversion of a three-dimensional scene.

[0005] This application provides the following solutions:

[0006] According to a first aspect, a method for style conversion of a three-dimensional model is provided, the method comprising: obtaining a Gaussian model corresponding to a three-dimensional scene and a color distribution of a target color style in a first color space; color decoding the Gaussian model to obtain a first color distribution of each Gaussian kernel in the first color space; using the color distribution of the target color style in the first color space, transforming the first color distribution of each Gaussian kernel in the first color space to obtain a second color distribution of each Gaussian kernel in the first color space; encoding the second color distribution of each Gaussian kernel in the first color space to obtain a Gaussian model corresponding to the target color style.

[0007] According to a second aspect, an image processing method is provided, the method comprising: determining a target perspective and a Gaussian model corresponding to a target color style, wherein the Gaussian model corresponding to the target color style is obtained by using the conversion method described in the first aspect above; rendering the target perspective using the Gaussian model corresponding to the target color style to obtain an image corresponding to the target color style.

[0008] According to a third aspect, a style conversion device for a three-dimensional model is provided, and the conversion device includes: a model acquisition unit, configured to acquire a Gaussian model corresponding to a three-dimensional scene; a target acquisition unit, configured to acquire a color distribution of a target color style in a first color space; a color decoding unit, configured to perform color decoding on the Gaussian model to obtain a first color distribution of each Gaussian kernel in the first color space; a color transformation unit, configured to transform the first color distribution of each Gaussian kernel in the first color space using the color distribution of the target color style in the first color space to obtain a second color distribution of each Gaussian kernel in the first color space; and a color encoding unit, configured to encode the second color distribution of each Gaussian kernel in the first color space to obtain a Gaussian model corresponding to the target color style.

[0009] According to a fourth aspect, an image processing device is provided, the device comprising:

[0010] a viewing angle determination unit configured to determine a target viewing angle;

[0011] A model determination unit is configured to determine a Gaussian model corresponding to a target color style, wherein the Gaussian model corresponding to the target color style is obtained by using the conversion device as described in the third aspect above;

[0012] The image rendering unit is configured to use the Gaussian model corresponding to the target color style to render the target perspective to obtain an image corresponding to the target color style.

[0013] According to a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the first and second aspects are implemented.

[0014] According to a sixth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, and the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the first and second aspects above.

[0015] According to a seventh aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method described in any one of the first and second aspects above.

[0016] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0017] The present application decodes the Gaussian model to obtain a first color distribution of the Gaussian kernel in the first color space, transforms the first color distribution of the Gaussian kernel using the color distribution of the target color style in the first color space to obtain a second color distribution, and obtains the Gaussian model corresponding to the target color distribution by encoding the second color distribution. The present application transforms the color distribution of the Gaussian kernel using the target color style, and provides an automated color style conversion method for a three-dimensional scene, which realizes the migration of a three-dimensional scene to a specific target style.

[0018] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A system architecture diagram applicable to the embodiments of the present application;

[0021] Figure 2 A flowchart of a style conversion method for a three-dimensional model provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the process of the style conversion method of a three-dimensional model provided in an embodiment of the present application;

[0023] Figure 4 A structural block diagram of a style conversion device for a three-dimensional model provided in an embodiment of the present application;

[0024] Figure 5 A structural block diagram of an image processing device provided in an embodiment of the present application;

[0025] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0027] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0028] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0029] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0030] 3D scene conversion is a computer vision and machine learning technology that can convert the color style of a 3D model to another color style, thereby generating a 3D model with new style features while retaining the main shape features of the original 3D model. The image rendered based on the 3D model with this new style feature also presents a new color style accordingly. The current 3D model color style conversion technology mainly relies on the manual operation of 3D modelers. This method not only consumes a lot of manpower costs, but also has certain limitations in technical effects.

[0031] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system architecture on which the present application is based is first described. Figure 1 An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system architecture may include: a user device, a three-dimensional model style conversion device and an image processing device located at a map application server.

[0032] The style conversion device of the three-dimensional model can convert the color style of the established Gaussian model of the three-dimensional scene by using the method provided by the embodiment of the present application in the offline or online stage to obtain a Gaussian model of another color style. In this way, in addition to the original Gaussian model of the color style, a Gaussian model of a three-dimensional scene can also have at least one Gaussian model of another color style.

[0033] The user can input the target color style through the user device, and the user device sends the target color style to the image processing device on the application server side. The image processing device can adopt the method provided in the embodiment of the present application, obtain the Gaussian model corresponding to the target color style based on the style conversion device of the three-dimensional model, and use the Gaussian model corresponding to the target color style to render the target perspective to obtain the image corresponding to the target style. The server side can send the image corresponding to the target style to the user terminal in response to the request of the user terminal.

[0034] Among them, user equipment may include but is not limited to: smart mobile terminals, smart home devices, wearable devices, PC (Personal Computer), etc. Among them, smart mobile devices may include such as mobile phones, tablet computers, laptops, PDAs (Personal Digital Assistants), Internet cars, etc. Smart home devices may include smart TVs, smart refrigerators, etc. Wearable devices may include such as smart watches, smart glasses, virtual reality devices, augmented reality devices, mixed reality devices (i.e., devices that can support virtual reality and augmented reality), etc.

[0035] The style conversion device and image processing device of the 3D model can be set as an independent server, or as a server group, or can be set as a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system to solve the defects of difficult management and weak service scalability in traditional physical hosts and virtual private servers (VPS). Figure 1 In addition to the illustrated architecture, the 3D reconstruction device may also be disposed on a computer terminal with relatively strong computing capabilities.

[0036] It should be understood that Figure 1 The user equipment and the style conversion device and image processing device of the three-dimensional model in the embodiment are only illustrative. According to the implementation requirements, there may be any number of user equipment and the style conversion device and image processing device of the three-dimensional model.

[0037] Figure 2 A flow chart of a method for converting the style of a three-dimensional model provided in an embodiment of the present application. The method can be performed by Figure 1 The style conversion device of the three-dimensional model in the system shown is performed. Figure 2 As shown in , the method may include the following steps:

[0038] Step 201: Obtain a Gaussian model corresponding to a three-dimensional scene and a color distribution of a target color style in a first color space.

[0039] Step 202: Perform color decoding on the Gaussian model to obtain a first color distribution of each Gaussian kernel in a first color space.

[0040] Step 203: using the color distribution of the target color style in the first color space, transform the first color distribution of each Gaussian kernel in the first color space to obtain the second color distribution of each Gaussian kernel in the first color space.

[0041] Step 204: Encode the second color distribution of each Gaussian kernel in the first color space to obtain a Gaussian model corresponding to the target color style.

[0042] As can be seen from the above process, the present application decodes the Gaussian model to obtain the first color distribution of the Gaussian kernel in the first color space, transforms the first color distribution of the Gaussian kernel using the color distribution of the target color style in the first color space to obtain the second color distribution, and obtains the Gaussian model corresponding to the target color distribution by encoding the second color distribution. The present application transforms the color distribution of the Gaussian kernel using the target color style, and provides an automated scene conversion method, which realizes the conversion of a three-dimensional scene to a specific target color style.

[0043] The following is a detailed description of each step in the above process and the effects that can be further produced in conjunction with the embodiments. It should be noted that the limitations such as "first" and "second" involved in the present disclosure do not have limitations in terms of size, order and quantity, and are only used to distinguish them in name, for example, "first color distribution" and "second color distribution" are used to distinguish two color distributions.

[0044] First, the above step 201, namely, "obtaining the Gaussian model corresponding to the three-dimensional scene and the color distribution of the target color style in the first color space", is described in detail in conjunction with the embodiment.

[0045] For ease of understanding, the three concepts of three-dimensional scene, three-dimensional model and Gaussian model involved in the embodiments of the present application are first briefly introduced.

[0046] The so-called 3D scene refers to a three-dimensional space environment created by 3D modeling technology, which is a simulation of the spatial relationship of the real world. For example, in the field of maps, a 3D scene is a simulation of the spatial relationship between buildings, terrain, vegetation, etc. in the real world.

[0047] The so-called three-dimensional model is a model expression of part or all of a three-dimensional scene.

[0048] The so-called Gaussian model is one of the expressions of a three-dimensional model. The present application performs color style conversion processing on the Gaussian model corresponding to a three-dimensional scene. The Gaussian model is a three-dimensional model obtained by three-dimensional Gaussian splatting (3D Gaussian Splatting).

[0049] The three-dimensional Gaussian splattering process is named because it uses three-dimensional Gaussian distribution as the basic unit of rendering. It is a method of restoring three-dimensional scenes based on three-dimensional Gaussian volumes, and has a good new perspective synthesis effect. The Gaussian model is essentially composed of a series of Gaussian kernels, which can be mapped to a specific perspective in the subsequent rendering process of the three-dimensional real scene model to generate an image. These Gaussian kernels are represented by Gaussian distribution, color and opacity, so a series of Gaussian kernels are actually encoded as Gaussian distribution, color distribution and opacity distribution respectively. After these distributions are initialized, they are continuously optimized through a series of training iterations. The Gaussian kernels obtained in this process constitute the Gaussian model corresponding to the three-dimensional scene.

[0050] In image processing and color theory, "color space" refers to a three-dimensional coordinate system used to define and represent colors. It describes how colors are combined and how to reproduce these colors in digital systems or physical devices. Different color spaces describe colors in different ways. For example, the RGB color space creates colors by combining the three basic colors of red, green, and blue, while the LAB color space describes colors based on human visual perception of color using combinations of lightness (L), green to red (A), and blue to yellow (B).

[0051] The color distribution of each Gaussian kernel can be represented by parameters such as mean, covariance, etc. The color distribution defines the color characteristics presented by the Gaussian kernel, which is usually the probability density distribution of the Gaussian kernel in the color space.

[0052] The target color style is the target style of the three-dimensional scene conversion. The target color style of the present application is represented by the color distribution in the first color space, and the subsequent conversion steps are guided according to the color distribution of the target color style in the first color space.

[0053] The first color space is a color space used to represent the color distribution of the Gaussian kernel when the application performs a color distribution transformation on the Gaussian kernel during a three-dimensional scene conversion. The first color space can be a space commonly used in the art for describing colors, such as a LAB color space, an RGB color space, and a CIE color space. Preferably, the first color space can be a LAB color space, which is a color model designed to be close to human visual perception, and it can better represent the color range and color differences seen by the human eye.

[0054] The LAB color space consists of three channels:

[0055] The L (Lightness) channel describes the brightness of the color, ranging from 0 (pure black) to 100 (pure white). The L channel has nothing to do with color and only represents the brightness information of the color.

[0056] The A (Green-Red axis) channel describes the degree to which the color tends to be between green and red. Positive values ​​tend to be green, and negative values ​​tend to be red. There is no fixed maximum or minimum limit for the value of the A channel, and the common range is [-128,127].

[0057] The B (Blue-Yellow axis) channel describes the color's tendency between blue and yellow, with positive values ​​tending toward yellow and negative values ​​tending toward blue. Like the A channel, the b channel value has no fixed maximum or minimum limit, and the common range is [-128,127].

[0058] The color distribution of the target color style in the first color space can be obtained in a variety of ways. For example, the target color style can be provided to the user for selection by the style conversion device of the three-dimensional model, and the color distribution of these styles can be pre-stored in the device and provided to the user for selection when the user has a scene conversion requirement.

[0059] As an implementable manner, when the target color style is represented by the color style of the target image, the color distribution of the target color style in the first color space can be obtained by acquiring the target image and counting the color distribution of the target image in the first color space.

[0060] As another feasible method, the target style can be described in the form of text, and then the target image can be generated according to the text. Specifically, the target text is obtained, and the target image corresponding to the target text is obtained using an image generation model (e.g., a large language model with a text-generated image function), and the color distribution of the target image in the first color space is counted.

[0061] Furthermore, if the target image is represented by the color distribution of the second color space, the color distribution of the target image in the second color space is statistically analyzed; the color distribution of the target image in the second color space is converted to the first color space to obtain the color distribution of the target image in the first color space.

[0062] The second color space is a color space different from the first color space, and the second color space may also be a space commonly used in the art for describing colors. For example, the second color space may be an RGB color space widely used in the art for color representation of electronic display screens.

[0063] The above step 202, namely "color decoding the Gaussian model to obtain a first color distribution of each Gaussian kernel in a first color space", is described in detail below in conjunction with an embodiment.

[0064] In the Gaussian model, each Gaussian kernel can represent a color distribution. By color decoding the Gaussian model, the color distribution of each Gaussian kernel in a specific color space can be obtained. Among them, color decoding of the Gaussian model can be achieved through a variety of techniques, for example, using a Gaussian mixture model to soft cluster color data, each Gaussian distribution represents a color cluster, and the parameters of the Gaussian distribution, such as the mean and covariance, are determined by the expectation maximization algorithm to achieve color decoding.

[0065] As an implementable manner, the spherical harmonic function of each Gaussian kernel in the Gaussian model may be decoded to obtain the color distribution of each Gaussian kernel in the first color space.

[0066] Spherical harmonics are a set of orthogonal basis functions defined on a sphere that can be used to expand any function on a sphere. In the field of computer graphics and vision, spherical harmonics are often used to efficiently represent and process ambient lighting and color distribution. In the Gaussian model, the color of each Gaussian kernel can be represented by a spherical harmonic function. This means that the color of each Gaussian kernel can be described by a set of spherical harmonic coefficients that define the color distribution when viewed from different directions. To obtain the color distribution of each Gaussian kernel in the first color space, the spherical harmonic coefficients need to be decoded into actual color values. This usually involves multiplying the spherical harmonic coefficients with the spherical harmonic function and accumulating the results to reconstruct the color in a specific direction. Finally, the spherical harmonic function and the corresponding coefficients can be used to render the scene to calculate the color distribution at different viewing angles.

[0067] In practical applications, the order of spherical harmonics is usually selected to be less than 3, because higher orders can provide more accurate color distribution, but the calculation cost will also increase significantly. By reasonably selecting the order and optimizing the calculation method, the consumption of computing resources can be effectively controlled while ensuring the rendering quality. Preferably, the first-order spherical harmonic coefficients can be used for calculation in this application.

[0068] As another possible approach, Figure 3 A process diagram of a style conversion method for a three-dimensional model provided in an embodiment of the present application, the present application can also decode the Gaussian model, for example, decode the spherical harmonics of each Gaussian kernel in the Gaussian model, and obtain the first color distribution of each Gaussian kernel in the second color space; convert the color distribution of each Gaussian kernel in the second color space to the first color space, and obtain the first color distribution of the Gaussian kernel in the first color space. For example, first decode the spherical harmonics of each Gaussian kernel in the Gaussian model to obtain the color distribution of each Gaussian kernel in the RGB color space (referred to as the first color distribution in the embodiment of the present application), and then convert it to the LAB color space to obtain the first color distribution of each Gaussian kernel in the LAB color space.

[0069] It should be noted that in the embodiment of the present application, converting the first color distribution of the second color space to the first color distribution of the first color space is an optional process. If the color representation used by the Gaussian model itself is in the first color space, the above conversion is not required. For example, if the Gaussian model itself uses the first color distribution of the LAB space, the above color space conversion process is not required.

[0070] The above step 203, i.e., "using the color distribution of the target color style in the first color space to transform the first color distribution of each Gaussian kernel in the first color space to obtain the second color distribution of each Gaussian kernel in the first color space" is described in detail below in conjunction with the embodiments.

[0071] The present application uses the target color style to guide the color transformation of the Gaussian model, and transforms the Gaussian kernel from the first color distribution to the second color distribution in the first color space. This process can be implemented using a machine learning model, for example, by using a convolutional neural network in deep learning to learn the characteristics of the target color style and apply it to the color distribution of the Gaussian kernel.

[0072] As an implementable method, the color distribution of the target color style in the first color space can be used as the migration target, and the color distribution of each Gaussian kernel in the first color space can be used as the migration object; the migration process is performed using the color distribution mean and standard deviation of the migration target to obtain the second color distribution of each Gaussian kernel in the first color space. The migration process can be implemented in a variety of ways, for example, applying a color migration algorithm, such as the Reinhard algorithm or the Welsh algorithm, etc., to achieve color migration by matching the color statistical characteristics of the migration target and the migration object.

[0073] As an implementable method, the migration processing can be: subtracting the color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain the third color distribution of each channel; scaling the third color distribution of each channel according to the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the migration object to obtain the fourth color distribution of each channel; superimposing the color distribution mean of the migration target to the fourth color distribution of each channel to obtain the second color distribution of each Gaussian kernel in the first color space.

[0074] Specifically, the second color distribution g of each Gaussian kernel in the first color space of each channel * It can be expressed as:

[0075]

[0076] in, is the color distribution mean of the migration object; is the standard deviation of the color distribution of the migration object; is the color distribution mean of the migration target; is the standard deviation of the color distribution of the migration target; g is the first color distribution of the Gaussian kernel in the first color space of each channel, and the third color distribution is The fourth color distribution is Each channel is processed according to the method shown in formula (1), and the second color distribution in each channel after conversion can be obtained, which together constitutes the processed Gaussian model G * The color of each Gaussian kernel in .

[0077] The present application can also use the source image corresponding to the Gaussian model for migration processing. The so-called source image refers to the image used when constructing the Gaussian model. Usually, multiple images are used when constructing the Gaussian model, and the source image can be one of the multiple images selected. First, the source image corresponding to the Gaussian model and the color distribution of the source image in the first color space are obtained, and then the migration processing is performed using the color distribution difference between the source image in the first color space and the migration target.

[0078] The target image may be obtained by processing the source image, and the processing method may be manual processing or Diffusion-based model processing, etc. The color transformation from the source image to the target image indicates the color transformation demand for the Gaussian model.

[0079] Among them, the migration processing can be: subtracting the color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain the third color distribution of each channel; using the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the source image, respectively scaling the third color distribution of each channel to obtain the fifth color distribution of each channel; using the difference between the color distribution mean of the migration target and the color distribution mean of the source image to obtain the adjusted mean; superimposing the adjusted mean and the color distribution mean of the migration object on the fifth color distribution of each channel to obtain the second color distribution of each channel of each Gaussian kernel in the first color space.

[0080] Specifically, the second color distribution g of each Gaussian kernel in the first color space of each channel * It can be expressed as:

[0081]

[0082] in, is the color distribution mean of the migration object; is the standard deviation of the color distribution of the migration target; is the standard deviation of the color distribution of the migration object; is the standard deviation of the color distribution of the source image; the coefficient C is used to control the intensity of the color transformation; g is the first color distribution of the Gaussian kernel in the first color space of each channel; the third color distribution is The fifth color distribution is The adjusted mean is Each channel is processed according to the method shown in formula (2), and the second color distribution in each channel after conversion can be obtained, which together constitutes the processed Gaussian model G * The color of each Gaussian kernel in .

[0083] After the migration process, if the value obtained by a certain channel is not within the valid value range of the channel, the value of each channel can be clipped as needed to customize the control of the color change range. Taking the first color space as the LAB color space as an example, the value range of the L channel is [0,100]; the value range of the A channel is [-128,127]; the value range of the B channel is [-128,127]. The clipping process of the present application can be that if the value obtained by a certain channel exceeds the valid value range of the channel, the value is selected as the endpoint value of the valid value range.

[0084] The above step 204, namely "encoding each Gaussian kernel in the second color distribution in the first color space to obtain a Gaussian model corresponding to the target color style", is described in detail below in conjunction with an embodiment.

[0085] After obtaining the second color distribution, the present application encodes the second color distribution to obtain the Gaussian model corresponding to the target color style. This process can be implemented in a variety of ways. For example, Gaussian kernel density estimation can be used to simulate the color distribution. Gaussian kernel density estimation is a non-parametric method for estimating a probability density function. It constructs a probability density function for the entire data set by placing a Gaussian kernel around each data point. By selecting data points in the color space, for each data point, it is regarded as the center of the probability distribution and represented by a kernel function. Then, the kernel functions of all data points are added to obtain a density estimate of the entire data set. After obtaining the density estimate, the target color style can be encoded by adjusting the parameters of the Gaussian kernel (such as the mean and variance).

[0086] As an implementable manner, the second color distribution of each Gaussian kernel in the first color space may be encoded to obtain the spherical harmonic function of each Gaussian kernel.

[0087] If the spherical harmonic function of the Gaussian kernel needs to be represented by a second color space, the second color distribution of each Gaussian kernel in the first color space can be converted to the second color space to obtain the second color distribution of each Gaussian kernel in the second color space; the second color distribution of each Gaussian kernel in the second color space is encoded to obtain the spherical harmonic function of each Gaussian kernel.

[0088] In the present application, the style conversion method embodiment of the three-dimensional model given above can be applied to an image processing method, and the method includes: determining a Gaussian model corresponding to a target perspective and a target color style, wherein the Gaussian model corresponding to the target color style is obtained by using a conversion method such as any one of the embodiments given above; rendering the target perspective using the Gaussian model corresponding to the target color style to obtain an image corresponding to the target color style.

[0089] The target viewing angle may be a viewing angle specified by a user, a default viewing angle, or a viewing angle obtained according to certain rules.

[0090] When determining the Gaussian model corresponding to the target color style, the image generation device may adopt an online generation mode or an offline generation mode.

[0091] When the online generation mode is adopted, the user terminal can directly specify the target color style by providing the target image information. Specifically, the scene identification information and the target image information from the user terminal are determined, the color distribution of the target color style is determined using the target image, and the Gaussian model corresponding to the scene identification information is determined, and the Gaussian model corresponding to the scene identification information is converted using any of the conversion methods in the embodiments given above to obtain the Gaussian model corresponding to the target color style.

[0092] When the offline mode is adopted, the image generation device has pre-set a plurality of color styles and correspondingly generated more than one Gaussian model. The image generation device provides these color styles to the user for selection, and determines the corresponding Gaussian model according to the user's selection. Specifically, the scene identification information is received from the user end, and more than one Gaussian model corresponding to the scene identification information is determined, and the more than one Gaussian model corresponding to the scene identification is pre-converted by using the conversion method of any one of the embodiments given above; the color style information corresponding to more than one Gaussian model is sent to the user end, the target color style selected in the color style information sent by the user end is received, and the Gaussian model corresponding to the target color style is determined from the more than one Gaussian model.

[0093] The above method provided in the embodiment of the present application can be applied to a variety of application scenarios, including but not limited to: it can be applied to map, game and film production, and the style of the scene can be quickly changed by this method. For example, when rendering and displaying an image of a city in a map application, the city image with a modern color style can be changed into an image with a futuristic or retro color style, thereby improving production efficiency and reducing production costs. It can also be applied to interior design and architectural visualization. Designers can use this method to quickly preview interior design effects of different styles, or show the style changes of buildings in different historical periods, thereby improving work efficiency. In urban planning, this method can be used to simulate three-dimensional scenes at different stages of urban development, helping planners and decision makers to better understand the potential impact of urban development.

[0094] It can be seen from the description of the above embodiments that the technical solution provided by the present application can also have the following advantages:

[0095] (1) The encoding and decoding of the Gaussian kernel in the first color space can be achieved by encoding and decoding the spherical harmonic function of the Gaussian kernel. The spherical harmonic function can well express the color information of the Gaussian kernel, thereby improving the style conversion effect of the three-dimensional model.

[0096] (2) When the color distribution of the second color space is obtained by decoding the Gaussian model, the application converts it to the first color space, and then performs a color distribution transformation to obtain the second color distribution. After the color distribution transformation, the second color distribution of the first color space is converted back to the second color space for encoding. By converting the color space obtained by decoding the Gaussian model to a certain color space that is specific or more suitable for color distribution transformation, the flexibility and accuracy of the style conversion of the three-dimensional model are improved.

[0097] (3) The present application can obtain the target color style through the target image or target text, making the process of obtaining the target color style more flexible and simple, thereby improving the style conversion efficiency of the three-dimensional model.

[0098] (4) The present application can convert the color distribution of the target image in the second color space to the first color space, making it the same as the color distribution space of the Gaussian kernel to be converted, thereby reducing the requirements for the target image and improving the accuracy of the style conversion of the three-dimensional model.

[0099] (5) This application uses the color distribution of the target color style in the first color space as the migration target, and uses the color distribution of each Gaussian kernel in the first color space as the migration object for migration processing, thereby obtaining the second color distribution of each Gaussian kernel in the first color space. The color distribution transformation of the Gaussian kernel is achieved through the migration processing, and an effective automatic scene conversion method is provided, which improves the efficiency and accuracy of the style conversion of the three-dimensional model.

[0100] (6) The migration processing of the present application obtains the second color distribution of each Gaussian kernel in the first color space according to the color distribution mean of the migration object, the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the migration object, and the color distribution mean of the migration target. A migration processing method that effectively applies the color distribution of the migration target to the migration object is provided, thereby improving the accuracy of the style conversion of the three-dimensional model.

[0101] (7) The migration processing of the present application performs migration processing according to the color distribution difference between the source image corresponding to the Gaussian model in the first color space and the migration target. The second color distribution of each channel of each Gaussian kernel in the first color space is obtained according to the color distribution mean of the migration object, the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the source image, the color distribution mean of the migration target and the color distribution mean of the source image and the color distribution mean of the migration object. A migration processing method that effectively applies the color distribution transformation from the source image to the migration target to the migration object is provided, thereby improving the accuracy of the style conversion of the three-dimensional model.

[0102] (8) The present application can use the Gaussian model corresponding to the target color style to render the target perspective and obtain an image corresponding to the target color style. When determining the Gaussian model corresponding to the target color style, it can be determined based on the scene identification information and the target image information from the user end, or it can be determined by sending the color style information corresponding to more than one Gaussian model to the user end based on the scene identification information, and the target color style selected by the user end is determined. This improves the interactivity of image processing and reduces the difficulty of user operation.

[0103] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] According to an embodiment of another aspect, a style conversion device for a three-dimensional model is provided. Figure 4 A schematic block diagram of a style conversion device for a three-dimensional model according to an embodiment is shown, wherein the device is arranged at Figure 1 The server side of the architecture shown in Figure 1. Figure 4 As shown, the device 400 includes:

[0105] The model acquisition unit 401 is configured to acquire a Gaussian model corresponding to the three-dimensional scene.

[0106] The target acquisition unit 402 is configured to acquire the color distribution of the target color style in the first color space.

[0107] The color decoding unit 403 is configured to perform color decoding on the Gaussian model to obtain a first color distribution of each Gaussian kernel in a first color space.

[0108] The color transformation unit 404 is configured to transform the first color distribution of each Gaussian kernel in the first color space by using the color distribution of the target color style in the first color space to obtain a second color distribution of each Gaussian kernel in the first color space.

[0109] The color encoding unit 405 is configured to encode each Gaussian kernel in the second color distribution of the first color space to obtain a Gaussian model corresponding to the target color style.

[0110] As one of the feasible ways, when the color decoding unit 403 performs color decoding on the Gaussian model and obtains the first color distribution of each Gaussian kernel in the first color space, it can be configured as: decoding the spherical harmonic function of each Gaussian kernel in the Gaussian model to obtain the color distribution of each Gaussian kernel in the first color space; when the color encoding unit 405 encodes the second color distribution of each Gaussian kernel in the first color space, it can be configured as: encoding the second color distribution of each Gaussian kernel in the first color space to obtain the spherical harmonic function of each Gaussian kernel.

[0111] As one of the achievable methods, when the color decoding unit 403 decodes the spherical harmonic functions of each Gaussian kernel in the Gaussian model to obtain the first color distribution of each Gaussian kernel in the first color space, it can be configured as follows: decoding the spherical harmonic functions of each Gaussian kernel in the Gaussian model to obtain the first color distribution of each Gaussian kernel in the second color space; converting the color distribution of each Gaussian kernel in the second color space to the first color space to obtain the first color distribution of the Gaussian kernel in the first color space; when the color encoding unit 405 encodes the second color distribution of each Gaussian kernel in the first color space to obtain the spherical harmonic functions of each Gaussian kernel, it can be configured as follows: converting the second color distribution of each Gaussian kernel in the first color space to the second color space to obtain the second color distribution of each Gaussian kernel in the second color space; encoding the second color distribution of each Gaussian kernel in the second color space to obtain the spherical harmonic functions of each Gaussian kernel.

[0112] As one of the achievable methods, the target acquisition unit 402 can be configured to: acquire a target image and count the color distribution of the target image in the first color space when acquiring the color distribution of the target color style in the first color space; or, acquire a target text, use an image generation model to acquire a target image corresponding to the target text, and count the color distribution of the target image in the first color space.

[0113] As one of the achievable methods, the target acquisition unit 402 can be configured to: count the color distribution of the target image in the second color space; convert the color distribution of the target image in the second color space to the first color space to obtain the color distribution of the target image in the first color space when counting the color distribution of the target image in the first color space.

[0114] As one achievable manner, the first color space includes a LAB color space, and the second color space includes an RGB color space.

[0115] As one of the feasible ways, the color transformation unit 404 can be configured as follows when using the color distribution of the target color style in the first color space to transform the first color distribution of each Gaussian kernel in the first color space to obtain the second color distribution of each Gaussian kernel in the first color space: taking the color distribution of the target color style in the first color space as the migration target, and taking the color distribution of each Gaussian kernel in the first color space as the migration object; and performing migration processing using the color distribution mean and standard deviation of the migration target to obtain the second color distribution of each Gaussian kernel in the first color space.

[0116] As one of the feasible ways, the color transformation unit 404 can be configured during the migration process as follows: subtracting the color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain the third color distribution of each channel; scaling the third color distribution of each channel according to the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the migration object to obtain the fourth color distribution of each channel; superimposing the color distribution mean of the migration target to the fourth color distribution of each channel to obtain the second color distribution of each Gaussian kernel in the first color space.

[0117] As one of the achievable methods, the device 400 also includes: a source image acquisition unit 406, configured to: acquire a source image corresponding to the Gaussian model and the color distribution of the source image in the first color space; the color transformation unit 404 can be configured to: perform migration processing using the color distribution difference between the source image in the first color space and the migration target.

[0118] As one of the feasible ways, the color transformation unit 404 can be configured as follows during the migration process: subtracting the color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain the third color distribution of each channel; scaling the third color distribution of each channel by using the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the source image to obtain the fifth color distribution of each channel; obtaining the adjusted mean by using the difference between the color distribution mean of the migration target and the color distribution mean of the source image; and superimposing the fifth color distribution of each channel by the adjusted mean and the color distribution mean of the migration object to obtain the second color distribution of each channel of each Gaussian kernel in the first color space.

[0119] According to an embodiment of another aspect, an image processing apparatus is provided. Figure 5 A schematic block diagram of an image processing device according to an embodiment is shown. The device is arranged at Figure 1 The server side of the architecture shown in Figure 1. Figure 5 As shown, the device 500 includes: a model determination unit 501, an image rendering unit 502 and a viewing angle determination unit 503. The main functions of each component unit are as follows:

[0120] The viewing angle determining unit 503 is configured to determine a target viewing angle.

[0121] The model determination unit 501 is configured to determine a Gaussian model corresponding to a target color style, where the Gaussian model corresponding to the target color style is obtained by using any one of the aforementioned embodiments of the conversion device 400 .

[0122] The image rendering unit 502 is configured to use the Gaussian model corresponding to the target color style to render the target perspective, and obtain an image corresponding to the target color style.

[0123] As one of the achievable methods, the model determination unit 501 can be configured to: determine the scene identification information and target image information from the user terminal, use the target image to determine the color distribution of the target color style and determine the Gaussian model corresponding to the scene identification information, and convert the Gaussian model corresponding to the scene identification information using the method described in any one of the aforementioned method embodiments to obtain the Gaussian model corresponding to the target color style; or, receive the scene identification information from the user terminal, determine one or more Gaussian models corresponding to the scene identification information, and the one or more Gaussian models corresponding to the scene identification are pre-converted using the method described in any one of the aforementioned method embodiments; send color style information corresponding to more than one Gaussian models to the user terminal, receive the target color style selected from the color style information sent by the user terminal, and determine the Gaussian model corresponding to the target color style from one or more Gaussian models.

[0124] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0125] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0126] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0127] And an electronic device, comprising:

[0128] one or more processors; and

[0129] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0130] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the methods in the aforementioned method embodiments when executed by a processor.

[0131] in, Figure 6 The architecture of the electronic device is shown as an example, which may include a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, and a memory 620. The processor 610, the video display adapter 611, the disk drive 612, the input / output interface 613, the network interface 614, and the memory 620 may be communicatively connected via a communication bus 630.

[0132] Among them, the processor 610 can be implemented by a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in this application.

[0133] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store an operating system 621 for controlling the operation of the electronic device 600, and a basic input and output system (BIOS) 622 for controlling the low-level operation of the electronic device 600. In addition, a web browser 623, a data storage management system 624, and a style conversion device / image processing device 625 of a three-dimensional model, etc. can also be stored. The style conversion device / image processing device 625 of the three-dimensional model can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0134] The input / output interface 613 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0135] The network interface 614 is used to connect to a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0136] The bus 630 comprises a pathway for transmitting information between the various components of the device (eg, the processor 610, the video display adapter 611, the disk drive 612, the input / output interface 613, the network interface 614, and the memory 620).

[0137] It should be noted that, although the above device only shows a processor 610, a video display adapter 611, a disk drive 612, an input / output interface 613, a network interface 614, a memory 620, a bus 630, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0138] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0139] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A style transfer method for a three-dimensional model, characterized in that: The method comprises: Obtain the Gaussian model corresponding to the three-dimensional scene and the color distribution of the target color style in the first color space; Performing color decoding on the Gaussian model to obtain a first color distribution of each Gaussian kernel in the first color space; Using the color distribution of the target color style in the first color space, transforming the first color distribution of each Gaussian kernel in the first color space to obtain a second color distribution of each Gaussian kernel in the first color space; The second color distribution of each Gaussian kernel in the first color space is encoded to obtain a Gaussian model corresponding to the target color style.

2. The method according to claim 1, characterized in that Color decoding is performed on the Gaussian model to obtain a first color distribution of each Gaussian kernel in the first color space, including: Decoding the color representation of each Gaussian kernel in the Gaussian model to obtain a first color distribution of each Gaussian kernel in the second color space; Converting the color distribution of each Gaussian kernel in the second color space to the first color space to obtain a first color distribution of the Gaussian kernel in the first color space; The encoding of the second color distribution of each Gaussian kernel in the first color space includes: Converting the second color distribution of each Gaussian kernel in the first color space to a second color space to obtain a second color distribution of each Gaussian kernel in the second color space; A second color distribution of each Gaussian kernel in the second color space is encoded.

3. The method according to claim 1, characterized in that Acquiring the color distribution of the target color style in the first color space includes: Acquire a target image, and count the color distribution of the target image in the first color space; or, A target text is obtained, a target image corresponding to the target text is obtained using an image generation model, and color distribution of the target image in the first color space is counted.

4. The method according to claim 2, characterized in that: The first color space includes a LAB color space, and the second color space includes an RGB color space.

5. The method according to any one of claims 1 to 4, characterized in that Using the color distribution of the target color style in the first color space, transforming the first color distribution of each Gaussian kernel in the first color space to obtain the second color distribution of each Gaussian kernel in the first color space includes: Taking the color distribution of the target color style in the first color space as a migration target, and taking the color distribution of each Gaussian kernel in the first color space as a migration object; The migration object is subjected to migration processing by using the color distribution mean and standard deviation of the migration target to obtain a second color distribution of each Gaussian kernel in the first color space.

6. The method according to claim 5, characterized in that Using the color distribution mean and standard deviation of the migration target, performing migration processing on the migration object includes: Subtracting a color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain a third color distribution of each channel; According to the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the migration object, the third color distribution of each channel is scaled to obtain the fourth color distribution of each channel; The color distribution mean of the migration target is superimposed on the fourth color distribution of each channel to obtain the second color distribution of each Gaussian kernel in the first color space.

7. The method according to claim 5, characterized in that The method further includes: acquiring a source image corresponding to the Gaussian model and a color distribution of the source image in a first color space; Using the color distribution mean and standard deviation of the migration target, performing migration processing on the migration object includes: Subtracting a color distribution mean of the migration object from the color distribution of each channel of the migration object to obtain a third color distribution of each channel; Using the ratio of the color distribution standard deviation of the migration target to the color distribution standard deviation of the source image, respectively scale the third color distribution of each channel to obtain the fifth color distribution of each channel; Obtaining an adjusted mean value by using a difference between a color distribution mean value of the migration target and a color distribution mean value of the source image; The adjusted mean and the color distribution mean of the migration object are superimposed on the fifth color distribution of each channel to obtain the second color distribution of each channel of each Gaussian kernel in the first color space.

8. An image processing method, characterized in that: The method comprises: Determine a Gaussian model corresponding to a target viewing angle and a target color style, wherein the Gaussian model corresponding to the target color style is obtained by using the conversion method according to any one of claims 1 to 7; The target perspective is rendered using a Gaussian model corresponding to the target color style to obtain an image corresponding to the target color style.

9. A style conversion device for a three-dimensional model, characterized in that: The conversion device comprises: A model acquisition unit is configured to acquire a Gaussian model corresponding to the three-dimensional scene; A target acquisition unit, configured to acquire a color distribution of a target color style in a first color space; A color decoding unit is configured to perform color decoding on the Gaussian model to obtain a first color distribution of each Gaussian kernel in the first color space; A color conversion unit is configured to convert a first color distribution of each Gaussian kernel in the first color space by using the color distribution of the target color style in the first color space to obtain a second color distribution of each Gaussian kernel in the first color space; The color encoding unit is configured to encode the second color distribution of each Gaussian kernel in the first color space to obtain a Gaussian model corresponding to the target color style.

10. An image processing device, characterized in that: The device comprises: a viewing angle determination unit configured to determine a target viewing angle; A model determination unit, configured to determine a Gaussian model corresponding to a target color style, wherein the Gaussian model corresponding to the target color style is obtained by using the conversion device according to claim 9; The image rendering unit is configured to use the Gaussian model corresponding to the target color style to render the target perspective to obtain an image corresponding to the target color style.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images