Image processing method, device, electronic device, storage medium and program product

Abstract

The present disclosure provides an image processing method, device, electronic device, storage medium and program product, the method comprising: determining the original image taken by a virtual camera from a virtual scene at a current viewing angle, replacing the color information of the pixels in the original image with brightness information to obtain an image to be rendered; creating a spherical model that wraps the virtual camera, sampling a preset noise map through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute is absolute rotation; determining the regional noise map sampling result on the spherical model taken by the virtual camera from the inside of the spherical model at the current viewing angle, applying it to the image to be rendered, and obtaining a preset style image. The present disclosure fixes the noise map sampling result on the geometric model, and when the picture taken by the camera changes, the noise map sampling result at the same position of the geometric model will not change, thereby reducing the flickering and jitter of pixels.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, storage medium and program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] 1-bit is a minimalist retro pixel game art style, which was first created due to the limitation of computer functions and the inability of game screens to display details and complex textures. With the upgrade of hardware and the iteration of production tools, game art is no longer limited to 1-bit style, and 1-bit art style has become a retro pixel stylized art form.

[0004] However, although the 1-bit pixel style performs well in static images, in dynamic images, 1-bit pixels will jitter as the frames change, disrupting the visual experience. Summary of the invention

[0005] In view of this, the purpose of the present disclosure is to provide an image processing method, device, electronic device, storage medium and program product, which at least to a certain extent solve one of the technical problems in the related art.

[0006] Based on the above purpose, the first aspect of the exemplary embodiment of the present disclosure provides an image processing method, including:

[0007] Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0008] A spherical model wrapping the virtual camera is created, and a preset noise map is sampled through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation;

[0009] Determine a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, apply the sampling result of the regional noise map to the image to be rendered, and obtain a preset style image.

[0010] Based on the same inventive concept, a second aspect of the exemplary embodiment of the present disclosure provides an image processing method, including:

[0011] Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0012] Determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, sample a preset noise map based on the two-dimensional texture coordinates, and obtain a noise map sampling result;

[0013] The noise map sampling result is applied to the image to be rendered to obtain a preset style image.

[0014] Based on the same inventive concept, the third aspect of the exemplary embodiment of the present disclosure provides an image processing device, including:

[0015] An image preprocessing module is configured to determine an original image captured by a virtual camera from a virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0016] A noise map sampling module is configured to create a spherical model that wraps the virtual camera, and to sample a preset noise map through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation;

[0017] The sampling result application module is configured to determine the regional noise map sampling result on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and apply the regional noise map sampling result to the image to be rendered to obtain a preset style image.

[0018] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides an image processing device, including:

[0019] An image preprocessing module is configured to determine an original image captured by a virtual camera from a virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0020] a noise map sampling module, configured to determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, sample a preset noise map based on the two-dimensional texture coordinates, and obtain a noise map sampling result;

[0021] The sampling result application module is configured to apply the noise map sampling result to the image to be rendered to obtain a preset style image.

[0022] Based on the same inventive concept, the fifth aspect of the exemplary embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0023] Based on the same inventive concept, the sixth aspect of the exemplary embodiment of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0024] Based on the same inventive concept, a seventh aspect of the exemplary embodiments of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the method described in the first aspect.

[0025] As can be seen from the above, the image processing method, device, electronic device, storage medium and program product provided by the embodiment of the present disclosure include: determining the original image taken by the virtual camera from the virtual scene at the current viewing angle, extracting the brightness information of the original image, replacing the color information of the pixel points in the original image with the brightness information, and obtaining the image to be rendered; creating a spherical model that wraps the virtual camera, sampling the preset noise map through the UV coordinates of the spherical model, and obtaining the noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; determining the regional noise map sampling result on the spherical model taken by the virtual camera from the inside of the spherical model at the current viewing angle, applying the regional noise map sampling result to the image to be rendered, and obtaining the preset style image. The present disclosure fixes the noise map sampling result on the geometric model. When the picture taken by the camera changes, the noise map sampling result at the same position of the geometric model will not change, thereby reducing the flickering and jitter of the pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A schematic diagram of an application scenario of the image processing method provided by the exemplary embodiment of the present disclosure;

[0028] Figure 2 A schematic diagram of a flow chart of an image processing method provided by an exemplary embodiment of the present disclosure;

[0029] Figure 3 A schematic diagram of an image to be rendered provided for an exemplary embodiment of the present disclosure;

[0030] Figure 4 A schematic diagram of a sphere model provided for an exemplary embodiment of the present disclosure;

[0031] Figure 5 A schematic diagram of a noise map provided for an exemplary embodiment of the present disclosure;

[0032] Figure 6 Another schematic diagram of a noise map provided for an exemplary embodiment of the present disclosure;

[0033] Figure 7 A schematic diagram of a pixel style image provided for an exemplary embodiment of the present disclosure;

[0034] Figure 8 Another schematic diagram of a pixel style image provided for an exemplary embodiment of the present disclosure;

[0035] Fig. 9 A schematic diagram of an edge profile provided for an exemplary embodiment of the present disclosure;

[0036] Fig.10 A schematic diagram of edge contour coloring provided for an exemplary embodiment of the present disclosure;

[0037] Fig.11 Another schematic flow chart of an image processing method provided by an exemplary embodiment of the present disclosure;

[0038] Fig.12 A schematic diagram of the structure of an image processing device provided by an exemplary embodiment of the present disclosure;

[0039] Fig.13 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0041] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present application according to the prompt message.

[0042] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0043] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of the present application. Other methods that meet the relevant laws and regulations may also be applied to the implementation method of the present application.

[0044] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0045] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, the principles and spirit of the present disclosure will be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0047] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "one" or "a" before an element does not exclude the existence of multiple such elements.

[0048] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0049] As mentioned in the background technology, 1-bit is a minimalist retro pixel game art style, which was first created due to the limitation of computer functions and the inability of game screens to display details and complex textures. With the upgrade of hardware and the iteration of production tools, game art is no longer limited to 1-bit style, and 1-bit art style has become a retro pixel stylized art form.

[0050] In the related art, the method of realizing 1-bit pixel style is usually: based on the screen space sampling dither map (a noise map), the grayscale of the current screen image is compared with the grayscale of the dither map (the dither map is a linear grayscale map). If the image grayscale is greater than the dither, the color value of the image is 0.1, and if it is less than 0.9.

[0051] However, although the 1-bit pixel style performs well in static images, in dynamic images, 1-bit pixels will jitter as the frames change, disrupting the visual experience.

[0052] The inventors of the present disclosure have found that the causes of jitter in the above-mentioned related technologies are:

[0053] The method of sampling dither maps based on screen space performs well in relatively static game screens. However, when the camera angle rotates (i.e. dynamic screen), 1-bit pixels will jitter and flicker. This is because when using screen space sampling dither maps, the sampling result is fixed on the screen, not on the geometric model. This will cause the position of the same grayscale of the geometric model to change when the camera rendering picture changes, because the position of the dither pixel changes, the position of the 1-bit pixel calculated by grayscale comparison will also change, causing the picture to jitter and flicker.

[0054] In order to solve the above problems, the present disclosure provides an image processing solution, which specifically includes:

[0055] Determine an original image taken by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered; create a spherical model that wraps the virtual camera, sample a preset noise map through the UV coordinates of the spherical model, and obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; determine a regional noise map sampling result on the spherical model taken by the virtual camera from the inside of the spherical model at the current viewing angle, apply the regional noise map sampling result to the image to be rendered, and obtain a preset style image.

[0056] In the present invention, the noise map sampling results of the area can change with the movement of the virtual camera, so that the noise map sampling results are fixed on the geometric model. When the picture taken by the camera changes, the noise map sampling results at the same position of the geometric model will not change, thereby reducing the flickering and jitter of pixels when the picture frame changes.

[0057] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0058] refer to Figure 1 , which is a schematic diagram of an application scenario of the image processing method provided by an exemplary embodiment of the present disclosure.

[0059] This application scenario includes a terminal device 101, a server 102, and a data storage system 103. The terminal device 101, the server 102, and the data storage system 103 may be connected via a wired or wireless communication network to achieve data interaction.

[0060] The terminal device 101 may be an electronic device with data transmission and multimedia input / output functions close to the user side, including but not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA) or other electronic devices capable of realizing the above functions. The electronic device may include a processor and a display screen with a touch input function, the display screen is used to present a graphical user interface, the graphical user interface can display an application interface, and the processor is used to process application data, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0061] Server 102 and data storage system 103 can both be independent physical servers, or a server cluster or distributed system composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0062] In some exemplary embodiments, the image processing method may be executed on the terminal device 101 or the server 102 .

[0063] When the image processing method is run on the server 102 , the server 102 is used to provide image processing services to the user of the terminal device 101 , and a client for communicating with the server 102 is installed in the terminal device 101 .

[0064] The server 102 determines the original image taken by the virtual camera from the virtual scene at the current viewing angle, extracts the brightness information of the original image, replaces the color information of the pixels in the original image with the brightness information, and obtains the image to be rendered; creates a spherical model that wraps the virtual camera, samples the preset noise map through the UV coordinates of the spherical model, and obtains the noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; determines the regional noise map sampling result on the spherical model taken by the virtual camera from the inside of the spherical model at the current viewing angle, applies the regional noise map sampling result to the image to be rendered, and obtains a preset style image.

[0065] The server 102 sends the preset style image to the client, and the client displays the preset style image to the user.

[0066] The data storage system 103 is used to assist the terminal device 101 and the server 102 in storing data, for example, storing a certain number of noise maps for users to select.

[0067] Combine the following Figure 1 The image processing method according to the exemplary embodiment of the present disclosure is described by using the application scenario. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0068] refer to Figure 2 , which is a flowchart of an image processing method provided by an exemplary embodiment of the present disclosure.

[0069] The image processing method comprises the following steps:

[0070] Step S210: determine the original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered.

[0071] In specific implementation, the virtual scene is a simulation environment of the real world, or a semi-simulation and semi-fictional virtual environment, or a purely fictional virtual environment. The virtual scene targeted by the present disclosure is a three-dimensional virtual scene, and the virtual environment can be the sky, land, ocean, etc., wherein the land includes environmental elements such as deserts and cities. For example, in a sandbox 3D shooting game, the virtual scene is a 3D game world for players to control virtual objects to fight, and an exemplary virtual scene may include: at least one element of mountains, plains, rivers, lakes, oceans, deserts, skies, plants, buildings, and vehicles; for a 3D multiplayer online tactical competitive game, the virtual scene is a 3D terrain scene for virtual objects to fight, and an exemplary virtual scene may include: canyon-style mountains, lines, rivers, classrooms, tables and chairs, podiums and other elements.

[0072] In this exemplary embodiment, extracting brightness information of the original image includes:

[0073] Determine color information of pixels in the original image;

[0074] The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information.

[0075] In a specific implementation, the color information includes RGB values;

[0076] The color values ​​of different channels in the color information, including the R value, G value and B value in the RGB value;

[0077] The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information, including:

[0078] Y=0.2126×R+0.7152×G+0.0722×B;

[0079] Among them, Y represents the calculated brightness information, and R, G, and B represent the values ​​of the red, green, and blue channels, respectively.

[0080] Taking into account the different sensitivities of the human eye to different colors, green, which is the most sensitive to the human eye, is given the highest weight, followed by red, and blue is the lowest.

[0081] As an example, see Figure 3 , which is a schematic diagram of an image to be rendered provided by an exemplary embodiment of the present disclosure.

[0082] Step S220, creating a spherical model that wraps the virtual camera, sampling a preset noise map through the UV coordinates of the spherical model, and obtaining a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation.

[0083] As an example, see Figure 4 , which is a structural schematic diagram of a spherical model wrapping a virtual camera provided by an exemplary embodiment of the present disclosure.

[0084] There is a virtual camera 401 in the virtual scene, and a spherical model 402 is created in the virtual scene. The spherical model 402 wraps the virtual camera 401 therein, and the spherical model 402 is always within the field of view of the virtual camera 401 .

[0085] During specific implementation, a spherical model is created using a model creation tool in a three-dimensional modeling software or a game engine.

[0086] In the specific implementation, in the material editor, set the material of the sphere model to a matte material. Matte materials (MatteMaterial) usually do not reflect light, have no gloss, and look matte visually.

[0087] When implementing it, turn on the Two-Sided Rendering option in the material settings, so that the virtual camera can see the texture when looking at the spherical model from inside the spherical model.

[0088] In this exemplary embodiment, after creating the spherical model that wraps the virtual camera, the method further includes:

[0089] In response to determining that the position of the virtual camera has moved, determining a displacement matrix of the virtual camera;

[0090] Based on the displacement matrix of the virtual camera, updating the position of the spherical model;

[0091] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0092] Determine an original image captured by the virtual camera from the virtual scene at an updated position viewing angle;

[0093] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0094] Determine a sampling result of a regional noise map on the spherical model captured by the virtual camera from inside the spherical model with an updated position viewing angle.

[0095] In this exemplary embodiment, after creating the spherical model that wraps the virtual camera, the method further includes:

[0096] In response to determining that the shooting angle of the virtual camera rotates, controlling the spherical model not to rotate with the virtual camera;

[0097] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0098] Determine an original image captured from the virtual scene by the virtual camera to update the shooting angle;

[0099] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0100] The virtual camera is determined to update the shooting angle to obtain a sampling result of a regional noise map on the spherical model shot from inside the spherical model.

[0101] During the specific implementation, the spherical model is mounted under the virtual camera: the spherical model is taken as the child object (Child Object) of the virtual camera, so that the spherical model will move with the movement of the camera.

[0102] During specific implementation, the rotation of the spherical model is set to absolute rotation: Set the rotation mode of the spherical model to absolute rotation (Absolute Rotation), so that the rotation of the spherical model will not be affected by the parent object (in this case, the virtual camera).

[0103] In specific implementation, the camera moves and rotates: when the virtual camera moves, since the sphere model is a child object of the camera, it moves with the camera;

[0104] When the virtual camera rotates, since the sphere model is set to absolute rotation, it does not rotate with the camera, so the texture sampling results on the sphere model will change relative to the camera's perspective.

[0105] In specific implementation, since the spherical model does not rotate with the camera, the appearance of the noise map on the spherical model will change with the rotation of the camera, producing a dynamic effect.

[0106] During specific implementation, there may be multiple preset noise maps.

[0107] As an example, see Figure 5 and Figure 6 , showing two different noise maps.

[0108] Step S230, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the noise map sampling result to the image to be rendered to obtain a preset style image.

[0109] In this exemplary embodiment, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the sampling result of the regional noise map to the image to be rendered to obtain a preset style image includes:

[0110] For a pixel point in the image to be rendered, comparing the grayscale value of the regional noise map sampling result with the grayscale value of the image to be rendered;

[0111] In response to determining that the grayscale value of the regional noise map sampling result is greater than the grayscale value of the image to be rendered, adjusting the grayscale value of the corresponding pixel point to a first preset value;

[0112] or,

[0113] In response to determining that the grayscale value of the regional noise map sampling result is less than the grayscale value of the image to be rendered, the grayscale value of the corresponding pixel point is adjusted to a second preset value.

[0114] During specific implementation, the first preset value and the second preset value can be freely adjusted according to requirements.

[0115] During specific implementation, the second preset value is greater than 0, which can facilitate the subsequent addition of color tendency.

[0116] As an example, the first preset value is 0.9, and the second preset value is 0.1.

[0117] When implementing it, take the implementation in Unity engine as an example:

[0118] The noise map sampling result obtained by virtual camera rendering is rendered into render texture (a function in Unity engine) in real time and passed into post-processing material.

[0119] During specific implementation, different noise maps are sampled to obtain different preset style images.

[0120] As an example, see Figure 7 and Figure 8 , Figure 7 Demonstrated Figure 5 The pixel style image is obtained by sampling the noise map shown in the figure. Figure 8 Demonstrated Figure 6 The pixel style image obtained by sampling the noise map shown in the figure has different visual effects.

[0121] However, reference Figure 7 and Figure 8 , where simple binary pixel division will cause the dark part of the geometric body to merge with its shadow in some cases, losing the outline. Therefore, additional stroke processing can be performed, specifically:

[0122] In this exemplary embodiment, after determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the sampling result of the regional noise map to the image to be rendered to obtain a preset style image, the method further includes:

[0123] Determining an edge contour of the virtual model in the preset style image;

[0124] The edge contour is colored.

[0125] In this exemplary embodiment, determining the edge contour of the virtual model in the preset style image includes:

[0126] Performing convolution on the original image based on the detection operator to obtain gradient values ​​of pixels in the original image;

[0127] Based on the gradient values ​​of the pixels in the original image, an edge contour of the virtual model in the preset style image is determined.

[0128] As an example, see Fig. 9 , a schematic diagram showing the edge contour of the virtual model.

[0129] In this exemplary embodiment, the coloring process of the edge contour includes:

[0130] Determining the grayscale value of the edge contour in the preset style image;

[0131] Converting the grayscale value into an integer;

[0132] Invert the grayscale value converted to an integer;

[0133] The inverted grayscale value is applied to the edge contour.

[0134] In a specific implementation, inverting the grayscale value refers to converting the grayscale value into its complement. If the grayscale value ranges from 0 to 255, the inverted value can be calculated by 255-grayscale value.

[0135] In specific implementation, the grayscale of the screen rendering result is rounded and inverted through the round function to color the stroke. The grayscale is small, that is, the junction of the dark part and the shadow, and the stroke is white or other specified colors, while the grayscale is large, that is, the stroke of the bright part is black.

[0136] As an example, see Fig.10 , showing a schematic diagram outlining the outlines of geometric shapes, which makes the picture more stylized.

[0137] From the above, it can be seen that the image processing method provided by the embodiment of the present disclosure includes: determining the original image taken by the virtual camera from the virtual scene at the current viewing angle, extracting the brightness information of the original image, replacing the color information of the pixel points in the original image with the brightness information, and obtaining the image to be rendered; creating a spherical model that wraps the virtual camera, sampling a preset noise map through the UV coordinates of the spherical model, and obtaining a noise map sampling result; wherein the material attribute of the spherical model is a matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; determining the regional noise map sampling result on the spherical model taken by the virtual camera from the inside of the spherical model at the current viewing angle, and applying the regional noise map sampling result to the image to be rendered to obtain a preset style image.

[0138] In the present invention, the noise map sampling results can change with the movement of the virtual camera, so that the noise map sampling results are fixed on the geometric model. When the picture taken by the camera changes, the noise map sampling results at the same position of the geometric model will not change, thereby reducing the flickering and jitter of pixels when the picture frame changes.

[0139] Furthermore, 1-bit retro pixel art effects can be efficiently generated: there is no need to process art assets, and 1-bit art effects can be quickly generated through real-time calculation and rendering, reducing the workload of art.

[0140] Furthermore, it is highly flexible and dynamic: through custom parameters, the 1-bit effect can be adjusted at any time according to the scene requirements and is suitable for a variety of different 3D scenes.

[0141] In the above exemplary embodiment, a method of sampling the texture of the noise map by spherical surface is introduced. Next, a method of implementing it in post-processing material will be introduced. Specifically:

[0142] refer to Fig.11 , which is a flowchart of an image processing method provided by an exemplary embodiment of the present disclosure.

[0143] The image processing method comprises the following steps:

[0144] Step S810: determine the original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered.

[0145] In this exemplary embodiment, extracting brightness information of the original image includes:

[0146] Determine color information of pixels in the original image;

[0147] The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information.

[0148] Step S820, determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered; convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates; sample the preset noise map based on the two-dimensional texture coordinates.

[0149] In specific implementation, the camera's view frustum defines the area that the camera can "see" in three-dimensional space. The camera view frustum is a geometric body, similar to a cone, with the vertex located at the camera's position and the bottom surface is a two-dimensional plane, called the near clipping plane. The view frustum extends forward along the axis to form an infinitely far conical space. Objects within this conical space can be captured by the camera. The view frustum is defined by six planes, including four side planes, a near plane and a far plane. The near plane and the far plane are two parallel planes of the view frustum, which determine the range of the scene that the camera can see.

[0150] In a specific implementation, the direction vectors of the four corners of the view cone of the virtual camera are determined, and the direction vectors of the four corners of the view cone are interpolated to obtain the direction vectors of the pixels in the image to be rendered, including the following steps:

[0151] (1) Determine the coordinates of the four corners of the viewing cone:

[0152] First, we need to determine the positions of the four corners of the camera's view frustum in the world coordinate system. These corners are the points where the view frustum intersects the far plane. Assuming the camera is at the origin (0,0,0) and the camera is oriented along the negative Z axis, the coordinates of the four corners can be expressed as:

[0153] Upper left corner: (-x, y, z)

[0154] Upper right corner: (x,y,z)

[0155] Lower left corner: (-x,-y,z)

[0156] Lower right corner: (x,-y,z)

[0157] Where x, y, and z are calculated based on the camera's field of view (FOV), aspect ratio, and far plane distance.

[0158] (2) Calculate the direction vector:

[0159] For each corner point, you need to calculate the direction vector from the camera position to the corner point. Since the camera is at the origin, the direction vector is the coordinates of the corner point itself:

[0160] Upper left corner direction vector: (-x, y, z)

[0161] Upper right corner direction vector: (x,y,z)

[0162] Lower left corner direction vector: (-x,-y,z)

[0163] Lower right corner direction vector: (x,-y,z)

[0164] (3) Perform interpolation operation

[0165] In order to obtain the direction vector of the screen pixel (i.e. the pixel of the image to be rendered), it is necessary to interpolate between the direction vectors of the four corner points. This can be achieved by bilinear interpolation. Assuming that the coordinates of the screen pixel are ((u,v)), where (u) and (v) are both in the range of [0,1], then the interpolation formula is:

[0166] Direction vector = (-x+2ux,y-2vy,z)

[0167] (4) Normalized direction vector

[0168] Finally, the direction vector can also be normalized to ensure that its length is 1.

[0169] In this way, the direction vector of the screen pixel is calculated.

[0170] Step S830: Apply the noise map sampling result to the image to be rendered to obtain a preset style image.

[0171] In this exemplary embodiment, applying the noise map sampling result to the image to be rendered to obtain a preset style image includes:

[0172] For a pixel point in the image to be rendered, comparing the grayscale value of the noise map sampling result with the grayscale value of the image to be rendered;

[0173] In response to determining that the grayscale value of the noise map sampling result is greater than the grayscale value of the image to be rendered, adjusting the grayscale value of the corresponding pixel point to a first preset value;

[0174] or,

[0175] In response to determining that the grayscale value of the noise map sampling result is less than the grayscale value of the image to be rendered, the grayscale value of the corresponding pixel point is adjusted to a second preset value.

[0176] During specific implementation, the first preset value and the second preset value can be freely adjusted according to requirements.

[0177] During specific implementation, the second preset value is greater than 0, which can facilitate the subsequent addition of color tendency.

[0178] As an example, the first preset value is 0.9, and the second preset value is 0.1.

[0179] The method of the above embodiment and the corresponding method in the foregoing embodiment have corresponding beneficial effects, which will not be described in detail here.

[0180] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0181] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0182] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides an image processing device.

[0183] refer to Fig.12 , which is a structural schematic diagram of an image processing device provided by an exemplary embodiment of the present disclosure.

[0184] The image processing device comprises the following modules:

[0185] The image preprocessing module 910 is configured to determine an original image captured by the virtual camera from the virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0186] The noise map sampling module 920 is configured to create a spherical model that wraps the virtual camera, and to sample a preset noise map through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation;

[0187] The sampling result application module 930 is configured to determine the regional noise map sampling result on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and apply the regional noise map sampling result to the image to be rendered to obtain a preset style image.

[0188] In some exemplary embodiments, the image preprocessing module 910 is specifically configured to:

[0189] Determine color information of pixels in the original image;

[0190] The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information.

[0191] In some exemplary embodiments, the noise map sampling module 920 is further configured to:

[0192] In response to determining that the position of the virtual camera has moved, determining a displacement matrix of the virtual camera;

[0193] Based on the displacement matrix of the virtual camera, updating the position of the spherical model;

[0194] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0195] Determine an original image captured by the virtual camera from the virtual scene at an updated position viewing angle;

[0196] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0197] Determine a sampling result of a regional noise map on the spherical model captured by the virtual camera from inside the spherical model with an updated position viewing angle.

[0198] In some exemplary embodiments, the noise map sampling module 920 is further configured to:

[0199] In response to determining that the shooting angle of the virtual camera rotates, controlling the spherical model not to rotate with the virtual camera;

[0200] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0201] Determine an original image captured from the virtual scene by the virtual camera to update the shooting angle;

[0202] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0203] The virtual camera is determined to update the shooting angle to obtain a sampling result of a regional noise map on the spherical model shot from inside the spherical model.

[0204] In some exemplary embodiments, the sampling result application module 930 is specifically configured to:

[0205] For a pixel point in the image to be rendered, comparing the grayscale value of the noise map sampling result with the grayscale value of the image to be rendered;

[0206] In response to determining that the grayscale value of the noise map sampling result is greater than the grayscale value of the image to be rendered, adjusting the grayscale value of the corresponding pixel point to a first preset value;

[0207] or,

[0208] In response to determining that the grayscale value of the noise map sampling result is less than the grayscale value of the image to be rendered, the grayscale value of the corresponding pixel point is adjusted to a second preset value.

[0209] In some exemplary embodiments, the sampling result application module 930 is further configured to:

[0210] Determining an edge contour of the virtual model in the preset style image;

[0211] The edge contour is colored.

[0212] In some exemplary embodiments, the sampling result application module 930 is further configured to:

[0213] Performing convolution on the original image based on the detection operator to obtain gradient values ​​of pixels in the original image;

[0214] Based on the gradient values ​​of the pixels in the original image, an edge contour of the virtual model in the preset style image is determined.

[0215] In some exemplary embodiments, the sampling result application module 930 is further configured to:

[0216] Determining the grayscale value of the edge contour in the preset style image;

[0217] Converting the grayscale value into an integer;

[0218] Invert the grayscale value converted to an integer;

[0219] The inverted grayscale value is applied to the edge contour.

[0220] In some exemplary embodiments, the noise map sampling module 920 is further configured to:

[0221] Determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone, and obtain the direction vectors of the pixels in the image to be rendered;

[0222] Converting the direction vector of the pixel in the image to be rendered into a two-dimensional texture coordinate;

[0223] A preset noise map is sampled based on the two-dimensional texture coordinates.

[0224] In some exemplary embodiments:

[0225] The image preprocessing module 910 is configured to determine an original image captured by the virtual camera from the virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0226] The noise map sampling module 920 is configured to determine the direction vectors of the four corners of the view frustum of the virtual camera, interpolate the direction vectors of the four corners of the view frustum to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, and sample a preset noise map based on the two-dimensional texture coordinates to obtain a noise map sampling result;

[0227] The sampling result application module 930 is configured to apply the noise map sampling result to the image to be rendered to obtain a preset style image.

[0228] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0229] The device of the above embodiment is used to implement the corresponding image processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0230] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method described in any of the above embodiments is implemented.

[0231] Fig.13 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0232] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0233] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0234] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0235] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0236] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0237] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0238] The electronic device of the above embodiment is used to implement the corresponding image processing method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0239] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, so that the processor 1010 executes the following instructions when running:

[0240] Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0241] A spherical model wrapping the virtual camera is created, and a preset noise map is sampled through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation;

[0242] Determine a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, apply the sampling result of the regional noise map to the image to be rendered, and obtain a preset style image.

[0243] In a possible implementation manner, in the instruction executed by the processor 1010, extracting brightness information of the original image includes:

[0244] Determine color information of pixels in the original image;

[0245] The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information.

[0246] In a possible implementation manner, in the instructions executed by the processor 1010, after creating the spherical model that wraps the virtual camera, the method further includes:

[0247] In response to determining that the position of the virtual camera has moved, determining a displacement matrix of the virtual camera;

[0248] Based on the displacement matrix of the virtual camera, updating the position of the spherical model;

[0249] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0250] Determine an original image captured by the virtual camera from the virtual scene at an updated position viewing angle;

[0251] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0252] Determine a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model with an updated position viewing angle.

[0253] In a possible implementation manner, in the instructions executed by the processor 1010, after creating the spherical model that wraps the virtual camera, the method further includes:

[0254] In response to determining that the shooting angle of the virtual camera rotates, controlling the spherical model not to rotate with the virtual camera;

[0255] Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including:

[0256] Determine an original image captured from the virtual scene by the virtual camera to update the shooting angle;

[0257] Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes:

[0258] The virtual camera is determined to update the shooting angle to obtain a sampling result of a regional noise map on the spherical model shot from inside the spherical model.

[0259] In a possible implementation manner, in the instructions executed by the processor 1010, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the sampling result of the regional noise map to the image to be rendered to obtain a preset style image includes:

[0260] For a pixel point in the image to be rendered, comparing the grayscale value of the regional noise map sampling result with the grayscale value of the image to be rendered;

[0261] In response to determining that the grayscale value of the regional noise map sampling result is greater than the grayscale value of the image to be rendered, adjusting the grayscale value of the corresponding pixel point to a first preset value;

[0262] or,

[0263] In response to determining that the grayscale value of the regional noise map sampling result is less than the grayscale value of the image to be rendered, the grayscale value of the corresponding pixel point is adjusted to a second preset value.

[0264] In a possible implementation manner, in the instructions executed by the processor 1010, after determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, applying the sampling result of the noise map to the image to be rendered, and obtaining a preset style image, the method further includes:

[0265] Determining an edge contour of the virtual model in the preset style image;

[0266] The edge contour is colored.

[0267] In a possible implementation manner, in the instructions executed by the processor 1010, determining the edge contour of the virtual model in the preset style image includes:

[0268] Performing convolution on the original image based on the detection operator to obtain gradient values ​​of pixels in the original image;

[0269] Based on the gradient values ​​of the pixels in the original image, an edge contour of the virtual model in the preset style image is determined.

[0270] In a possible implementation manner, in the instruction executed by the processor 1010, the coloring process on the edge contour includes:

[0271] Determining the grayscale value of the edge contour in the preset style image;

[0272] Converting the grayscale value into an integer;

[0273] Invert the grayscale value converted to an integer;

[0274] The inverted grayscale value is applied to the edge contour.

[0275] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, so that the processor 1010 executes the following instructions when running:

[0276] Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered;

[0277] Determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, sample a preset noise map based on the two-dimensional texture coordinates, and obtain a noise map sampling result;

[0278] The noise map sampling result is applied to the image to be rendered to obtain a preset style image.

[0279] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the image processing method described in any of the above embodiments.

[0280] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0281] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0282] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the image processing method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0283] Based on the same inventive concept, corresponding to the image processing method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the image processing method. Corresponding to the execution subject corresponding to each step in each embodiment of the image processing method, the processor that executes the corresponding step can belong to the corresponding execution subject.

[0284] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the image processing method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0285] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0286] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0287] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0288] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0289] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0290] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0291] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0292] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0293] In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0294] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0295] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0296] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0297] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0298] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0299] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

[0300] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. An image processing method, characterized in that: include: Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered; A spherical model wrapping the virtual camera is created, and a preset noise map is sampled through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; Determine a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, apply the sampling result of the regional noise map to the image to be rendered, and obtain a preset style image.

2. The method according to claim 1, characterized in that The extracting brightness information of the original image includes: Determine color information of pixels in the original image; The color values ​​of different channels in the color information are weighted and summed with preset weight parameters to obtain the brightness information.

3. The method according to claim 1, characterized in that After creating the spherical model that wraps the virtual camera, the method further includes: In response to determining that the position of the virtual camera has moved, determining a displacement matrix of the virtual camera; Based on the displacement matrix of the virtual camera, updating the position of the spherical model; Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including: Determine an original image captured by the virtual camera from the virtual scene at an updated position viewing angle; Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes: Determine a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model with an updated position viewing angle.

4. The method according to claim 1, characterized in that After creating the spherical model that wraps the virtual camera, the method further includes: In response to determining that the shooting angle of the virtual camera rotates, controlling the spherical model not to rotate with the virtual camera; Then, determine the original image taken by the virtual camera from the virtual scene at the current viewing angle, including: Determine an original image captured from the virtual scene by the virtual camera to update the shooting angle; Then, determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle includes: The virtual camera is determined to update the shooting angle to obtain a sampling result of a regional noise map on the spherical model shot from inside the spherical model.

5. The method according to claim 1, characterized in that The step of determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the sampling result of the regional noise map to the image to be rendered to obtain a preset style image includes: For a pixel point in the image to be rendered, comparing the grayscale value of the regional noise map sampling result with the grayscale value of the image to be rendered; In response to determining that the grayscale value of the regional noise map sampling result is greater than the grayscale value of the image to be rendered, adjusting the grayscale value of the corresponding pixel point to a first preset value; or, In response to determining that the grayscale value of the regional noise map sampling result is less than the grayscale value of the image to be rendered, the grayscale value of the corresponding pixel point is adjusted to a second preset value.

6. The method according to claim 1, characterized in that After determining a sampling result of a regional noise map on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and applying the sampling result of the regional noise map to the image to be rendered to obtain a preset style image, the method further includes: Determining an edge contour of the virtual model in the preset style image; The edge contour is colored.

7. The method according to claim 6, characterized in that The determining the edge contour of the virtual model in the preset style image includes: Performing convolution on the original image based on the detection operator to obtain gradient values ​​of pixels in the original image; Based on the gradient values ​​of the pixels in the original image, an edge contour of the virtual model in the preset style image is determined.

8. The method according to claim 6, characterized in that The coloring process of the edge contour includes: Determining the grayscale value of the edge contour in the preset style image; Converting the grayscale value into an integer; Invert the grayscale value converted to an integer; The inverted grayscale value is applied to the edge contour.

9. An image processing method, characterized in that: include: Determine an original image captured by the virtual camera from the virtual scene at the current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered; Determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, sample a preset noise map based on the two-dimensional texture coordinates, and obtain a noise map sampling result; The noise map sampling result is applied to the image to be rendered to obtain a preset style image.

10. An image processing device, characterized in that: include: An image preprocessing module is configured to determine an original image captured by a virtual camera from a virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered; A noise map sampling module is configured to create a spherical model that wraps the virtual camera, and to sample a preset noise map through the UV coordinates of the spherical model to obtain a noise map sampling result; wherein the material attribute of the spherical model is matte material, the rendering attribute of the spherical model is double-sided rendering, and the rotation attribute of the spherical model is absolute rotation; The sampling result application module is configured to determine the regional noise map sampling result on the spherical model photographed by the virtual camera from inside the spherical model at the current viewing angle, and apply the regional noise map sampling result to the image to be rendered to obtain a preset style image.

11. An image processing device, characterized in that: include: An image preprocessing module is configured to determine an original image captured by a virtual camera from a virtual scene at a current viewing angle, extract brightness information of the original image, replace color information of pixels in the original image with the brightness information, and obtain an image to be rendered; a noise map sampling module, configured to determine the direction vectors of the four corners of the view cone of the virtual camera, interpolate the direction vectors of the four corners of the view cone to obtain the direction vectors of the pixels in the image to be rendered, convert the direction vectors of the pixels in the image to be rendered into two-dimensional texture coordinates, sample a preset noise map based on the two-dimensional texture coordinates, and obtain a noise map sampling result; The sampling result application module is configured to apply the noise map sampling result to the image to be rendered to obtain a preset style image.

12. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 8 or the method according to claim 9 is implemented.

13. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method of any one of claims 1 to 8 or the method of claim 9.

14. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 8 or the method according to claim 9.

Images