Image processing method and device, equipment and storage medium

Abstract

The invention provides an image processing method and device, equipment and a storage medium, and the method comprises the steps: determining the grid density information of a to-be-rendered model according to the vertex data of the to-be-rendered model, and determining the screen space proportion information of the to-be-rendered model according to the position information of the to-be-rendered model, and determining sampling weight information corresponding to the to-be-rendered model according to the grid density information and the current screen space proportion information, and sampling a map corresponding to the to-be-rendered model according to the sampling weight information to render the to-be-rendered model. According to the method, the UV coordinates of the to-be-rendered model do not need to be manually created, so that the workload is reduced, and the problem of UV seams is avoided. And the three-dimensional model with a larger area ratio in the screen can obtain higher rendering precision during rendering, and the three-dimensional model with a small area ratio can obtain lower rendering precision during rendering, so that the consumption of computing resources is reduced, and the rendering efficiency is further improved on the premise of ensuring the rendering effect.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] In the field of 3D model graphics rendering, image sampling is one of the key technologies to achieve high-quality visual effects. The current image sampling method is to first create the UV coordinates of the 3D model, then map the 2D texture image to the 3D model, and finally perform image sampling based on the UV coordinates.

[0003] However, this method requires manual creation of UV coordinates, which is labor-intensive and complex. In addition, when rendering complex scenes or performing real-time rendering, the rendering speed is slow due to the large consumption of computing resources. Summary of the invention

[0004] The purpose of this application is to provide an image processing method, device, equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problems of large computing resource consumption and slow rendering speed in the prior art.

[0005] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, the present application provides an image processing method, the method comprising:

[0007] Determining mesh density information of the model to be rendered according to vertex data of the model to be rendered;

[0008] Determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered mapped to the screen;

[0009] Determine sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information;

[0010] The texture corresponding to the model to be rendered is sampled according to the sampling weight information to render the model to be rendered.

[0011] In a second aspect, the present application provides an image processing device, the device comprising:

[0012] A first determination module, used to determine mesh density information of the model to be rendered according to vertex data of the model to be rendered;

[0013] A second determination module is used to determine screen space proportion information of the model to be rendered according to the position information of the model to be rendered, wherein the screen space proportion information is used to represent the area proportion of the model to be rendered mapped to the screen;

[0014] A third determination module is used to determine the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information;

[0015] A rendering module is used to sample the map corresponding to the model to be rendered according to the sampling weight information to render the model to be rendered.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the image processing method as described in any one of the first aspects.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method as described in any one of the first aspects are executed.

[0018] The beneficial effects of the present application are: compared with the existing method of image sampling based on UV coordinates, the present application does not need to manually create the UV coordinates of the model to be rendered, which greatly reduces the workload and does not cause the problem of UV seams. In addition, the grid density information represents the surface area ratio of the model to be rendered in the three-dimensional space, and the screen space ratio information represents the area ratio of the model to be rendered in the graphical user interface. Therefore, the three-dimensional model with a larger area ratio in the screen can obtain higher rendering accuracy when rendering, and the three-dimensional model with a smaller area ratio can obtain lower rendering accuracy when rendering, which reduces the consumption of computing resources and further improves the rendering efficiency while ensuring the rendering effect.

[0019] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown;

[0022] Figure 2 A flow chart for determining grid density information provided by an embodiment of the present application is shown;

[0023] Figure 3 A flowchart of determining screen space ratio information provided by an embodiment of the present application is shown;

[0024] Figure 4 A flow chart for determining screen position coordinates provided by an embodiment of the present application is shown;

[0025] Figure 5 Another flow chart for determining screen position coordinates provided by an embodiment of the present application is shown;

[0026] Figure 6 A flowchart of another method of determining screen space ratio information provided by an embodiment of the present application is shown;

[0027] Figure 7 A flow chart for determining sampling weight information provided by an embodiment of the present application is shown;

[0028] Figure 8 A flowchart of rendering color information in a map to a model to be rendered is shown in an embodiment of the present application;

[0029] Fig. 9 A flowchart of another method of rendering color information in a map to a model to be rendered is shown in an embodiment of the present application;

[0030] Fig.10 An example diagram of the effect after image sampling provided by an embodiment of the present application is shown;

[0031] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown;

[0032] Fig.12 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0034] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0035] The image processing method in one embodiment of the present disclosure can be run on a local terminal device or a server. When the image processing method is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0036] In an optional implementation, various cloud applications can be run under the cloud interaction system, such as cloud games. Taking cloud games as an example, cloud games refer to a game mode based on cloud computing. In the operation mode of cloud games, the operating body of the game program and the main body of the game screen presentation are separated, the storage and operation of the image processing method are completed on the cloud game server, and the role of the client device is used for receiving and sending data and presenting the game screen. For example, the client device can be a display device with data transmission function close to the user side, such as a mobile terminal, a TV, a computer, a handheld computer, etc.; but the cloud game server in the cloud is used for information processing. When playing the game, the player operates the client device to send an operation instruction to the cloud game server. The cloud game server runs the game according to the operation instruction, encodes and compresses the game screen and other data, and returns it to the client device through the network. Finally, the client device decodes and outputs the game screen.

[0037] In an optional embodiment, taking a game as an example, a local terminal device stores a game program and is used to present a game screen. The local terminal device is used to interact with the player through a graphical user interface, that is, the game program is downloaded and installed by an electronic device and run conventionally. The local terminal device may provide the graphical user interface to the player in a variety of ways, for example, it may be rendered and displayed on a display screen of the terminal, or provided to the player through a holographic projection. For example, the local terminal device may include a display screen and a processor, the display screen is used to present a graphical user interface, the graphical user interface includes a game screen, and the processor is used to run the game, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0038] In a possible implementation, an embodiment of the present invention provides an image processing method that provides a graphical user interface through a terminal device, wherein the terminal device may be the local terminal device mentioned above, or may be a client device in the cloud interaction system mentioned above.

[0039] In the prior art, when image sampling is performed based on UV coordinates, the process generally includes:

[0040] S1, model preparation, manually create UV coordinates for the 3D model;

[0041] S2, texture mapping, mapping the two-dimensional texture of the map onto the three-dimensional model;

[0042] S3. During the rendering process, image sampling is performed according to the UV coordinates.

[0043] However, the process of creating UV coordinates for 3D models is cumbersome and not suitable for automated processes. In addition, during the rendering process, calculations need to be performed based on the UV coordinates, which requires a lot of calculations, especially when rendering complex scenes or performing real-time rendering. The computing resources consumed are large, which will also affect the rendering efficiency. In addition, the method of image sampling based on UV coordinates often has obvious UV seams, and the rendering effect needs to be improved.

[0044] Based on this, the present application proposes an image processing method, which performs image sampling based on the model's own mesh density and world space position, so that users do not need to manually create UV coordinates, realizes automatic image sampling, and improves rendering speed.

[0045] The method of the present application can be applied to the game running stage. When the game scene changes or the virtual objects in the current virtual scene need to be rendered in the real-time game scene, image sampling can be performed based on the mesh density of the virtual object's model itself and the world space position based on the image processing method of the present application, and rapid rendering of the virtual objects in the current virtual scene can be achieved.

[0046] In another implementation, the method of the present application can also be applied to the game design stage. In the process of rendering the model of the virtual object, image sampling can be performed based on the method of the present application, thereby improving the rendering efficiency.

[0047] The following combination Figure 1 The image processing method of the present application is described with reference to Figure 1 , the method comprising:

[0048] S101 . Determine mesh density information of a model to be rendered according to vertex data of the model to be rendered.

[0049] The model to be rendered can be a three-dimensional model of a virtual object that needs to be rendered in a virtual scene. Rendering refers to the processing of the visual effects of the virtual model after the virtual model is completed. For example, the lighting, materials, textures, colors, environments, etc. of the virtual model can be set so that the virtual model can present corresponding visual effects in the game scene.

[0050] The model to be rendered may be a three-dimensional model designed by three-dimensional software, or may be obtained by establishing a three-dimensional geometric structure by using a two-dimensional image.

[0051] The vertex data of the model to be rendered includes the position coordinates of the vertices of the model to be rendered, the number of faces in the mesh of the model to be rendered, the area of ​​each face, etc.

[0052] The mesh density information is used to characterize the surface area ratio of the model to be rendered in the three-dimensional space. In one possible implementation, the mesh density information of the model to be rendered can be characterized by the number of vertices and the number of faces of the model to be rendered. Exemplarily, the ratio of the number of vertices to the number of faces of the model to be rendered can be used as the mesh density information of the model to be rendered. In another possible implementation, the mesh density information of the model to be rendered can also be characterized by the number of vertices and the area of ​​the model to be rendered. For example, the ratio of the number of vertices to the area of ​​the model to be rendered is used as the mesh density information.

[0053] S102 . Determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered mapped to the screen.

[0054] If the current graphical user interface changes, which causes the size and proportion of the model to be rendered on the screen to change, the screen space usage information of the model to be rendered will change. At this time, the area usage of the model to be rendered on the screen can be re-determined based on the current position information of the model to be rendered.

[0055] The position information of the model to be rendered may be the position information of the model to be rendered in three-dimensional space, such as the world position coordinates of each vertex in the model to be rendered in a world coordinate system.

[0056] In a possible implementation, the pixel ratio of the model to be rendered in the screen may be used as the screen space occupation information of the model to be rendered.

[0057] S103: Determine sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information.

[0058] When the user adjusts the virtual objects in the graphical user interface, such as scaling the virtual scene or adjusting the viewing angle of the virtual scene, the screen space ratio information of the model to be rendered may change. At this time, the sampling weight information corresponding to the model to be rendered may be re-determined based on the new current screen space ratio information.

[0059] The sampling weight information corresponding to the model to be rendered indicates how to perform image sampling on the texture of the model to be rendered.

[0060] S104 . Sample the texture corresponding to the model to be rendered according to the sampling weight information to render the model to be rendered.

[0061] Based on the sampling weight information, a two-dimensional coordinate system can be constructed on the surface of the model to be rendered. By projecting the pixel points in the two-dimensional map into the two-dimensional coordinate system, image projection from the pixel points in the two-dimensional map to the vertices of the surface of the model to be rendered can be achieved.

[0062] For a more complex model to be rendered, the map corresponding to the model to be rendered may include multiple maps, and the map may be a texture map, a color map, or a light and shadow map, etc. For example, it may include a basic texture map and at least one detail texture map, or multiple areas in the model to be rendered may include corresponding maps respectively.

[0063] It should be noted that the rendering method based on UV coordinates depends on the size of the current sampling map, and the accuracy is the same for each 3D model in the virtual scene. Therefore, if the rendering accuracy is high, there will be a problem of high computing resource consumption. If the accuracy is low, there will be a problem of poor rendering effect.

[0064] In the embodiment of the present application, compared with the existing method of image sampling based on UV coordinates, the present application does not need to manually create the UV coordinates of the model to be rendered, which greatly reduces the workload and does not cause the problem of UV seams. In addition, the grid density information represents the surface area ratio of the model to be rendered in the three-dimensional space, and the screen space ratio information represents the area ratio of the model to be rendered in the graphical user interface. Therefore, the three-dimensional model with a larger area ratio in the screen can obtain higher rendering accuracy when rendering, and the three-dimensional model with a small area ratio can obtain lower rendering accuracy when rendering, which reduces the consumption of computing resources and further improves the rendering efficiency while ensuring the rendering effect.

[0065] The following is a further explanation of determining the mesh density information of the model to be rendered according to the vertex data of the model to be rendered.

[0066] In the first implementation, the ratio of the number of vertices of the model to be rendered to the surface area of ​​the model can be used as mesh density information. Figure 2 As shown, the above step S101 includes:

[0067] S201 . Determine a model surface area, a number of model faces, and a number of vertices of a model to be rendered based on vertex data.

[0068] S202: Using the ratio of the number of vertices to the model surface area or the ratio of the number of vertices to the number of model faces as mesh density information of the model to be rendered.

[0069] The model surface area may be the sum of the surface areas of all meshes in the model to be rendered, and the number of vertices may be the total number of vertices in the model to be rendered. The number of vertices and the model surface area of ​​different models are different. The ratio of the number of vertices to the model surface area can be used to represent the surface area ratio of the model to be rendered in three-dimensional space.

[0070] In the second implementation, the total number of faces and vertices of the model to be rendered can also be determined based on the vertex data, and the ratio of the number of vertices to the total number of faces of the model to be rendered is used as the mesh density information. The number of faces can be the total number of meshes in the model.

[0071] The following is a further explanation of determining the screen space ratio information of the model to be rendered according to the position information of the model to be rendered. Figure 3 As shown, the above step S102 includes:

[0072] S301 . Determine the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space.

[0073] The screen position coordinates may indicate the pixel positions of the vertices of the model to be rendered in the screen. Exemplarily, a screen space is constructed for the current screen, that is, the coordinate system of the screen, for example, the lower left corner of the screen is used as the coordinate origin, the horizontal rightward direction is the positive direction of the x-axis, and the vertical upward direction is the positive direction of the y-axis. The range of the screen space coordinates usually depends on the resolution of the screen. For example, on a screen with a resolution of 1920x1080, the range of the x-axis is 0 to 1920, and the range of the y-axis is 0 to 1080. The screen position coordinates may indicate the positions of the vertices of the model to be rendered in the screen space coordinates.

[0074] In a first implementation manner, the world position coordinates of each vertex in the model to be rendered may be converted to obtain the screen position coordinates of each vertex of the model to be rendered on the screen.

[0075] In the second implementation, the world position coordinates of some vertices in the model to be rendered may be transformed, for example, the world position coordinates of the edge vertices of the model to be rendered displayed on the screen may be transformed to obtain the screen position coordinates of the edge vertices on the screen.

[0076] S302: Determine screen space ratio information of the model to be rendered according to the screen position coordinates of the model to be rendered and the number of pixels in the screen.

[0077] Optionally, the screen position coordinates of the model to be rendered may represent the area occupied by the model to be rendered on the screen. Based on the number of pixels in the area occupied by the model to be rendered and the number of pixels in the screen, the screen space ratio information of the model to be rendered may be determined.

[0078] In a second possible implementation, the area of ​​the model to be rendered on the screen can be calculated based on the screen position coordinates of the model to be rendered, and the screen space ratio information of the model to be rendered can be determined based on the area of ​​the model to be rendered and the total area of ​​the screen.

[0079] The following is a further explanation of determining the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space. Figure 4 As shown, the above step S301 includes:

[0080] S401 , transforming the world position coordinates of the model to be rendered based on a first transformation matrix to obtain first position coordinates of the model to be rendered in a clipping space.

[0081] The first conversion matrix is ​​a conversion matrix from world space to clipping space, and is used to convert the coordinates of the model to be rendered in the world coordinate system into the clipping space.

[0082] In the process of converting the model to be rendered to the clipping space, the model located in the screen space, that is, the model visible on the screen, can be retained, and the model located outside the screen space, that is, the model invisible on the screen, can be discarded, thereby ensuring that only the model visible on the screen is rendered in the subsequent rendering process.

[0083] S402: Determine the screen position coordinates of the model to be rendered based on the first position coordinates of the model to be rendered in the clipping space.

[0084] The first position coordinates of the model to be rendered in the clipping space may be the position coordinates of each vertex of the model to be rendered in the clipping space.

[0085] like Figure 5 As shown, the process of determining the screen position coordinates of the model to be rendered based on the first position coordinates of the model to be rendered in the clipping space specifically includes:

[0086] S501: Determine whether the model to be rendered is within a visible range of the screen space according to a first position coordinate of the model to be rendered in the clipping space.

[0087] The visible range of the screen space can be represented by position coordinates. By comparing the first position coordinates with the position coordinate range of the visible range, it can be ensured that only the model within the visible range is rendered.

[0088] S502: If the model to be rendered is within the visible range of the screen space, transform the first position coordinates based on the second transformation matrix to obtain the screen position coordinates of the model to be rendered.

[0089] The second transformation matrix is ​​a transformation matrix from clipping space to screen space, and is used to transform the position coordinates of the clipping space into the screen space.

[0090] If the model to be rendered is not within the visible range of the screen space, the model to be rendered will not be converted. It should be noted that there is no strict order requirement between the steps of determining the screen space ratio information and determining the mesh density information in this application. In one possible implementation method, it is possible to first determine whether the model is within the visible range of the screen space in the process of determining the screen space ratio information, and use the model within the visible range of the screen space as the model to be rendered, and then determine the mesh density information of the model to be rendered.

[0091] The following is a further explanation of the above-mentioned method of determining the screen space ratio information of the model to be rendered according to the screen position coordinates of the model to be rendered and the number of pixels in the current screen. Figure 6 As shown, the above step S302 includes:

[0092] S601: Determine an occupied area of ​​the model to be rendered on the screen according to the screen position coordinates of multiple target vertices of the model to be rendered.

[0093] The target vertex may be an edge vertex of the model to be rendered in the graphical user interface. Exemplarily, all visible vertices of the model to be rendered from the camera perspective may be determined, and the visible vertex at the edge may be used as the target vertex of the model to be rendered.

[0094] S602: Taking the ratio of the number of pixels in the occupied area to the total number of pixels in the screen as screen space ratio information of the model to be rendered.

[0095] In a possible implementation, the ratio of the number of pixels in the occupied area to the total number of pixels on the screen may be used as screen space ratio information.

[0096] In another possible implementation, the area of ​​the occupied region may be calculated, and the ratio of the area of ​​the occupied region to the area of ​​the current graphical user interface may be used as the screen space ratio information.

[0097] The above step of determining the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information includes:

[0098] The sampling weight information corresponding to the model to be rendered is determined according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered.

[0099] In the first implementation method, sampling weight information can be determined according to mesh density information and current screen space usage information, the mesh ratio of the model to be rendered can be determined based on the mesh density information, the screen ratio of the model to be rendered can be determined based on the screen space usage information, and then a coordinate system can be constructed on the surface of the model to be rendered according to the mesh ratio and the screen ratio, and each vertex of the model to be rendered can be projected onto the coordinate system. Then, the color information of the texture corresponding to the model to be rendered can be mapped into the coordinate system to realize image sampling of the texture of the model to be rendered.

[0100] In the second implementation, the sampling weight information can also be determined based on the mesh density information and the screen position coordinates of each vertex of the model to be rendered. For example, the mesh density information can be used as the x-axis of the coordinate system, the screen position coordinates can be used as the y-axis, the screen position coordinates can be calculated based on a preset calculation strategy to obtain the y value, and the color information of the texture can be projected into the coordinate system to implement the sampling of the texture.

[0101] In the third implementation, the sampling weight information corresponding to the model to be rendered may also be determined according to the mesh density information, the current screen space ratio information, and the screen position coordinates of each vertex of the model to be rendered, such as Figure 7As shown, the above steps include:

[0102] S701: Determine a ratio of grid density information to screen space ratio information.

[0103] S702 : Determine sampling weight information corresponding to the model to be rendered according to the ratio and the screen position coordinates of each vertex of the model to be rendered.

[0104] Optionally, the ratio of the mesh density information to the screen space ratio information can be used as a parameter of the sampling weight information to uniquely distinguish the model to be rendered. The screen position coordinates of the model to be rendered are used as the second parameter of the sampling weight information to locate the position of the vertex in the model to be rendered.

[0105] Among them, the ratio of the mesh density information of the model to be rendered to the screen space ratio information can be adjusted in real time according to the changes of the model to be rendered in the current virtual scene, or can be pre-set to a fixed value.

[0106] As another possible implementation method, the screen position coordinates of each vertex can be calculated based on a preset calculation strategy to obtain the calculated value of each screen position coordinate, and the ratio of the grid density information and the screen space ratio information is used as the x value, and the calculated value is used as the y value to construct a coordinate system for the surface of the model to be rendered. The vertices on the model to be rendered are determined based on the x value and the y value, and the map is projected onto the corresponding vertices of the model to be rendered.

[0107] Optionally, the sampling weight information may further include a sampling control parameter for controlling the repetition of the sampling pattern during the image sampling process.

[0108] The following is a further description of rendering the texture corresponding to the model to be rendered to the model to be rendered according to the sampling weight information. Figure 8 As shown, the above step S104 includes:

[0109] S801, obtaining the color information of each pixel in the map.

[0110] The sampling map may be a four-dimensional continuous image. Compared with traditional maps, the four-dimensional continuous image can achieve higher precision by tiling the same precision in proportion and size. The color information of the pixel in the map may be a numerical representation of the color information in the image.

[0111] S802: Sample the color information of each pixel in the map according to the sampling weight information, and map the sampling result to the model to be rendered.

[0112] In one possible implementation, a coordinate system can be constructed on the model surface based on the sampling weight information, and the map can be scaled according to the proportion of the coordinate system so that the coordinate system of the map is the same as the coordinate system of the model surface. At this time, based on the position coordinates of the vertices on the model surface, the pixels with the same position can be determined on the map, and the color information of each pixel can be sampled to obtain the sampling results of each pixel. The sampling results of the pixel points can then be arranged in the order of the position coordinates of the vertices to complete the image sampling.

[0113] In another possible implementation, the coordinate system of the model surface and the coordinate system of the texture can be constructed separately, and a transformation matrix from the model surface coordinate system to the texture coordinate system can be established, and then the color information is mapped and rendered based on the transformation matrix. Specifically, Fig. 9 As shown, in the above step S802, the step of mapping the color information of each vertex in the texture to the model to be rendered according to the sampling weight information includes:

[0114] S901: Construct a coordinate transformation matrix according to sampling weight information.

[0115] Optionally, a two-dimensional model coordinate system can be constructed on the surface of the model to be rendered according to the sampling weight information, and a texture coordinate system of the two-dimensional texture can be constructed, and a coordinate transformation matrix can be calculated for the model coordinate system and the texture coordinate system. The coordinate transformation matrix is ​​used to determine the corresponding vertices of the pixel points in the texture in the model to be rendered.

[0116] S902: Determine, according to the coordinate conversion matrix, a first vertex corresponding to a first pixel point in the map in the model to be rendered.

[0117] S903: Sample the color information of the first pixel, and map the sampling result to the first vertex.

[0118] The first pixel point can be any point in the texture. The position coordinates of the first pixel point in the texture coordinate system are transformed according to the coordinate transformation matrix to obtain the calculated position coordinates, which are the position coordinates of the vertex corresponding to the first pixel point in the model to be rendered, that is, the position coordinates of the first vertex.

[0119] In another possible implementation, the vertices of the model to be rendered may be transformed based on a coordinate transformation matrix to obtain the corresponding pixels of the vertices of the model to be rendered in the map, and the color information of the pixels may be projected onto the vertices of the model to be rendered.

[0120] like Fig.10 , which is an example of the effect of image sampling based on conventional methods and image sampling based on the image processing method of the present application. Fig.10, image sampling based on conventional methods may have seam errors and artistic effect errors in UV sampling marked by the dotted box in the figure, while the sampling results obtained based on the method of the present application are better, without seam errors and artistic errors, and after comparison, the method of the present application is faster and more efficient.

[0121] Based on the same inventive concept, an image processing device corresponding to the image processing method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned image processing method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0122] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application is shown. Fig.11 The device includes a first determination module 1101, a second determination module 1102, a third determination module 1103 and a rendering module 1104.

[0123] A first determination module, used to determine mesh density information of the model to be rendered according to vertex data of the model to be rendered;

[0124] A second determination module is used to determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered on the screen;

[0125] A third determination module is used to determine the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information;

[0126] The rendering module is used to sample the texture corresponding to the model to be rendered according to the sampling weight information to render the model to be rendered.

[0127] In a feasible implementation scheme, the first determining module 1101 is specifically used for:

[0128] Determine the model surface area, the number of model faces and the number of vertices of the model to be rendered based on the vertex data;

[0129] The ratio of the number of vertices to the model surface area or the ratio of the number of vertices to the number of model faces is used as the mesh density information of the model to be rendered.

[0130] In a feasible implementation manner, the second determining module 1102 is specifically configured to:

[0131] Determine the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space;

[0132] The screen space ratio information of the model to be rendered is determined according to the screen position coordinates of the model to be rendered and the number of pixels in the screen.

[0133] In a feasible implementation manner, the second determining module 1102 is specifically configured to:

[0134] The world position coordinates of the model to be rendered are transformed based on the first transformation matrix to obtain the first position coordinates of the model to be rendered in the clipping space;

[0135] Based on the first position coordinates of the model to be rendered in the clipping space, the screen position coordinates of the model to be rendered are determined.

[0136] In a feasible implementation manner, the second determining module 1102 is specifically configured to:

[0137] Determine whether the model to be rendered is within the visible range of the screen space according to the first position coordinates of the model to be rendered in the clipping space;

[0138] If the model to be rendered is within the visible range of the screen space, the first position coordinates are transformed based on the second transformation matrix to obtain the screen position coordinates of the model to be rendered.

[0139] In a feasible implementation manner, the second determining module 1102 is specifically configured to:

[0140] Determine the occupied area of ​​the model to be rendered on the screen according to the screen position coordinates of multiple target vertices of the model to be rendered;

[0141] The ratio of the number of pixels in the occupied area to the total number of pixels on the screen is used as the screen space ratio information of the model to be rendered.

[0142] In a feasible implementation scheme, the third determination module 1103 is specifically used for:

[0143] The sampling weight information corresponding to the model to be rendered is determined according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered.

[0144] In a feasible implementation scheme, the third determination module 1103 is specifically used for:

[0145] Determine the ratio of mesh density information to screen space ratio information;

[0146] According to the ratio and the screen position coordinates of each vertex of the model to be rendered, the sampling weight information corresponding to the model to be rendered is determined.

[0147] In a feasible implementation manner, the rendering module 1104 is specifically used for:

[0148] Get the color information of each pixel in the map;

[0149] According to the sampling weight information, the color information of each pixel in the map is sampled, and the sampling result is mapped to the model to be rendered.

[0150] In a feasible implementation manner, the rendering module 1104 is specifically used for:

[0151] Construct a coordinate transformation matrix based on the sampling weight information;

[0152] Determine, according to the coordinate conversion matrix, a first vertex corresponding to a first pixel point in the texture in the model to be rendered;

[0153] The color information of the first pixel is sampled, and the sampling result is mapped to the first vertex of the model to be rendered.

[0154] Compared with the existing method of image sampling based on UV coordinates, this application does not need to manually create the UV coordinates of the model to be rendered, which greatly reduces the workload and does not cause the problem of UV seams. In addition, the grid density information represents the surface area ratio of the model to be rendered in the three-dimensional space, and the screen space ratio information represents the area ratio of the model to be rendered in the graphical user interface. Therefore, the three-dimensional model with a larger area ratio on the screen can obtain higher rendering accuracy when rendering, and the three-dimensional model with a smaller area ratio can obtain lower rendering accuracy when rendering, which reduces the consumption of computing resources and further improves the rendering efficiency while ensuring the rendering effect.

[0155] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown, including: a processor 1201, a storage medium 1202 and a bus 1203, wherein the storage medium 1202 stores machine-readable instructions executable by the processor 1201. When the electronic device runs the image processing method in the embodiment, the processor 1201 communicates with the storage medium 1202 via the bus 1203, and the processor 1201 executes the machine-readable instructions. The processor 1201 performs the following steps in the preamble of the method item:

[0156] Determine mesh density information of the model to be rendered according to vertex data of the model to be rendered;

[0157] Determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered on the screen;

[0158] Determine the sampling weight information corresponding to the model to be rendered based on the mesh density information and the current screen space usage information;

[0159] The texture corresponding to the model to be rendered is sampled according to the sampling weight information to render the model to be rendered.

[0160] In a feasible implementation manner, when the processor 1201 determines the mesh density information of the model to be rendered according to the vertex data of the model to be rendered, it is specifically used to:

[0161] Determine the model surface area, the number of model faces and the number of vertices of the model to be rendered based on the vertex data;

[0162] The ratio of the number of vertices to the model surface area or the ratio of the number of vertices to the number of model faces is used as the mesh density information of the model to be rendered.

[0163] In a feasible implementation manner, when the processor 1201 determines the screen space proportion information of the model to be rendered according to the position information of the model to be rendered, it is specifically used to:

[0164] Determine the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space;

[0165] The screen space ratio information of the model to be rendered is determined according to the screen position coordinates of the model to be rendered and the number of pixels in the screen.

[0166] In a feasible implementation manner, when the processor 1201 determines the screen position coordinates of the model to be rendered in the current graphical user interface according to the world position coordinates of the model to be rendered in the three-dimensional space, it is specifically used to:

[0167] The world position coordinates of the model to be rendered are transformed based on the first transformation matrix to obtain the first position coordinates of the model to be rendered in the clipping space;

[0168] Based on the first position coordinates of the model to be rendered in the clipping space, the screen position coordinates of the model to be rendered are determined.

[0169] In a feasible implementation manner, when the processor 1201 determines the screen position coordinates of the model to be rendered based on the first position coordinates of the model to be rendered in the clipping space, it is specifically used to:

[0170] Determine whether the model to be rendered is within the visible range of the screen space according to the first position coordinates of the model to be rendered in the clipping space;

[0171] If the model to be rendered is within the visible range of the screen space, the first position coordinates are transformed based on the second transformation matrix to obtain the screen position coordinates of the model to be rendered.

[0172] In a feasible implementation manner, when the processor 1201 determines the screen space proportion information of the model to be rendered according to the screen position coordinates of the model to be rendered and the number of pixels in the current graphical user interface, it is specifically used to:

[0173] Determine the occupied area of ​​the model to be rendered on the screen according to the screen position coordinates of multiple target vertices of the model to be rendered;

[0174] The ratio of the number of pixels in the occupied area to the total number of pixels on the screen is used as the screen space ratio information of the model to be rendered.

[0175] In a feasible implementation manner, when the processor 1201 determines the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space occupation information, it is specifically used to:

[0176] The sampling weight information corresponding to the model to be rendered is determined according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered.

[0177] In a feasible implementation scheme, when the processor 1201 determines the sampling weight information corresponding to the model to be rendered according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered, it is specifically used to:

[0178] Determine the ratio of mesh density information to screen space ratio information;

[0179] According to the ratio and the screen position coordinates of each vertex of the model to be rendered, the sampling weight information corresponding to the model to be rendered is determined.

[0180] In a feasible implementation manner, when the processor 1201 renders the texture corresponding to the model to be rendered to the model to be rendered according to the sampling weight information, it is specifically used to:

[0181] Get the color information of each pixel in the map;

[0182] According to the sampling weight information, the color information of each pixel in the map is sampled, and the sampling result is mapped to the model to be rendered.

[0183] In a feasible implementation manner, the processor 1201 is specifically used to:

[0184] Construct a coordinate transformation matrix based on the sampling weight information;

[0185] Determine, according to the coordinate conversion matrix, a first vertex corresponding to a first pixel point in the texture in the model to be rendered;

[0186] The color information of the first pixel is sampled, and the sampling result is mapped to the first vertex of the model to be rendered.

[0187] This application does not need to manually create UV coordinates of the model to be rendered, which reduces the workload and does not cause UV seam problems. In addition, the grid density information represents the surface area ratio of the model to be rendered in the three-dimensional space, and the screen space ratio information represents the area ratio of the model to be rendered in the graphical user interface. Therefore, the three-dimensional model with a larger area ratio on the screen can obtain higher rendering accuracy when rendering, and the three-dimensional model with a smaller area ratio can obtain lower rendering accuracy when rendering, which reduces the consumption of computing resources and further improves the rendering efficiency while ensuring the rendering effect.

[0188] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:

[0189] Determine mesh density information of the model to be rendered according to vertex data of the model to be rendered;

[0190] Determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered on the screen;

[0191] Determine the sampling weight information corresponding to the model to be rendered based on the mesh density information and the current screen space usage information;

[0192] The texture corresponding to the model to be rendered is sampled according to the sampling weight information to render the model to be rendered.

[0193] In a feasible implementation manner, when the processor determines the mesh density information of the model to be rendered according to the vertex data of the model to be rendered, the processor is specifically used to:

[0194] Determine the model surface area, the number of model faces and the number of vertices of the model to be rendered based on the vertex data;

[0195] The ratio of the number of vertices to the model surface area or the ratio of the number of vertices to the number of model faces is used as the mesh density information of the model to be rendered.

[0196] In a feasible implementation manner, when the processor determines the screen space proportion information of the model to be rendered according to the position information of the model to be rendered, the processor is specifically used to:

[0197] Determine the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space;

[0198] The screen space ratio information of the model to be rendered is determined according to the screen position coordinates of the model to be rendered and the number of pixels in the screen.

[0199] In a feasible implementation manner, when the processor determines the screen position coordinates of the model to be rendered in the current graphical user interface according to the world position coordinates of the model to be rendered in the three-dimensional space, it is specifically used to:

[0200] The world position coordinates of the model to be rendered are transformed based on the first transformation matrix to obtain the first position coordinates of the model to be rendered in the clipping space;

[0201] Based on the first position coordinates of the model to be rendered in the clipping space, the screen position coordinates of the model to be rendered are determined.

[0202] In a feasible implementation manner, when the processor determines the screen position coordinates of the model to be rendered based on the first position coordinates of the model to be rendered in the clipping space, the processor is specifically used to:

[0203] Determine whether the model to be rendered is within the visible range of the screen space according to the first position coordinates of the model to be rendered in the clipping space;

[0204] If the model to be rendered is within the visible range of the screen space, the first position coordinates are transformed based on the second transformation matrix to obtain the screen position coordinates of the model to be rendered.

[0205] In a feasible implementation manner, when the processor determines the screen space proportion information of the model to be rendered according to the screen position coordinates of the model to be rendered and the number of pixels in the current graphical user interface, it is specifically used to:

[0206] Determine the occupied area of ​​the model to be rendered on the screen according to the screen position coordinates of multiple target vertices of the model to be rendered;

[0207] The ratio of the number of pixels in the occupied area to the total number of pixels on the screen is used as the screen space ratio information of the model to be rendered.

[0208] In a feasible implementation manner, when the processor determines the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information, it is specifically used to:

[0209] The sampling weight information corresponding to the model to be rendered is determined according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered.

[0210] In a feasible implementation scheme, when the processor determines the sampling weight information corresponding to the model to be rendered according to the mesh density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered, it is specifically used to:

[0211] Determine the ratio of mesh density information to screen space ratio information;

[0212] According to the ratio and the screen position coordinates of each vertex of the model to be rendered, the sampling weight information corresponding to the model to be rendered is determined.

[0213] In a feasible implementation manner, when the processor renders the texture corresponding to the model to be rendered to the model to be rendered according to the sampling weight information, it is specifically used to:

[0214] Get the color information of each pixel in the map;

[0215] According to the sampling weight information, the color information of each pixel in the map is sampled, and the sampling result is mapped to the model to be rendered.

[0216] In a feasible implementation scheme, when the processor samples the color information of each pixel in the map according to the sampling weight information and maps the sampling result to the model to be rendered, it is specifically used to:

[0217] Construct a coordinate transformation matrix based on the sampling weight information;

[0218] Determine, according to the coordinate conversion matrix, a first vertex corresponding to a first pixel point in the texture in the model to be rendered;

[0219] The color information of the first pixel is sampled, and the sampling result is mapped to the first vertex of the model to be rendered.

[0220] This application does not need to manually create UV coordinates of the model to be rendered, which reduces the workload and does not cause UV seam problems. In addition, the grid density information represents the surface area ratio of the model to be rendered in the three-dimensional space, and the screen space ratio information represents the area ratio of the model to be rendered in the graphical user interface. Therefore, the three-dimensional model with a larger area ratio on the screen can obtain higher rendering accuracy when rendering, and the three-dimensional model with a smaller area ratio can obtain lower rendering accuracy when rendering, which reduces the consumption of computing resources and further improves the rendering efficiency while ensuring the rendering effect.

[0221] In the embodiment of the present application, the computer program can also execute other machine-readable instructions when run by the processor to execute other methods described in the embodiment. For the specific execution method steps and principles, please refer to the description of the embodiment, which will not be repeated here.

[0222] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0223] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0224] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0225] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0226] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0227] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Determining mesh density information of the model to be rendered according to vertex data of the model to be rendered; Determine screen space ratio information of the model to be rendered according to the position information of the model to be rendered, where the screen space ratio information is used to represent the area ratio of the model to be rendered mapped to the screen; Determine sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information; The texture corresponding to the model to be rendered is sampled according to the sampling weight information to render the model to be rendered.

2. The method according to claim 1, characterized in that The step of determining mesh density information of the model to be rendered according to vertex data of the model to be rendered includes: Determine the model surface area, the number of model faces and the number of vertices of the model to be rendered based on the vertex data; The ratio of the number of vertices to the surface area of ​​the model or the ratio of the number of vertices to the number of faces of the model is used as the mesh density information of the model to be rendered.

3. The method according to claim 1, characterized in that The step of determining screen space proportion information of the model to be rendered according to the position information of the model to be rendered includes: Determining the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space; The screen space ratio information of the model to be rendered is determined according to the screen position coordinates of the model to be rendered and the number of pixels in the screen.

4. The method according to claim 3, characterized in that The step of determining the screen position coordinates of the model to be rendered on the screen according to the world position coordinates of the model to be rendered in the three-dimensional space includes: The world position coordinates of the model to be rendered are transformed based on the first transformation matrix to obtain the first position coordinates of the model to be rendered in the clipping space; Based on the first position coordinates of the model to be rendered in the clipping space, the screen position coordinates of the model to be rendered are determined.

5. The method according to claim 4, characterized in that The determining the screen position coordinates of the model to be rendered based on the first position coordinates of the model to be rendered in the clipping space includes: Determining whether the model to be rendered is within a visible range of the screen space according to the first position coordinates of the model to be rendered in the clipping space; If the model to be rendered is within the visible range of the screen space, the first position coordinates are transformed based on a second transformation matrix to obtain the screen position coordinates of the model to be rendered.

6. The method according to claim 3, characterized in that The determining, according to the screen position coordinates of the model to be rendered and the number of pixels in the screen, screen space proportion information of the model to be rendered includes: Determining an occupied area of ​​the model to be rendered on the screen according to the screen position coordinates of a plurality of target vertices of the model to be rendered; The ratio of the number of pixels in the occupied area to the total number of pixels in the screen is used as the screen space ratio information of the model to be rendered.

7. The method according to claim 1, characterized in that Determining sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information includes: The sampling weight information corresponding to the model to be rendered is determined according to the grid density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered.

8. The method according to claim 7, characterized in that The determining, according to the grid density information, the current screen space ratio information and the screen position coordinates of each vertex of the model to be rendered, sampling weight information corresponding to the model to be rendered includes: Determine a ratio of the grid density information to the screen space ratio information; The sampling weight information corresponding to the model to be rendered is determined according to the ratio and the screen position coordinates of each vertex of the model to be rendered.

9. The method according to claim 1, characterized in that: The step of sampling the map corresponding to the model to be rendered according to the sampling weight information includes: Obtaining color information of each pixel in the map; According to the sampling weight information, the color information of each pixel in the map is sampled, and the sampling result is mapped to the model to be rendered.

10. The method according to claim 9, characterized in that The step of sampling the color information of each pixel in the map according to the sampling weight information and mapping the sampling result to the model to be rendered includes: Constructing a coordinate transformation matrix according to the sampling weight information; Determine, according to the coordinate conversion matrix, a first vertex corresponding to a first pixel point in the map in the model to be rendered; The color information of the first pixel is sampled, and the sampling result is mapped to the first vertex of the model to be rendered.

11. An image processing device, characterized in that: include: A first determining module, used to determine mesh density information of the model to be rendered according to vertex data of the model to be rendered; A second determination module is used to determine screen space proportion information of the model to be rendered according to the position information of the model to be rendered, wherein the screen space proportion information is used to represent the area proportion of the model to be rendered mapped to the screen; A third determination module is used to determine the sampling weight information corresponding to the model to be rendered according to the grid density information and the current screen space ratio information; A rendering module is used to sample the map corresponding to the model to be rendered according to the sampling weight information to render the model to be rendered.

12. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the image processing method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 10 are executed.

Images