Image processing method and device, electronic equipment, storage medium and program product

Abstract

This application provides an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes. The method includes: acquiring an original image of the user interface of the virtual scene; acquiring an interface texture region corresponding to the texture coordinate system of the user interface, and acquiring the first texture coordinates of each sampling point of the interface texture region in the texture coordinate system; performing perspective transformation processing on the first texture coordinates of each sampling point in the texture coordinate system based on perspective graphics to obtain the second texture coordinates of each sampling point in the texture coordinate system; performing sampling processing on the original image based on the second texture coordinates of each sampling point to obtain perspective rendering data for each sampling point; and generating a rendered image of the user interface with perspective effect based on the perspective rendering data of each sampling point. This application enables sampling entirely based on the original image of the user interface, thereby improving processing flexibility.

Description

Technical Field

[0001] This application relates to image processing technology for virtual scenes, and more particularly to an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes. Background Technology

[0002] Display technologies based on graphics processing hardware have expanded the channels for perceiving the environment and acquiring information. In particular, multimedia technologies for virtual scenes, with the help of human-computer interaction engine technology, can realize diverse interactions between virtual objects controlled by users or artificial intelligence according to actual application needs. They have various typical application scenarios. For example, in virtual scenes such as games, they can simulate the real battle process between virtual objects.

[0003] In related technologies, the virtual scene can be presented in three dimensions, thus providing users with an excellent visual experience. In addition to the virtual scene, the visual presentation of the user interface also affects the user's visual experience. Therefore, related technologies will render the user interface and the virtual scene together, so that a three-dimensional perspective effect can also be achieved in the user interface. However, the co-rendering method in related technologies leads to low rendering flexibility and requires high computing resources. Summary of the Invention

[0004] This application provides an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes, which can sample images entirely based on the original images of the user interface, thereby improving processing flexibility and saving computing resources.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an image processing method for a virtual scene, including:

[0007] Obtain the original image of the user interface of the virtual scene;

[0008] Obtain the interface texture region corresponding to the texture coordinate system of the user interface, and obtain the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system;

[0009] A perspective transformation based on perspective graphics is performed on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system.

[0010] Based on the second texture coordinates of each sampling point, sampling processing is performed on the original image to obtain perspective rendering data for each sampling point;

[0011] Based on the perspective rendering data of each of the sampling points, a user interface with perspective effect is rendered to the screen.

[0012] This application provides an image processing apparatus for a virtual scene, comprising:

[0013] The acquisition module is used to acquire the original image of the user interface of the virtual scene;

[0014] The coordinate module is used to obtain the interface texture region corresponding to the texture coordinate system of the user interface, and to obtain the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system.

[0015] The transformation module is used to perform perspective transformation processing based on perspective graphics on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system.

[0016] A sampling module is used to perform sampling processing on the original image based on the second texture coordinates of each sampling point to obtain perspective rendering data for each sampling point.

[0017] The rendering module is used to render a user interface with perspective effect to the screen based on the perspective rendering data of each of the sampling points.

[0018] In the above scheme, the acquisition module is further configured to: call the rendering engine to perform planar rendering processing on the user interface of the virtual scene to obtain an original image for output, wherein the virtual scene is a three-dimensional virtual scene, and the original image does not have the perspective effect.

[0019] In the above scheme, the transformation module is further configured to: construct a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture region; transform the first texture coordinates of each sampling point in the texture coordinate system to obtain the first perspective coordinates of each sampling point in the perspective coordinate system; perform a transformation process based on the perspective graphic on the first perspective coordinates of each sampling point to obtain the second perspective coordinates of each sampling point; and perform a transformation process from the perspective coordinate system to the texture coordinate system on the second perspective coordinates of each sampling point to obtain the second texture coordinates of each sampling point.

[0020] In the above scheme, the transformation module is further configured to: obtain a target horizontal axis that passes through the target point pointed to by the specified texture coordinates and is parallel to the horizontal axis of the texture coordinate system; obtain a perspective graphic that is located within the interface texture area, is symmetrical about the target horizontal axis, and conforms to the perspective parameters; obtain the focal point of the perspective graphic on the target horizontal axis as the origin; obtain a target vertical axis that is perpendicular to the target horizontal axis and passes through the origin; and generate a perspective coordinate axis composed of the target vertical axis and the target horizontal axis.

[0021] In the above scheme, when the perspective parameters include a first perspective length, a second perspective length, and a perspective angle, the transformation module is further configured to: obtain the base of a trapezoid with the target point as its midpoint, a length equal to the first perspective length, and perpendicular to the target horizontal axis; obtain the top of a trapezoid parallel to the base of the trapezoid, a length equal to the second perspective length, and bisected by the target horizontal axis; and use the isosceles trapezoid characterized by the top of the trapezoid, the base of the trapezoid, and the perspective angle as the perspective graphic.

[0022] In the above scheme, before transforming the first texture coordinate of each sampling point in the texture coordinate system to obtain the first perspective coordinate of each sampling point in the perspective coordinate system, the transformation module is further configured to: obtain the number of pixels on the horizontal edge and the number of pixels on the vertical edge of the user interface; perform grid division processing on the interface texture area based on the number of pixels on the horizontal edge and the number of pixels on the vertical edge to obtain multiple intersection points of the interface texture area as the sampling points.

[0023] In the above scheme, when the perspective graphic is an isosceles trapezoid, the transformation module is further configured to: obtain the perspective coordinate transformation relationship between the isosceles trapezoid and the interface texture region in the perspective coordinate system; and perform transformation processing on the first perspective coordinate of each sampling point based on the perspective coordinate transformation relationship to obtain the second perspective coordinate of each sampling point.

[0024] In the above scheme, the sampling module is further configured to: perform initial sampling processing on the original image based on the second texture coordinates of each sampling point to obtain the initial sampling result of each sampling point; and perform anti-aliasing processing in the vertical direction on the initial sampling result of each sampling point to obtain the perspective rendering data of each sampling point.

[0025] In the above scheme, the sampling module is further configured to: perform the following processing for each sampling point: obtain the base color value and transparency value corresponding to the second texture coordinate in the original image; when the second texture coordinate is not outside the coordinate range corresponding to the interface texture area, combine the base color value and the transparency value to form the initial sampling result of the sampling point; when the second texture coordinate is outside the coordinate range corresponding to the interface texture area, update the transparency value of the sampling point to zero, and combine the base color value and the updated transparency value to form the initial sampling result of the sampling point.

[0026] In the above scheme, the sampling module is further configured to: perform the following processing for each sampling point: when the sampling point has two adjacent sampling points in the vertical direction, obtain the initial sampling result of each of the adjacent sampling points, and fuse the initial sampling results of the two adjacent sampling points with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point; when the sampling point has one adjacent sampling point in the vertical direction, obtain the initial sampling result of the one adjacent sampling point, and fuse the initial sampling result of the one adjacent sampling point with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point.

[0027] This application provides an image processing method for a virtual scene, including:

[0028] The virtual scene is displayed in the human-computer interaction interface;

[0029] The user interface of the virtual scene is displayed in the human-computer interaction interface, wherein the user interface has a perspective effect and is rendered by executing the image processing method of the virtual scene provided in the embodiments of this application.

[0030] This application provides an image processing apparatus for a virtual scene, the apparatus comprising:

[0031] The display module is used to display the virtual scene in the human-computer interaction interface;

[0032] The display module is further configured to display the user interface of the virtual scene in the human-computer interaction interface, wherein the user interface has a perspective effect and is rendered by executing the image processing method of the virtual scene provided in the embodiments of this application.

[0033] This application provides an electronic device, including:

[0034] Memory, used to store executable instructions;

[0035] The processor, when executing executable instructions stored in the memory, implements the image processing method for virtual scenes provided in the embodiments of this application.

[0036] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the image processing method for a virtual scene provided in this application.

[0037] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the image processing method for a virtual scene provided in this application.

[0038] The embodiments of this application have the following beneficial effects:

[0039] The perspective effect is achieved by transforming the texture coordinates of the sampling points based on perspective graphics, and then sampling the original image of the user interface based on the transformed second texture coordinates. Since the rendering is entirely based on the user interface, sampling overhead can be saved and the flexibility of processing timing can be improved. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of image file coordinates provided in an embodiment of this application;

[0041] Figure 2 This is a schematic diagram of the user interface rendering process for related technologies;

[0042] Figure 3 This is a schematic diagram illustrating the application mode of the image processing method for virtual scenes provided in the embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of an electronic device for an image processing method for applying virtual scenes provided in an embodiment of this application;

[0044] Figures 5A-5C This is a flowchart illustrating the image processing method for a virtual scene provided in an embodiment of this application;

[0045] Figures 6A-6B This is a schematic diagram of the interface of the image processing method for virtual scenes provided in the embodiments of this application;

[0046] Figure 7 This is a flowchart illustrating the image processing method for a virtual scene provided in an embodiment of this application;

[0047] Figure 8 This is a perspective trapezoidal diagram of the image processing method for a virtual scene provided in the embodiments of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0050] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0052] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0053] 1) Perspective effect: A two-dimensional projection effect obtained by rotating a planar image along a vertical axis.

[0054] 2) UV coordinates: See Figure 1 , Figure 1 This is a schematic diagram of image file coordinates provided in the embodiments of this application. The UV coordinate means that all image files are a two-dimensional plane, with the horizontal direction being U and the vertical direction being V. Through this two-dimensional plane UV coordinate system, any pixel on the image can be located.

[0055] 3) Patch: refers to a graphic formed by four points with three-dimensional coordinates.

[0056] 4) Texture: Here, it specifically refers to a two-dimensional texture, which is a texture map that can be sampled. When sampling, the input coordinates (X, Y) are used to output RGB or RGBA colors.

[0057] See Figure 2 , Figure 2This is a schematic diagram of the rendering process of the user interface of the related technology. In step 101, the two-dimensional user interface is set as a surface with three-dimensional coordinates. In step 102, the surface and the virtual scene are rendered in three dimensions together. In step 103, the user interface and the virtual scene are output at the same time.

[0058] In related technologies, the user interface is treated as a surface in a 3D scene. Since user interfaces are mostly made of semi-transparent materials, rendering these materials requires additional performance overhead. Furthermore, the actual depth of the user interface is zero, making depth testing difficult. Rendering the user interface and the 3D scene together may cause 3D lighting to affect the normal display of the user interface. In summary, the rendering methods in these technologies must be bound to the virtual scene, making it impossible to perform lightweight perspective processing on the user interface at any time, resulting in low flexibility and requiring significant computational resources.

[0059] This application provides an image processing method, apparatus, electronic device, computer-readable storage medium, and computer program product for virtual scenes. It can sample the original image of the user interface in two-dimensional space to achieve a three-dimensional perspective effect, thereby improving the flexibility of the perspective-based user interface and saving computing resources. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices).

[0060] To facilitate a clearer understanding of the image processing method for virtual scenes provided in this application, an exemplary implementation scenario of the image processing method for virtual scenes provided in this application will be described first. The virtual scene can be output entirely based on the terminal, or output based on the collaboration between the terminal and the server.

[0061] In some embodiments, the virtual scene can be an environment for game characters to interact. For example, it can be a place where game characters can fight each other. By controlling the actions of virtual objects, two parties can interact in the virtual scene, allowing users to relieve life stress during the game. For example, it can be a place where users can build and repair buildings in the virtual scene through a user interface, allowing users to learn while having fun during the game.

[0062] In another implementation scenario, see Figure 3 , Figure 3 This is a schematic diagram of the application mode of the image processing method for virtual scenes provided in the embodiments of this application. It is applied to the terminal 400 and the server 200. Generally, it is suitable for application modes that rely on the computing power of the server 200 to complete the calculation of virtual scenes and output the virtual scenes on the terminal 400.

[0063] As an example, let's take a simulated building repair app as an example. Server 200 acquires the original image of the user interface of the virtual scene (the building to be repaired, such as the Great Wall). The user interface can be the user's interface (e.g., a repair interface, which includes multiple repair controls). Server 200 acquires the interface texture region corresponding to the texture coordinate system of the user interface, and constructs a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture region. Server 200 acquires the first perspective coordinate of each sampling point in the interface texture region in the perspective coordinate system, and performs perspective transformation processing based on perspective graphics on the first perspective coordinate of each sampling point to obtain the first texture coordinate of each sampling point in the texture coordinate system. Based on the first texture coordinate of each sampling point, server 200 performs sampling processing on the original image to obtain the perspective rendering data of each sampling point. Based on the perspective rendering data of each sampling point, server 200 generates a rendered image of the user interface with perspective effect. Server 200 sends the rendered image to terminal 400 so that the game client in terminal 400 can present the user interface to the user by outputting the rendered image.

[0064] As an example, terminal 400 acquires the original image of the user interface (e.g., a repair interface, which includes multiple repair controls) of a virtual scene (a building to be repaired, such as the Great Wall); acquires the interface texture region corresponding to the texture coordinate system of the user interface, and constructs a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture region; acquires the first perspective coordinate of each sampling point of the interface texture region in the perspective coordinate system, and performs perspective transformation processing based on perspective graphics on the first perspective coordinate of each sampling point to obtain the first texture coordinate of each sampling point in the texture coordinate system; performs sampling processing on the original image based on the first texture coordinate of each sampling point to obtain the perspective rendering data of each sampling point; and generates a rendered image of the user interface with perspective effect based on the perspective rendering data of each sampling point, so that the game client in terminal 400 presents the user interface to the user by outputting the rendered image during runtime.

[0065] In some embodiments, the terminal 400 can implement the image processing method for the virtual scene provided in this application embodiment by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a game APP (i.e., the client mentioned above); it can also be a mini-program, that is, a program that only needs to be downloaded to a browser environment to run; or it can be a game mini-program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plugin.

[0066] The embodiments of this application can be implemented with the help of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.

[0067] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.

[0068] As an example, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, or smartwatch, but is not limited to these. Terminal 400 and server 200 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0069] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device for the image processing method of applying virtual scenes provided in the embodiments of this application. The description takes the electronic device as a terminal as an example. Figure 4 The terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 440.

[0070] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0071] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0072] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0073] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0074] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0075] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0076] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0077] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0078] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0079] In some embodiments, the image processing apparatus for virtual scenes provided in this application can be implemented in software. Figure 4 An image processing device 455 for a virtual scene stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: acquisition module 4551, coordinate module 4552, transformation module 4553, sampling module 4554, and rendering module 4555. These modules are logically related and can be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0080] The image processing method for virtual scenes provided in this application will be described in conjunction with exemplary applications and implementations of the terminals provided in the embodiments of this application.

[0081] See Figure 5A , Figure 5A This is a flowchart illustrating the image processing method for a virtual scene provided in this application embodiment, which will be combined with... Figure 5A Steps 301 to 305 are described below.

[0082] In step 301, the original image of the user interface of the virtual scene is obtained.

[0083] The user interface (UI) is the medium through which a system and its users interact and exchange information. It converts information from its internal form to a form that is acceptable to humans. A UI can be a user interface for operation, or a display interface for prompts and information, etc.

[0084] In some embodiments, obtaining the original image of the user interface of the virtual scene in step 301 can be achieved by the following technical solution: calling the rendering engine to perform planar rendering processing on the user interface of the virtual scene to obtain the original image for output, wherein the virtual scene is a three-dimensional virtual scene, and the original image does not have perspective effect. By directly using the original image as the basis for subsequent sampling, the number of sampling times can be reduced and the image processing efficiency can be improved.

[0085] As an example, consider a game. The virtual scene in the game is a 3D virtual scene, while the user interface is a 2D user interface. To achieve a perspective effect in the 2D user interface, the original image of the 2D user interface (which does not have perspective) can be used as the basis for subsequent sampling. The rendering pipeline of a rendering engine (such as Unreal Engine) renders the original image of the 2D user interface as the texture to be sampled. The original image of the 2D user interface does not have perspective. See [link to documentation]. Figure 6A , Figure 6A The image shown is the original image, which is used as the basis for sampling.

[0086] In step 302, the interface texture region corresponding to the texture coordinate system of the user interface is obtained, and the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system is obtained.

[0087] As an example, the texture coordinate system is a UV coordinate system with m as the origin; see [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of image file coordinates provided in an embodiment of this application. UV coordinates refer to the fact that all image files are on a two-dimensional plane, with the horizontal direction represented by U and the vertical direction by V. Through this two-dimensional plane's UV coordinate system, any pixel on the image can be located. The interface texture region is the area constrained by set coordinates in the texture coordinate system, for example... Figure 8 The interface texture area shown is a square, which is constrained by the coordinates (0,0), (0,1), (1,0) and (1,1).

[0088] In step 303, a perspective transformation based on perspective graphics is performed on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system.

[0089] In some embodiments, see Figure 5B , Figure 5B This is a flowchart illustrating the image processing method for a virtual scene provided in this application embodiment. In step 303, a perspective transformation based on perspective graphics is performed on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system. Figure 5B Steps 3031 to 3034 are implemented.

[0090] In step 3031, a perspective coordinate system is constructed based on the specified texture coordinates of the interface texture region and perspective parameters.

[0091] In some embodiments, the perspective coordinate system is a newly constructed coordinate system used for perspective transformation. A new coordinate system (perspective coordinate system XOY) can be constructed based on the known texture coordinate system and perspective parameters. In step 3031, the perspective coordinate system is constructed according to the specified texture coordinates and perspective parameters of the interface texture region. This can be achieved through the following technical solutions: Obtain the target horizontal axis that passes through the target point pointed to by the specified texture coordinate and is parallel to the horizontal axis of the texture coordinate system. The specified texture coordinate is located within the interface texture region, or it can be located on the boundary of the interface texture region. The specified texture coordinate is point f on a square, and the coordinates of point f in the texture coordinate system are (1, 0.5). The target horizontal axis passes through point f in the texture coordinate system (UV coordinate system), and the target horizontal axis is parallel to the horizontal axis (U axis) of the texture coordinate system. Obtain a perspective graphic that is symmetrical about the target horizontal axis within the interface texture region and conforms to the perspective parameters. The perspective graphic can be a trapezoid or a triangle. For example, the perspective graphic is... Figure 8 The isosceles trapezoid shown is symmetrical about the horizontal axis of the target and conforms to perspective parameters, which can be the included angles and side lengths related to the trapezoid, for example... Figure 8 The angle α between the hypotenuse of the trapezoid and the x-axis is used to construct an isosceles trapezoid symmetrical about the x-axis, based on the perspective angle 2a and the set length h of the upper side of the trapezoid. The focal point of the perspective figure on the target horizontal axis is taken as the origin, and the intersection (focal point) of the two hypotenuses of the trapezoid on the x-axis is taken as the origin O(0,0) of the perspective coordinate system XOY. The target vertical axis, perpendicular to the target horizontal axis and passing through the origin, is obtained. Since the origin of the perspective coordinate system has been obtained, and the perspective coordinate system is a rectangular coordinate system, the target vertical axis is perpendicular to the target horizontal axis. Figure 8 The target's vertical axis (Y-axis) is shown, and the final result is a perspective coordinate system composed of the target's vertical axis and horizontal axis. Constructing an XOY coordinate system before performing perspective transformations allows for subsequent perspective transformations within that system, reducing computational complexity and improving processing efficiency.

[0092] In some embodiments, when the perspective parameters include a first perspective length, a second perspective length, and a perspective angle, the aforementioned acquisition of a perspective graphic that is symmetrical about the target horizontal axis within the interface texture area and conforms to the perspective parameters can be achieved through the following technical solution: acquiring the base of a trapezoid with the target point as its midpoint, a length equal to the first perspective length, and perpendicular to the target horizontal axis; acquiring the top of a trapezoid that is parallel to the base, a length equal to the second perspective length, and is bisected by the target horizontal axis; and using the isosceles trapezoid characterized by the top, base, and perspective angle as the perspective graphic. By using the isosceles trapezoid as the perspective graphic in this application embodiment, subsequent transformations can be performed based on the isosceles trapezoid, and the final rendering can visually create a perspective effect where elements converge from near to far to a certain point.

[0093] As an example, see Figure 8 The first perspective length is the length of the longer lower side of the isosceles trapezoid, and the second perspective length is the length of the shorter upper side. Using an isosceles trapezoid, a perspective effect of gradually focusing from near to far can be created. Similarly, an isosceles trapezoid can be used as a perspective figure. Perspective parameters can also be the lengths of the isosceles sides of an isosceles triangle and the included angle between those sides. The target point is... Figure 8 Point f shown here, the first perspective length can be the side length of the square, which is 1, or the first perspective length can be less than 1, for example, 0.8. The length of the upper side of the trapezoid is the second perspective length, such as... Figure 8 The second perspective length is h. The intersection of the two hypotenuses of the isosceles trapezoid on the X-axis is the focal point, so that any hypotenuse forms a perspective angle α with the X-axis.

[0094] In step 3032, the first texture coordinates of each sampling point in the texture coordinate system are transformed to obtain the first perspective coordinates of each sampling point in the perspective coordinate system.

[0095] In some embodiments, before step 3032, the number of pixels on the horizontal and vertical edges of the user interface is obtained; the interface texture region is then divided into grids based on the number of pixels on the horizontal and vertical edges, resulting in multiple intersection points of the interface texture region as sampling points. This embodiment allows the number of sampling points to be controlled according to the actual parameters of the user interface, avoiding invalid sampling and thus improving sampling efficiency.

[0096] As an example, see Figure 8 The interface texture area is Figure 8 In the texture coordinate system, the square is usually set to a side length of 1. The four vertices of the square are located at (0, 0), (0, 1), (1, 0), and (1, 1) respectively. Since screen resolutions vary, such as 1024*512 or 1920*1480, assuming the screen resolution is 100*50, the vertical side has 50 pixels and the horizontal side has 100 pixels. When performing meshing, 98 horizontal dividing lines need to be drawn on the square, and 48 vertical dividing lines need to be drawn on the square, thus forming 5000 intersection points. The intersection points include the intersection points between dividing lines and the intersection points between the dividing lines and the sides of the square. Therefore, sampling is required for 5000 sampling points.

[0097] In step 3033, the first perspective coordinates of each sampling point are transformed based on the perspective graphic to obtain the second perspective coordinates of each sampling point.

[0098] In some embodiments, when the perspective graphic is an isosceles trapezoid, step 3033 involves transforming the first perspective coordinates of each sampling point based on the perspective graphic to obtain the second perspective coordinates of each sampling point. This can be achieved through the following technical solution: obtaining the perspective coordinate transformation relationship between the isosceles trapezoid and the interface texture region in the perspective coordinate system; and transforming the first perspective coordinates of each sampling point based on the perspective coordinate transformation relationship to obtain the second perspective coordinates of each sampling point. The embodiments of this application can reduce the computational load of perspective transformation. Performing perspective transformation in the XOY coordinate system can reduce computational complexity and improve transformation processing efficiency.

[0099] As an example, see Figure 8 The perspective coordinate transformation relationship between an isosceles trapezoid and the interface texture area is the same as the transformation relationship of mapping points in the isosceles trapezoid to the square. The transformation relationship can be fitted by selecting the transformation between vertices. For example, stretching point n to point m and point p to point q, the coordinates of point m in the perspective coordinate system can be calculated. The coordinates of point n in the perspective coordinate system are: The coordinates of point q in the perspective coordinate system can be calculated as follows: The coordinates of point p in the perspective coordinate system are: Based on the above four key points, we can fit the perspective coordinate transformation relationship from the coordinates of point n to the coordinates of point m, and the perspective coordinate transformation relationship from the coordinates of point p to the coordinates of point q. The perspective coordinate transformation relationship can be found in formulas (1) and (2):

[0100]

[0101]

[0102] Where x represents the x-axis coordinate of sampling point n in the square in the new coordinate system, y represents the y-axis coordinate of sampling point n in the square in the new coordinate system, X 1 The x-axis and y-axis coordinates of the sampled point n in the new coordinate system after stretching are represented by... 1 This represents the y-axis coordinate of the sampled point n in the new coordinate system after stretching.

[0103] The above transformation of the first perspective coordinates of each sampling point based on the perspective coordinate transformation relationship can be achieved by substituting the first perspective coordinates into formulas (1) and (2) to obtain the second perspective coordinates (X) of each sampling point. 1 Y 1 ).

[0104] In step 3034, the second perspective coordinates of each sampling point are transformed from the perspective coordinate system to the texture coordinate system to obtain the second texture coordinates of each sampling point.

[0105] As an example, after performing a perspective transformation, it is necessary to transform back from the new coordinate system to the UV coordinate system (texture coordinate system), see formulas (3) and (4):

[0106]

[0107]

[0108] Where x represents the x-axis coordinate of sampling point n in the square in the new coordinate system, y represents the y-axis coordinate of sampling point n in the square in the new coordinate system, X 1 The x-axis and y-axis coordinates of the sampled point n in the new coordinate system after stretching are represented by... 1 The X represents the y-axis coordinate of the sampled point n in the new coordinate system after stretching. 2 This represents the U-axis coordinate of the stretched sampling point n in the UV coordinate system, Y... 2 This represents the V-axis coordinate of the sampled point n in the UV coordinate system after stretching.

[0109] In step 304, based on the second texture coordinates of each sampling point, sampling processing is performed on the original image to obtain perspective rendering data for each sampling point.

[0110] In some embodiments, the step 304, which involves sampling the original image based on the second texture coordinates of each sampling point to obtain perspective rendering data for each sampling point, can be achieved through the following technical solution: performing initial sampling processing on the original image based on the second texture coordinates of each sampling point to obtain initial sampling results for each sampling point; and performing anti-aliasing processing in the vertical direction on the initial sampling results for each sampling point to obtain perspective rendering data for each sampling point. Through the embodiments of this application, the sampling results can be corrected to avoid image aliasing and improve subsequent rendering effects.

[0111] As an example, after performing perspective coordinate transformation, the original image is sampled based on the second texture coordinates to obtain an initial sampling result. This initial sampling result includes the red, yellow, and blue color values ​​corresponding to the second texture coordinates obtained from the original image, as well as the transparency value. At this point, the upper and lower boundaries of the user interface may change from horizontal to diagonal lines, leading to aliasing. Since the aliasing caused by perspective is entirely horizontal, Gaussian blur-based anti-aliasing processing can be performed using adjacent pixels in the vertical direction.

[0112] In some embodiments, the initial sampling process based on the second texture coordinates of each sampling point in the original image to obtain the initial sampling result of each sampling point can be achieved through the following technical solution: For each sampling point, perform the following processing: obtain the base color value and transparency value corresponding to the second texture coordinates in the original image; when the second texture coordinates are not outside the coordinate range corresponding to the interface texture region, combine the base color value and transparency value to form the initial sampling result of the sampling point; when the second texture coordinates are outside the coordinate range corresponding to the interface texture region, update the transparency value of the sampling point to zero, and combine the base color value and the updated transparency value to form the initial sampling result of the sampling point. Boundary detection can clear the transparency of sampling points outside the coordinate range of the interface texture region to zero, thereby ensuring that the sampling points corresponding to the subsequent sampling results used for rendering all originate from within the trapezoid.

[0113] As an example, before performing anti-aliasing, boundary detection needs to be performed on channel A to clear the opacity of sampling points that exceed the coordinate range of the interface texture area. If sampling point A exceeds the coordinate range of the interface texture area (i.e., X... 2 <0, or X 2 >1, or Y 2 <0, or Y 2 >1), then sampling point A on the screen is a point outside the trapezoid. Since the final visual effect of forming an isosceles trapezoid is required, there should be no color outside the trapezoid, so the transparency needs to be cleared to zero.

[0114] In some embodiments, the vertical anti-aliasing processing of the initial sampling results of each sampling point to obtain the perspective rendering data of each sampling point can be achieved through the following technical solution: For each sampling point, the following processing is performed: when a sampling point has two adjacent sampling points in the vertical direction, the initial sampling results of each adjacent sampling point are obtained, and the initial sampling results of the two adjacent sampling points are fused with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point; when a sampling point has one adjacent sampling point in the vertical direction, the initial sampling result of the one adjacent sampling point is obtained, and the initial sampling result of the one adjacent sampling point is fused with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point. By fusing the sampling results of adjacent pixels in the vertical direction, the image achieves an anti-aliasing effect in the vertical direction.

[0115] As an example, for a certain sampling point, the sampled values ​​of the previous pixel (initial sampling result), the next pixel (initial sampling result), and the sampling value of this sampling point (initial sampling result) in the vertical position are weighted and averaged. The resulting image after weighted averaging will be slightly blurred in the vertical direction, thus achieving the anti-aliasing effect. The perspective rendering color of each sampling point after anti-aliasing is shown in formula (5):

[0116]

[0117] Among them, rgba(X) on the left side of the equation 2 ,Y 2 ) is the final rendered color value of a sample point A after anti-aliasing, and rgba(X) on the right side of the equation 2 (Y-0.5*PixelSize) 2 ) represents the sampled value of the vertically adjacent sampled point A below it, and rgba(X) on the right side of the equation 2 ,Y 2 ) represents the sampled value at sampling point A, and rgba(X) on the right side of the equation 2 (Y + 0.5 * PixelSize) 2 The value represents the sampled value of the vertically adjacent sampled point above sampled point A. After boundary aliasing, the final rendered color after anti-aliasing can be directly rendered onto the screen to obtain the final rendered image used for output.

[0118] In step 305, based on the perspective rendering data of each sampling point, the user interface with perspective effect is rendered to the screen.

[0119] The calculated perspective rendering data is transmitted to the graphics hardware via the data bus, so that the graphics hardware can perform pixel filling calculations and render the data to the frame buffer on the screen.

[0120] See Figure 5C , Figure 5C This is a flowchart illustrating the image processing method for a virtual scene provided in this application embodiment, which will be combined with... Figure 5C The steps shown are explained.

[0121] In step 401, a virtual scene is displayed in the human-computer interaction interface.

[0122] In step 402, a user interface of the virtual scene is displayed in the human-computer interaction interface, wherein the user interface has a perspective effect and is obtained by performing the image processing method of the virtual scene provided in the embodiments of this application.

[0123] This application achieves a three-dimensional perspective effect by sampling the original image in screen space within a two-dimensional space. Because it employs a purely two-dimensional screen space rendering method, the overhead is limited to a single sampling operation. Furthermore, since it is entirely based on two-dimensional screen space post-processing, the processing timing is flexible, allowing for plug-and-play operation and making it suitable for dynamically iterative game development scenarios.

[0124] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0125] Taking a simulated building restoration app as an example, the server acquires the original image of the user interface of the virtual scene (the building to be restored, such as the Great Wall). The user interface can be the user's interface (e.g., a restoration interface containing multiple restoration controls). The server acquires the interface texture region corresponding to the texture coordinate system of the user interface and constructs a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture region. The server acquires the first perspective coordinates of each sampling point in the interface texture region within the perspective coordinate system and performs perspective transformation processing on the first perspective coordinates of each sampling point based on perspective graphics to obtain the first texture coordinates of each sampling point in the texture coordinate system. Based on the first texture coordinates of each sampling point, the server performs sampling processing on the original image to obtain the perspective rendering data for each sampling point. Based on the perspective rendering data of each sampling point, the server generates a rendered image of the user interface with perspective effects. The server sends the rendered image to the terminal so that the game client on the terminal can present the user interface to the user by outputting the rendered image during runtime.

[0126] In some embodiments, see Figures 6A-6B , Figures 6A-6B This is a schematic diagram of the interface of the image processing method for virtual scenes provided in this application embodiment. The image processing method for virtual scenes provided in this application embodiment can be applied to building repair APP. The image processing method for virtual scenes provided in this application embodiment can achieve a three-dimensional perspective effect in a two-dimensional user interface. Figure 6A This shows a 2D user interface without 3D perspective effects applied. Figure 6B This demonstrates a 2D user interface with a 3D perspective effect.

[0127] The basic principle of the image processing method for virtual scenes provided in this application embodiment is as follows: based on the perspective effect of Y-axis rotation, the final image is an isosceles trapezoid, thereby calculating the UV coordinates in the sampling space corresponding to the isosceles trapezoid in the screen space, and then sampling the original image of the original two-dimensional user interface based on the calculation results.

[0128] See Figure 7 , Figure 7This is a flowchart illustrating the image processing method for a virtual scene provided in this application embodiment. In step 201, the image of the original user interface is acquired. In step 202, a new coordinate system (perspective coordinate system) is constructed. In step 203, the perspective trapezoid corresponding to the new coordinate system is mapped to the square in the texture coordinate system. In step 204, boundary aliasing is performed to obtain the rendering color value of each sampling point.

[0129] The specific implementation process is explained in detail below:

[0130] Specifically, step 201 involves rendering the original image of the two-dimensional user interface as the texture to be sampled using the rendering pipeline of a rendering engine (such as Unreal Engine). The two-dimensional user interface in the original image does not have a perspective effect.

[0131] For step 202, first construct a new coordinate system XOY, such that its X-axis passes through the point f(1, 0.5) in the texture coordinate system (UV coordinate system). In the new coordinate system, point f is represented as... See Figure 8 , Figure 8 This is a perspective trapezoidal diagram of the image processing method for a virtual scene provided in this application embodiment, where the UV coordinates default to point m as the origin; a refers to... Figure 8 The angle between the hypotenuse of the trapezoid and the x-axis is used to construct a trapezoid symmetrical about the x-axis based on the perspective angle 2a and the set length h of the upper side of the trapezoid. The intersection of the two hypotenuses of the trapezoid is taken as the origin O(0,0) of the new coordinate system XOY.

[0132] Regarding step 203, the purpose of step 203 is to use a trapezoid to represent all the contents of the interface texture area [0, 1] in the texture coordinate system. Essentially, it is to stretch the trapezoid into a square, and the square represents the entire screen (the screen can be rectangular, so it can also be stretched into a rectangle). Then, the texture is sampled according to the coordinates of the square.

[0133] Let the side length of the square be 1, and the length of the upper side of the trapezoid be h. For each sampling point in the square, calculate the coordinates of the sampling point in the new coordinate system. For example, the coordinates of point m in the new coordinate system can be obtained as follows: The coordinates of point n in the new coordinate system are: To stretch point n to point m, refer to formulas (6) and (7):

[0134]

[0135]

[0136] Where x represents the x-axis coordinate of sampling point n in the square in the new coordinate system, y represents the y-axis coordinate of sampling point n in the square in the new coordinate system, X 1The x-axis and y-axis coordinates of the sampled point n in the new coordinate system after stretching are represented by... 1 This represents the y-axis coordinate of the sampled point n in the new coordinate system after stretching.

[0137] Then transform back to the UV coordinate system (texture coordinate system) from the new coordinate system, see formulas (8) and (9):

[0138]

[0139]

[0140] Where x represents the x-axis coordinate of sampling point n in the square in the new coordinate system, y represents the y-axis coordinate of sampling point n in the square in the new coordinate system, X 1 The x-axis and y-axis coordinates of the sampled point n in the new coordinate system after stretching are represented by... 1 The X represents the y-axis coordinate of the sampled point n in the new coordinate system after stretching. 2 This represents the U-axis coordinate of the stretched sampling point n in the UV coordinate system, Y... 2 This represents the V-axis coordinate of the sampled point n in the UV coordinate system after stretching.

[0141] By stretching the trapezoid into a square, we can obtain, as shown below. Figure 7 The square shown can be used to stretch point n within the square to point m, and then the coordinates of this square can be used to sample the texture. Since the parameters used when sampling the texture need to be UV coordinates, the coordinate values ​​in the UV coordinate system after stretching need to be calculated using formulas (8) and (9).

[0142] Regarding step 204, after perspective processing is performed in step 203, the upper and lower boundaries of the user interface change from horizontal to diagonal lines, resulting in aliasing. Since the aliasing caused by perspective is all in the horizontal direction, anti-aliasing processing based on Gaussian blur can be performed in the vertical direction using adjacent pixels.

[0143] Before performing anti-aliasing, boundary detection needs to be performed on channel A to clear the opacity of sampling points that are outside the texture coordinate range. If sampling point A is outside the texture coordinate range (i.e., X... 2 <0, or X 2 >1, or Y 2 <0, or Y 2 >1), then sampling point A on the screen is outside the trapezoid. The final desired effect is a trapezoidal shape, and there should be no color outside the trapezoid, so the opacity needs to be zero.

[0144] For a given sampling point, the sampled values ​​of the preceding and following pixels in the vertical direction are weighted and averaged together. The resulting image will be slightly blurred in the vertical direction after weighted averaging, thus achieving an anti-aliasing effect. The final color of each sampling point after anti-aliasing is given by formula (10):

[0145]

[0146] Among them, rgba(X) on the left side of the equation 2 ,Y 2 ) is the final rendered color value of a sample point A after anti-aliasing, and rgba(X) on the right side of the equation 2 (Y-0.5*PixelSize) 2 ) represents the sampled value of the vertically adjacent sampled point A below it, and rgba(X) on the right side of the equation 2 ,Y 2 ) represents the sampled value at sampling point A, and rgba(X) on the right side of the equation 2 (Y + 0.5 * PixelSize) 2 The value represents the sampled value of the vertically adjacent sampled point above sampled point A. After boundary aliasing, the final rendered color after anti-aliasing can be directly rendered onto the screen to obtain the final rendered image used for output.

[0147] This application achieves a three-dimensional perspective effect by sampling the original image in screen space within a two-dimensional space. Because it employs a purely two-dimensional screen space rendering method, the overhead is limited to a single sampling operation. Furthermore, since it is entirely based on two-dimensional screen space post-processing, the processing timing is flexible, allowing for plug-and-play operation and making it suitable for dynamically iterative game development scenarios.

[0148] The following description continues to illustrate the exemplary structure of the virtual scene image processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4As shown, the software modules in the image processing device 455 of the virtual scene stored in the memory 450 may include: an acquisition module 4551, used to acquire the original image of the user interface of the virtual scene; a coordinate module 4552, used to acquire the interface texture region corresponding to the texture coordinate system of the user interface, and acquire the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system; a transformation module 4553, used to perform perspective transformation processing based on perspective graphics on the first texture coordinate of each sampling point in the texture coordinate system to obtain the second texture coordinate of each sampling point in the texture coordinate system; a sampling module 4554, used to perform sampling processing on the original image based on the second texture coordinate of each sampling point to obtain the perspective rendering data of each sampling point; and a rendering module 4555, used to render the user interface with perspective effect to the screen based on the perspective rendering data of each sampling point.

[0149] In some embodiments, the acquisition module 4551 is further configured to: call the rendering engine to perform planar rendering processing on the user interface of the virtual scene to obtain an original image for output, wherein the virtual scene is a three-dimensional virtual scene and the original image does not have a perspective effect.

[0150] In some embodiments, the transformation module 4553 is further configured to: construct a perspective coordinate system based on the specified texture coordinates of the interface texture region and perspective parameters; transform the first texture coordinates of each sampling point in the texture coordinate system to obtain the first perspective coordinates of each sampling point in the perspective coordinate system; perform a perspective-based transformation on the first perspective coordinates of each sampling point to obtain the second perspective coordinates of each sampling point; and perform a transformation from the perspective coordinate system to the texture coordinate system on the second perspective coordinates of each sampling point to obtain the second texture coordinates of each sampling point.

[0151] In some embodiments, the transformation module 4553 is further configured to: obtain a target horizontal axis that passes through the target point pointed to by the specified texture coordinates and is parallel to the horizontal axis of the texture coordinate system; obtain a perspective graphic that is located within the interface texture area, is symmetrical about the target horizontal axis, and conforms to perspective parameters; obtain the focal point of the perspective graphic on the target horizontal axis as the origin; obtain a target vertical axis that is perpendicular to the target horizontal axis and passes through the origin; and generate a perspective coordinate axis composed of the target vertical axis and the target horizontal axis.

[0152] In some embodiments, when the perspective parameters include a first perspective length, a second perspective length, and a perspective angle, the transformation module 4553 is further configured to: obtain the base of a trapezoid with the target point as its midpoint, a length equal to the first perspective length, and perpendicular to the target horizontal axis; obtain the top of a trapezoid parallel to the base of the trapezoid, a length equal to the second perspective length, and bisected by the target horizontal axis; and use the isosceles trapezoid characterized by the top of the trapezoid, the base of the trapezoid, and the perspective angle as a perspective graphic.

[0153] In some embodiments, before transforming the first texture coordinates of each sampling point in the texture coordinate system to obtain the first perspective coordinates of each sampling point in the perspective coordinate system, the transformation module 4553 is further configured to: obtain the number of pixels on the horizontal side and the number of pixels on the vertical side of the user interface; perform grid division processing on the interface texture area based on the number of pixels on the horizontal side and the number of pixels on the vertical side to obtain multiple intersection points of the interface texture area as sampling points.

[0154] In some embodiments, when the perspective graphic is an isosceles trapezoid, the transformation module 4553 is further configured to: obtain the perspective coordinate transformation relationship between the isosceles trapezoid and the interface texture area in the perspective coordinate system; and perform transformation processing on the first perspective coordinate of each sampling point based on the perspective coordinate transformation relationship to obtain the second perspective coordinate of each sampling point.

[0155] In some embodiments, the sampling module 4554 is further configured to: perform initial sampling processing on the original image based on the second texture coordinates of each sampling point to obtain the initial sampling result of each sampling point; and perform anti-aliasing processing in the vertical direction on the initial sampling result of each sampling point to obtain the perspective rendering data of each sampling point.

[0156] In some embodiments, the sampling module 4554 is further configured to: perform the following processing for each sampling point: obtain the base color value and transparency value corresponding to the second texture coordinate in the original image; when the second texture coordinate is not outside the coordinate range corresponding to the interface texture area, combine the base color value and transparency value to form the initial sampling result of the sampling point; when the second texture coordinate is outside the coordinate range corresponding to the interface texture area, update the transparency value of the sampling point to zero, and combine the base color value and the updated transparency value to form the initial sampling result of the sampling point.

[0157] In some embodiments, the sampling module 4554 is further configured to: perform the following processing for each sampling point: when the sampling point has two adjacent sampling points in the vertical direction, obtain the initial sampling result of each adjacent sampling point, and fuse the initial sampling results of the two adjacent sampling points with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point; when the sampling point has one adjacent sampling point in the vertical direction, obtain the initial sampling result of one adjacent sampling point, and fuse the initial sampling result of one adjacent sampling point with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point.

[0158] The following description continues to illustrate the exemplary structure of the virtual scene image processing device provided in the embodiments of this application as a software module. In some embodiments, the software module in the virtual scene image processing device stored in the memory may include: a display module for displaying the virtual scene in a human-computer interaction interface; the display module is also used for displaying the user interface of the virtual scene in the human-computer interaction interface, wherein the user interface has a perspective effect, and the user interface is obtained by executing the virtual scene image processing method provided in the embodiments of this application.

[0159] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the virtual scene image processing method described above in this application.

[0160] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the image processing method for a virtual scene provided in this application. For example, ... Figures 5A-5C The image processing method for the virtual scene is shown.

[0161] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0162] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0163] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0164] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0165] In summary, this application's embodiments achieve a perspective effect by transforming the texture coordinates of sampling points based on perspective graphics and then sampling the original image of the user interface based on the transformed second texture coordinates. Since the rendering is entirely based on sampling the user interface, sampling overhead is saved, and the flexibility of processing timing is improved.

[0166] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An image processing method for a virtual scene, characterized in that, The method includes: Obtain the original image of the user interface of the virtual scene; Obtain the interface texture region in the texture coordinate system corresponding to the user interface, and obtain the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system; A perspective transformation based on perspective graphics is performed on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system. Based on the second texture coordinates of each sampling point, sampling processing is performed on the original image to obtain perspective rendering data for each sampling point; Based on the perspective rendering data of each of the sampling points, a user interface with perspective effect is rendered to the screen.

2. The method according to claim 1, characterized in that, The step of performing a perspective transformation based on perspective graphics on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system includes: Construct a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture area; The first texture coordinate of each sampling point in the texture coordinate system is transformed to obtain the first perspective coordinate of each sampling point in the perspective coordinate system. The first perspective coordinates of each sampling point are transformed based on the perspective graphic to obtain the second perspective coordinates of each sampling point; The second perspective coordinates of each sampling point are transformed from the perspective coordinate system to the texture coordinate system to obtain the second texture coordinates of each sampling point.

3. The method according to claim 2, characterized in that, The step of constructing a perspective coordinate system based on the specified texture coordinates and perspective parameters of the interface texture region includes: Obtain the target horizontal axis that passes through the target point pointed to by the specified texture coordinates and is parallel to the horizontal axis of the texture coordinate system; Obtain a perspective graphic that is symmetrical about the target horizontal axis within the interface texture area and conforms to the perspective parameters; The origin is obtained by taking the focal point of the perspective image on the target's horizontal axis; Obtain the target vertical axis that is perpendicular to the target horizontal axis and passes through the origin; Generate a perspective coordinate axis consisting of the target's vertical axis and the target's horizontal axis.

4. The method according to claim 3, characterized in that, When the perspective parameters include a first perspective length, a second perspective length, and a perspective angle, obtaining a perspective graphic that is symmetrical about the target horizontal axis and conforms to the perspective parameters within the interface texture area includes: Obtain the base of a trapezoid with the target point as its midpoint, a length equal to the first perspective length, and perpendicular to the horizontal axis of the target. Obtain the upper edge of the trapezoid that is parallel to the bottom edge of the trapezoid, has a length equal to the second perspective length, and is equally bisected by the target horizontal axis; The isosceles trapezoid, characterized by the upper side, the lower side, and the included perspective angle, is used as the perspective figure.

5. The method according to claim 2, characterized in that, Before transforming the first texture coordinate of each sampling point in the texture coordinate system to obtain the first perspective coordinate of each sampling point in the perspective coordinate system, the method further includes: Obtain the number of pixels on the horizontal and vertical sides of the user interface; The interface texture region is divided into grids based on the number of pixels on the horizontal edge and the number of pixels on the vertical edge, and multiple intersection points of the interface texture region are obtained as sampling points.

6. The method according to claim 2, characterized in that, When the perspective drawing is an isosceles trapezoid, the step of performing a transformation process on the first perspective coordinates of each sampling point based on the perspective drawing to obtain the second perspective coordinates of each sampling point includes: Obtain the perspective coordinate transformation relationship between the isosceles trapezoid and the interface texture region in the perspective coordinate system; Based on the perspective coordinate transformation relationship, the first perspective coordinates of each sampling point are transformed to obtain the second perspective coordinates of each sampling point.

7. The method according to claim 1, characterized in that, The process of sampling the original image based on the second texture coordinates of each sampling point to obtain perspective rendering data for each sampling point includes: Based on the second texture coordinates of each sampling point, initial sampling processing is performed on the original image to obtain the initial sampling result of each sampling point; Vertical anti-aliasing processing is performed on the initial sampling results of each sampling point to obtain the perspective rendering data of each sampling point.

8. The method according to claim 7, characterized in that, The initial sampling process, based on the second texture coordinates of each sampling point, is performed on the original image to obtain the initial sampling result for each sampling point, including: For each of the sampling points, the following processing is performed: Obtain the base color value and transparency value corresponding to the second texture coordinates in the original image; When the second texture coordinates are not outside the coordinate range corresponding to the interface texture area, the basic color value and the transparency value are used to form the initial sampling result of the sampling point; When the second texture coordinate is outside the coordinate range corresponding to the interface texture area, the transparency value of the sampling point is updated to zero, and the base color value and the updated transparency value are combined to form the initial sampling result of the sampling point.

9. The method according to claim 7, characterized in that, The process of performing vertical anti-aliasing processing on the initial sampling results of each sampling point to obtain perspective rendering data for each sampling point includes: Perform the following processing for each sampling point: When the sampling point has two adjacent sampling points in the vertical direction, the initial sampling result of each adjacent sampling point is obtained, and the initial sampling results of the two adjacent sampling points are fused with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point. When the sampling point has an adjacent sampling point in the vertical direction, the initial sampling result of the adjacent sampling point is obtained, and the initial sampling result of the adjacent sampling point is fused with the initial sampling result of the sampling point to obtain the perspective rendering data of the sampling point.

10. An image processing method for a virtual scene, characterized in that, The method includes: The virtual scene is displayed in the human-computer interaction interface; The user interface of the virtual scene is displayed in the human-computer interaction interface, wherein the user interface has a perspective effect, and the user interface is rendered by performing the image processing method of the virtual scene according to any one of claims 1 to 9.

11. An image processing device for a virtual scene, characterized in that, The device includes: The acquisition module is used to acquire the original image of the user interface of the virtual scene; The coordinate module is used to obtain the interface texture region corresponding to the texture coordinate system of the user interface, and to obtain the first texture coordinate of each sampling point of the interface texture region in the texture coordinate system. The transformation module is used to perform perspective transformation processing based on perspective graphics on the first texture coordinates of each sampling point in the texture coordinate system to obtain the second texture coordinates of each sampling point in the texture coordinate system. A sampling module is used to perform sampling processing on the original image based on the second texture coordinates of each sampling point to obtain perspective rendering data for each sampling point. The rendering module is used to render a user interface with perspective effect to the screen based on the perspective rendering data of each of the sampling points.

12. An image processing device for a virtual scene, characterized in that, The device includes: The display module is used to display the virtual scene in the human-computer interaction interface; The display module is further configured to display the user interface of the virtual scene in the human-computer interaction interface, wherein the user interface has a perspective effect, and the user interface is rendered by performing the image processing method of the virtual scene according to any one of claims 1 to 9.

13. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the image processing method for the virtual scene according to any one of claims 1 to 9 or 10.

14. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the image processing method for the virtual scene according to any one of claims 1 to 9 or 10.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the image processing method for the virtual scene as described in any one of claims 1 to 9 or 10.

Images