Method, device and application of screen space reflection with continuous non-planar imaging surface

Abstract

The application provides a screen space reflection method, device and application for realizing continuous non-planar imaging surface, which expands the means for calculating the reflection range of the scene on the basis of the original screen space reflection method, adopts a spherical projection mode to generate a reflection map without reflection picture to better avoid the splicing disorder caused by the curvature change of the bending part on the non-planar imaging surface, expands the calculation range of the reflection scene to intercept additional materials at the splicing part of the two pictures, so that better reflection map can be obtained when the observation point angle is relatively extreme, and the problem that the reflection content cannot be spliced is further reduced, and secondly, more reasonable angle data is used for distortion processing, each non-planar imaging surface is divided into many small blocks, the plane normal data of the non-planar imaging surface is replaced by the plane of each small block, and the reflection map with more reasonable distortion processing is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of rendering, in particular to a method and device for realizing screen space reflection with continuous non-planar imaging surface and application thereof. BACKGROUND

[0002] Screen Space Reflection (SSR) is a way of generating reflection effect in real time rendering applied on the material with "reflection" property. It can reflect other objects in the scene when representing materials such as smooth surface (metal, smooth ground), water surface (lake, ground water), etc. It can greatly improve the picture quality and richness of realism, and thus provide users with a sense of reality when displaying virtual pictures.

[0003] The principle of SSR algorithm is to solve the reflection vector according to the normal and view information of each pixel of the object in the screen space, step along the reflection vector in the screen space, and judge whether the depth of the stepped coordinate and the depth of the object stored in the depth buffer intersect. If they intersect, the color of the object at the intersection point is taken as the final reflection color. The current SSR algorithm only calculates the range of the scene seen in the current screen space. For example, if there is an object with reflection material in the screen space within the current viewport range, the pre-rendered image information without reflection is distorted to form reflection layer information according to the spatial relationship between the reflection object and the screen plane, and then integrated into the output image.

[0004] However, the existing SSR algorithm does not perform well on non-planar imaging surface. Since the spatial angle between the reflection object and the screen plane is calculated when the image is distorted, the reflection layer information calculated on the non-planar surface will be broken or offset, which will cause the generated transmission picture to have a strong tearing effect. Moreover, the scene range of the existing SSR algorithm only includes the visible part in the viewport, and the visible part on the non-planar surface is different at different viewing angles, which will cause some reflection pictures to have a strange tearing effect. The different visible parts in different viewports also cause the generated reflection map to be unable to be spliced. SUMMARY

[0005] The embodiments of the present application provide a method and device for realizing screen space reflection with continuous non-planar imaging surface and application thereof. The scene range is expanded and the picture angle during distortion processing is corrected to realize continuous imaging of screen space method on non-planar imaging surface, reduce the tearing effect of picture on non-planar imaging surface, and thus improve the picture quality of the imaging picture.

[0006] In a first aspect, the embodiments of the present application provide a method for realizing screen space reflection with continuous non-planar imaging surface, comprising the following steps:

[0007] mapping a virtual space with a three-dimensional coordinate system of a physical imaging space, wherein the physical imaging space is provided with at least one non-planar imaging surface, and each non-planar imaging surface is composed of a plurality of curved imaging planes;

[0008] obtaining an observation coordinate point of an observation point in a world coordinate system in which the physical imaging space is located, and obtaining a pose of an imaging plane group composed of the imaging planes in the world coordinate system;

[0009] connecting lines from the observation coordinate point to vertices of each imaging plane of the imaging plane group in the world coordinate system, and calculating a maximum horizontal angle and a maximum pitch angle of a view cone of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane;

[0010] rendering a spherical picture according to the maximum horizontal angle and the maximum pitch angle;

[0011] obtaining a horizontal angle and a pitch angle of each imaging plane with respect to the observation coordinate point, wherein the horizontal angle and / or the pitch angle of a current imaging plane is enlarged by a certain proportion at a boundary position of each imaging plane;

[0012] cutting a reflection-free picture from the spherical picture according to the horizontal angle and the pitch angle;

[0013] dividing the imaging planes into at least two partition planes, using vector data of a line from the observation coordinate point to the partition planes as plane normal data of the partition planes, and performing distortion on the reflection-free picture based on the plane normal data of each partition plane to obtain a reflection picture.

[0014] In a second aspect, the embodiments of the present application provide a device for realizing screen space reflection with continuous non-planar imaging surface, comprising:

[0015] a mapping unit configured to map a virtual space with a three-dimensional coordinate system of a physical imaging space, wherein the physical imaging space is provided with at least one non-planar imaging surface, and each non-planar imaging surface is composed of a plurality of curved imaging planes;

[0016] a coordinate obtaining unit configured to obtain an observation coordinate point of an observation point in a world coordinate system in which the physical imaging space is located, and obtain a pose of an imaging plane group composed of the imaging planes in the world coordinate system;

[0017] a view positioning unit configured to calculate a maximum horizontal angle and a maximum pitch angle of a view cone formed with the observation coordinate point as a starting point in each imaging plane by connecting the observation coordinate point to a vertex of each imaging plane in the world coordinate system;

[0018] a spherical picture rendering unit configured to render a spherical picture according to the maximum horizontal angle and the maximum pitch angle;

[0019] an angle obtaining unit configured to obtain a horizontal angle and a pitch angle of each imaging plane with the observation coordinate point, wherein the horizontal angle and / or the pitch angle of a current imaging plane is enlarged by a set proportion at a boundary position of each imaging plane;

[0020] a cutting unit configured to cut a reflection-free picture from the spherical picture according to the horizontal angle and the pitch angle;

[0021] a reflection unit configured to divide the imaging plane into at least two partition planes, use vector data of a line connecting the observation coordinate point to the partition planes as plane normal data of the partition planes, and obtain a reflection picture by distorting the reflection-free picture based on the plane normal data of each partition plane.

[0022] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, characterized in that the memory stores a computer program, and the processor is configured to run the computer program to execute the method for realizing screen space reflection with continuous non-planar imaging surface.

[0023] In a fourth aspect, an embodiment of the present application provides a readable storage medium, characterized in that the readable storage medium stores a computer program, and the computer program includes program codes for controlling a process to execute the process, and the process includes the method for realizing screen space reflection with continuous non-planar imaging surface.

[0024] The main contributions and innovative points of the present application are as follows:

[0025] The embodiments of the present application improve and optimize the prior art screen space reflection method. Firstly, the range of the scene for calculating reflection is expanded by means of spherical projection to generate a reflection map without reflection picture to better avoid the problem of splicing disorder caused by the curvature change at the bending part of the non-planar imaging surface. The range of the scene for calculating reflection is expanded to intercept additional materials at the splicing part of two pictures to make the reflection map better when the observation point angle is relatively extreme, and further reduce the problem that the reflection content cannot be spliced. Secondly, more reasonable angle data is used for distortion processing. Each non-planar imaging surface is divided into many small blocks, the plane of each small block is used to replace the plane normal data of the non-planar imaging surface, and a reflection map with more reasonable distortion processing is generated.

[0026] The original scene range is expanded.

[0027] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, purposes and advantages of the present application more clear and easy to understand. BRIEF DESCRIPTION OF DRAWINGS

[0028] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, the schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0029] Figure 1 It is a flow chart of the screen space reflection method for realizing the continuous non-planar imaging surface according to the embodiments of the present application;

[0030] Figure 2 It is a schematic diagram of the view pyramid of the observation coordinate point according to an embodiment of the present application;

[0031] Figure 3 It is a schematic diagram of the projection area of the spherical surface intercepted by the maximum horizontal included angle and the maximum pitch included angle, and the projection picture of the virtual scene picture formed in the projection area;

[0032] Figure 4 It is a structural block diagram of the screen space reflection device for realizing the continuous non-planar imaging surface according to the embodiments of the present application;

[0033] Figure 5 It is a hardware structure schematic diagram of the electronic device according to the embodiments of the present application. DETAILED DESCRIPTION

[0034] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description below refers to the accompanying drawings, which show, by way of example, specific embodiments with which some embodiments of the present description are in accordance. These embodiments are described in such detail, to enable those skilled in the art to practice the embodiments of this description in conjunction with current technology. However, it is to be understood that various changes can be made within the scope of the present description, and it will be apparent to those of ordinary skill in the art that the embodiments described herein are not the only way to implement a method in accordance with one or more embodiments of the present description. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present description as detailed in the appended claims.

[0035] It is to be understood that the steps of the methods need not be performed in the order described in this specification. In some other embodiments, the steps of the methods can be performed in an order other than that described in this specification. Also, a single step described in this specification can be split into multiple steps in other embodiments, and multiple steps described in this specification can be combined into a single step in other embodiments.

[0036] Embodiment One

[0037] As shown in the figure, the present solution provides a screen space reflection method for realizing continuous non-planar imaging surface, comprising the following steps: Figure 1

[0038] establishing a mapping of a virtual space to a three-dimensional coordinate system of a physical imaging space, wherein the physical imaging space is provided with at least one non-planar imaging surface, and each non-planar imaging surface is composed of a plurality of curved imaging planes;

[0039] obtaining an observation coordinate point of an observation point in a world coordinate system, and obtaining a pose of an imaging plane group composed of the imaging planes in the world coordinate system;

[0040] connecting lines from the observation coordinate point to vertices of each imaging plane of the imaging plane group in the world coordinate system, and calculating a maximum horizontal angle and a maximum pitch angle of a view cone of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane;

[0041] rendering a spherical picture according to the maximum horizontal angle and the maximum pitch angle;

[0042] obtaining a horizontal angle and a pitch angle of each imaging plane with respect to the observation coordinate point, wherein the horizontal angle and / or the pitch angle of the current imaging plane is enlarged by a certain proportion at a boundary position of each imaging plane;

[0043] cutting a reflection-free picture from the spherical picture according to the horizontal angle and the pitch angle;

[0044] ​The imaging plane is divided into at least two sub-zones, and the vector data of the line connecting the observation coordinate point and the sub-zone plane is used as the plane normal data of the sub-zone plane. The reflection image is obtained by distorting the non-reflection image based on the plane normal data of each sub-zone plane.

[0045] The present solution improves the prior art screen space reflection technology to better adapt to the reflection of non-planar imaging planes, so that the reflection image on the non-planar imaging plane is continuous and has no tearing. In other words, the present solution is particularly suitable for the reflection scenario of non-planar imaging planes.

[0046] In an embodiment of the present solution, the non-planar imaging plane refers to an imaging plane composed of multiple curved imaging planes, that is, at least two imaging planes on the non-planar imaging plane are not located in the same horizontal direction to form a bend, including but not limited to curved surfaces and angle surfaces. It should be noted that the physical imaging space provided by the present solution includes at least one non-planar imaging plane, but the physical imaging space is not limited to being composed of only non-planar imaging planes. For example, the imaging plane of a certain physical imaging space at a certain viewing angle is a non-planar imaging plane, while the imaging plane at other viewing angles is a planar imaging plane.

[0047] In order to realize the projection of the virtual image, in the step of "establishing the mapping between the modeling coordinate system of the virtual space and the world coordinate system of the physical imaging space", the mapping between the modeling coordinate system of the virtual space and the world coordinate system of the physical imaging space is established. In some embodiments, both the modeling coordinate system and the world coordinate system are three-dimensional coordinate systems, and the mapping between the modeling coordinate system and the world coordinate system is established in the same dimension.

[0048] Since the reflection effect of an object is related to the observation angle of the observation point, the present solution needs to know the observation coordinate point of the observation point in the world coordinate system. In some embodiments, the user is at a specific position in the physical imaging space, and the coordinates of the specific position in the world coordinate system are collected as the observation coordinate point. Of course, the reflection effect of an object is also related to the position of the imaging plane group, so the present solution also needs to obtain the pose of the imaging plane group composed of the imaging plane group in the world coordinate system. It should be particularly noted that the planar imaging plane in the physical imaging space is directly added to the imaging plane group as an imaging plane. Specifically, in the step of "obtaining the pose of the imaging plane group composed of the imaging plane group in the world coordinate system", the position and angle of each imaging plane in the world coordinate system are obtained as the pose of the imaging plane.

[0049] After learning the pose of each imaging plane, the observation coordinate point and the vertex of each imaging plane are connected, and the vertex of the imaging plane refers to the point on the edge of the imaging plane. Correspondingly, in the step of "calculating the maximum horizontal angle and the maximum pitch angle of the view cone of the imaging plane group based on the horizontal angle and the pitch angle in each imaging plane", the horizontal angle and the pitch angle of the current imaging plane are obtained by connecting the observation coordinate point with the vertex of each imaging plane, and the maximum horizontal angle is obtained by taking the incremental value of the horizontal angle of all imaging planes in the imaging plane group, and the maximum pitch angle is obtained by taking the incremental value of the pitch angle of all imaging planes in the imaging plane group. In some embodiments, each horizontal angle corresponds to a connecting line with each pitch angle corresponding to a connecting line to form a view cone with the observation coordinate point as the starting point. That is, each imaging plane has only one set of horizontal angle and pitch angle for each observation point, and the present scheme is to obtain the maximum horizontal angle and the maximum pitch angle of the imaging plane group of the non-planar imaging surface to intercept the spherical picture.

[0050] The schematic diagram of the observation point and the imaging plane group forming the maximum horizontal angle and the maximum pitch angle is shown in Figure 2 Figure 2 The thick solid line in the figure is the horizontal projection of the imaging plane group of the non-planar imaging surface on the horizontal plane, the maximum horizontal angle is obtained by summarizing the horizontal angle of each imaging plane, and the maximum pitch angle is obtained by summarizing the pitch angle of each imaging plane. It can be seen that the maximum horizontal angle and the maximum pitch angle formed under the view angle of different observation points are different.

[0051] After obtaining the maximum horizontal angle and the maximum pitch angle, a spherical picture needs to be rendered, wherein the spherical picture is a virtual picture of 360° panorama without reflection effect, and the reflection map is generated using the spherical picture subsequently. Specifically, in the embodiments of the present scheme, the spherical picture is generated in the form of equirectangular projection.

[0052] Specifically, in the step of "rendering a spherical picture according to the maximum horizontal angle and the maximum pitch angle", the projection area of the world coordinate system is intercepted according to the maximum horizontal angle and the maximum pitch angle, the Cartesian coordinate system of the world coordinate system is converted into a spherical coordinate system to obtain the spherical coordinate of the projection area in the spherical coordinate system, the spherical coordinate is represented in the form of a matrix to obtain a spherical projection matrix, and the spherical picture is rendered according to the spherical projection matrix. The reason why the present scheme adopts spherical projection to generate the reflection map is that it can better avoid the problem of reflection content not being spliced due to sudden change of curvature at the bending part.

[0053] As shown in Figure 3 ​The projection area of the sphere is intercepted by the maximum horizontal included angle and the maximum pitch included angle, and a projection picture of the virtual scene picture is formed in the projection area. Figure 3 If the blue block in the virtual scene picture is an object, it forms a corresponding projection in the projection area.

[0054] The three-dimensional coordinate system of the world coordinate system is a Cartesian coordinate system, in which the Z axis is upward, and each coordinate point of the modeling coordinate can be described as (x, y, z). Each coordinate point in the spherical coordinate system is represented by latitude θ (zenith angle), longitude φ (azimuth angle), and spherical radius r, and each coordinate point of the spherical coordinate is represented as (θ, φ, r).

[0055] The formula for converting the three-dimensional coordinate system to the spherical coordinate system is as follows:

[0056]

[0057]

[0058]

[0059] The above formula is expressed in the form of a matrix, and the formula for obtaining the spherical projection matrix of the spherical coordinate is as follows:

[0060]

[0061] According to the above steps, the spherical projection matrix has been obtained, but for rendering, the final carrier is not a sphere, but a two-dimensional plane description, so the spherical projection matrix needs to be converted to a two-dimensional plane in the step of rendering the spherical picture according to the spherical projection matrix. In other words, in the step of "rendering the spherical picture according to the spherical projection matrix", the spherical projection matrix is expressed as a two-dimensional plane coordinate system, and then the spherical picture is rendered. That is, the projection area of the Cartesian coordinate system is first converted to the spherical coordinate system, and then the spherical projection matrix on the spherical coordinate system is converted to the two-dimensional plane coordinate system before rendering. The advantage of this is that it can better cope with the problem of reflection content that cannot be spliced due to changes in curvature at the bending part.

[0062] Specifically, the change of the X-axis coordinate in the two-dimensional plane coordinate system corresponds to the change of the longitude φ in the spherical coordinate system, and the change of the Y-axis coordinate corresponds to the change of the latitude θ in the spherical coordinate system. For a map with a length of L and a width of H, first determine the scaling relationship between the longitude and the X-axis, and the scaling relationship between the latitude and the Y-axis. The scaling relationship between the longitude and the X-axis is: kx = [φ(max) - φ(min)] / L; the scaling relationship between the latitude and the Y-axis is: ky = [θ(max) - θ(min)] / H; for any point (φ, θ) on the spherical picture, the corresponding X-axis coordinate is: x = (φ - φ(min)) * kx; the corresponding Y-axis coordinate is: y = (θ - θ(min)) * ky.i , θ i ): Yi = θi / k y ; then, the information collected at the point (φ i , θ i ) is actually plotted as a point (Xi, Yi) on the graph. For a complete sphere, the values of φ(max)-φ(min) and θ(max)-θ(min) are both 2π, and for a partial sphere, the corresponding cut-off coordinates are used.

[0063] In the step of "obtaining the horizontal and pitch angles of each imaging plane with the observation coordinate point, and expanding the horizontal and / or pitch angles of the current imaging plane by a certain percentage at the boundary position of each imaging plane", the boundary position of each imaging plane refers to the position of each imaging plane with the adjacent imaging plane, and the horizontal and / or pitch angles of the boundary position are expanded by a certain percentage. In some embodiments, the percentage is 30%. The advantage is that additional materials are taken at the splicing position of adjacent imaging planes, so that when the observation point angle is close to the limit, the reflection map will not be out of the view range, and the possibility of splicing the reflection content is further reduced. Correspondingly, in the step of "cutting the non-reflection picture from the spherical picture according to the horizontal and pitch angles", the picture is cut according to the expanded horizontal and pitch angles.

[0064] In addition, since there are more splices between the imaging planes of the non-planar imaging plane and the imaging planes, in order to reduce the tearing problem caused by the splicing, the present scheme uses more reasonable angle data for distortion processing. Specifically, in the step of "dividing the imaging plane into at least two partition planes", the imaging plane is divided into at least two partition planes, and the number of divisions is not limited. It is generally believed that the more the number of divisions is, the less the mutation of the reflection splicing is, so the specific number of divisions can be adjusted according to the actual rendering computing power. Of course, in some embodiments, the imaging plane is not divided into at least two partition planes, and the number of partition planes divided at the boundary position of the imaging plane is greater than the number of partition planes divided in other regions of the imaging plane.

[0065] For the divided partition planes, the vector data of the line connecting the observation coordinate point and the partition plane is used as the plane normal data of the partition plane for each partition plane, that is, the present scheme independently distorts each partition plane, and converts the single-threaded distortion processing of the original imaging plane into multi-threaded distortion processing of multiple partition planes. In some embodiments, multiple asynchronous threads are used to process the distortion processing of the partition planes.

[0066] After the reflection pictures of each partition plane are obtained, corresponding reflection maps are generated according to the proportion that the partition planes correspond to the imaging plane, the reflection objects in the non-reflection picture are spliced according to the map range and edge transparency of the reflection maps, so as to produce continuous reflection effect.

[0067] Embodiment Two

[0068] Based on the same concept, referring to Figure 4 , the application also proposes a screen space reflection device for realizing continuous non-planar imaging plane, comprising:

[0069] A mapping unit is configured to establish mapping of a virtual space and a three-dimensional coordinate system of a physical imaging space, wherein the physical imaging space is provided with at least one non-planar imaging plane, and each non-planar imaging plane is composed of a plurality of curved imaging planes.

[0070] A coordinate acquisition unit is configured to acquire observation coordinate points of an observation point in a world coordinate system in which the physical imaging space is located, and acquire the posture of an imaging plane group composed of the imaging planes in the world coordinate system.

[0071] A frustum positioning unit is configured to connect the observation coordinate points to the vertices of each imaging plane of the imaging plane group in the world coordinate system, and calculate the maximum horizontal angle and the maximum pitch angle of the frustum of the imaging plane group based on the horizontal angle and the pitch angle in each imaging plane.

[0072] A spherical picture rendering unit is configured to render a spherical picture according to the maximum horizontal angle and the maximum pitch angle.

[0073] An angle acquisition unit is configured to acquire the horizontal angle and the pitch angle of each imaging plane with the observation coordinate points, wherein the horizontal angle and / or the pitch angle of the current imaging plane is enlarged by a certain proportion at the boundary position of each imaging plane.

[0074] A cutting unit is configured to cut a non-reflection picture from the spherical picture according to the horizontal angle and the pitch angle.

[0075] A reflection unit is configured to divide the imaging plane into at least two partition planes, use the vector data of the line connecting the observation coordinate points to the partition planes as the plane normal data of the partition planes, and distort the non-reflection picture based on the plane normal data of each partition plane to obtain a reflection picture.

[0076] The content of embodiment two is the same as that of embodiment one, which will not be described here.

[0077] Embodiment Three

[0078] The embodiment also provides an electronic device, referring to Figure 5 , comprising a memory 304 and a processor 302, the memory 304 stores a computer program, and the processor 302 is configured to execute the computer program to perform the steps in any of the above-mentioned screen space reflection method embodiments for realizing a non-planar imaging surface.

[0079] Specifically, the processor 302 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the application.

[0080] The memory 304 can include mass storage for data or instructions. By way of example, and not limitation, the memory 304 can include a Hard Disk Drive (HDD), a floppy disk drive, a solid state drive (SSD), flash memory, a USB drive, a Zip Drive, or a combination of two or more of these. The memory 304 can be removable or non-removable (or fixed) as appropriate. The memory 304 can be internal or external to the data processing apparatus as appropriate. In particular embodiments, the memory 304 is a Non-Volatile memory. In particular embodiments, the memory 304 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask programmed ROM, a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Electrically Alterable Read-Only Memory (EAROM), or a FLASH, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Out Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0081] The memory 304 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 302.

[0082] The processor 302 implements the non-planar imaging surface continuous screen space reflection method of any of the above embodiments by reading and executing the computer program instructions stored in the memory 304.

[0083] Optionally, the electronic device described above can further include a transmission device 306 connected to the processor 302 and an input / output device 308 connected to the processor 302.

[0084] The transmission device 306 can be used to receive or send data via a network. Specific examples of the network can include wired or wireless networks provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (NIC) that can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0085] The input / output device 308 is used to input or output information. In the present embodiment, the input information can be observation coordinate points, etc., and the output information can be reflection pictures, etc.

[0086] Optionally, in the present embodiment, the processor 302 can be configured to perform the following steps by computer program:

[0087] establishing a mapping of a virtual space to a three-dimensional coordinate system of a physical imaging space, wherein the physical imaging space is provided with at least one non-planar imaging surface, and each non-planar imaging surface is composed of a plurality of curved imaging planes;

[0088] obtaining an observation coordinate point of an observation point in a world coordinate system in which the physical imaging space is located, and obtaining a pose of an imaging plane group composed of the imaging planes in the world coordinate system;

[0089] connecting lines from the observation coordinate point to vertices of each imaging plane of the imaging plane group in the world coordinate system, and calculating a maximum horizontal angle and a maximum pitch angle of a view cone of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane;

[0090] rendering a spherical picture according to the maximum horizontal angle and the maximum pitch angle;

[0091] obtaining horizontal and vertical angles between each imaging plane and the observation coordinate point, wherein the horizontal and / or vertical angles of the current imaging plane are enlarged by a certain proportion at the boundary position of each imaging plane;

[0092] cutting the reflection-free image from the spherical image according to the horizontal and vertical angles;

[0093] dividing the imaging plane into at least two sub-zones, using the vector data of the line connecting the observation coordinate point and the sub-zone as the plane normal data of the sub-zone, and obtaining the reflection image by distorting the reflection-free image based on the plane normal data of each sub-zone.

[0094] It should be noted that the specific examples in the embodiments can refer to the examples described in the above embodiments and optional implementation manners, which will not be described here again.

[0095] Generally, various embodiments can be implemented in hardware or special-purpose circuits, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented in firmware or software which can be executed by a controller, microprocessor or other computing device, but the application is not limited thereto. While various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controler or other computing devices, or some combination thereof.

[0096] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions to implement certain aspects of the embodiments. The program product can include one or more computer-executable components tangibly embodied on a non-transitory medium such as one or more of storage devices mentioned above. The one or more computer-executable components can include, but are not limited to, program instructions, state information, databases, or

[0097] Those skilled in the art should understand that each technical feature of the above embodiments can be combined arbitrarily, and for the sake of brevity, each technical feature in the above embodiments is not described in all possible combinations, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0098] The above embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as the limitation of the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for achieving continuous screen space reflection of a non-planar imaging surface, characterized in that, The method comprises the following steps: establishing a mapping of a three-dimensional coordinate system of a virtual space and a physical imaging space, wherein at least one non-planar imaging surface is arranged in the physical imaging space, and each non-planar imaging surface is composed of a plurality of curved imaging planes; obtaining an observation coordinate point of an observation point in a world coordinate system in which the physical imaging space is located, and obtaining a pose of an imaging plane group composed of the imaging planes in the world coordinate system; connecting lines from the observation coordinate point to vertices of each imaging plane of the imaging plane group in the world coordinate system, and obtaining a maximum horizontal angle and a maximum pitch angle of a view cone of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane; connecting the observation coordinate point to the vertices of each imaging plane to obtain the horizontal angle and the pitch angle of the current imaging plane, taking a maximum horizontal angle by taking an incremental value of the horizontal angles of all the imaging planes in the imaging plane group, taking a maximum pitch angle by taking an incremental value of the pitch angles of all the imaging planes in the imaging plane group, and obtaining a spherical coordinate of a projection region in a spherical coordinate system by converting a Cartesian coordinate system of the world coordinate system into the spherical coordinate system according to the maximum horizontal angle and the maximum pitch angle, obtaining a spherical projection matrix by using a matrix form to represent the spherical coordinate, rendering the spherical projection matrix represented in a two-dimensional plane coordinate system to obtain a spherical picture; obtaining a horizontal angle and a pitch angle of each imaging plane with respect to the observation coordinate point, wherein the horizontal angle and / or the pitch angle of the current imaging plane are enlarged by a certain proportion at a boundary position of each imaging plane; obtaining a reflection-free picture by cutting the spherical picture according to the horizontal angle and the pitch angle; segmenting the imaging plane into at least two partition planes, using vector data of a line connecting the observation coordinate point to the partition planes as plane normal data of the partition planes, and obtaining a reflection picture by distorting the reflection-free picture based on the plane normal data of each partition plane.

2. The method of claim 1, wherein, In the step of "obtaining a maximum horizontal angle and a maximum pitch angle of a view cone of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane", the observation coordinate point is connected to the vertices of each imaging plane to obtain the horizontal angle and the pitch angle of the current imaging plane, a maximum horizontal angle is obtained by taking an incremental value of the horizontal angles of all the imaging planes in the imaging plane group, and a maximum pitch angle is obtained by taking an incremental value of the pitch angles of all the imaging planes in the imaging plane group.

3. The method of claim 1, wherein, The boundary position of each imaging plane refers to a position of each imaging plane with respect to a neighboring imaging plane.

4. The method of claim 1, wherein, According to a proportion of the partition planes corresponding to the imaging planes, a corresponding reflection map is generated for the reflection picture, and reflection objects in the reflection-free picture are spliced according to a map range and an edge transparency of the reflection map.

5. The method of claim 1, wherein, The number of partition planes segmented at the boundary position of the imaging plane is greater than the number of partition planes segmented at other regions of the imaging plane.

6. The method of claim 1, wherein, The at least two imaging planes on the non-planar imaging surface are not located in the same horizontal direction and form a bend.

7. A screen space reflection apparatus that realizes a non-planar imaging surface continuity, characterized by, The method comprises the following steps: The mapping unit is configured to establish a mapping between a virtual space and a three-dimensional coordinate system of a physical imaging space, wherein at least one non-planar imaging surface is arranged in the physical imaging space, and each non-planar imaging surface is composed of a plurality of curved imaging planes. The coordinate acquisition unit is configured to acquire an observation coordinate point of an observation point in a world coordinate system in which the physical imaging space is located, and acquire an attitude of an imaging plane group composed of the imaging planes in the world coordinate system. The frustum positioning unit is configured to connect the observation coordinate point to vertexes of the imaging planes of the imaging plane group in the world coordinate system, and calculate a maximum horizontal angle and a maximum pitch angle of a frustum of the imaging plane group based on a horizontal angle and a pitch angle in each imaging plane. The spherical picture rendering unit is configured to acquire the horizontal angle and the pitch angle of the current imaging plane by connecting the observation coordinate point to the vertexes of each imaging plane, acquire a maximum horizontal angle by taking a sum of the horizontal angles of all the imaging planes in the imaging plane group, acquire a maximum pitch angle by taking a sum of the pitch angles of all the imaging planes in the imaging plane group, obtain a spherical coordinate of the projection region in the spherical coordinate system by converting a Cartesian coordinate system of the world coordinate system into the spherical coordinate system according to the maximum horizontal angle and the maximum pitch angle, obtain a spherical projection matrix by representing the spherical coordinate in the form of a matrix, perform rendering by representing the spherical projection matrix in a two-dimensional plane coordinate system to obtain a spherical picture. The angle acquisition unit is configured to acquire the horizontal angle and the pitch angle of each imaging plane with respect to the observation coordinate point, and to expand the horizontal angle and / or the pitch angle of the current imaging plane by a certain proportion at a boundary position of each imaging plane. The intercepting unit is configured to intercept a reflection-free picture from the spherical picture according to the horizontal angle and the pitch angle. The reflection unit is configured to divide the imaging plane into at least two sub-zoned planes, use vector data of a line connecting the observation coordinate point to the sub-zoned planes as plane normal data of the sub-zoned planes, and perform distortion on the reflection-free picture based on the plane normal data of each sub-zoned plane to obtain a reflection picture. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for realizing screen space reflection of a continuous non-planar imaging surface according to any one of claims 1 to 6.

9. A readable storage medium, characterized by, The readable storage medium stores a computer program, and the computer program includes program code for controlling a process to execute the process, and the process includes the method for realizing screen space reflection of a continuous non-planar imaging surface according to any one of claims 1 to 6.

Images