Image display method and device, electronic equipment and medium

Abstract

The invention provides an image display method, relates to a virtual world, and can be used in the field of financial science and technology. The image display method comprises the steps that a first visual angle of a virtual camera and a shot image under the first visual angle are acquired, and the first visual angle is a first person visual angle of a user in a virtual world; selecting a second view angle from the first view angles, wherein the second view angle is smaller than the first view angle; rendering the shot image at the second visual angle to obtain a rendered image at the second visual angle; and displaying an image at the first view angle, wherein the image at the first view angle comprises the rendered image at the second view angle. The invention further provides an image display device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to a virtual world and can be used in the field of financial technology, and more specifically to an image display method, device, electronic device and medium. Background Art

[0002] As the concept of the virtual world is booming, the banking industry has also launched various functions with virtual world tags on mobile clients to provide customers with a first-person immersive business experience. However, with the continuous updating of image technology, the industry has set higher standards for images and has launched 4K, 8K and even higher-definition images. However, on the one hand, this situation has brought huge performance requirements and pressure to the host system and customer mobile phones. On the other hand, the network signal in scenarios such as subways and remote areas is unstable and information transmission is slow. Both reasons may cause customers to have a poor experience when using mobile clients due to untimely image processing and transmission. Summary of the invention

[0003] In view of the above problems, the present disclosure provides an image display method, device, electronic device and medium with improved image processing efficiency.

[0004] According to a first aspect of the present disclosure, an embodiment of the present disclosure provides an image display method, including: obtaining a first perspective of a virtual camera, and a captured image under the first perspective, the first perspective being the first-person perspective of a user in a virtual world; selecting a second perspective within the first perspective, the second perspective being smaller than the first perspective; rendering the captured image under the second perspective to obtain a rendered image of the second perspective; and displaying the image under the first perspective, the image under the first perspective including the rendered image of the second perspective.

[0005] According to an embodiment of the present disclosure, the second viewing angle is greater than half of the first viewing angle.

[0006] According to an embodiment of the present disclosure, selecting a second perspective within a first perspective includes: determining a horizontal rotation angle of a device held by a user in a horizontal direction and a vertical rotation angle in a vertical direction; calculating a left selection position of the left boundary of the second perspective according to the horizontal rotation angle and the position of the left boundary of the first perspective, or calculating a right selection position of the right boundary of the second perspective according to the horizontal rotation angle and the position of the right boundary of the first perspective; calculating an upper selection position of the upper boundary of the second perspective according to the vertical rotation angle and the position of the upper boundary of the first perspective, or calculating a lower selection position of the lower boundary of the second perspective according to the vertical rotation angle and the position of the lower boundary of the first perspective; selecting the second perspective from the first perspective using the left selection position and the upper selection position as the boundaries of the second perspective, or using the left selection position and the lower selection position as the boundaries of the second perspective, or using the right selection position and the upper selection position as the boundaries of the second perspective, or using the right selection position and the lower selection position as the boundaries of the second perspective.

[0007] According to an embodiment of the present disclosure, before rendering the captured image at the second viewing angle, the method further includes: determining a first resolution of the captured image at the second viewing angle, where the first resolution is higher than a resolution of the captured image at the first viewing angle.

[0008] According to an embodiment of the present disclosure, obtaining the first viewing angle of a virtual camera and a captured image at the first viewing angle includes: obtaining the first viewing angle of the virtual camera; determining objects within the first viewing angle; calculating the position of each object relative to the virtual camera; establishing a perspective projection matrix based on the position; calculating the projection position of the object on the display screen of a device held by the user based on the perspective projection matrix; and generating a captured image within the first viewing angle based on the projection position.

[0009] According to an embodiment of the present disclosure, generating a captured image within a first perspective according to a projection position includes: determining perspective effect parameters of an object; confirming a display image of the object according to the perspective effect parameters; scaling the display image of the object according to the perspective effect parameters; and combining the scaled display images of all objects into a captured image within the first perspective according to the projection position.

[0010] According to an embodiment of the present disclosure, calculating the projection position of an object on a display screen of a device held by a user according to a perspective projection matrix includes: calculating the normalized device coordinates of the object according to the coordinate position of the object in the virtual world and the perspective projection matrix; determining the projection position of the object on the display screen according to the normalized device coordinates and size parameters of the display screen;

[0011] According to an embodiment of the present disclosure, rendering the captured image under the second perspective includes: obtaining the projection position and scaling ratio of each object in the captured image within the second perspective; rendering the image of the object according to the projection position, scale and geometric texture data of the object.

[0012] According to an embodiment of the present disclosure, the horizontal rotation angle of the device held by the user in the horizontal direction and the vertical rotation angle in the vertical direction are calculated by data from sensors of the device held by the user, and the sensors include: at least one of an accelerometer sensor, a gyroscope sensor, and a magnetometer sensor.

[0013] According to an embodiment of the present disclosure, when the data of the sensor changes, the second viewing angle and / or the first viewing angle are updated.

[0014] In a second aspect of the present disclosure, an embodiment of the present disclosure provides an image display device, including: an acquisition module, used to acquire a first perspective of a virtual camera, and a captured image at the first perspective, the first perspective being the first-person perspective of a user in a virtual world; a selection module, used to select a second perspective within the first perspective, the second perspective being smaller than the first perspective; a rendering module, used to render the captured image at the second perspective to obtain a rendered image of the second perspective; and a display module, used to display the image at the first perspective, the image at the first perspective including the rendered image of the second perspective.

[0015] In a third aspect of the present disclosure, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned image display method.

[0016] In a fourth aspect of the present disclosure, an embodiment of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned image display method.

[0017] According to the embodiments of the present disclosure, the present disclosure has at least one of the following beneficial effects: by selecting the second perspective within the first perspective, only the captured images within the second perspective are rendered, and the captured images within the second perspective use high-resolution pictures, and the portion of the first perspective excluding the second perspective is not rendered or is only incompletely rendered, and the captured images within the first perspective excluding the second perspective use the lowest level of resolution supported by the system, which reduces the amount of pictures that need to be rendered and reduces the requirements for the resolution of displayed pictures, reduces the memory occupied by displayed pictures, improves the image processing speed and transmission speed, and reduces the performance pressure of the server in processing captured images. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0019] Figure 1 The application scenario diagram of the image display method according to the embodiment of the present disclosure is schematically shown;

[0020] Figure 2 A flowchart of an image display method according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A schematic diagram schematically illustrates an area of ​​a display portion according to an embodiment of the present disclosure;

[0022] Figure 4Schematically shows a flow chart of selecting a second viewing angle within a first viewing angle according to an embodiment of the present disclosure;

[0023] Figure 5 A schematic diagram schematically shows a method for selecting a position of a second viewing angle boundary when a device held by a user is tilted upward according to an embodiment of the present disclosure;

[0024] Figure 6 A schematic diagram schematically shows a method for selecting a position of a second viewing angle boundary when a device held by a user is tilted downward according to an embodiment of the present disclosure;

[0025] Figure 7 A flowchart of acquiring a captured image at a first viewing angle according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 8 A flowchart of generating a captured image within a first viewing angle according to a projection position according to an embodiment of the present disclosure is schematically shown;

[0027] Fig. 9 A structural block diagram schematically shows an image display device according to an embodiment of the present disclosure; and

[0028] Fig.10 A block diagram of an electronic device suitable for implementing an image display method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0029] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] Figure 1 The application scenario diagram of the image display method according to the embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103. A network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and a server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0035] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal devices 101, 102, 103 can be installed with various motion sensors, such as accelerometer sensors, gyroscope sensors, magnetometer sensors, etc. The above multiple sensors can reflect the dynamic position of the terminal device in space in real time, such as horizontal tilt, turn, forward and backward, etc.

[0036] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting virtual functions of a virtual world, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0037] Server 105 may be a server that provides various services, such as a background management server for the virtual world screens browsed by users using terminal devices 101, 102, and 103 (only as an example). The virtual world is a multi-platform and multi-application ecosystem that integrates virtual reality, augmented reality, and mixed reality technologies, and uses artificial intelligence, big data, and other technologies to create and manage virtual environments. It can provide a more immersive and interactive experience, allowing users to conduct various activities in the virtual world, such as handling various banking services and understanding various bank products through the virtual world, beyond the limitations of the real world.

[0038] The backend management server can analyze and process the received user request and other data, and feed back the processing results (such as the corresponding virtual world image screen, business processing screen and options, etc. obtained according to the user request or the user terminal sensor data) to the terminal device.

[0039] It should be noted that the image display method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the image display device provided in the embodiment of the present disclosure can generally be set in the server 105. The image display method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image display device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0041] The following will be based on Figure 1 The scene described by Figure 2 to Figure 8 The image display method of the disclosed embodiment is described in detail.

[0042] Figure 2 The flowchart of the image display method according to the embodiment of the present disclosure is schematically shown.

[0043] like Figure 2 As shown, the image display of this embodiment includes operations S210 to S240, and the transaction processing method can be executed in a server.

[0044] In operation S210, a first viewing angle of a virtual camera and a captured image under the first viewing angle are acquired, where the first viewing angle is a first-person viewing angle of a user in a virtual world.

[0045] According to the actual needs of the business, set the perspective of the character in the virtual world, that is, the position and angle of the virtual camera, simulate the user's perspective in the virtual world, and thus determine the range of the first-person perspective. The first-person perspective is usually achieved through virtual devices, such as head-mounted displays, handheld controllers, motion capture devices, smartphones, and tablets.

[0046] In the first-person perspective, users can feel the environment, dynamics and interactions in the virtual world, and complete interactive operations such as business processing and business consultation in the virtual world.

[0047] In operation S220, a second viewing angle is selected within the first viewing angle, and the second viewing angle is smaller than the first viewing angle.

[0048] The second viewing angle is within the first viewing angle, and its range is smaller than the first viewing angle. The displayed image within the range of the second viewing angle only occupies a part of the total image captured within the first viewing angle; preferably, the projection of the captured image of the second viewing angle on the screen only occupies half of the captured image of the first viewing angle. The second viewing angle can be similar to the viewing angle of a glasses frame, and the aspect ratio can use a common glasses ratio such as 16:9. In some embodiments, the size of the second viewing angle can be adjusted according to actual conditions. Preferably, the second viewing angle is greater than half of the second viewing angle. In some embodiments, the aspect ratio of the second viewing angle can be adjusted accordingly according to the actual conditions of the display device, for example, it can be 4:3, 10:7, etc. The proportion of the second viewing angle is greater than one-half, which ensures that most of the displayed content can be rendered, and reduces the rendering pressure while ensuring the user experience.

[0049] The selection position of the second perspective can be determined based on the sensor data on the display device used by the user. When the sensor data are all in the initial state, the second perspective is located at the center of the first perspective, and the second perspective is selected from the first perspective according to the preset aspect ratio and size. When the sensor data is not in the initial state, the selection position of the second perspective is calculated based on the sensor data.

[0050] In operation S230, the captured image at the second viewing angle is rendered to obtain a rendered image at the second viewing angle.

[0051] Only the images captured from the first perspective are rendered. The rendering methods may include the following methods:

[0052] Rasterization: After determining the position and size of the captured object on the screen, the color value of each pixel is calculated based on properties such as lighting and texture.

[0053] Ray Tracing: By tracing the propagation path of light in the scene, calculating the intersection points and interactions between light and objects, the color value of the pixel is determined. Ray tracing can simulate the refraction, reflection, shadow and other phenomena of light, and can produce realistic rendering effects.

[0054] Rendering Pipeline: The rendering process is divided into multiple stages, each of which handles a specific task. Common stages include geometry processing, lighting calculation, texture mapping, etc. By processing these stages in parallel, the rendering process can be accelerated.

[0055] Volume Rendering: Applicable to rendering scenes containing dense volume data. It generates corresponding two-dimensional images by sampling the volume data and calculating light propagation.

[0056] Real-time Rendering: Real-time rendering usually uses rasterization technology and improves rendering performance through a series of optimization methods (such as primitive culling, cascade shadows, LOD, etc.).

[0057] According to an embodiment of the present disclosure, before rendering the captured image at the second viewing angle, a first resolution of the captured image at the second viewing angle may be determined, and the first resolution is higher than the resolution of the captured image at the first viewing angle. For example, the resolution of the captured image at the second viewing angle may be 4K high-definition resolution, while the resolution of the image at the first viewing angle only needs to be processed to the minimum level supported by the current system, or the resolution of the image at the first viewing angle may be set lower than the first resolution according to actual business needs.

[0058] In operation S240, an image at a first viewing angle is displayed, and the image at the first viewing angle includes a rendered image at a second viewing angle.

[0059] According to the embodiments of the present disclosure, by selecting the second perspective within the first perspective, only the captured image within the second perspective is rendered, and the part of the first perspective excluding the second perspective is not rendered or is only incompletely rendered, thereby reducing the amount of pictures that need to be rendered, improving the image processing speed and transmission speed, and reducing the performance pressure of the server in processing the captured images.

[0060] Figure 3 A schematic diagram schematically illustrates an area of ​​a display portion according to an embodiment of the present disclosure.

[0061] like Figure 3 As shown, the image under the first viewing angle is divided into two parts, one of which is the second viewing angle part A (i.e. Figure 3 The other is the part B within the first perspective excluding the second perspective. The captured image of part A is fully rendered, and the captured image of part B does not need to be rendered or does not need to be fully rendered. Only the image within the second perspective has been fully rendered. Similarly, the captured image in part A is of high resolution, such as 4K, 8K, etc.; the captured image in part B only needs to be processed to the lowest level supported by the current system, such as 512P, etc. According to an embodiment of the present disclosure, the captured image within the second perspective uses a high-resolution image, and the captured image within the first perspective excluding the part of the second perspective uses the lowest level of resolution supported by the system, which reduces the resolution requirement for the displayed image, reduces the memory occupied by the displayed image, further improves the image processing speed and transmission speed, and reduces the performance pressure of the server in processing the captured image.

[0062] Figure 4 Schematically shows a flow chart of selecting a second viewing angle within a first viewing angle according to an embodiment of the present disclosure;

[0063] like Figure 4 As shown, the operation S220 may include steps S410 to S460;

[0064] In operation S410, a horizontal rotation angle of a device held by a user in a horizontal direction and a vertical rotation angle in a vertical direction are determined according to sensor data of a device used by a user;

[0065] Before obtaining sensor data, the sensors in the hardware of the device used by the user are first determined. The sensor types supported by the device currently used by the user can be determined by initiating a background call by the application.

[0066] The sensor includes at least one of an accelerometer sensor, a gyroscope sensor, and a magnetometer sensor. In some embodiments, if there are other sensors that can achieve the same effect as one of the above sensors or the effect of the sum of multiple sensors, other sensors can also be used to replace one or more of the above sensors.

[0067] An accelerometer is a sensor that measures the acceleration of an object. It can detect the acceleration of a mobile phone in three axes (x, y, and z). By reading the value of the accelerometer sensor, you can get the current acceleration of the mobile phone. The accelerometer value is usually measured in gravitational acceleration (g), which indicates the acceleration of the mobile phone on each axis relative to the gravity of the earth.

[0068] A gyroscope is a sensor used to measure the rotation speed and angle of an object. By monitoring the values ​​of the gyroscope sensor, the angular velocity of the phone's rotation around three axes (x, y, z) can be obtained. Based on the angular velocity data of the gyroscope sensor, the rotation angle of the phone on the three axes can be calculated by integration.

[0069] A magnetometer is a sensor that measures the strength of the surrounding magnetic field. Combining the data from the accelerometer and magnetometer allows for magnetic field calibration and calculation of the phone's orientation, i.e. the phone's facing direction.

[0070] According to the embodiments of the present disclosure, only a small amount of sensor data is required for calculation, and the data processing speed is fast.

[0071] In operation S420, after determining the sensor type of the device used by the user, the application initiates a background call to obtain data from each sensor. A sensor fusion algorithm, such as a Kalman filter or a quaternion rotation algorithm, is used to process and integrate the data of multiple sensors, perform posture calculation and posture control, and determine the posture data of the device currently used by the user, such as the horizontal rotation angle of the device held by the user in the horizontal direction and the vertical rotation angle in the vertical direction.

[0072] In operation S430, a left selected position of a left boundary of the second viewing angle is calculated according to the horizontal rotation angle and the position of the left boundary of the first viewing angle, or a right selected position of a right boundary of the second viewing angle is calculated according to the horizontal rotation angle and the position of the right boundary of the first viewing angle;

[0073] In operation S440, an upper selected position of an upper boundary of the second viewing angle is calculated according to the vertical rotation angle and the position of the upper boundary of the first viewing angle, or a lower selected position of a lower boundary of the second viewing angle is calculated according to the vertical rotation angle and the position of the lower boundary of the first viewing angle.

[0074] In operation S450, the second perspective is selected from the first perspective with the left selection position and the upper selection position as the boundary of the second perspective, or with the left selection position and the lower selection position as the boundary of the second perspective, or with the right selection position and the upper selection position as the boundary of the second perspective, or with the right selection position and the lower selection position as the boundary of the second perspective.

[0075] Since the size and length and width of the second viewing angle have been preset, it is only necessary to determine the positions of two adjacent boundaries in the second viewing angle to select the desired second viewing angle from the first viewing angle.

[0076] According to different sensor data, the selection position of the second perspective in the first perspective is also different.

[0077] According to the embodiment of the present disclosure, the selection of the second viewing angle varies according to different sensor data, thereby ensuring that the second viewing angle is always located in a position within the first viewing angle that conforms to the user's daily interaction habits.

[0078] Figure 5 A schematic diagram schematically shows a method for selecting a position of a second viewing angle boundary when a device held by a user is tilted upward according to an embodiment of the present disclosure.

[0079] like Figure 5As shown, the X-axis is the horizontal axis and the Y-axis is the vertical axis. When the device used by the user is tilted upward, the first-person perspective is a bird's-eye view. At this time, the position of the second perspective should be lower than the first perspective. To facilitate the calculation of the position, the selection position of the lower boundary of the second perspective is calculated based on the vertical rotation angle shown and the lower boundary of the first perspective; the left selection position of the left boundary of the second perspective is calculated based on the horizontal rotation angle and the left boundary of the first perspective, or the right selection position of the right boundary of the second perspective is calculated based on the horizontal rotation angle and the right boundary of the first perspective.

[0080] The second perspective is selected from the first perspective based on the lower selection position and the left selection position as the boundary of the second perspective, or the lower selection position and the right selection position as the boundary of the second perspective. In actual operation, for the convenience of calculation, when the device used by the user has an offset angle to the right, the right selection position is calculated first, and the second perspective is selected from the first perspective with the right selection position and the lower selection position as the boundary; when the device used by the user has an offset angle to the left, the left selection position is calculated first, and the second perspective is selected from the first perspective with the left selection position and the lower selection position as the boundary; when the device used by the user has no horizontal offset, selection can be made through the left selection position or the right selection position.

[0081] Figure 6 A schematic diagram schematically shows a method for selecting a position of a second viewing angle boundary when a device held by a user is tilted downward according to an embodiment of the present disclosure.

[0082] like Figure 6 As shown, the X-axis is the horizontal axis and the Y-axis is the vertical axis. When the device used by the user is tilted downward, the first-person perspective is an upward perspective. At this time, the position of the second perspective should be slightly above the first perspective. To facilitate the calculation of the position, the selection position of the lower boundary of the second perspective is calculated based on the vertical rotation angle shown and the lower boundary of the first perspective; the left selection position of the left boundary of the second perspective is calculated based on the horizontal rotation angle and the left boundary of the first perspective, or the right selection position of the right boundary of the second perspective is calculated based on the horizontal rotation angle and the right boundary of the first perspective.

[0083] The second perspective is selected from the first perspective according to the lower selected position and the left selected position as the boundary of the second perspective, or the lower selected position and the right selected position as the boundary of the second perspective. In actual operation, for the convenience of calculation, when the device used by the user has an offset angle to the left or right, the left selected position or the right selected position is preferentially selected, and the selection condition is the same as the above case when the device used by the user is tilted upward.

[0084] When the horizontal rotation angle of the device used by the user is to the left or right, the method for selecting the second viewing angle is similar to the above content and will not be described in detail here.

[0085] The lower selection position of the lower boundary of the second perspective does not exceed the lower boundary of the first perspective, the left selection position of the left boundary of the second perspective does not exceed the left boundary of the first perspective, the right selection position of the right boundary of the second perspective does not exceed the right boundary of the first perspective, and the upper selection position of the upper boundary of the second perspective does not exceed the upper boundary of the first perspective.

[0086] Figure 7 The flowchart of acquiring a captured image at a first viewing angle according to an embodiment of the present disclosure is schematically shown.

[0087] like Figure 7 As shown, operation S210 may include steps S710 to S760.

[0088] In operation S710, a first perspective of a virtual camera is obtained. The first perspective of a virtual camera is a first-person perspective of a user in a virtual world. The first perspective of a virtual camera is determined by various parameters of a display device currently used by the user and a line of sight position of a preset first perspective in the virtual world. The various parameters of the display device may be, for example, the size of a display screen, sensor data of a position and angle sensor, etc.

[0089] In operation S720, objects located in the first viewing angle are determined. According to the captured content of the first viewing angle, each object in the captured content is determined in sequence. The object may be, for example, a virtual window, a virtual floor, a virtual wall, a virtual operating table, a virtual button, etc. After the object is determined, parameter information of each object may be obtained, and the parameter information of each object itself may include: coordinate information, texture information, illumination information, material information, object orientation information, color information, size information, etc.

[0090] In operation S730, the position of each object relative to the virtual camera is calculated. The relative position information of each object relative to the camera is calculated based on the object coordinate information, the coordinate information of the virtual camera in the virtual world, and the direction and angle information of the first perspective, and the relative position information includes the distance information between the object and the virtual camera and the offset information between the object and the first perspective.

[0091] In operation S740, a perspective projection matrix is ​​established according to the position. According to the actual needs of the business, the parameters of the perspective projection of the object on the display screen are set, and the parameters of the perspective projection may include, for example, the field of view angle, the screen aspect ratio, the distance between the near clipping plane and the meta clipping plane, etc.; the perspective projection matrix is ​​calculated using the parameters of the perspective projection.

[0092] In operation S750, the projection position of the object on the display screen of the device held by the user is calculated according to the perspective projection matrix. The perspective projection matrix is ​​a key step in mapping the three-dimensional scene in the virtual world to the display screen of the device used by the user. At the same time, according to the coordinate position of the virtual camera in the virtual world and the angular direction of the first perspective, the view matrix of the virtual camera is generated, and the view matrix is ​​used to convert the object from the world coordinate system to the camera coordinate system.

[0093] Traverse the objects within the viewing angle, and multiply the vertex coordinates of the object by the view matrix and the perspective projection matrix according to the coordinate information of each object in the virtual world to obtain the projected coordinates, that is, project the object coordinates in the virtual world into the standard cube (viewing volume). Divide the projected coordinates by the w component in the homogeneous coordinates, and offset the projected coordinates from the center of the viewing volume to a cube in the range of [-1, 1], that is, convert the projected coordinates into normalized device coordinates (NDC).

[0094] The x and y components in the normalized device coordinates are mapped to the two-dimensional coordinate system of the display screen. The normalized device coordinates are converted into pixel coordinates on the screen through the width and height of the screen, thus obtaining the projection position of the object on the display screen of the device held by the user.

[0095] In operation S760, a photographed image within the first viewing angle is generated according to the projection position.

[0096] Figure 8 The flowchart of generating a captured image within a first viewing angle according to a projection position according to an embodiment of the present disclosure is schematically shown.

[0097] like Figure 8 As shown, step S760 may include steps S810 to S840.

[0098] In operation S810, perspective effect parameters of an object are determined. The perspective effect parameters may include, for example, a projection position of the object, a scaling ratio, texture information of the object, and a relative position between the object and a virtual camera.

[0099] In operation S820, the display image of the object is determined according to the perspective effect parameters. Different objects have different texture information, light and dark information, etc. on the image to be displayed according to their different locations. The closer the distance, the more texture details, and the farther the distance, the fewer texture details. After determining the texture information such as the texture details, the display image of the object can be obtained.

[0100] In operation S830, the displayed image of the object is scaled according to the perspective effect parameter. The size of the displayed image of the object is different according to the relative distance between the position of the object and the virtual camera. The closer the distance, the larger the displayed image, and the farther the distance, the smaller the displayed image. The scaled size of the object on the real screen is determined according to the relative position of the object and the virtual camera, and the real image is scaled according to the scaled size.

[0101] In operation S840, the zoomed display pictures of all objects are combined into a captured image within the first viewing angle according to the projection positions.

[0102] According to an embodiment of the present disclosure, when the sensor data of the device used by the user changes, the second perspective and / or the first perspective are updated. When the coordinates of the virtual camera corresponding to the device held by the user remain unchanged in the virtual world, when the downward looking angle of the device held by the user increases, the selection position of the second perspective is updated, and the lower selection position of the updated second perspective should be closer to the lower boundary of the first perspective than the lower selection position before the update. When the lower boundary of the second perspective coincides with the lower boundary of the first perspective, and the downward looking angle of the device held by the user continues to increase, the orientations of the first perspective and the second perspective in the virtual world are synchronously updated, and the relative positions of the boundary of the first perspective and the boundary of the second perspective remain unchanged.

[0103] When the coordinates of the virtual camera corresponding to the device held by the user in the virtual world change, the coordinate positions and shooting contents of the first perspective and the second perspective in the virtual world are synchronously updated according to the changed coordinates of the virtual camera.

[0104] Similarly, when the angle of the device used by the user changes upward, left, or right, the first viewing angle and the second viewing angle change in a similar manner to the above-mentioned downward-looking situation, which will not be elaborated here.

[0105] Based on the above image display method, the present disclosure also provides an image display device. Fig. 9 The device is described in detail.

[0106] Fig. 9 The structure block diagram of the image display device according to the embodiment of the present disclosure is schematically shown.

[0107] like Fig. 9 As shown, the image display device 900 of this embodiment includes an acquisition module 910 , a selection module 920 , a rendering module 930 and a display module 940 .

[0108] The acquisition module 910 is used to acquire the first perspective of the virtual camera and the captured image under the first perspective, where the first perspective is the first-person perspective of the user in the virtual world. In one embodiment, the acquisition module 910 can be used to perform the operation S210 described above, which will not be repeated here.

[0109] The selection module 920 is used to select a second viewing angle within the first viewing angle, where the second viewing angle is smaller than the first viewing angle. In one embodiment, the selection module 920 may be used to perform the operation S220 described above, which will not be described in detail herein.

[0110] The rendering module 930 is used to render the captured image at the second viewing angle to obtain a rendered image at the second viewing angle. In one embodiment, the rendering module 930 may be used to perform the operation S230 described above, which will not be described in detail herein.

[0111] The display module 940 is used to display an image at a first viewing angle, where the image at the first viewing angle includes a rendered image at a second viewing angle. In one embodiment, the display module 940 may be used to perform the operation S240 described above, which will not be described in detail herein.

[0112] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 910, the selection module 920, the rendering module 930 and the display module 940 can be combined in one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the selection module 920, the rendering module 930 and the display module 940 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 910, the selection module 920, the rendering module 930 and the display module 940 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0113] Fig.10 A block diagram of an electronic device suitable for implementing an image display method according to an embodiment of the present disclosure is schematically shown.

[0114] like Fig.10As shown, the electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0115] In RAM 1003, various programs and data required for the operation of electronic device 1000 are stored. Processor 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. Processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1002 and / or RAM 1003. It should be noted that the program can also be stored in one or more memories other than ROM 1002 and RAM 1003. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0116] According to an embodiment of the present disclosure, the electronic device 1000 may further include an input / output (I / O) interface 1005, which is also connected to the bus 1004. The electronic device 1000 may further include one or more of the following components connected to the I / O interface 1005: an input portion 1006 including a keyboard, a mouse, etc.; an output portion 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk, etc.; and a communication portion 1009 including a network interface card such as a LAN card, a modem, etc. The communication portion 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage portion 1008 as needed.

[0117] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0118] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1002 and / or RAM 1003 described above and / or one or more memories other than ROM 1002 and RAM 1003.

[0119] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the item recommendation method provided by the embodiment of the present disclosure.

[0120] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1001. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0121] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1009, and / or installed from the removable medium 1011. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0122] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the processor 1001, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0123] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0124] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0126] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image display method, comprising: Acquire a first perspective of the virtual camera and a captured image under the first perspective, where the first perspective is a first-person perspective of the user in the virtual world; Selecting a second viewing angle within the first viewing angle, wherein the second viewing angle is smaller than the first viewing angle; Rendering the captured image at the second viewing angle to obtain a rendered image at the second viewing angle; The image at the first perspective is displayed, wherein the image at the first perspective includes the rendered image at the second perspective. 2 . The image display method according to claim 1 , wherein the second viewing angle is greater than half of the first viewing angle.

3. The image display method according to claim 1, wherein selecting a second viewing angle within the first viewing angle comprises: Determine the horizontal rotation angle of the device held by the user in the horizontal direction and the vertical rotation angle in the vertical direction; Calculating a left selected position of the left boundary of the second viewing angle according to the horizontal rotation angle and the position of the left boundary of the first viewing angle, or calculating a right selected position of the right boundary of the second viewing angle according to the horizontal rotation angle and the position of the right boundary of the first viewing angle; Calculating an upper selected position of an upper boundary of the second viewing angle according to the vertical rotation angle and the position of an upper boundary of the first viewing angle, or calculating a lower selected position of a lower boundary of the second viewing angle according to the vertical rotation angle and the position of a lower boundary of the first viewing angle; The second perspective is selected from the first perspective according to the left selection position and the upper selection position, or according to the left selection position and the lower selection position, or according to the right selection position and the upper selection position, or according to the right selection position and the lower selection position.

4. The image display method according to claim 1, before rendering the captured image at the second viewing angle, further comprising: A first resolution of the image captured at the second viewing angle is determined, where the first resolution is higher than a resolution of the image captured at the first viewing angle.

5. The image display method according to claim 1, wherein the step of acquiring a first viewing angle of the virtual camera and a captured image under the first viewing angle comprises: Get the first-person perspective of the virtual camera; determining an object located within the first viewing angle; Calculating the position of each of the objects relative to the virtual camera; Establishing a perspective projection matrix according to the position; Calculating the projection position of the object on the display screen of the device held by the user according to the perspective projection matrix; A captured image within the first viewing angle is generated according to the projection position.

6. The image display method according to claim 5, wherein generating the captured image within the first viewing angle according to the projection position comprises: Determining perspective effect parameters of the object; Determining a display image of the object according to the perspective effect parameters; scaling the displayed image of the object according to the perspective effect parameter; The zoomed display pictures of all the objects are combined into a captured image within the first viewing angle according to the projection position.

7. The image display method according to claim 5, wherein calculating the projection position of the object on the display screen of the device held by the user according to the perspective projection matrix comprises: Calculating the normalized device coordinates of the object according to the coordinate position of the object in the virtual world and the perspective projection matrix; The projection position of the object on the display screen is determined according to the normalized device coordinates and the size parameters of the display screen.

8. The image display method according to claim 6, wherein rendering the captured image at the second viewing angle comprises: Acquire the projection position and the scaling ratio of each object in the captured image within the second viewing angle; An image of the object is rendered according to the projection position, the scale, and the geometric texture data of the object.

9. The image display method according to claim 3, wherein the horizontal rotation angle of the device held by the user in the horizontal direction and the vertical rotation angle of the device held by the user in the vertical direction are calculated from data of a sensor of the device held by the user, and the sensor comprises: At least one of an accelerometer sensor, a gyroscope sensor, and a magnetometer sensor.

10. The image display method according to claim 9, further comprising: When the data of the sensor changes, the second viewing angle and / or the first viewing angle are updated.

11. An image display device, comprising: An acquisition module, used to acquire a first viewing angle of a virtual camera and a captured image under the first viewing angle, wherein the first viewing angle is a first-person viewing angle of a user in a virtual world; A selection module, configured to select a second viewing angle within the first viewing angle, wherein the second viewing angle is smaller than the first viewing angle; A rendering module, used for rendering the captured image at the second viewing angle to obtain a rendered image at the second viewing angle; as well as The display module is used to display the image at the first viewing angle, where the image at the first viewing angle includes the rendered image at the second viewing angle.

12. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 10.

Images