Panoramic picture encoding method, apparatus, display system, and electronic device

Abstract

This invention relates to the field of virtual display technology and discloses a panoramic image encoding method, apparatus, display system, and electronic device. The method includes: determining key display positions of the panoramic image; rotating the panoramic image to bring the key display positions to the extreme points of the panoramic image; unfolding the rotated panoramic image into a planar image; and encoding the planar image to generate the encoded result of the panoramic image. Through the above method, this invention improves the display effect of panoramic images.

Description

Technical Field

[0001] This invention relates to the field of virtual display technology, specifically to a panoramic image encoding method, apparatus, display system, and electronic device. Background Technology

[0002] Virtual reality (VR) technology uses computers to simulate virtual environments to enhance user immersion. As VR technology continues to develop, its applications are expanding.

[0003] In related technologies, virtual display technology first captures images from different angles using multiple lenses, then renders these images into a panoramic view using rendering technology. After unfolding the panoramic view into a planar image, the planar image is encoded and transmitted to a display device for display. However, in implementing the embodiments of this invention, the inventors discovered that in related technologies, invalid information occupies a large display area while valid information occupies a small display area in the panoramic image displayed by the display device, resulting in poor display quality. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a panoramic image encoding method, apparatus, display system and electronic device to solve the problem of poor panoramic image display effect in the prior art.

[0005] According to one aspect of the present invention, a panoramic image encoding method is provided, the method comprising:

[0006] Determine the key display positions of the panoramic image;

[0007] The panoramic view is rotated to bring the key display position to the extreme point of the panoramic view.

[0008] The rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image.

[0009] In one alternative approach, rotating the panoramic view includes:

[0010] The rotation axis, rotation angle, and rotation direction for rotating the panoramic image are determined based on the key display position.

[0011] The panoramic image is rotated according to the rotation axis, the rotation angle, and the rotation direction to rotate the key display position to the extreme position of the panoramic image.

[0012] In an alternative approach, the method further includes:

[0013] The plane containing the key display position, the center of the panoramic image, and the north pole of the panoramic image is defined as the rotation plane of the panoramic image.

[0014] The axis perpendicular to the plane of rotation and passing through the center of the sphere in the panoramic view is defined as the axis of rotation.

[0015] In an alternative approach, the method further includes:

[0016] Determine the rotation arc corresponding to the rotation plane, wherein the rotation arc is a circular arc on the sphere of the panoramic image that intersects with the rotation plane;

[0017] Identify two directed arcs on the rotation arc that point from the key display position to the pole position;

[0018] Determine the shorter of the two directed spherical arcs;

[0019] The direction of the shorter directional arc is determined as the rotation direction.

[0020] In an alternative embodiment, the two directed arcs point towards the North Pole, and the method further includes:

[0021] Determine the spherical coordinates (x, y, z) of the key display position;

[0022] If z > 0, then the rotation angle of the panoramic image is determined as follows:

[0023] If z < 0, then the rotation angle of the panoramic image is determined as follows:

[0024] In an alternative embodiment, after encoding the planar image to generate the encoded panoramic image, the method further includes:

[0025] Obtain the rotation information of the panoramic image, wherein the rotation information includes the rotation axis, rotation direction, and rotation angle of the panoramic image;

[0026] The rotation information is added as additional enhancement information to the encoding result.

[0027] According to another aspect of the present invention, a panoramic image encoding apparatus is provided, the apparatus comprising:

[0028] The determination module is used to determine the key display positions of the panoramic image;

[0029] A rotation module is used to rotate the panoramic image to rotate the key display position to the extreme position of the panoramic image;

[0030] The generation module is used to unfold the rotated panoramic image into a planar image, and encode the planar image to generate the encoded result of the panoramic image.

[0031] According to another aspect of the present invention, a panoramic image display system is provided, the system comprising a display device and the aforementioned encoding device:

[0032] The encoding device is used to send the encoding result of the panoramic image to the display device, and the display device is used to decode the encoding result of the panoramic image and display the decoded panoramic image.

[0033] According to another aspect of the present invention, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0034] The memory is used to store at least one executable instruction that causes the processor to perform the operation of the panoramic image encoding method described above.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, characterized in that the storage medium stores at least one executable instruction, which, when executed on an electronic device, causes the electronic device to perform the operation of the panoramic image encoding method described above.

[0036] In this embodiment of the invention, the region where the extreme point of the panoramic image is located is the pixel concentration area when the panoramic image is displayed. However, the extreme point of the panoramic image is often not located in the key display area of ​​the panoramic image, affecting the display effect of the key display area. When encoding the panoramic image, the key display position of the panoramic image is first determined, and then the panoramic image is rotated based on the key display position to rotate the key display position to the extreme point of the panoramic image. Finally, the rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image. In the above embodiment, by rotating the key display position of the panoramic image to the extreme point, the key display area of ​​the panoramic image can have a better display effect.

[0037] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0038] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0039] Figure 1 A flowchart illustrating the panoramic image encoding method provided in an embodiment of the present invention is shown;

[0040] Figure 2 A schematic diagram of the rotation of the panoramic image provided in an embodiment of the present invention is shown;

[0041] Figure 3 A schematic diagram of the panoramic image encoding device provided in an embodiment of the present invention is shown;

[0042] Figure 4 A schematic diagram of the panoramic image display system provided in an embodiment of the present invention is shown;

[0043] Figure 5 A schematic diagram illustrating the display effect of the panoramic image provided in an embodiment of the present invention is shown;

[0044] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0045] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0046] In related technologies, virtual display technology first captures images from different angles using multiple lenses, then uses rendering technology to render these images into a panoramic view. After unfolding the panoramic view into a flat image, the flat image is encoded and transmitted to a display device for display. When unfolding a panoramic view into a flat image, common projection techniques such as equidistant cylindrical projection or normal-axis isometric cylindrical projection are generally used. However, these common projection techniques have several drawbacks. For example, world maps are a common application of equidistant cylindrical projection. Equidistant cylindrical projection unfolds a panoramic view into a flat image using rectangular spaces divided by longitude and latitude. Its biggest problem is highly uneven distortion, which worsens with increasing latitude. When encoding the unfolded flat image, the high-latitude regions with severe distortion have higher encoding quality. Therefore, when the display device restores the flat image to a panoramic view, the high-latitude regions occupy a larger portion of the image, retain more effective information, and are clearer, while the low-latitude regions occupy a smaller portion, retain less effective information, and are blurrier. In other words, the pixels in the high-latitude regions where the North and South Poles are located are very dense, resulting in a clearer image, while the pixels in the low-latitude regions where the equator is located are relatively sparse, resulting in a blurrier image.

[0047] However, in practical applications, the key display area of ​​a panoramic image is often concentrated in the center of the viewer's eye, meaning that important content is usually distributed in the low-latitude region where the equator is located. The equator, which should be a concentrated area of ​​effective information, instead receives the smallest area, while the top and bottom, areas where ineffective information accumulates, receive a very large area. This results in the final effect of the panoramic image display where the ceiling and floor are the clearest, while the main image is the most blurry. To solve the inefficient encoding problem of high ineffective information density and low effective information density in related technologies when displaying panoramic images, this invention rotates the key display area of ​​the panoramic image to the high-latitude region where the North and South Poles are located before unfolding the panoramic image into a planar image using projection technology. This allows the main stage area to occupy a large screen coverage on the display device after the panoramic image is unfolded into a planar image and encoded, while the ceiling, floor, and sides of the stage occupy a smaller screen coverage.

[0048] Figure 1 A flowchart of a panoramic image encoding method according to an embodiment of the present invention is shown, which is executed by an electronic device. The memory of the electronic device is used to store at least one executable instruction that causes the processor of the electronic device to perform the operation of the panoramic image encoding method described above.

[0049] like Figure 1 As shown, the method includes the following steps:

[0050] Step 110: Determine the key display positions of the panoramic image.

[0051] In a panoramic view, the key display position is the center of the key display area. The key display area is the main display area of ​​the panoramic view; for example, in a stage panoramic view, the key display area is the main stage area, while the ceiling, floor, and surrounding areas are not key display areas. Furthermore, the key display position can be automatically identified or manually specified. Before determining the key display position, virtual reality cameras need to be deployed in appropriate locations on the shooting site. First, multiple images from different directions are captured by the virtual display cameras. Then, rendering technology is used to stitch and overlay these multiple images from different directions to create a panoramic view in the form of a dome screen. A virtual display camera can include multiple lenses, each used to capture images from different directions. For example, a virtual display camera can include 4 to 8 lenses. Furthermore, a key display position calibration function can be added to the shooting software of the virtual reality camera. Calibration lines for key display positions can be set and pointed to the main shooting area, such as the competition venue, performance stage, or main scenic area of ​​a scenic spot. The shooting software, combined with the initial three-dimensional coordinates, automatically records the three-dimensional coordinates M(x, y, z) of the key display position. These three-dimensional coordinates M(x, y, z) are the key display positions of the panoramic image.

[0052] Before determining the key display positions of the panoramic image, a virtual reality rendering server or rendering chip can be used to conventionally unfold the panoramic image along the meridians to obtain a conventional planar image. This conventional planar image is used for image adjustment, composition reference, and preview monitoring.

[0053] Step 120: Rotate the panoramic image to bring the key display position to the extreme point of the panoramic image.

[0054] Specifically, when rotating the panoramic image, the rotation axis, rotation angle, and rotation direction can be determined based on the key display position. The panoramic image is then rotated according to these parameters to bring the key display position to the extreme point of the panoramic image. Furthermore, the plane containing the key display position, the center of the panoramic image, and the extreme point of the panoramic image can be defined as the rotation plane of the panoramic image, and the axis perpendicular to the rotation plane and passing through the center of the panoramic image can be defined as the rotation axis. For example, if the 3D coordinates corresponding to the key display position are M(x, y, z), when the 3D coordinates M(x, y, z) are located in the first or eighth quadrant, the plane of rotation is a plane passing through the center of the sphere, orthogonal to the xoy plane, and with a direction offset by an angle θ from the positive x-axis to the positive y-axis in the xoy plane; when the 3D coordinates M(x, y, z) are located in the second or seventh quadrant, the plane of rotation is a plane passing through the center of the sphere, orthogonal to the xoy plane, and with a direction offset by an angle θ from the positive x-axis to the positive y-axis in the xoy plane; when the 3D coordinates M(x, y, z) are located in the third or sixth quadrant, the plane of rotation is a plane passing through the center of the sphere, orthogonal to the xoy plane, and with a direction offset by an angle θ from the negative x-axis to the negative y-axis in the xoy plane; when the 3D coordinates M(x, y, z) are located in the fourth or fifth quadrant, the plane of rotation is a plane passing through the center of the sphere, orthogonal to the xoy plane, and with a direction offset by an angle θ from the positive x-axis to the negative y-axis in the xoy plane; θ can be determined by the following formula:

[0055]

[0056] Figure 2 This diagram illustrates the rotation of a panoramic view provided in an embodiment of the present invention. Please refer to [link / reference needed]. Figure 2 The key display position's three-dimensional coordinates M(x, y, z) are located in the first quadrant (x>0, y>0, z>0). The rotation plane is the gray plane marked in the figure, which is the plane where the three-dimensional coordinates M(x, y, z) are located. The included angle θ is shown in the figure. The rotation axis is a straight line passing through the center of the sphere and orthogonal to the rotation plane.

[0057] In determining the rotation direction of the panoramic image, the rotation arc corresponding to the rotation plane can be determined first. The rotation arc is the circular arc on the sphere of the panoramic image that intersects with the rotation plane. Then, two directed arcs on the rotation arc pointing from the key display position to the extreme position can be determined. The shorter of the two directed spherical arcs can be further determined, and the direction of the shorter directed arc can be determined as the rotation direction of the panoramic image.

[0058] In a preferred embodiment, both directed arcs point towards the North Pole, therefore the shorter directed arc also points towards the North Pole. When determining the rotation angle for the panoramic view, first determine the spherical coordinates (x, y, z) of the key display position; if z > 0, meaning the three-dimensional coordinates M(x, y, z) of the key display position are located in any of the first, second, third, and fourth quadrants, then the rotation angle of the panoramic view is... Determined as If z < 0, meaning the 3D coordinates M(x, y, z) of the key display position are located in any of the fifth, sixth, seventh, or eighth quadrants, then the rotation angle of the panoramic image will be adjusted. Determined as Furthermore, after rotating the key display position of the panoramic image to the extreme position of the panoramic image according to the determined rotation direction, rotation axis and rotation angle, the rotation information of the panoramic image can be saved. This rotation information can include the rotation direction, rotation axis and rotation angle.

[0059] Step 130: Unfold the rotated panoramic image into a planar image, and encode the planar image to generate the encoded result of the panoramic image.

[0060] One method is to unfold the rotated panoramic image into a planar image using projection techniques such as equidistant cylindrical projection or positive-axis isometric cylindrical projection. For example, when unfolding the rotated panoramic image into a planar image using equidistant cylindrical projection, the longitude (w) and latitude (h) of the panoramic image can be equally divided to obtain a spherical grid. Then, a right-angle transformation is used to convert the spherical grid into a planar grid, thus obtaining the planar image. Taking a virtual reality camera set up between the stage and the audience seating as an example, after unfolding the rotated panoramic image into a planar image, the stage area and the audience area occupy a very large area on both the north and south poles of the planar image, the ceiling and floor occupy a smaller area near the equator, and the remaining part is filled with annular areas of low value density on both sides of the audience's field of vision when facing the stage.

[0061] High-resolution, high-complexity, and high-bitrate encoding methods can be used to encode and encapsulate the planar image. After encoding the planar image to generate the encoded panoramic image, the rotation information of the panoramic image can be obtained. The rotation information includes the rotation axis, rotation direction, and rotation angle of the panoramic image. This rotation information is then added as supplementary enhancement information to the encoded result using SEI (Supplemental Enhancement Information) technology. The encoded planar image is then transmitted to the display device for transcoding and playback. Alternatively, lower-resolution, lower-complexity, and lower-bitrate encoding methods can be used to encapsulate the aforementioned conventional planar image, and the encoded result of the conventional planar image is transmitted to the display device for monitoring and playback.

[0062] In this embodiment of the invention, the region where the extreme point of the panoramic image is located is the pixel concentration area when the panoramic image is displayed. However, the extreme point of the panoramic image is often not located in the key display area of ​​the panoramic image, affecting the display effect of the key display area. When encoding the panoramic image, the key display position of the panoramic image is first determined, and then the panoramic image is rotated based on the key display position to rotate the key display position to the extreme point of the panoramic image. Finally, the rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image. In the above embodiment, by rotating the key display position of the panoramic image to the extreme point, the key display area of ​​the panoramic image can have a better display effect.

[0063] Figure 3 A schematic diagram of the panoramic image encoding device according to an embodiment of the present invention is shown. Figure 3 As shown, the device 200 includes: a determining module 210, a rotating module 220, and a generating module 230.

[0064] The determining module 210 is used to determine the key display position of the panoramic image; the rotating module 220 is used to rotate the panoramic image to rotate the key display position to the extreme position of the panoramic image; and the generating module 230 is used to unfold the rotated panoramic image into a planar image and encode the planar image to generate the encoded result of the panoramic image.

[0065] In one alternative embodiment, the rotating module 220 is used for:

[0066] The rotation axis, rotation angle, and rotation direction for rotating the panoramic image are determined based on the key display position.

[0067] The panoramic image is rotated according to the rotation axis, the rotation angle, and the rotation direction to rotate the key display position to the extreme position of the panoramic image.

[0068] In one alternative embodiment, the rotating module 220 is used for:

[0069] The plane containing the key display position, the center of the panoramic image, and the north pole of the panoramic image is defined as the rotation plane of the panoramic image.

[0070] The axis perpendicular to the plane of rotation and passing through the center of the sphere in the panoramic view is defined as the axis of rotation.

[0071] In one alternative embodiment, the rotating module 220 is used for:

[0072] Determine the rotation arc corresponding to the rotation plane, wherein the rotation arc is a circular arc on the sphere of the panoramic image that intersects with the rotation plane;

[0073] Identify two directed arcs on the rotation arc that point from the key display position to the pole position;

[0074] Determine the shorter of the two directed spherical arcs;

[0075] The direction of the shorter directional arc is determined as the rotation direction.

[0076] In one alternative embodiment, the rotating module 320 is used for:

[0077] Determine the spherical coordinates (x, y, z) of the key display position;

[0078] If z > 0, then the rotation angle of the panoramic image is determined as follows:

[0079] If z < 0, then the rotation angle of the panoramic image is determined as follows:

[0080] In an alternative embodiment, the device 200 further includes an adding module for obtaining rotation information of the panoramic image after encoding the planar image to generate the panoramic image, wherein the rotation information includes the rotation axis, rotation direction, and rotation angle of the panoramic image.

[0081] The rotation information is added as additional enhancement information to the encoding result.

[0082] In this embodiment of the invention, the region where the extreme point of the panoramic image is located is the pixel concentration area when the panoramic image is displayed. However, the extreme point of the panoramic image is often not located in the key display area of ​​the panoramic image, affecting the display effect of the key display area. When encoding the panoramic image, the key display position of the panoramic image is first determined, and then the panoramic image is rotated based on the key display position to rotate the key display position to the extreme point of the panoramic image. Finally, the rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image. In the above embodiment, by rotating the key display position of the panoramic image to the extreme point, the key display area of ​​the panoramic image can have a better display effect.

[0083] Figure 4 A schematic diagram of the panoramic image display system according to an embodiment of the present invention is shown. Figure 4 As shown, the system 300 includes the encoding device 310 and the display device 320 in the above embodiments.

[0084] The encoding device 310 is used to send the encoding result of the panoramic image to the display device 320, and the display device 320 is used to decode the encoding result of the panoramic image and display the decoded panoramic image.

[0085] Furthermore, after decoding the encoded result of the panoramic image, the display device 320 can restore the decoded planar image to a panoramic image using inverse projection techniques such as inverse equidistant cylindrical projection. The rotation axis, direction, and angle of the panoramic image rotation by the encoding device 310 are read from the decoded additional enhancement information. The restored panoramic image is then rotated in reverse, returning the key display positions to their initial positions, and the inversely rotated panoramic image is then played and displayed.

[0086] Figure 5 This diagram illustrates the display effect of a panoramic image provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 5 When the panoramic image provided in this embodiment of the invention is displayed, the main display areas such as the stage and audience area occupy a larger display area at the North and South Poles, while the non-main display areas such as the ceiling, floor, and sides of the stage occupy a smaller display area near the equator.

[0087] In this embodiment of the invention, the encoding device of the panoramic image display system can rotate the key display position of the panoramic image to the extreme position of the panoramic image, unfold the rotated panoramic image into a planar image, encode the planar image to generate the encoding result of the panoramic image, and send the encoding result to the display device. The display device can decode the encoding result sent by the encoding device and display the decoded panoramic image. Through the above method, the effective display area occupies a large display area in the panoramic image displayed by the display device, while the ineffective display area occupies a small display area. This embodiment of the invention can significantly improve the display quality of the panoramic image and enhance its viewing experience without changing the encoding consumption.

[0088] Figure 6 The diagram shows a schematic of the electronic device structure according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0089] like Figure 6 As shown, the electronic device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0090] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements, such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps described above in the panoramic image encoding method embodiment.

[0091] Specifically, program 410 may include program code, which includes computer-executable instructions.

[0092] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0093] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0094] Specifically, program 410 can be called by processor 402 to cause the electronic device to perform the following operations:

[0095] Determine the key display positions of the panoramic image;

[0096] The panoramic view is rotated to bring the key display position to the extreme point of the panoramic view.

[0097] The rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image.

[0098] In an alternative manner, the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations:

[0099] The rotation axis, rotation angle, and rotation direction for rotating the panoramic image are determined based on the key display position.

[0100] The panoramic image is rotated according to the rotation axis, the rotation angle, and the rotation direction to rotate the key display position to the extreme position of the panoramic image.

[0101] In an alternative manner, the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations:

[0102] The plane containing the key display position, the center of the panoramic image, and the north pole of the panoramic image is defined as the rotation plane of the panoramic image.

[0103] The axis perpendicular to the plane of rotation and passing through the center of the sphere in the panoramic view is defined as the axis of rotation.

[0104] In an alternative manner, the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations:

[0105] Determine the rotation arc corresponding to the rotation plane, wherein the rotation arc is a circular arc on the sphere of the panoramic image that intersects with the rotation plane;

[0106] Identify two directed arcs on the rotation arc that point from the key display position to the pole position;

[0107] Determine the shorter of the two directed spherical arcs;

[0108] The direction of the shorter directional arc is determined as the rotation direction.

[0109] In an alternative approach, the two directed arcs point to the North Pole location, and the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations to determine the spherical coordinates (x, y, z) of the key display location;

[0110] If z > 0, then the rotation angle of the panoramic image is determined as follows:

[0111] If z < 0, then the rotation angle of the panoramic image is determined as follows:

[0112] In an alternative manner, the program 410 is invoked by the processor 402 to cause the electronic device to perform the following operations: after encoding the planar image to generate the panoramic image, obtaining the rotation information of the panoramic image, wherein the rotation information includes the rotation axis, rotation direction and rotation angle of the panoramic image;

[0113] The rotation information is added as additional enhancement information to the encoding result.

[0114] In this embodiment of the invention, the region where the extreme point of the panoramic image is located is the pixel concentration area when the panoramic image is displayed. However, the extreme point of the panoramic image is often not located in the key display area of ​​the panoramic image, affecting the display effect of the key display area. When encoding the panoramic image, the key display position of the panoramic image is first determined, and then the panoramic image is rotated based on the key display position to rotate the key display position to the extreme point of the panoramic image. Finally, the rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image. In the above embodiment, by rotating the key display position of the panoramic image to the extreme point, the key display area of ​​the panoramic image can have a better display effect.

[0115] This invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on an electronic device, causes the electronic device to perform the panoramic image encoding method in any of the above method embodiments.

[0116] This invention provides a panoramic image encoding device for performing the above-described panoramic image encoding method.

[0117] This invention provides a computer program that can be called by a processor to enable an electronic device to execute the panoramic image encoding method in any of the above method embodiments.

[0118] This invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed on a computer, cause the computer to perform the panoramic image encoding method in any of the above method embodiments.

[0119] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0120] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0121] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0122] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0123] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A panoramic image encoding method, characterized in that, The method includes: Determine the key display position of the panoramic image; wherein, the key display position of the panoramic image is the center position of the key display area of ​​the panoramic image; The rotation axis, rotation angle, and rotation direction for rotating the panoramic image are determined based on the key display position. The panoramic image is rotated according to the rotation axis, rotation angle, and rotation direction to rotate the key display position to the extreme point of the panoramic image. The rotation plane of the panoramic image is determined by the plane containing the key display position, the center of the panoramic image, and the north pole of the panoramic image. The rotation axis is an axis perpendicular to the rotation plane and passing through the center of the panoramic image. When determining the rotation direction of the panoramic image, the rotation arc corresponding to the rotation plane is determined; the rotation arc is an arc on the sphere of the panoramic image that intersects the rotation plane. Two directed arcs on the rotation arc pointing from the key display position to the extreme point are determined. The shorter of the two directed arcs is determined. The direction of the shorter directed arc is determined as the rotation direction. The rotated panoramic image is unfolded into a planar image, and the planar image is encoded to generate the encoded result of the panoramic image.

2. The method according to claim 1, characterized in that, The two directed arcs point towards the North Pole, and the method further includes: Determine the spherical coordinates (x, y, z) of the key display position; If z > 0, then the rotation angle of the panoramic image is set to 90°. ; If z < 0, then the rotation angle of the panoramic image is determined to be 90° + 0. .

3. The method according to claim 1, characterized in that, After encoding the planar image to generate the encoded panoramic image, the method further includes: Obtain the rotation information of the panoramic image, wherein the rotation information includes the rotation axis, rotation direction, and rotation angle of the panoramic image; The rotation information is added as additional enhancement information to the encoding result.

4. A panoramic image encoding device, characterized in that, The device includes: The determining module is used to determine the key display position of the panoramic image; wherein, the key display position of the panoramic image is the center position of the key display area of ​​the panoramic image; A rotation module is used to determine the rotation axis, rotation angle, and rotation direction for rotating the panoramic image based on the key display position; to rotate the panoramic image according to the rotation axis, rotation angle, and rotation direction to rotate the key display position to the extreme point of the panoramic image; wherein, the rotation plane of the panoramic image is determined by the plane containing the key display position, the center of the panoramic image, and the north pole of the panoramic image; the rotation axis is an axis perpendicular to the rotation plane and passing through the center of the panoramic image; when determining the rotation direction of the panoramic image, the rotation arc corresponding to the rotation plane is determined, the rotation arc being an arc on the sphere of the panoramic image that intersects the rotation plane; two directed arcs on the rotation arc pointing from the key display position to the extreme point are determined; the shorter of the two directed arcs is determined; and the direction of the shorter directed arc is determined as the rotation direction; The generation module is used to unfold the rotated panoramic image into a planar image, and encode the planar image to generate the encoded result of the panoramic image.

5. A panoramic image display system, characterized in that, The system includes a display device and an encoding device as described in claim 4: The encoding device is used to send the encoding result of the panoramic image to the display device, and the display device is used to decode the encoding result of the panoramic image and display the decoded panoramic image.

6. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation of the panoramic image encoding method as described in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on the electronic device, causes the electronic device to perform the panoramic image encoding method as described in any one of claims 1-3.

Images