Panoramic video compression method and device, computer device and storage medium

Abstract

The application relates to a panoramic video compression method and device, computer equipment and a storage medium. The computer equipment obtains first position information of a center point of a viewing angle of a current frame panoramic image in the panoramic image, and second position information of the center point of the panoramic image; determines a first mapping relationship according to the first position information and the second position information; converts the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image with the center point of the viewing angle as the center; and compresses the intermediate panoramic image to obtain a target panoramic image. In the embodiment, the current frame panoramic image is converted to ensure that the center point of the current viewing angle of the viewer is located at the center of the panoramic image when the transmission bandwidth of the panoramic video is reduced, and the converted panoramic image is compressed on this basis to reduce the transmission bandwidth of the current frame panoramic image, thereby greatly improving the definition of the compressed panoramic video.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of panoramic video compression, and in particular to a panoramic video compression method and device, a computer device and a storage medium. BACKGROUND

[0002] With the development of panoramic video technology, the application of panoramic video is more and more extensive, such as live broadcast, remote conference, etc. When a scene is presented by using panoramic video, the panoramic spherical video is usually projected into a planar video, and then the scene is presented in the form of the planar video.

[0003] In the prior art, in the process of saving or transmitting the panoramic video, a special projection method is used to project the panoramic spherical video into a planar video. In general, when the panoramic video is transmitted, a complete panoramic video picture is transmitted, but the viewer can only view a picture of one viewing angle at the same time. Therefore, when it is necessary to ensure that the picture of the viewing angle is of high definition, a panoramic video of higher resolution needs to be transmitted. However, under the limitation of the transmission bandwidth of the panoramic video, for the collected high-definition panoramic video, the original high-definition panoramic video needs to be transmitted after reducing the overall transmission bandwidth by down-sampling, which will result in a decrease in the definition of the picture of the viewing angle viewed by the viewer. SUMMARY

[0004] Therefore, it is necessary to provide a panoramic video compression method, device, computer device and storage medium which can reduce the transmission bandwidth of the panoramic video while improving the definition of the picture of the viewing angle currently viewed by the viewer.

[0005] In a first aspect, a panoramic video compression method is provided, which comprises:

[0006] obtaining first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image, and second position information of a center point of the panoramic image;

[0007] determining a first mapping relationship according to the first position information and the second position information;

[0008] converting the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing angle;

[0009] compressing the intermediate panoramic image to obtain a target panoramic image.

[0010] In one embodiment, the intermediate panoramic image is compressed to obtain a target panoramic image, which comprises:

[0011] According to the preset second mapping relationship, the intermediate panoramic image is compressed to obtain a compressed panoramic image.

[0012] The target panoramic image is determined according to the compressed panoramic image.

[0013] In one embodiment, according to the preset second mapping relationship, the intermediate panoramic image is compressed to obtain a compressed panoramic image, including:

[0014] According to the preset second mapping relationship, the intermediate panoramic image is compressed along the horizontal direction to obtain a compressed panoramic image; the height of the compressed panoramic image is equal to the height of the intermediate panoramic image, and the width of the compressed panoramic image is half of the width of the intermediate panoramic image.

[0015] In one embodiment, the distance between a pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image.

[0016] In one embodiment, when the compressed panoramic image is a rhombus, the target panoramic image is determined according to the compressed panoramic image, including:

[0017] The compressed panoramic image is divided along the horizontal center line and the vertical center line to obtain four divided images;

[0018] The four divided images are recombined to obtain the target panoramic image.

[0019] In one embodiment, the four divided images are recombined to obtain the target panoramic image, including:

[0020] The adjacent two divided images are respectively rotated according to the preset rotation direction and rotation angle to obtain the target panoramic image.

[0021] In one embodiment, the second mapping relationship is a mapping relationship constructed according to a preset S-shaped curve, and the S-shaped curve satisfies the following conditions:

[0022] The S-shaped curve passes through three fixed points (-1, -1), (0, 0), and (+1, +1);

[0023] The S-shaped curve is centrally symmetric with respect to the origin;

[0024] The slope of the S-shaped curve at the origin is maximum.

[0025] In one of the embodiments, the first position information comprises first position coordinates of a center point of the viewing perspective in the panoramic image, and the second position information comprises second position coordinates of a center point of the panoramic image; a first mapping relationship is determined according to the first position information and the second position information, comprising:

[0026] The first position coordinates are projected into a coordinate system in which a spherical image corresponding to the panoramic image is located, to obtain third position coordinates;

[0027] The second position coordinates are projected into the coordinate system in which the spherical image is located, to obtain fourth position coordinates;

[0028] The first mapping relationship is determined according to the third position coordinates and the fourth position coordinates.

[0029] In a second aspect, a panoramic video compression device is provided, comprising:

[0030] An acquisition module is configured to acquire first position information of a center point of a viewing perspective in a current frame panoramic image of a panoramic video, and second position information of a center point of the panoramic image;

[0031] A first determination module is configured to determine a first mapping relationship according to the first position information and the second position information;

[0032] A second determination module is configured to convert the panoramic image according to the first mapping relationship, to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing perspective;

[0033] A third determination module is configured to compress the intermediate panoramic image, to obtain a target panoramic image.

[0034] In a third aspect, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0035] First position information of a center point of a viewing perspective in a current frame panoramic image of a panoramic video, and second position information of a center point of the panoramic image are acquired;

[0036] A first mapping relationship is determined according to the first position information and the second position information;

[0037] The panoramic image is converted according to the first mapping relationship, to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing perspective;

[0038] The intermediate panoramic image is compressed, to obtain a target panoramic image.

[0039] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0040] obtaining first position information of a center point of a viewing angle in a current frame panoramic image of the panoramic video in the panoramic image, and second position information of the center point of the panoramic image;

[0041] determining a first mapping relationship according to the first position information and the second position information;

[0042] converting the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing angle;

[0043] compressing the intermediate panoramic image to obtain a target panoramic image.

[0044] The panoramic video compression method, device, computer device and storage medium described above, the computer device obtains first position information of a center point of a viewing angle in a current frame panoramic image of the panoramic video in the panoramic image, and second position information of the center point of the panoramic image; and determines a first mapping relationship according to the first position information and the second position information; and then converts the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image centered on the center point of the viewing angle; and then compresses the intermediate panoramic image to obtain a target panoramic image; that is, in the embodiment of the present application, the computer device reduces the transmission bandwidth of the panoramic video by first converting the obtained current frame panoramic image to ensure that the center point of the current viewing angle of the viewer is located at the center of the panoramic image; and on this basis, the converted panoramic image is compressed to reduce the transmission bandwidth of the current frame panoramic image; therefore, the panoramic image processed according to the process can not only reduce the transmission bandwidth of the panoramic video in the transmission or storage process, but also ensure the clarity of the picture corresponding to the current viewing angle of the viewer, greatly improving the clarity of the compressed panoramic video, and improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 An application environment diagram of the panoramic video compression method in one embodiment;

[0046] Figure 2 A flowchart of the panoramic video compression method in one embodiment;

[0047] Figure 3 A structure diagram of the current frame panoramic image in one embodiment;

[0048] Figure 4 A flowchart of the panoramic video compression method in another embodiment;

[0049] Figure 5 a flowchart of a panoramic video compression method in another embodiment;

[0050] Figure 6 a structural diagram of a panoramic image segmentation and reorganization in an embodiment;

[0051] Figure 7 a structural diagram of an S-shaped curve in an embodiment;

[0052] Figure 8 a flowchart of a panoramic video compression method in another embodiment;

[0053] Figure 9 a structural block diagram of a panoramic video compression device in an embodiment;

[0054] Figure 10 an internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0055] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0056] The panoramic video compression method provided by the present application can be applied to a computer device as shown in Figure 1 , which can be but is not limited to any type of terminal with video function, such as a general camera, a panoramic camera, a projector, a personal computer, a notebook computer, a virtual reality (VR) acquisition device, etc. The internal structural diagram of the computer device is as shown in Figure 1 , which includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus.

[0057] In an embodiment, as shown in Figure 2 , a panoramic video compression method is provided, which is described below by taking the computer device in Figure 1 as an example, including the following steps:

[0058] Step 201, obtaining first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image, and second position information of the center point of the panoramic image.

[0059] In the process of transmission or storage, the panoramic video usually needs to be converted into a form of video frames, each frame panoramic image can be a panoramic picture formed by using any form of projection, for example: cylindrical projection, cube projection, strip projection, etc. In the embodiment of the present application, the panoramic video needs to be converted into a rectangular picture using cylindrical projection. In the case where the panoramic video is projected using a projection mode other than cylindrical projection, the panoramic picture under the other projection mode needs to be converted into a rectangular picture under cylindrical projection.

[0060] In an optional implementation of the embodiment, when compressing each frame panoramic image respectively, the panoramic image of the current frame of the panoramic video can be acquired first, and the first position information of the center point of the viewing angle of the viewer in the current frame in the panoramic image of the current frame and the second position information of the center point of the panoramic image of the current frame can be acquired. The first position information of the center point of the viewing angle of the current frame can be acquired from the computer device currently used by the viewer, and the second position information of the center point of the panoramic image of the current frame can be determined according to the width and height of the panoramic image of the current frame. For example, as shown in FIG. 2, the panoramic image of the current frame is A, the center point of the viewing angle of the current frame can be point c in the panoramic image, and the center point of the panoramic image of the current frame can be point q in the panoramic image. The first position information of the center point c of the viewing angle can include the coordinates corresponding to the c point on the two-dimensional plane, and the second position information of the center point q of the panoramic image can include the coordinates corresponding to the q point on the two-dimensional plane, which can be represented as (W / 2, H / 2), where W is the width of the panoramic image of the current frame, and H is the height of the panoramic image of the current frame. Figure 3 Figure 3 Figure 3

[0061] Step 202, determining a first mapping relationship according to the first position information and the second position information.

[0062] In order to ensure that the clarity of the picture corresponding to the viewing angle of the viewer is maintained in the process of compression, after the center point of the viewing angle of the viewer in the current frame panoramic image is acquired, if it is determined that the center point of the viewing angle is not the center point of the current frame panoramic image, the current frame panoramic image can be converted to obtain a panoramic image with the center point of the viewing angle as the center.

[0063] ​​​Optionally, after obtaining the first position information of the center point of the viewing angle of the current frame in the panoramic image and the second position information of the center point of the panoramic image, a first mapping relationship can be determined according to the first position information and the second position information; for example, the first position can be translated in the horizontal direction and in the vertical direction to reach the second position according to the first position information and the second position information to determine the first mapping relationship; the method for determining the first mapping relationship is not limited in the embodiment, and the mapping between the first position information and the second position information can be determined according to the first mapping relationship.

[0064] In step 203, the panoramic image is converted according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing angle.

[0065] After the first mapping relationship is obtained according to the first position information and the second position information, the panoramic image can be converted according to the first mapping relationship, that is, each pixel position information of the panoramic image is converted according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is the panoramic image centered on the center point of the viewing angle of the current frame panoramic image.

[0066] In step 204, the intermediate panoramic image is compressed to obtain a target panoramic image.

[0067] In an optional implementation of the embodiment, after the intermediate panoramic image centered on the center point of the viewing angle of the current frame panoramic image is obtained, the intermediate panoramic image can be compressed to obtain a target panoramic image; the target panoramic image is the current frame panoramic image compressed to reduce the transmission bandwidth of the current frame panoramic image under the premise of maintaining the clarity of the center point of the viewing angle. Optionally, the intermediate panoramic image can be compressed by using the same compression rate when the intermediate panoramic image is compressed; different compression rates can be used to compress the intermediate panoramic image according to different positions of the intermediate panoramic image; the picture within a certain range of the center point of the viewing angle can not be compressed, and the picture outside the certain range of the center point of the viewing angle can be compressed; the compression method is not limited in the embodiment.

[0068] In the panoramic video compression method, the computer device obtains first position information of a center point of a viewing angle of a current frame of the panoramic video in a panoramic image and second position information of a center point of the panoramic image, determines a first mapping relationship according to the first position information and the second position information, converts the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image with the center point of the viewing angle as a center, and compresses the intermediate panoramic image to obtain a target panoramic image. That is, in the embodiment, the computer device first converts the obtained current frame of the panoramic image to ensure that the center point of the current viewing angle of the viewer is located at the center of the panoramic image when reducing the transmission bandwidth of the panoramic video, and then compresses the converted panoramic image to reduce the transmission bandwidth of the current frame of the panoramic image. Therefore, the panoramic image processed according to the process can not only reduce the transmission bandwidth of the panoramic video in the transmission or storage process, but also ensure the definition of the corresponding picture of the current viewing angle of the viewer, greatly improve the definition of the compressed panoramic video, and improve the user experience.

[0069] Figure 4 A flowchart of a panoramic video compression method in another embodiment is shown. The embodiment relates to an optional implementation process of compressing the intermediate panoramic image to obtain a target panoramic image. As shown in Figure 4 the above embodiment, step 204 includes:

[0070] Step 401: compressing the intermediate panoramic image according to a preset second mapping relationship to obtain a compressed panoramic image.

[0071] In an optional implementation of the embodiment, when compressing the intermediate panoramic image, the intermediate panoramic image can be compressed in the horizontal direction, or in the vertical direction, or in both the horizontal and vertical directions, or in any at least one direction in a two-dimensional plane. The embodiment does not limit the compression direction. In addition, when the intermediate panoramic image is compressed in different directions, the preset second mapping relationship is different, and the compressed panoramic image obtained is naturally different. Optionally, when the intermediate panoramic image is compressed in the horizontal direction according to the preset second mapping relationship, the compressed panoramic image obtained has a different width and the same height as the intermediate panoramic image. When the intermediate panoramic image is compressed in the vertical direction according to the preset second mapping relationship, the compressed panoramic image obtained has the same width and a different height as the intermediate panoramic image. When the intermediate panoramic image is compressed in both the horizontal and vertical directions according to the preset second mapping relationship, the compressed panoramic image obtained has different width and height as the intermediate panoramic image. Optionally, the preset second mapping relationship can be a mapping relationship between the intermediate panoramic image and the compressed panoramic image corresponding to a same compression rate, or a mapping relationship between the intermediate panoramic image and the compressed panoramic image corresponding to different compression rates. According to the preset second mapping relationship, the compressed panoramic image obtained can be rectangular, square, circular, or the like. The embodiment does not limit the preset second mapping relationship.

[0072] At step 402, the target panoramic image is determined according to the compressed panoramic image.

[0073] After obtaining the compressed panoramic image, the target panoramic image can be determined according to the compressed panoramic image. Optionally, when the compressed panoramic image is rectangular or square, the compressed panoramic image can be determined as the target panoramic image. When the compressed panoramic image is in any other shape except rectangular or square, the compressed panoramic image can be transformed to obtain a panoramic image in a rectangular or square shape as the target panoramic image. The embodiment does not limit the manner of converting the compressed panoramic image in any other shape except rectangular or square to the target panoramic image in a rectangular or square shape, as long as the target panoramic image in a rectangular or square shape can be obtained.

[0074] In this embodiment, the computer device compresses the intermediate panoramic image according to the preset second mapping relationship to obtain a compressed panoramic image, and determines the target panoramic image according to the compressed panoramic image. That is, in this embodiment, after obtaining the intermediate panoramic image centered on the center point of the viewing angle, the computer device can compress the intermediate panoramic image in the horizontal direction according to the preset second mapping relationship to obtain a compressed panoramic image, so that the height of the compressed panoramic image is consistent with the height of the intermediate panoramic image, and the width of the compressed panoramic image is smaller than the width of the intermediate panoramic image, so as to reduce the transmission bandwidth in the transmission or storage process of the panoramic video, and avoid waste of transmission bandwidth.

[0075] In an optional embodiment of the present application, when the intermediate panoramic image is compressed, the intermediate panoramic image is compressed in the horizontal direction according to the preset second mapping relationship to obtain a compressed panoramic image, the height of the compressed panoramic image is equal to the height of the intermediate panoramic image, and the width of the compressed panoramic image is half of the width of the intermediate panoramic image. That is, in this embodiment, after the intermediate panoramic image is compressed in the horizontal direction according to the preset second mapping relationship, the transmission bandwidth of the compressed panoramic image obtained is half of the transmission bandwidth of the intermediate panoramic image, greatly reducing the transmission bandwidth in the transmission or storage process of the panoramic video.

[0076] In an optional embodiment of the present application, the distance between the pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image. That is, when the computer device compresses the intermediate panoramic image, the compression rate of the pixel point closer to the center of the intermediate panoramic image is lower, and the compression rate of the pixel point closer to the edge of the intermediate panoramic image is higher, that is, according to the relationship that the distance between the pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image, the clarity of the viewing angle center frame can be improved to a greater extent, and the clarity of the frame outside the viewing angle is reduced.

[0077] Figure 5 The flowchart of the panoramic video compression method in another embodiment is shown. In this embodiment, when the compressed panoramic image obtained in step 401 is a rhombus, an optional implementation process for determining the target panoramic image according to the compressed panoramic image is provided. Based on the above embodiment, as shown in Figure 5 , the step 402 includes:

[0078] Step 501, the compressed panoramic image is divided along the horizontal center line and the vertical center line to obtain four divided images.

[0079] In one optional implementation of this embodiment, when the compressed panoramic image is rhomboid, the compressed panoramic image can be segmented along the horizontal and vertical center lines to obtain four segmented images; for example: Figure 6 As shown, when the height of the rhombus is the same as its width, dividing the rhombus along the horizontal and vertical center lines will yield four right triangles of equal size.

[0080] Step 502: Reassemble the four segmented images to obtain the panoramic image of the target.

[0081] In one optional implementation of this embodiment, the four segmented images are recombined to obtain the target panoramic image; alternatively, two adjacent segmented images can be rotated according to preset rotation directions and angles to obtain the target panoramic image; for example: Figure 6 As shown, the right-angled triangle A in the upper left corner can be rotated 90° counterclockwise around the upper vertex of the rhombus, and the right-angled triangle B in the lower left corner can be rotated 90° clockwise around the lower vertex of the rhombus. This reassembles the rhombus into a rectangle, resulting in a rectangular target panoramic image. The height of this target panoramic image is equal to the height of the intermediate panoramic image, and the width of the target panoramic image is one-quarter of the width of the intermediate panoramic image, thus compressing the image by 75%. This target panoramic image not only reduces its size to one-quarter of its original size but also maintains the clarity of the image near the center of the viewing angle. Although the image has been segmented and reassembled, the panoramic video compression method of this application utilizes the characteristics of cylindrical projection, ensuring that the images on both sides of the stitching point originate from the same object in the panoramic image. Therefore, the reassembled image has no visible stitching seams and also improves the efficiency of subsequent panoramic video encoding.

[0082] Optionally, when recombining the four segmented images, the upper left right triangle can be rotated 90° clockwise around the left vertex of the rhombus, and the upper right right triangle can be rotated 90° counterclockwise around the right vertex of the rhombus, so that the rhombus can be recombined into a rectangle, resulting in a target panoramic image with a rectangular shape; the height of the target panoramic image is half the height of the middle panoramic image, and the width of the target panoramic image is half the width of the middle panoramic image.

[0083] In this embodiment, when the compressed panoramic image is rhomboid, the computer device segments the compressed panoramic image along the horizontal and vertical center lines to obtain four segmented images, and then reassembles the four segmented images to obtain the target panoramic image. In other words, the computer device can segment and reassemble a rhomboid compressed panoramic image to obtain a rectangular target panoramic image, thereby satisfying the planar rectangular transmission condition of panoramic video during transmission or storage, which greatly improves the intelligence and scalability of the computer device.

[0084] In an optional embodiment of this application, the second mapping relationship can be a mapping relationship constructed based on a preset S-curve, and the S-curve satisfies the following conditions: the S-curve passes through three fixed points (-1, -1), (0, 0), and (+1, +1); the S-curve is symmetrical about the origin; and the slope of the S-curve is the largest at the origin. For example: assuming the second mapping relationship is f2: p′→p″, where: p′: (x, y) is a point on the intermediate panoramic image, p″: (x′, y) is a point on the compressed panoramic image, the vertical coordinate remains unchanged, and the calculation process of the horizontal coordinate x′ is as follows:

[0085]

[0086]

[0087] x2=(1-|y1|)·S(x1) (3)

[0088]

[0089] Where W is the width of the intermediate panoramic image, H is the height of the intermediate panoramic image, x1, y1, and x2 are intermediate variables, and S(·) is the S-curve function mentioned above. According to the above formulas (1)-(4), the coordinates of each point on the intermediate panoramic image converted to the coordinates of a point on the compressed panoramic image can be calculated. For the S-curve, optionally, as follows: Figure 7 As shown, the S-shaped curve can be a cubic Bézier curve determined by four nodes (-1,-1), (+0.25,-1), (-0.25,+1), and (+1,+1). Based on the S-shaped curve, a unique point (x,S(x)) can be found on the curve for any x∈[-1,1], and its ordinate is the value of S(x).

[0090] In the embodiment, the second mapping relationship can be a mapping relationship constructed according to a preset S-shaped curve. According to the requirement met by the preset S-shaped curve, the pixel closer to the center point of the intermediate panoramic image has a lower compression rate in the compression process, and the pixel closer to the edge point of the intermediate panoramic image has a higher compression rate, so that the definition of the picture near the center point of the compressed panoramic image is higher, and the effect of panoramic image compression is greatly improved, and the definition of the picture at the viewing angle center point is ensured.

[0091] The first position information can include a first position coordinate of the center point of the viewing angle in the panoramic image, and the second position information can include a second position coordinate of the center point of the panoramic image. When the first mapping relationship is determined according to the first position information and the second position information, the first mapping relationship can be determined according to the first position coordinate and the second position coordinate. Figure 8 The flowchart of the panoramic video compression method in another embodiment is shown. The embodiment relates to an optional implementation process of determining the first mapping relationship according to the first position information and the second position information. As shown in the above embodiment, Figure 8 The step 202 includes:

[0092] In step 801, the first position coordinate is projected into the coordinate system of the spherical image corresponding to the panoramic image to obtain a third position coordinate.

[0093] In step 802, the second position coordinate is projected into the coordinate system of the spherical image to obtain a fourth position coordinate.

[0094] According to the definition of cylindrical projection, the projected plane coordinate and the corresponding spherical coordinate can be one-to-one corresponding according to the third mapping relationship. It is assumed that the third mapping relationship can be represented as: wherein p is a plane coordinate, is a spherical coordinate; then, the third mapping relationship The first position coordinate is projected into the coordinate system of the spherical image corresponding to the panoramic image to obtain a third position coordinate, and the second position coordinate is projected into the coordinate system of the spherical image to obtain a fourth position coordinate. For example, the first position coordinate is c, and the third position coordinate is The second position coordinate is q, and the fourth position coordinate is

[0095] In step 803, the first mapping relationship is determined according to the third position coordinate and the fourth position coordinate.

[0096] After obtaining the coordinates of the center point of the viewing angle and the center point of the panoramic image, respectively, in their third and fourth position coordinates on the sphere, the computer device, in order to ensure that the center point of the viewing angle falls on the center point of the panoramic image (i.e., corresponding to the sphere), rotates and transforms the third position coordinates to the fourth position coordinates. Optionally, a three-dimensional rotation transformation R can be determined based on the third and fourth position coordinates. This three-dimensional rotation transformation R can be a rotation transformation matrix, such that the third position coordinates can be rotated and transformed to the fourth position coordinates according to the three-dimensional rotation transformation R, thus obtaining... Alternatively, the three-dimensional rotational transformation R can be obtained in the following way, i.e. Rotate it around the Z-axis so that it falls into the same direction as the Z-axis. The same vertical plane (i.e., the XZ plane), then rotated around the Y-axis, so that... and The two rotations are combined and transformed to obtain the three-dimensional rotation transformation R. This embodiment does not limit the method of obtaining the three-dimensional rotation transformation R.

[0097] On the surface of the sphere Rotation transformation Then, after an inverse transformation of the third mapping relationship, the coordinates on the sphere can be mapped back to the planar coordinates. That is, based on the three-dimensional rotation transformation R and the above-mentioned third mapping relationship, the first mapping relationship can be determined. Then, based on the first mapping relationship, the panoramic image of the current frame can be converted into an intermediate panoramic image centered on the center point of the viewing angle. Assume that the first mapping relationship is f1:p→p′, where p is any pixel coordinate on the panoramic image of the current frame, and p′ is any pixel coordinate on the corresponding intermediate panoramic image. p′ can be expressed by formula (5).

[0098]

[0099] In this embodiment, the computer device projects the first position coordinates onto the coordinate system of the spherical image corresponding to the panoramic image to obtain the third position coordinates, and projects the second position coordinates onto the coordinate system of the spherical image to obtain the fourth position coordinates. Then, based on the third and fourth position coordinates, the first mapping relationship is determined. That is, in this embodiment, the computer device maps the center point coordinates of the viewing angle on the planar panoramic image and the center point coordinates of the panoramic image onto the sphere, and based on the relationship between the two coordinates on the sphere, finally determines the first mapping relationship between the center point coordinates of the viewing angle on the planar panoramic image and the center point coordinates of the panoramic image. The rotation change on the sphere can make the first mapping relationship more accurate, thereby making the image clarity of the compressed intermediate panoramic image centered on the center point of the viewing angle higher.

[0100] It should be understood that, although Figures 2-8 the steps in the flowcharts are shown in sequential order, such that one step necessarily precedes another step, the steps are not necessarily performed in the order shown by the arrows. The execution of the steps is not necessarily limited to the order shown, unless specifically stated otherwise in this document. Moreover, Figures 2-8 at least some of the steps in the flowcharts can include multiple steps or multiple stages, which are not necessarily performed at the same time, but can be performed at different times. The execution of the steps or stages is not necessarily sequential, but can be performed in rotation or alternation with at least some of the other steps or stages.

[0101] In one embodiment, as shown in Figure 9 a panoramic video compression apparatus is provided, comprising: an obtaining module 901, a first determining module 902, a second determining module 903, and a third determining module 904, wherein:

[0102] The obtaining module 901 is configured to obtain first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image, and second position information of the center point of the panoramic image.

[0103] The first determining module 902 is configured to determine a first mapping relationship according to the first position information and the second position information.

[0104] The second determining module 903 is configured to convert the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is centered on the center point of the viewing angle.

[0105] The third determining module 904 is configured to compress the intermediate panoramic image to obtain a target panoramic image.

[0106] In one embodiment, the third determining module 904 is specifically configured to compress the intermediate panoramic image according to a preset second mapping relationship to obtain a compressed panoramic image; and determine the target panoramic image according to the compressed panoramic image.

[0107] In one embodiment, the third determining module 904 is specifically configured to compress the intermediate panoramic image in a horizontal direction according to a preset second mapping relationship to obtain a compressed panoramic image; the height of the compressed panoramic image is equal to the height of the intermediate panoramic image, and the width of the compressed panoramic image is half of the width of the intermediate panoramic image.

[0108] In one of the embodiments, the distance between the pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image.

[0109] In one of the embodiments, when the compressed panoramic image is a rhombus, the third determining module 904 is specifically configured to split the compressed panoramic image along a horizontal center line and a vertical center line to obtain four split images; and recombine the four split images to obtain the target panoramic image.

[0110] In one of the embodiments, the third determining module 904 is specifically configured to rotate two adjacent split images respectively according to a preset rotation direction and a rotation angle to obtain the target panoramic image.

[0111] In one of the embodiments, the second mapping relationship is a mapping relationship constructed according to a preset S-shaped curve, and the S-shaped curve satisfies the following conditions: the S-shaped curve passes through three fixed points (-1, -1), (0, 0) and (+1, +1); the S-shaped curve is centrally symmetric with respect to the origin; and the slope of the S-shaped curve at the origin is maximum.

[0112] In one of the embodiments, the first position information includes a first position coordinate of the center point of the viewing angle in the panoramic image, and the second position information includes a second position coordinate of the center point of the panoramic image; the first determining module 902 is specifically configured to project the first position coordinate into a coordinate system in which a spherical image corresponding to the panoramic image is located to obtain a third position coordinate; project the second position coordinate into the coordinate system in which the spherical image is located to obtain a fourth position coordinate; and determine the first mapping relationship according to the third position coordinate and the fourth position coordinate.

[0113] The specific limitations of the panoramic video compression device can be referred to the limitations of the panoramic video compression method in the foregoing, which will not be repeated here. Each module in the panoramic video compression device can be realized by software, hardware and combinations thereof in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above modules by the processor.

[0114] In one embodiment, a computer device is provided, which can be a terminal with video function, and the internal structure diagram thereof can be as shown in Figure 10As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless mode can be achieved through WIFI, operator network, NFC (Near Field Communication) or other technologies. The computer program is executed by the processor to implement a panoramic video compression method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0115] Those skilled in the art can understand that, Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0116] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0117] Obtain the first position information of the center point of the viewing angle of the current frame panoramic image of the panoramic video in the panoramic image, and the second position information of the center point of the panoramic image;

[0118] According to the first position information and the second position information, determine the first mapping relationship;

[0119] According to the first mapping relationship, the panoramic image is converted to obtain an intermediate panoramic image. The intermediate panoramic image is centered on the center point of the viewing angle;

[0120] The intermediate panoramic image is compressed to obtain a target panoramic image.

[0121] In one embodiment, the processor executing the computer program further implements the following steps: according to a preset second mapping relationship, the intermediate panoramic image is compressed to obtain a compressed panoramic image; and according to the compressed panoramic image, the target panoramic image is determined.

[0122] In one embodiment, the processor, when executing the computer program, also implements the following steps: compressing the intermediate panoramic image in a horizontal direction according to a preset second mapping relationship to obtain a compressed panoramic image; the compressed panoramic image has a same height as the intermediate panoramic image and a width that is half of the width of the intermediate panoramic image.

[0123] In one embodiment, the processor, when executing the computer program, also implements the following steps: the distance between a pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image.

[0124] In one embodiment, the processor, when executing the computer program, also implements the following steps: in a case where the compressed panoramic image is a rhombus, the compressed panoramic image is split along a horizontal center line and a vertical center line to obtain four split images; and the four split images are recombined to obtain the target panoramic image.

[0125] In one embodiment, the processor, when executing the computer program, also implements the following steps: the two adjacent split images are rotated according to preset rotation directions and rotation angles respectively to obtain the target panoramic image.

[0126] In one embodiment, the processor, when executing the computer program, also implements the following steps: the second mapping relationship is a mapping relationship constructed according to a preset S-shaped curve, and the S-shaped curve satisfies the following conditions: the S-shaped curve passes through three fixed points (-1, -1), (0, 0) and (+1, +1); the S-shaped curve is centrally symmetric with respect to the origin; and the slope of the S-shaped curve at the origin is maximum.

[0127] In one embodiment, the processor, when executing the computer program, also implements the following steps: the first position information includes first position coordinates of the center point of the viewing angle in the panoramic image, and the second position information includes second position coordinates of the center point of the panoramic image; the first position coordinates are projected into a coordinate system in which the spherical image corresponding to the panoramic image is located to obtain third position coordinates; the second position coordinates are projected into the coordinate system in which the spherical image is located to obtain fourth position coordinates; and the first mapping relationship is determined according to the third position coordinates and the fourth position coordinates.

[0128] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0129] obtain first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image and second position information of a center point of the panoramic image;

[0130] determine a first mapping relationship according to the first position information and the second position information;

[0131] convert the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image; the intermediate panoramic image is centered on a center point of the viewing angle;

[0132] compress the intermediate panoramic image to obtain a target panoramic image.

[0133] In one embodiment, the computer program, when executed by the processor, further implements the following steps: according to a preset second mapping relationship, compressing the intermediate panoramic image to obtain a compressed panoramic image; and determining the target panoramic image according to the compressed panoramic image.

[0134] In one embodiment, the computer program, when executed by the processor, further implements the following steps: according to a preset second mapping relationship, compressing the intermediate panoramic image along the horizontal direction to obtain a compressed panoramic image; the height of the compressed panoramic image is equal to the height of the intermediate panoramic image, and the width of the compressed panoramic image is half of the width of the intermediate panoramic image.

[0135] In one embodiment, the computer program, when executed by the processor, further implements the following steps: the distance between a pixel point in the intermediate panoramic image and the center pixel point of the intermediate panoramic image is positively correlated with the compression rate of the intermediate panoramic image.

[0136] In one embodiment, the computer program, when executed by the processor, further implements the following steps: in the case that the compressed panoramic image is a rhombus, the compressed panoramic image is divided along a horizontal center line and a vertical center line to obtain four divided images; and the four divided images are recombined to obtain the target panoramic image.

[0137] In one embodiment, the computer program, when executed by the processor, further implements the following steps: rotating two adjacent divided images according to a preset rotation direction and a rotation angle respectively to obtain the target panoramic image.

[0138] In one embodiment, the computer program, when executed by the processor, further implements the following steps: the second mapping relationship is a mapping relationship constructed according to a preset S-shaped curve, and the S-shaped curve satisfies the following conditions: the S-shaped curve passes through three fixed points (-1, -1), (0, 0), and (+1, +1); the S-shaped curve is centrally symmetric with respect to the origin; and the slope of the S-shaped curve at the origin is maximum.

[0139] In one embodiment, the computer program, when executed by the processor, further implements the following steps: the first position information comprises first position coordinates of a center point of the viewing perspective in the panoramic image, and the second position information comprises second position coordinates of a center point of the panoramic image; the first position coordinates are projected into a coordinate system in which the spherical image corresponding to the panoramic image is located to obtain third position coordinates; the second position coordinates are projected into the coordinate system in which the spherical image is located to obtain fourth position coordinates; and the first mapping relationship is determined according to the third position coordinates and the fourth position coordinates.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0141] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0142] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent of the present application should be subject to the appended claims.

Claims

1. A panoramic video compression method characterized by, The method comprises: obtaining first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image, and second position information of a center point of the panoramic image, wherein, in a case where the panoramic video is projected in a projection mode other than a cylindrical projection, it is necessary to convert the panoramic image in the other projection mode into a rectangular image in the cylindrical projection; determining a first mapping relationship according to the first position information and the second position information; converting the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image, wherein the intermediate panoramic image is centered on the center point of the viewing angle; compressing the intermediate panoramic image to obtain a target panoramic image; the step of compressing the intermediate panoramic image to obtain a target panoramic image comprises: compressing the intermediate panoramic image according to a preset second mapping relationship to obtain a compressed panoramic image, wherein a distance between a pixel point in the intermediate panoramic image and a center pixel point of the intermediate panoramic image is positively correlated with a compression rate of the intermediate panoramic image; determining the target panoramic image according to the compressed panoramic image, wherein the target panoramic image reduces transmission bandwidth of the panoramic video in a transmission process compared with the current frame panoramic image.

2. The method of claim 1, wherein, the step of compressing the intermediate panoramic image according to a preset second mapping relationship to obtain a compressed panoramic image comprises: compressing the intermediate panoramic image in a horizontal direction according to a preset second mapping relationship to obtain a compressed panoramic image, wherein a height of the compressed panoramic image is equal to a height of the intermediate panoramic image, and a width of the compressed panoramic image is half of a width of the intermediate panoramic image.

3. The method of claim 1, wherein, if the compressed panoramic image is a rhombus, the step of determining the target panoramic image according to the compressed panoramic image comprises: segmenting the compressed panoramic image along a horizontal center line and a vertical center line to obtain four segmented images; recombining the four segmented images to obtain the target panoramic image.

4. The method of claim 3, wherein, the step of recombining the four segmented images to obtain the target panoramic image comprises: rotating two adjacent segmented images according to a preset rotation direction and a rotation angle to obtain the target panoramic image.

5. The method of claim 1, wherein, the second mapping relationship is a mapping relationship constructed according to a preset S-shaped curve, and the S-shaped curve satisfies the following conditions: the S-shaped curve passes through three fixed points (-1, -1), (0, 0) and (+1, +1); the S-shaped curve is centrally symmetric with respect to the origin; a slope of the S-shaped curve at the origin is maximum.

6. The method of claim 1, wherein, the first position information comprises first position coordinates of the center point of the viewing angle in the panoramic image, and the second position information comprises second position coordinates of the center point of the panoramic image; the step of determining a first mapping relationship according to the first position information and the second position information comprises: projecting the first position coordinates into a coordinate system in which a spherical image corresponding to the panoramic image is located to obtain third position coordinates; projecting the second position coordinates into the coordinate system in which the spherical image is located to obtain fourth position coordinates; According to the third position coordinate and the fourth position coordinate, the first mapping relationship is determined.

7. A panoramic video compression apparatus characterized by comprising: The device comprises: An acquisition module is configured to acquire first position information of a center point of a viewing angle of a current frame panoramic image of a panoramic video in the panoramic image, and second position information of a center point of the panoramic image, wherein in a case where the panoramic video is projected in a projection mode other than a cylindrical projection, the panoramic image in the other projection mode needs to be converted into a rectangular image in the cylindrical projection. A first determination module is configured to determine a first mapping relationship according to the first position information and the second position information. A second determination module is configured to convert the panoramic image according to the first mapping relationship to obtain an intermediate panoramic image, wherein the intermediate panoramic image is centered on the center point of the viewing angle. A third determination module is configured to compress the intermediate panoramic image to obtain a target panoramic image. The compression of the intermediate panoramic image to obtain the target panoramic image comprises: According to a preset second mapping relationship, the intermediate panoramic image is compressed to obtain a compressed panoramic image, wherein a distance between a pixel point in the intermediate panoramic image and a center pixel point of the intermediate panoramic image is positively correlated with a compression rate of the intermediate panoramic image. The target panoramic image is determined according to the compressed panoramic image, wherein the target panoramic image has reduced transmission bandwidth of the panoramic video in a transmission process compared with the current frame panoramic image.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images