Methods, devices and storage media for panoramic video transmission

Abstract

This invention relates to the field of image processing technology and provides a method, apparatus, and storage medium for transmitting panoramic video. The method includes: acquiring a first projected image of the region of interest of a client user based on the current panoramic video frame; encoding the first projected image and the current panoramic video frame at a reduced resolution and sending them to the client, thereby reducing network resource overhead and transmission latency during panoramic video transmission and ensuring the clarity of the region of interest.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method, apparatus and storage medium for transmitting panoramic video. Background Technology

[0002] Panoramic video is an emerging data format that, compared to traditional video, features a wider field of view, higher resolution, and larger data volume. Current technologies typically encode stitched panoramic or raw fisheye video into a standard data stream format for transmission. The client then decodes the data stream to view the area of ​​interest. However, transmitting such high-resolution, high-quality data incurs significant network resource overhead and introduces a certain degree of network transmission latency. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, and storage medium for transmitting panoramic video, aiming to solve the problems of network resource waste and transmission delay in the transmission of panoramic video in the prior art.

[0004] On one hand, the present invention provides a method for transmitting panoramic video, the method comprising the following steps:

[0005] Based on the current panoramic video frame, obtain the first projected image of the region of interest for the client user;

[0006] The first projected image and the current panoramic video frame after being reduced in resolution are encoded and sent to the client.

[0007] Optionally, the step of acquiring the first projected image of the client user's region of interest based on the current panoramic video frame includes:

[0008] A latitude and longitude mapping table is generated based on the preset calibration parameters of the panoramic shooting device;

[0009] Obtain the first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system, wherein the second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client.

[0010] Establish a rotation matrix based on the Euler angles used to characterize the region of interest;

[0011] The first coordinate information is rotated according to the rotation matrix to obtain the second coordinate information;

[0012] The latitude and longitude information is mapped to each of the second coordinate information to obtain the latitude and longitude information;

[0013] The latitude and longitude mapping table is used to perform coordinate mapping on the latitude and longitude information to obtain the third coordinate information of the region of interest in the current panoramic video frame.

[0014] The first projected image is obtained by acquiring the pixel values ​​corresponding to each third coordinate information based on the current panoramic video frame.

[0015] Optionally, the target three-dimensional coordinate system is a right-handed coordinate system, with the origin at the location of the panoramic shooting device. The first coordinate information is calculated as follows:

[0016] x = (i - rayW × 0.5)

[0017] y = (j - rayH × 0.5)

[0018] z=tan(0.5π-0.5×fov×π / 180°)×rayW×0.5

[0019] Where (i, j) represents the two-dimensional coordinates of the second projected image in the two-dimensional projection coordinate system, i and j are both positive numbers, the two-dimensional projection coordinate system takes the upper left corner of the image displayed on the client's screen as the origin, the X-axis and Y-axis of the two-dimensional projection coordinate system are in the same direction as the X-axis and Y-axis of the right-hand coordinate system, (x, y, z) represents the three-dimensional coordinates of the two-dimensional coordinate point (i, j) in the right-hand coordinate system, fov represents the field of view of the image displayed on the client's screen, rayW represents the width of the client's screen, and rayH represents the height of the client's screen;

[0020] The rotation matrix is ​​calculated as follows:

[0021]

[0022] r(0,0) = cos(pitch) * cos(yaw)

[0023] r(0,1)=sin(roll)*sin(pitch)*cos(yaw)-cos(roll)*sin(yaw)

[0024] r(0,2)=cos(roll)*sin(pitch)*cos(yaw)+sin(roll)*sin(yaw)

[0025] r(1,0)=cos(pitch)*sin(yaw)

[0026] r(1,1)=sin(roll)*sin(pitch)*sin(yaw)+cos(roll)*cos(yaw)

[0027] r(1,2)=cos(roll)*sin(pitch)*sin(yaw)-sin(roll)*cos(yaw)

[0028] r(2,0)=-sin(pitch)

[0029] r(2,1)=sin(roll)*cos(pitch)

[0030] r(2,2)=cos(roll)*cos(pitch)

[0031] Where R represents the rotation matrix, and yaw, pitch, and roll represent the yaw angle, pitch angle, and roll angle of the region of interest, respectively;

[0032] The second coordinate information is calculated as follows:

[0033] x0=x×r(0,0)+y×r(0,1)+z×r(0,2)

[0034] y0=x×r(1,0)+y×r(1,1)+z×r(1,2)

[0035] z0=x×r(2,0)+y×r(2,1)+z×r(2,2)

[0036] Where (x0, y0, z0) represents the three-dimensional coordinates of the projection point in the second coordinate information;

[0037] The latitude and longitude information is calculated as follows:

[0038]

[0039]

[0040] Where θ1 represents longitude and θ2 represents latitude.

[0041] Optionally, the step of encoding the first projected image and the current panoramic video frame after reducing its resolution and then sending them to the client includes:

[0042] Get two fisheye images of the current panoramic video frame;

[0043] The two fisheye images, reduced in resolution, are combined with the first projected image, and the combined image is encoded and sent to the client.

[0044] Optionally, the step of encoding the first projected image and the current panoramic video frame after reducing its resolution and then sending them to the client includes:

[0045] Obtain the planar projection image of the current panoramic video frame;

[0046] The reduced-resolution planar projection image is combined with the first projection image, and the combined image is encoded and sent to the client.

[0047] Optionally, the method further includes:

[0048] The resolution of the current panoramic video frame after resolution reduction is determined based on the acquired target environment information.

[0049] On the other hand, the present invention provides a panoramic video transmission device, the device comprising:

[0050] The projection image acquisition unit is used to acquire the first projection image of the region of interest of the client user based on the current panoramic video frame;

[0051] The video sending unit is used to encode the first projected image and the current panoramic video frame after reducing the resolution and then send them to the client.

[0052] Optionally, the projected image acquisition unit further includes:

[0053] The mapping table generation unit is used to generate a latitude and longitude mapping table based on the preset calibration parameters of the panoramic shooting device;

[0054] The first information acquisition unit is used to acquire the first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system. The second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client.

[0055] A rotation matrix establishment unit is used to establish a rotation matrix based on Euler angles used to characterize the region of interest.

[0056] The second information acquisition unit is used to rotate each of the first coordinate information according to the rotation matrix to obtain each of the second coordinate information; and

[0057] The latitude and longitude mapping unit is used to perform latitude and longitude mapping on each of the second coordinate information to obtain each latitude and longitude information;

[0058] The third information acquisition unit is used to perform coordinate mapping on the latitude and longitude information using the latitude and longitude mapping table to obtain the third coordinate information of the region of interest in the current panoramic video frame; and

[0059] The projection image generation unit is used to obtain the pixel values ​​corresponding to each of the third coordinate information based on the current panoramic video frame, and to obtain the first projection image.

[0060] On the other hand, the present invention also provides a panoramic shooting device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0061] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0062] This invention acquires a first projected image of the region of interest for the client user based on the current panoramic video frame, encodes the first projected image and the current panoramic video frame at reduced resolution, and sends them to the client. This reduces network resource overhead and transmission latency during panoramic video transmission while ensuring the clarity of the region of interest. Attached Figure Description

[0063] Figure 1A This is a flowchart illustrating the implementation of the panoramic video transmission method provided in Embodiment 1 of the present invention.

[0064] Figure 1B This is a schematic diagram of the right-handed coordinate system and the two-dimensional projected coordinate system provided in Embodiment 1 of the present invention;

[0065] Figure 1C This is an example diagram showing the display effect of the encoded image provided in Embodiment 1 of the present invention;

[0066] Figure 2 This is a schematic diagram of the structure of the panoramic video transmission device provided in Embodiment 2 of the present invention; and

[0067] Figure 3 This is a schematic diagram of the panoramic shooting device provided in Embodiment 3 of the present invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0069] The specific implementation of the present invention will be described in detail below with reference to specific embodiments:

[0070] Example 1:

[0071] Figure 1A The implementation flow of the panoramic video transmission method provided in Embodiment 1 of the present invention is illustrated. For ease of explanation, only the parts related to the embodiments of the present invention are shown, and are described in detail below:

[0072] In step S101, the first projected image of the region of interest of the client user is obtained based on the current panoramic video frame.

[0073] This invention applies to panoramic shooting devices, which can be surveillance cameras, cameras, mobile phones, tablets, wearable devices, in-vehicle devices, laptops, etc. This application does not limit the specific type of panoramic shooting device. The aforementioned client can be a video playback device such as a mobile phone, tablet, wearable device, in-vehicle device, laptop, super mobile personal computer, or server.

[0074] In this embodiment of the invention, the client acquires the user's region of interest (ROI) and sends the acquired ROI to the panoramic shooting device. The client can be used to acquire the ROI input by the user, and it can also predict the user's ROI for the next period within each ROI calculation cycle, sending the predicted ROI to the panoramic shooting device.

[0075] The panoramic shooting device performs projection calculations on the current panoramic video frame based on the acquired region of interest (ROI) of the client user, obtaining a first projected image of the ROI. This first projected image is typically a projection of the spherical image of the ROI in the current panoramic video frame in linear projection mode.

[0076] When acquiring the first projected image of the region of interest (ROI) of the client user based on the current panoramic video frame, optionally, a latitude and longitude mapping table is generated according to the calibration parameters preset by the panoramic shooting device. The first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system is obtained. Then, a rotation matrix is ​​established according to the Euler angles used to represent the ROI. The first coordinate information is rotated according to the rotation matrix to obtain the second coordinate information. The second coordinate information is then mapped to latitude and longitude to obtain the latitude and longitude information. The pre-set latitude and longitude mapping table is used to map the latitude and longitude information to obtain the third coordinate information of the ROI in the current panoramic video frame. The pixel values ​​corresponding to the third coordinate information are obtained according to the current panoramic video frame to obtain the first projected image. Thus, the acquisition of the ROI image is achieved through coordinate conversion. The second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client. The horizontal and vertical coordinates of the latitude and longitude mapping table represent the latitude and longitude of the sphere centered on the panoramic shooting device, respectively. The values ​​in the latitude and longitude mapping table are the specific coordinates of the panoramic video frame. The calibration parameters mentioned above may include intrinsic and extrinsic parameters and distortion parameters. The intrinsic and extrinsic parameters may include parameters such as field of view and focal length.

[0077] like Figure 1BAs shown, optionally, the target's three-dimensional coordinate system is a right-handed coordinate system, with the origin at the location of the panoramic shooting device and the Z-axis pointing directly in front of the panoramic shooting device. Figure 1B In the right-hand coordinate system, Xd, Yd, and Zd are the three axes, and Od is the origin of the right-hand coordinate system, which is the center of the spherical model. In linear projection mode, the second projected image is obtained by taking a view from directly in front of the panoramic shooting device according to the client's screen size. The two-dimensional coordinates of the projection point of the second projected image in the two-dimensional projection coordinate system are represented by (i, j). The X-axis and Y-axis of the two-dimensional projection coordinate system are in the same direction as the X-axis and Y-axis of the right-hand coordinate system, respectively. The origin of the two-dimensional projection coordinate system is the upper left corner of the image displayed on the client's screen. Figure 1B In this system, Xp and Yp are the two axes of a two-dimensional projected coordinate system, and Od is the center of the circle. The calculation method for the first coordinate information is as follows:

[0078] x = (i - rayW × 0.5)

[0079] y = (j - rayH × 0.5)

[0080] z=tan(0.5π-0.5×fov×π / 180°)×rayW×0.5

[0081] Where i and j are both positive numbers, (x,y,z) represents the three-dimensional coordinates of the two-dimensional coordinate point (i,j) in the right-hand coordinate system, fov represents the field of view of the image displayed on the client screen, rayW represents the width of the client screen, and rayH represents the height of the client screen.

[0082] After obtaining the aforementioned second coordinate information, the region of interest sent by the client is received, and the rotation matrix is ​​calculated based on the Euler angles of the region of interest. The calculation method for the rotation matrix is ​​as follows:

[0083]

[0084] r(0,0) = cos(pitch) * cos(yaw)

[0085] r(0,1)=sin(roll)*sin(pitch)*cos(yaw)-cos(roll)*sin(yaw)

[0086] r(0,2)=cos(roll)*sin(pitch)*cos(yaw)+sin(roll)*sin(yaw)

[0087] r(1,0)=cos(pitch)*sin(yaw)

[0088] r(1,1)=sin(roll)*sin(pitch)*sin(yaw)+cos(roll)*cos(yaw)

[0089] r(1,2)=cos(roll)*sin(pitch)*sin(yaw)-sin(roll)*cos(yaw)

[0090] r(2,0)=-sin(pitch)

[0091] r(2,1)=sin(roll)*cos(pitch)

[0092] r(2,2)=cos(roll)*cos(pitch)

[0093] Where R represents the rotation matrix, and yaw, pitch, and roll represent the yaw angle, pitch angle, and roll angle of the region of interest, respectively.

[0094] After calculating the rotation matrix, the first coordinate information is rotated according to the rotation matrix to obtain the second coordinate information. The calculation method for the second coordinate information is as follows:

[0095] x0=x×r(0,0)+y×r(0,1)+z×r(0,2)

[0096] y0=x×r(1,0)+y×r(1,1)+z×r(1,2)

[0097] z0=x×r(2,0)+y×r(2,1)+z×r(2,2)

[0098] Where (x0, y0, z0) represents the three-dimensional coordinates of the projection point in the second coordinate information.

[0099] After obtaining the aforementioned second coordinate information, latitude and longitude mapping can be performed on each of the second coordinate information to obtain the latitude and longitude information. The latitude and longitude information is calculated as follows:

[0100]

[0101]

[0102] Where θ1 represents longitude and θ2 represents latitude.

[0103] After obtaining the latitude and longitude information, the coordinates of each latitude and longitude information can be mapped according to the latitude and longitude mapping table to obtain the third coordinate information of the region of interest in the current panoramic video frame. Finally, the pixel values ​​corresponding to each third coordinate information are obtained according to the current panoramic video frame, and the first projected image can be obtained based on the pixel values ​​corresponding to each third coordinate information.

[0104] In step S102, the first projected image and the current panoramic video frame after being reduced in resolution are encoded and sent to the client.

[0105] In an embodiment of the present invention, optionally, two fisheye images of the current panoramic video frame are acquired, the two fisheye images with reduced resolution are combined with the first projection image, and the combined image is encoded and sent to the client, so as to ensure the image clarity of the area of ​​interest to the user while ensuring that the user can observe global information from the two fisheye images. Figure 1C This is an example of the display effect after combining two fisheye images with a first projected image after reducing their resolution.

[0106] Optionally, the planar projection image of the current panoramic video frame is obtained, the planar projection image after being reduced in resolution is combined with the first projection image, and the combined image is encoded and sent to the client, so as to ensure the image clarity of the area of ​​interest to the user while ensuring that the user can observe global information from the planar projection image.

[0107] Optionally, the display mode selected by the client user is obtained. If it is a fisheye display mode, the two fisheye images with reduced resolution and the first projection image are combined, and the combined image is encoded and sent to the client. If it is a tiled display mode, the planar projection image with reduced resolution and the first projection image are combined, and the combined image is encoded and sent to the client, so as to enrich the display modes of the panoramic video frames after the resolution is reduced.

[0108] The resolution of the current panoramic video frame after resolution reduction can be a fixed resolution or can be flexibly set by the client user. Optionally, the resolution of the current panoramic video frame after resolution reduction can be determined based on the acquired target environment information, thereby improving the flexibility of adjusting the resolution of the current panoramic video frame. The target environment information may include the client's network environment and / or hardware and software environment. The network environment may include network bandwidth, network transmission latency, etc., while the hardware and software environment may include the client's data processing capabilities, client power consumption, and the minimum resolution specified by the client user.

[0109] In this embodiment of the invention, a first projected image of the region of interest of the client user is obtained based on the current panoramic video frame. The first projected image and the current panoramic video frame after being reduced in resolution are encoded and sent to the client, thereby reducing network resource overhead and transmission latency during panoramic video transmission and ensuring the clarity of the region of interest.

[0110] Example 2:

[0111] Figure 2 The structure of the panoramic video transmission device provided in Embodiment 2 of the present invention is shown. For ease of explanation, only the parts related to the embodiments of the present invention are shown, including:

[0112] The projection image acquisition unit is used to acquire a first projection image of the region of interest of the client user based on the current panoramic video frame; and

[0113] The video transmission unit is used to encode the first projected image and the current panoramic video frame after reducing its resolution and then send them to the client.

[0114] Optionally, the projected image acquisition unit further includes:

[0115] The mapping table generation unit is used to generate a latitude and longitude mapping table based on the preset calibration parameters of the panoramic shooting device;

[0116] The first information acquisition unit is used to acquire the first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system. The second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client.

[0117] A rotation matrix establishment unit is used to establish a rotation matrix based on Euler angles used to characterize the region of interest.

[0118] The second information acquisition unit is used to rotate each first coordinate information according to the rotation matrix to obtain each second coordinate information;

[0119] The latitude and longitude mapping unit is used to map each second coordinate information to latitude and longitude to obtain each latitude and longitude information;

[0120] The third information acquisition unit is used to perform coordinate mapping on the latitude and longitude information using a latitude and longitude mapping table to obtain the third coordinate information of the region of interest in the current panoramic video frame; and

[0121] The projection image generation unit is used to obtain the pixel values ​​corresponding to each third coordinate information based on the current panoramic video frame to obtain the first projection image.

[0122] In this embodiment of the invention, each unit of the panoramic video transmission device can be implemented by a corresponding hardware or software unit. Each unit can be an independent hardware or software unit, or it can be integrated into a single hardware or software unit, which is not intended to limit the invention. Specific implementation methods for each unit of the panoramic video transmission device can be found in the description of the foregoing method embodiments, and will not be repeated here.

[0123] Example 3:

[0124] Figure 3 The structure of the panoramic shooting device provided in Embodiment 3 of the present invention is shown. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0125] The panoramic shooting device 3 of this embodiment includes a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, it implements the steps described in the above method embodiments, for example... Figure 1A The steps S101 to S102 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The functions of units 21 and 22 shown.

[0126] In this embodiment of the invention, a first projected image of the region of interest of the client user is obtained based on the current panoramic video frame. The first projected image and the current panoramic video frame after being reduced in resolution are encoded and sent to the client, thereby reducing network resource overhead and transmission latency during panoramic video transmission and ensuring the clarity of the region of interest.

[0127] Example 4:

[0128] In this embodiment of the invention, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps in the method embodiments described above. For example... Figure 1A The steps S101 to S102 are shown. Alternatively, when the computer program is executed by a processor, it implements the functions of each unit in the above-described apparatus embodiments, for example... Figure 2 The functions of units 21 and 22 shown.

[0129] In this embodiment of the invention, a first projected image of the region of interest of the client user is obtained based on the current panoramic video frame. The first projected image and the current panoramic video frame after being reduced in resolution are encoded and sent to the client, thereby reducing network resource overhead and transmission latency during panoramic video transmission and ensuring the clarity of the region of interest.

[0130] The computer-readable storage medium in embodiments of the present invention may include any entity or device capable of carrying computer program code, a recording medium, such as ROM / RAM, disk, optical disk, flash memory, etc.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for transmitting panoramic video, characterized in that, The method includes the following steps: The first projected image of the region of interest of the client user is obtained based on the current panoramic video frame, wherein the first projected image is formed by coordinate transformation; The first projected image and the current panoramic video frame after being reduced in resolution are combined, and the combined image is encoded and sent to the client. The client displays the first projected image and the current panoramic video frame after being reduced in resolution simultaneously in the same window.

2. The method as described in claim 1, characterized in that, The step of acquiring the first projected image of the client user's region of interest based on the current panoramic video frame includes: A latitude and longitude mapping table is generated based on the preset calibration parameters of the panoramic shooting device; Obtain the first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system, wherein the second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client. Establish a rotation matrix based on the Euler angles used to characterize the region of interest; The first coordinate information is rotated according to the rotation matrix to obtain the second coordinate information; The latitude and longitude information is mapped to each of the second coordinate information to obtain the latitude and longitude information; The latitude and longitude mapping table is used to perform coordinate mapping on the latitude and longitude information to obtain the third coordinate information of the region of interest in the current panoramic video frame. The first projected image is obtained by acquiring the pixel values ​​corresponding to each third coordinate information based on the current panoramic video frame.

3. The method as described in claim 2, characterized in that, The target's three-dimensional coordinate system is a right-handed coordinate system, with the origin at the location of the panoramic shooting device. The first coordinate information is calculated as follows: in,( i , j () represents the two-dimensional coordinates of the second projected image in the two-dimensional projected coordinate system. i, j All are positive numbers. The two-dimensional projection coordinate system takes the upper left corner of the image displayed on the client's screen as its origin. The X-axis and Y-axis of the two-dimensional projection coordinate system are in the same direction as the X-axis and Y-axis of the right-hand coordinate system, respectively. x , y , z ) represents a two-dimensional coordinate point ( i,j The three-dimensional coordinates in the right-handed coordinate system are: fov represents the field of view of the image displayed on the client's screen, rayW represents the width of the client's screen, and rayH represents the height of the client's screen. The rotation matrix is ​​calculated as follows: in, Represents the rotation matrix, , and These represent the yaw angle, pitch angle, and roll angle of the region of interest, respectively. The second coordinate information is calculated as follows: in,( x 0, y 0, z 0) represents the three-dimensional coordinates of the projection point in the second coordinate information; The latitude and longitude information is calculated as follows: in, Indicates longitude. Indicates latitude.

4. The method as described in claim 1, characterized in that, The step of encoding the first projected image and the current panoramic video frame after reducing its resolution and then sending them to the client includes: Get two fisheye images of the current panoramic video frame; The two fisheye images, reduced in resolution, are combined with the first projected image, and the combined image is encoded and sent to the client.

5. The method as described in claim 1, characterized in that, The step of encoding the first projected image and the current panoramic video frame after reducing its resolution and then sending them to the client includes: Obtain the planar projection image of the current panoramic video frame; The reduced-resolution planar projection image is combined with the first projection image, and the combined image is encoded and sent to the client.

6. The method as described in claim 1, characterized in that, The method further includes: The resolution of the current panoramic video frame after resolution reduction is determined based on the acquired target environment information.

7. A panoramic video transmission device, characterized in that, The device includes: The projection image acquisition unit is used to acquire a first projection image of the region of interest of the client user based on the current panoramic video frame, wherein the first projection image is formed by coordinate transformation; The video sending unit is used to combine the first projected image and the current panoramic video frame after reducing the resolution, encode the combined image and send it to the client, and the client displays the first projected image and the current panoramic video frame after reducing the resolution in the same window at the same time.

8. The apparatus as claimed in claim 7, characterized in that, The projection image acquisition unit further includes: The mapping table generation unit is used to generate a latitude and longitude mapping table based on the preset calibration parameters of the panoramic shooting device; The first information acquisition unit is used to acquire the first coordinate information of each pixel of the second projected image in the target three-dimensional coordinate system. The second projected image is a projected image obtained when the panoramic shooting device is viewed from the front according to the screen size of the client. A rotation matrix establishment unit is used to establish a rotation matrix based on Euler angles used to characterize the region of interest. The second information acquisition unit is used to rotate each of the first coordinate information according to the rotation matrix to obtain each of the second coordinate information; and The latitude and longitude mapping unit is used to perform latitude and longitude mapping on each of the second coordinate information to obtain each latitude and longitude information; The third information acquisition unit is used to perform coordinate mapping on the latitude and longitude information using the latitude and longitude mapping table to obtain the third coordinate information of the region of interest in the current panoramic video frame; and The projection image generation unit is used to obtain the pixel values ​​corresponding to each of the third coordinate information based on the current panoramic video frame, and to obtain the first projection image.

9. A panoramic shooting device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images