Panoramic video evaluation method and device and electronic equipment

Abstract

The invention relates to the technical field of data processing, in particular to a panoramic video evaluation method and device and electronic equipment, and is used for solving the problem of how to evaluate panoramic videos with different configuration parameters. The method comprises the following steps: acquiring a panoramic video and a historical user view angle corresponding to each moment when a user watches the panoramic video; based on the historical user viewing angles and the panoramic video, determining a theoretical image seen by the user at each historical user viewing angle and a theoretical bandwidth occupied by transmission of the theoretical image; performing video preprocessing on the panoramic video to obtain a preprocessed panoramic video; determining transmission information during transmission of the preprocessed panoramic video, an actual image seen by each target user view angle and an actual bandwidth occupied by transmission of the actual image based on the preprocessed panoramic video and the historical user view angle; and based on the target parameter, generating an evaluation result of the preprocessed panoramic video.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a panoramic video evaluation method, device and electronic device. Background Art

[0002] In recent years, with the popularization of virtual reality (VR) head-mounted display hardware and the improvement of network infrastructure, more and more users are watching panoramic videos through VR head-mounted displays to experience the immersive experience brought by panoramic videos.

[0003] In order to ensure the experience of panoramic videos, panoramic videos need to be evaluated before they are deployed to users for viewing. For this purpose, each service provider that provides panoramic videos has a set of evaluation methods that suit them. When comparing the evaluation results, since different service providers provide different configuration parameters for panoramic videos, and different service providers have their own evaluation methods, it is impossible to intuitively compare the evaluation results of different configuration parameters.

[0004] Therefore, how to evaluate panoramic videos with different configuration parameters has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present disclosure provides a panoramic video evaluation method, device and electronic device, which are used to solve the problem of how to evaluate panoramic videos with different configuration parameters in the related art.

[0006] In order to achieve the above objectives, the present disclosure provides the following technical solutions:

[0007] In a first aspect, the present disclosure provides a method for evaluating a panoramic video, comprising: acquiring a panoramic video and a historical user perspective corresponding to each moment when a user watches the panoramic video; determining, based on the historical user perspective and the panoramic video, a theoretical image seen by the user at each historical user perspective and a theoretical bandwidth occupied by transmitting the theoretical image; performing video preprocessing on the panoramic video to obtain a preprocessed panoramic video; wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video; based on the preprocessed panoramic video and the historical user perspective, determining transmission information when transmitting the preprocessed panoramic video, an actual image seen by each target user perspective, and an actual bandwidth occupied by transmitting the actual image; wherein the target user perspective is calculated based on the historical user perspective; and generating an evaluation result of the preprocessed panoramic video based on the target parameters; wherein the target parameters include at least one of the transmission information, the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth.

[0008] As an optional implementation of the present disclosure, based on the preprocessed panoramic video and the historical user perspective, the transmission information when transmitting the preprocessed panoramic video, the actual image seen from the perspective of each target user, and the actual bandwidth occupied by transmitting the actual image are determined, including: performing perspective prediction based on the historical user perspective to obtain the predicted user perspective, and using the predicted user perspective as the target user perspective; based on the target user perspective and the preprocessed panoramic video, the transmission information when transmitting the preprocessed panoramic video, the actual image seen from the perspective of each target user, and the actual bandwidth occupied by transmitting the actual image.

[0009] As an optional implementation mode of the present disclosure, based on the preprocessed panoramic video and the historical user perspectives, the transmission information when transmitting the preprocessed panoramic video, the actual image seen from the perspective of each target user, and the actual bandwidth occupied by transmitting the actual image are determined, including: based on the preprocessed panoramic video, determining the video frames to be transmitted in the next cycle; based on the video frames and the historical user perspectives of the next cycle, determining the actual image seen by the user from the perspective of the target user corresponding to each video frame, and the actual bandwidth occupied by transmitting the actual image; simulating the transmission of the video frames to determine the total transmission data volume and the number of freezes in the next cycle; based on the total transmission data volume and the number of freezes, determining the transmission information when transmitting the preprocessed panoramic video.

[0010] As an optional implementation of the present disclosure, the target parameters include a theoretical image, an actual image, a theoretical bandwidth and an actual bandwidth, and the theoretical image and the actual image correspond to each other one by one; based on the target parameters, an evaluation result of the preprocessed panoramic video is generated, including: based on the theoretical image and the actual image, determining a first total number of pixels contained in the theoretical image and a second total number of pixels contained in the actual image; based on the first total number, the second total number, the theoretical bandwidth and the actual bandwidth, determining the evaluation result of the panoramic video.

[0011] As an optional implementation of the present disclosure, the target parameters include a theoretical image and an actual image; based on the target parameters, an evaluation result of the preprocessed panoramic video is generated, including: based on the theoretical image and the actual image, determining the peak signal-to-noise ratio and structural similarity between the theoretical image and the actual image; based on one or more of the peak signal-to-noise ratio and the structural similarity, obtaining an evaluation result of the panoramic video.

[0012] As an optional implementation of the present disclosure, the target parameters include transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes; based on the target parameters, an evaluation result of the pre-processed panoramic video is generated, including: determining the evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes.

[0013] As an optional implementation of the present disclosure, a panoramic video is preprocessed to obtain a preprocessed panoramic video, including: format conversion of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format; encoding the encoded panoramic video according to preset encoding parameters to obtain a preprocessed panoramic video.

[0014] As an optional implementation of the present disclosure, the projection method includes: any one of rectangular spherical projection, cube map projection, equiangular cube map projection, pyramid prism projection, and barrel distortion projection.

[0015] As an optional implementation of the present disclosure, the encoding parameters include: rectangular block size, quantization parameter, preset configuration parameters of video encoding, picture group size, intra-frame coded frame, forward and backward predicted coded frame, bidirectional predicted interpolated coded frame, one or more of the video coding standards.

[0016] In a second aspect, the present disclosure provides an evaluation device for panoramic videos, comprising: an acquisition unit, for acquiring panoramic videos and historical user perspectives corresponding to each moment when a user watches the panoramic videos; a processing unit, for determining, based on the historical user perspectives acquired by the acquisition unit and the panoramic videos acquired by the acquisition unit, a theoretical image seen by the user at each historical user perspective and a theoretical bandwidth occupied by transmitting the theoretical image; the processing unit is further used to perform video preprocessing on the panoramic video acquired by the acquisition unit to obtain a preprocessed panoramic video; wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video; the processing unit is further used to determine, based on the preprocessed panoramic video acquired by the acquisition unit and the historical user perspectives acquired by the acquisition unit, transmission information when transmitting the preprocessed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image; wherein the target user perspective is calculated based on the historical user perspective; the processing unit is further used to generate an evaluation result of the preprocessed panoramic video based on the target parameters; wherein the target parameters include at least one of the transmission information, the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth.

[0017] As an optional implementation of the present disclosure, the processing unit is specifically used to predict the perspective based on the historical user perspective acquired by the acquisition unit, obtain the predicted user perspective, and use the predicted user perspective as the target user perspective; the processing unit is specifically used to determine the transmission information when transmitting the preprocessed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image based on the target user perspective and the preprocessed panoramic video acquired by the acquisition unit.

[0018] As an optional implementation of the present disclosure, the processing unit is specifically used to determine the video frames to be transmitted in the next cycle based on the preprocessed panoramic video; the processing unit is specifically used to simulate the transmission of the video frames to determine the total transmission data volume and the number of freezes in the next cycle; the processing unit is specifically used to determine the actual image seen by the user from the target user perspective corresponding to each video frame, and the actual bandwidth occupied by transmitting the actual image based on the video frames and the historical user perspectives of the next cycle acquired by the acquisition unit; the processing unit is specifically used to determine the transmission information when transmitting the preprocessed panoramic video based on the total transmission data volume and the number of freezes.

[0019] As an optional implementation of the present disclosure, the target parameters include a theoretical image, an actual image, a theoretical bandwidth and an actual bandwidth, and the theoretical image and the actual image correspond to each other one by one; the processing unit is specifically used to determine a first total number of pixels contained in the theoretical image and a second total number of pixels contained in the actual image based on the theoretical image and the actual image; the processing unit is specifically used to determine an evaluation result of the panoramic video based on the first total number, the second total number, the theoretical bandwidth and the actual bandwidth.

[0020] As an optional implementation of the present disclosure, the target parameters include a theoretical image and an actual image; a processing unit is specifically used to determine the peak signal-to-noise ratio and structural similarity between the theoretical image and the actual image based on the theoretical image and the actual image; the processing unit is specifically used to obtain an evaluation result of the panoramic video based on one or more of the peak signal-to-noise ratio and the structural similarity.

[0021] As an optional implementation of the present disclosure, the target parameters include transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes; the processing unit is specifically used to determine the evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes.

[0022] As an optional implementation of the present disclosure, the processing unit is specifically used to convert the format of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format; the processing unit is specifically used to encode the encoded panoramic video according to preset encoding parameters to obtain a preprocessed panoramic video.

[0023] As an optional implementation of the present disclosure, the projection method includes: any one of rectangular spherical projection, cube map projection, equiangular cube map projection, pyramid prism projection, and barrel distortion projection.

[0024] As an optional implementation of the present disclosure, the encoding parameters include: rectangular block size, quantization parameter, preset configuration parameters of video encoding, picture group size, intra-frame coded frame, forward and backward predicted coded frame, bidirectional predicted interpolated coded frame, one or more of the video coding standards.

[0025] In a third aspect, the present disclosure provides an electronic device, comprising: a memory and a processor, the memory being used to store a computer program; the processor being used to enable the electronic device to implement the panoramic video evaluation method provided in the first aspect when executing the computer program.

[0026] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computing device, the computing device implements the panoramic video evaluation method provided in the first aspect above.

[0027] In a fifth aspect, the present disclosure provides a computer program product. When the computer program product is executed on a computer, the computer implements the panoramic video evaluation method provided in the first aspect.

[0028] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the panoramic video evaluation device, or may be packaged separately from the processor of the panoramic video evaluation device, and the present disclosure does not limit this.

[0029] The description of the second, third, fourth and fifth aspects in the present disclosure can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0030] In the present disclosure, the name of the panoramic video evaluation device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of the present disclosure, they belong to the scope of the claims of the present disclosure and their equivalent technologies.

[0031] These and other aspects of the present disclosure will become more apparent from the following description.

[0032] Compared with the prior art, the technical solution provided by the present invention has the following advantages:

[0033] The panoramic video evaluation method provided by the present disclosure can determine the transmission information when transmitting the preprocessed panoramic video, the actual image seen from the perspective of each target user, and the actual bandwidth occupied by transmitting the actual image, based on the historical user perspective corresponding to each moment when the user watches the panoramic video and the preprocessed panoramic video. Afterwards, based on the theoretical image seen from each historical user perspective when the user watches the preprocessed panoramic video, and at least one of the theoretical bandwidth, actual image, and actual bandwidth occupied by transmitting the theoretical image, an evaluation result of the preprocessed panoramic video is generated. Since the panoramic videos with different configuration parameters are subjected to video preprocessing to obtain the preprocessed panoramic videos with the same configuration parameters, the evaluation results of the generated preprocessed panoramic videos can be directly compared, thereby solving the problem of how to evaluate the panoramic videos with different configuration parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 A schematic diagram of a right-handed Cartesian coordinate system for a panoramic video evaluation method provided by an embodiment of the present disclosure;

[0037] Figure 2 One of the flowcharts of the panoramic video evaluation method provided in the embodiment of the present disclosure;

[0038] Figure 3 A schematic diagram of a video block of a panoramic video evaluation method provided by an embodiment of the present disclosure;

[0039] Figure 4 A second flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0040] Figure 5 A third flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0041] Figure 6 A fourth flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0042] Figure 7 A fifth flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0043] Figure 8 A sixth flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0044] Fig. 9 A seventh flowchart of a panoramic video evaluation method provided in an embodiment of the present disclosure;

[0045] Fig.10 A schematic diagram of the structure of a panoramic video evaluation device provided in an embodiment of the present disclosure;

[0046] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0047] Fig.12 A schematic diagram of the structure of a computer program product of a panoramic video evaluation method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0050] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0051] H.264 in the disclosed embodiment is also the tenth part of MPEG-4, which is a highly compressed digital video codec standard proposed by the Joint Video Team (JVT) jointly composed of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG).

[0052] H.265 in the embodiments of the present disclosure is a new video coding standard developed by ITU-T VCEG following H.264.

[0053] In the process of encoding and decoding each video frame in the embodiment of the present disclosure, in order to improve efficiency, a video frame is divided into multiple slices or tiles, and each Slice or Tile of a video frame is encoded and decoded independently, and the direct dependence on each other is not high, thereby achieving the effect of quickly encoding and decoding a video frame; the difference between slices and tiles lies in the different division methods. The slice method divides the video into strips, while the tile is a rectangle.

[0054] The preset parameter in the embodiment of the present disclosure is used to specify preset configuration parameters of video encoding. The preset configuration is a set of pre-defined encoding parameters that can be easily applied to different video encoding tasks.

[0055] The JSON (JavaScript Object Notation) in the embodiments of the present disclosure is a lightweight data exchange format.

[0056] In the embodiment of the present disclosure, Figure 1 As shown, a right-handed Cartesian coordinate system in three-dimensional space is established with the center point of the head of the user wearing the head-mounted display device as the center. Among them, roll refers to rotation around the X axis, pitch refers to rotation forward and backward around the Y axis, and yaw refers to left and right rotation around the Z axis.

[0057] Exemplarily, the panoramic video evaluation method provided by the embodiment of the present disclosure is described by taking the server 1 as an example of an execution subject of the panoramic video evaluation method provided by the embodiment of the present disclosure.

[0058] Figure 2 is a flow chart of a panoramic video evaluation method according to an exemplary embodiment. Figure 2 As shown, the method includes the following S11-S15.

[0059] S11. Obtain the panoramic video and the historical user perspective corresponding to each moment when the user watches the panoramic video.

[0060] In some examples, when a user needs to watch a specified panoramic video, the server 2 storing the specified panoramic video can be accessed through the head-mounted display device. Afterwards, server 2 sends the video data of the specified panoramic video. Therefore, after receiving the video data sent by server 2, the head-mounted display device decodes the video data to obtain decoded data. Afterwards, the head-mounted display device renders the decoded data, so that the user can watch the specified panoramic video through the head-mounted display device. In this process, the head-mounted display device can upload the user's perspective when the user watches the specified panoramic video to server 2, so that server 2 stores the user's perspective when the head-mounted display device plays the specified panoramic video.

[0061] In some examples, server 1 may obtain a panoramic video and a historical user perspective corresponding to each moment when a user watches the panoramic video from server 2. The panoramic video may be any specified panoramic video.

[0062] In some examples, the panoramic video and the historical user perspective corresponding to each moment when the user watches the panoramic video can be collected by the system.

[0063] S12. Based on the historical user perspectives and the panoramic video, determine the theoretical image seen by the user at each historical user perspective, and the theoretical bandwidth occupied by transmitting the theoretical image.

[0064] In some examples, when server 2 transmits a specified panoramic video to a head mounted display device, server 2 may record the theoretical bandwidth occupied by transmitting different theoretical images, and record the correspondence between the theoretical bandwidth and the theoretical image. In this way, server 1 may send query information to server 2 to obtain the correspondence corresponding to the panoramic video. Afterwards, the theoretical image is queried in the correspondence to obtain the theoretical bandwidth occupied by transmitting the theoretical image.

[0065] In some examples, server 1 performs frame alignment of a historical user perspective with a panoramic video, and then captures an image in a video frame corresponding to the historical user perspective based on the historical user perspective to obtain a theoretical image seen by the historical user perspective.

[0066] In some examples, the server 1 obtains the theoretical image seen by the user at each historical user perspective by inputting the historical user perspective and the panoramic video into the perspective analysis model. The training process of the perspective analysis model is as follows:

[0067] Obtain training sample data and labeling results of the training sample data; wherein the training sample data includes: a specified panoramic video and an actual user perspective corresponding to each moment when a user watches the specified panoramic video, and the labeling results include: an actual image seen by the user from the actual user perspective corresponding to each moment.

[0068] The training sample data is input into the neural network model for learning, and the prediction results of the neural network model for the training sample data are obtained.

[0069] Based on the prediction results and labeling results, the network parameters of the neural network model are adjusted until the neural network model converges to obtain a perspective analysis model.

[0070] In some examples, the server 1 may determine the theoretical bandwidth occupied by transmitting the theoretical image according to the image size of the theoretical image and the transmission duration.

[0071] S13, performing video preprocessing on the panoramic video to obtain a preprocessed panoramic video, wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video.

[0072] In some examples, the configuration parameters include at least one of a projection format and a coding parameter; wherein the projection format includes any one of: Equirectangular (ERP), Cubemap (CMP), Equi-Angular Cubemap (EAC), Truncated square pyramid (TSP), and Barrel; and the coding parameters include any one of: Tile size, Quantization parameter (QP), preset configuration parameter (Preset parameter) of video coding, GoP size, Intra-Picture (I-frame), Predictive-Picture (P-frame), Bi-directional interpolated prediction frame (B-frame), and one or more of the video coding standards.

[0073] In some examples, the video coding standard includes any of H.264 and H.265.

[0074] In some examples, video preprocessing includes one or more of format conversion and encoding. Among them, format conversion of the panoramic video includes: format conversion of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format. In this way, panoramic videos of different projection formats can be converted into panoramic videos of the same projection format. Encoding the panoramic video includes format conversion of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format. Thereafter, the encoded panoramic video is encoded according to preset encoding parameters to obtain a preprocessed panoramic video. In this way, the panoramic video of the same projection format can be converted into a panoramic video encoded with the same encoding parameters, thereby facilitating the analysis of the evaluation results of the panoramic videos with different configuration parameters.

[0075] In some examples, the generated pre-processed panoramic video can be a JSON file.

[0076] S14: Based on the pre-processed panoramic video and the historical user perspectives, determine the transmission information when transmitting the pre-processed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image, wherein the target user perspective is calculated based on the historical user perspective.

[0077] In some examples, the transmission information includes at least the number of freezes and the total amount of data transmitted. In order to obtain the transmission information when transmitting the preprocessed panoramic video, the actual image seen from each historical user's perspective, and the actual bandwidth occupied by transmitting the actual image, the server 1 can determine the video frame to be transmitted in the next cycle based on the preprocessed panoramic video. Based on the video frame and the historical user perspective of the next cycle, determine the actual image seen by the user from the historical user perspective corresponding to each video frame, and the actual bandwidth occupied by transmitting the actual image. Simulate the transmission of the video frame to determine the total amount of data transmitted and the number of freezes in the next cycle. Based on the total amount of data transmitted and the number of freezes, determine the transmission information when transmitting the preprocessed panoramic video.

[0078] In some examples, the target user's perspective is equal to the historical user's perspective. In this case, the server 1 can determine the actual image seen by each target user's perspective based on the historical user's perspective. In this way, compared with the server 1 sending the panoramic image to the head-mounted display device, the server 1 can only send the actual image seen by each target user's perspective to the head-mounted display device, which can greatly reduce the amount of data sent by the server 1 to the head-mounted display device and reduce the occupancy rate of network resources.

[0079] In some examples, the target user perspective is equal to the predicted user perspective. At this time, the server 1 can predict the perspective based on the historical user perspective at the current moment (e.g., motion trace of the historical user perspective, such as roll, pitch, and yaw; at the same time, a regression algorithm (linear regression (LR), support vector regression (SVR), ridge regression (RR), or deep learning algorithm (long short-term memory (LSTM)) is used to predict the perspective of the historical user perspective at the current moment) to obtain the predicted user perspective, such as the user perspective (pitch, yaw, roll) of the next cycle. In this way, the actual image actually seen by the user can be determined based on the user perspective of the next cycle and the pre-processed panoramic video, and the actual image can be transmitted to the head-mounted display device, which can greatly reduce the amount of data sent by the server 1 to the head-mounted display device and reduce the occupancy rate of network resources.

[0080] In some examples, the historical user perspective at the current moment can be input into the perspective prediction model to obtain the predicted user perspective. The training process of the perspective prediction model is as follows:

[0081] Obtain training sample data and labeling results of the training sample data; wherein the training sample data includes the actual user perspective corresponding to each moment when the user watches the specified panoramic video, and the labeling results include the actual user perspective at the current moment and the actual user perspective at the next moment.

[0082] The training sample data is input into the neural network model for training, and the prediction results of the neural network model for the training sample data are obtained.

[0083] Based on the prediction results and labeling results, the network parameters of the neural network model are adjusted until the neural network model converges to obtain a view prediction model.

[0084] It should be noted that the process of determining the actual image seen by the user from the historical user perspective corresponding to each video frame based on the video frame and the historical user perspective of the next cycle is similar to the process of determining the theoretical image seen by the user in each historical user perspective based on the historical user perspective and the panoramic video, and will not be repeated here.

[0085] It should be noted that the process of determining the actual bandwidth occupied by transmitting the actual image is similar to the process of determining the theoretical bandwidth occupied by transmitting the theoretical image, and will not be described in detail here.

[0086] S15: Generate an evaluation result of the preprocessed panoramic video based on the target parameter, wherein the target parameter includes at least one of transmission information, theoretical image, actual image, theoretical bandwidth and actual bandwidth.

[0087] In some examples, the target parameters include a theoretical image, an actual image, a theoretical bandwidth, and an actual bandwidth, and the theoretical image and the actual image correspond one to one. Based on the theoretical image and the actual image, the server 1 determines a first total number of pixels contained in the theoretical image (i.e., a first total number of effective pixels (pixels contained in the image actually seen by the user's naked eyes)) and a second total number of pixels contained in the actual image. Then, based on the first total number, the second total number, the theoretical bandwidth, and the actual bandwidth, the evaluation result of the panoramic video is determined, such as: We define It is the bandwidth consumed by transmitting effective pixels per unit. B_others represents the actual bandwidth, and B_fov represents the theoretical bandwidth; PN_others represents the second total number, and PN_fov represents the first total number. Afterwards, Norm(B) is used as the evaluation result of the panoramic video.

[0088] In some examples, the first total is equal to the resolution within a field of view (FoV) area (ie, a historical user viewing angle of a theoretical image corresponding to the first total).

[0089] In some examples, Indicates the average number of valid pixels of the panoramic video during playback, p h represents the pixel density of the video frame that has not been downsampled in the target user's perspective when the corresponding video frame of the pre-processed panoramic video is viewed from the target user's perspective, p l It represents the pixel density of the downsampled video frame in the target user's field of view when the corresponding video frame of the preprocessed panoramic video is viewed from the target user's field of view. w1 represents the area of ​​the video frame that is not downsampled in the target user's field of view. w2 represents the area of ​​the video frame that is downsampled in the target user's field of view. N represents the total number of video frames used. w1+w2=1.

[0090] In some examples, the target parameters include a theoretical image and an actual image. The server 1 determines a peak signal-to-noise ratio (PSNR) and a structural similarity (SSIM) between the theoretical image and the actual image based on the theoretical image and the actual image. Based on one or more of the peak signal-to-noise ratio and the structural similarity, an evaluation result of the panoramic video is obtained.

[0091] In some examples, the target parameters include transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes. Server 1 determines the evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes. For example: based on the ratio of the total amount of transmitted data to the total number of data frames corresponding to the total amount of transmitted data, determine the average amount of data for each video frame corresponding to the total amount of transmitted data; based on the ratio of the amount of data of the current video frame to the total number of tiles contained in the current video frame, determine the average amount of data for each tile in the current video frame; based on the amount of data of the current video frame, determine the amount of data for each tile; based on the sum of the difference in the amount of data of adjacent tiles, determine the first quality difference value of adjacent tiles in the current video frame; based on the sum of the difference in the amount of data of tiles at the same position in the nth frame image corresponding to the current video block and the next video block, determine the second quality difference value of tiles at the same position in the current video block and the next video block (for example: a video block contains n video frames, and the video frames in the same video block are divided into tiles in the same way, such as Figure 3 The video frame in the same video block shown is divided into 4 Tiles, namely Tile1, Tile2, Tile3 and Tile4. When calculating the second quality difference value, the first difference between the data amount of Tile1 of the first frame image in the current video block and the data amount of Tile1 of the first frame image in the next video block, the second difference between the data amount of Tile2 of the first frame image in the current video block and the data amount of Tile2 of the first frame image in the next video block, the third difference between the data amount of Tile3 of the first frame image in the current video block and the data amount of Tile3 of the first frame image in the next video block, and the fourth difference between the data amount of Tile4 of the first frame image in the current video block and the data amount of Tile4 of the first frame image in the next video block are calculated respectively; based on the sum of the first difference, the second difference, the third difference and the fourth difference, the first difference value between the current video block and the next video block is obtained. Similarly, server 1 traverses and calculates the nth difference value between the nth frame image of the current video block and the nth frame image of the next video block (N represents the total number of video frames contained in the current video block, n∈[1,N], n and N are both integers greater than or equal to 1); then, based on the ratio of the sum of the difference values ​​(the first difference value, the second difference value, ..., the sum of the Nth difference value) and N, a second quality difference value is obtained); based on the number of freezes, the average data amount, the first quality difference value and the second quality difference value, an evaluation result of the panoramic video is generated.

[0092] In some examples, the number of freezes, the average data volume, the first quality difference value, and the second quality difference value may be substituted into an evaluation formula to obtain an evaluation score; wherein the evaluation formula includes:

[0093] max{Qw i (I1+I2)-ws ×T stall}.

[0094] Among them, T stall represents the number of freezes, Q represents the average data volume, I1 represents the first quality difference value, I2 represents the second quality difference value, and w i and w s is a constant.

[0095] In this way, the number of freezes, the average quality value, the first quality difference value and the second quality difference value can be brought into the evaluation formula, and by setting w i and w s The size of Qw i (I1+I2)-w s ×T stall The value of Qw is the largest. i (I1+I2)-w s ×T stall The maximum value of is taken as the evaluation score, and the evaluation score is taken as the evaluation result of the panoramic video.

[0096] In some examples, the total amount of transmitted data and the number of freezes can be input into the evaluation model to obtain the evaluation result of the panoramic video. The training process of the evaluation model is as follows:

[0097] The training sample data and the marking results of the training sample data are obtained; wherein the training sample data includes the total transmission data volume and the number of freezes of the panoramic video in each historical cycle, and the marking results include the actual evaluation results of each cycle.

[0098] The training sample data is input into the neural network model for learning, and the prediction results of the neural network model for the training sample data are obtained.

[0099] The network parameters of the neural network model are adjusted based on the prediction results and the labeling results until the neural network model converges to obtain an evaluation model.

[0100] In some examples, the target parameters include transmission information, theoretical images, actual images, theoretical bandwidth, and actual bandwidth. At this time, the server 1 can determine the first total number of pixels contained in the theoretical image (i.e., the first total number of effective pixels (pixels contained in the image actually seen by the user's naked eyes)) based on the theoretical image and the actual image, and the second total number of pixels contained in the actual image. Afterwards, based on the first total number, the second total number, the theoretical bandwidth, and the actual bandwidth, a first result is determined. At the same time, based on the theoretical image and the actual image, the peak signal-to-noise ratio and structural similarity between the theoretical image and the actual image are determined. Afterwards, based on the peak signal-to-noise ratio and structural similarity, a second result is determined. At the same time, based on the total amount of transmitted data and the number of freezes, a third result is determined. Afterwards, the first result, the second result, and the third result are summarized to obtain an evaluation result of the panoramic video.

[0101] It should be noted that the process of determining the first result based on the first total, the second total, the theoretical bandwidth and the actual bandwidth is similar to the process of determining the evaluation result of the panoramic video based on the first total, the second total, the theoretical bandwidth and the actual bandwidth, and will not be repeated here.

[0102] At the same time, the process of determining the second result based on the peak signal-to-noise ratio and the structural similarity is similar to the process of obtaining the evaluation result of the panoramic video based on one or more of the peak signal-to-noise ratio and the structural similarity, and will not be repeated here.

[0103] At the same time, the process of determining the third result based on the total amount of transmitted data and the number of freezes is similar to the process of determining the evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes, and will not be repeated here.

[0104] From the above, it can be seen that the panoramic video evaluation method provided by the embodiment of the present disclosure performs video preprocessing on panoramic videos with different configuration parameters to obtain preprocessed panoramic videos with the same configuration parameters. The evaluation results of the generated preprocessed panoramic videos can be directly compared, which is convenient for users to evaluate panoramic videos with different transmission strategies (such as projection formats, encoding parameters) to find the best transmission strategy and ensure user experience.

[0105] As an optional implementation of the present disclosure, Figure 2 ,like Figure 4 As shown, the above S14 can be specifically implemented through the following S140 and S141.

[0106] S140: Perform perspective prediction based on historical user perspectives to obtain a predicted user perspective, and use the predicted user perspective as a target user perspective.

[0107] S141. Based on the target user perspective and the pre-processed panoramic video, determine transmission information when transmitting the pre-processed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image.

[0108] As an optional implementation of the present disclosure, Figure 2 ,like Figure 5 As shown, the above S14 can be specifically implemented through the following S142-S145.

[0109] S142: Determine a video frame to be transmitted in the next cycle based on the pre-processed panoramic video.

[0110] S143. Based on the video frame and the historical user perspective of the next period, determine the actual image seen by the user at the target user perspective corresponding to each video frame, and the actual bandwidth occupied by transmitting the actual image.

[0111] S144, simulate the transmission of the video frame to determine the total transmission data volume and the number of freezes in the next transmission cycle.

[0112] In some examples, in order to ensure the authenticity of the simulated transmission, a rate adaptive algorithm can be used to simulate the transmission of video frames to determine the total amount of transmitted data and the number of freezes in the next transmission cycle.

[0113] S145. Determine transmission information when transmitting the pre-processed panoramic video based on the total transmission data volume and the number of freezes.

[0114] As an optional implementation of the present disclosure, the target parameters include a theoretical image, an actual image, a theoretical bandwidth and an actual bandwidth, and the theoretical image and the actual image correspond one to one; Figure 2 ,like Figure 6 As shown, the above S15 can be specifically implemented through the following S150 and S151.

[0115] S150. Based on the theoretical image and the actual image, determine a first total number of pixels in the theoretical image and a second total number of pixels in the actual image.

[0116] S151. Determine an evaluation result of the panoramic video based on the first total, the second total, the theoretical bandwidth, and the actual bandwidth.

[0117] As an optional implementation of the present disclosure, the target parameters include a theoretical image and an actual image; Figure 2 ,like Figure 7 As shown, the above S15 can be specifically implemented through the following S152 and S153.

[0118] S152: Based on the theoretical image and the actual image, determine the peak signal-to-noise ratio and structural similarity between the theoretical image and the actual image.

[0119] S153: Obtain an evaluation result of the panoramic video based on one or more of the peak signal-to-noise ratio and the structural similarity.

[0120] As an optional implementation of the present disclosure, the target parameter includes transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes; Figure 2 ,like Figure 8 As shown, the above S15 can be specifically implemented through the following S154.

[0121] S154: Determine an evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes.

[0122] As an optional implementation of the present disclosure, Figure 2 ,like Fig. 9 As shown, the above S13 can be specifically implemented through the following S130 and S131.

[0123] S130: Convert the format of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format.

[0124] S131. Encode the panoramic video according to preset encoding parameters to obtain a preprocessed panoramic video.

[0125] As an optional implementation of the present disclosure, the projection method includes: any one of rectangular spherical projection, cube map projection, equiangular cube map projection, pyramid prism projection, and barrel distortion projection.

[0126] As an optional implementation of the present disclosure, the encoding parameters include: rectangular block size, quantization parameter, preset configuration parameters of video encoding, picture group size, intra-frame coded frame, forward and backward predicted coded frame, bidirectional predicted interpolated coded frame, one or more of the video coding standards.

[0127] The above mainly introduces the solution provided by the embodiment of the present invention from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0128] The embodiment of the present invention can divide the functional modules of the panoramic video evaluation device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0129] like Fig.10 As shown, an embodiment of the present invention provides a schematic diagram of the structure of a panoramic video evaluation device 10. The panoramic video evaluation device 10 includes an acquisition unit 101 and a processing unit 102.

[0130] The acquisition unit 101 is used to acquire a panoramic video and a historical user perspective corresponding to each moment when the user watches the panoramic video; the processing unit 102 is used to determine the theoretical image seen by the user at each historical user perspective and the theoretical bandwidth occupied by transmitting the theoretical image based on the historical user perspectives obtained by the acquisition unit 101 and the panoramic video obtained by the acquisition unit 101; the processing unit 102 is also used to perform video preprocessing on the panoramic video obtained by the acquisition unit 101 to obtain a preprocessed panoramic video; wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video. The processing unit 102 is also used to determine the transmission information when transmitting the preprocessed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image based on the preprocessed panoramic video obtained by the acquisition unit 101 and the historical user perspectives obtained by the acquisition unit 101; wherein the target user perspective is calculated based on the historical user perspective; the processing unit 102 is also used to generate an evaluation result of the preprocessed panoramic video based on the target parameters; wherein the target parameters include at least one of the transmission information, the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth.

[0131] As an optional implementation of the present disclosure, the processing unit 102 is specifically used to predict the perspective based on the historical user perspective acquired by the acquisition unit 101, obtain the predicted user perspective, and use the predicted user perspective as the target user perspective; the processing unit 102 is specifically used to determine the transmission information when transmitting the preprocessed panoramic video, the actual image seen by each target user perspective, and the actual bandwidth occupied by transmitting the actual image based on the target user perspective and the preprocessed panoramic video acquired by the acquisition unit 101.

[0132] As an optional implementation of the present disclosure, the processing unit 102 is specifically used to determine the video frames to be transmitted in the next cycle based on the preprocessed panoramic video; the processing unit 102 is specifically used to simulate the transmission of the video frames to determine the total transmission data volume and the number of freezes in the next cycle; the processing unit 102 is specifically used to determine the actual image seen by the user from the target user perspective corresponding to each video frame, and the actual bandwidth occupied by transmitting the actual image based on the video frames and the historical user perspectives of the next cycle acquired by the acquisition unit 101; the processing unit 102 is specifically used to determine the transmission information when transmitting the preprocessed panoramic video based on the total transmission data volume and the number of freezes.

[0133] As an optional implementation of the present disclosure, the target parameters include a theoretical image, an actual image, a theoretical bandwidth and an actual bandwidth, and the theoretical image and the actual image correspond to each other one by one; the processing unit 102 is specifically used to determine a first total number of pixels contained in the theoretical image and a second total number of pixels contained in the actual image based on the theoretical image and the actual image; the processing unit 102 is specifically used to determine an evaluation result of the panoramic video based on the first total number, the second total number, the theoretical bandwidth and the actual bandwidth.

[0134] As an optional implementation of the present disclosure, the target parameters include a theoretical image and an actual image; the processing unit 102 is specifically used to determine the peak signal-to-noise ratio and structural similarity between the theoretical image and the actual image based on the theoretical image and the actual image; the processing unit 102 is specifically used to obtain an evaluation result of the panoramic video based on one or more of the peak signal-to-noise ratio and the structural similarity.

[0135] As an optional implementation of the present disclosure, the target parameters include transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes; the processing unit 102 is specifically used to determine the evaluation result of the panoramic video based on the total amount of transmitted data and the number of freezes.

[0136] As an optional implementation of the present disclosure, the processing unit 102 is specifically used to convert the format of the panoramic video to generate an encoded panoramic video encoded according to a preset projection format; the processing unit 102 is specifically used to encode the encoded panoramic video according to preset encoding parameters to obtain a preprocessed panoramic video.

[0137] As an optional implementation of the present disclosure, the projection method includes: any one of rectangular spherical projection, cube map projection, equiangular cube map projection, pyramid prism projection, and barrel distortion projection.

[0138] As an optional implementation of the present disclosure, the encoding parameters include: rectangular block size, quantization parameter, preset configuration parameters of video encoding, picture group size, intra-frame coded frame, forward and backward predicted coded frame, bidirectional predicted interpolated coded frame, one or more of the video coding standards.

[0139] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and its role will not be repeated here.

[0140] Of course, the panoramic video evaluation device 10 provided in the embodiment of the present invention includes but is not limited to the above modules, for example, the panoramic video evaluation device 10 may also include a storage unit 103. The storage unit 103 may be used to store the program code of the panoramic video evaluation device 10, and may also be used to store data generated by the panoramic video evaluation device 10 during operation, such as data in a write request, etc.

[0141] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Fig.11 As shown, the electronic device may include: at least one processor 51 , a memory 52 , a communication interface 53 and a communication bus 54 .

[0142] Combine the following Fig.11 A detailed introduction to the various components of electronic equipment:

[0143] The processor 51 is the control center of the electronic device, and may be a processor or a general term for multiple processing elements. For example, the processor 51 is a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more DSPs, or one or more field programmable gate arrays (FPGAs).

[0144] In a specific implementation, as an embodiment, the processor 51 may include one or more CPUs, such as Fig.11 Also, as an embodiment, the electronic device 10 may include multiple processors, such as Fig.1151 and processor 56 are shown in FIG. Each of these processors may be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0145] The memory 52 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 52 may exist independently and be connected to the processor 51 via a communication bus 54. The memory 52 may also be integrated with the processor 51.

[0146] In a specific implementation, the memory 52 is used to store the data of the present invention and execute the software program of the present invention. The processor 51 can execute various functions of the air conditioner by running or executing the software program stored in the memory 52 and calling the data stored in the memory 52.

[0147] The communication interface 53 uses any transceiver-like device to communicate with other devices or communication networks, such as Radio Access Network (RAN), Wireless Local Area Networks (WLAN), terminals, the cloud, etc. The communication interface 53 may include an acquisition unit to implement the acquisition function.

[0148] The communication bus 54 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0149] As an example, combining Fig.10 The function of the acquisition unit 101 in the panoramic video evaluation device 10 is similar to Fig.11 The function of the communication interface 53 in the panoramic video evaluation device 10 is the same as that of the processing unit 102. Fig.11 The function of the processor 51 in the panoramic video evaluation device 10 is the same as that of the storage unit 103. Fig.11 The function of the memory 52 in is the same.

[0150] Another embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the method shown in the above method embodiment.

[0151] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0152] Fig.12 The diagram schematically shows a conceptual partial view of a computer program product provided by an embodiment of the present invention, where the computer program product includes a computer program for executing a computer process on a computing device.

[0153] In one embodiment, the computer program product is provided using a signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S11-S15 can be undertaken by one or more instructions associated with the signal bearing medium 410. In addition, Fig.12 The program instructions in also describe example instructions.

[0154] In some examples, signal bearing medium 410 may include computer readable medium 411, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), and the like.

[0155] In some implementations, signal bearing medium 410 may include computer recordable medium 412 such as, but not limited to, memory, read / write (R / W) CD, R / W DVD, and the like.

[0156] In some implementations, signal bearing medium 410 may include communication medium 413 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0157] The signal bearing medium 410 may be communicated by a wireless form of communication medium 413 (eg, a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0158] In some examples, such as for Fig.10 The described panoramic video evaluation apparatus 10 may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable medium 411 , computer-recordable medium 412 , and / or communication medium 413 .

[0159] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0160] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0162] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0163] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0164] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A panoramic video evaluation method, characterized in that: include: Obtaining a panoramic video and a historical user perspective corresponding to each moment when the user watches the panoramic video; Based on the historical user perspectives and the panoramic video, determining a theoretical image seen by the user at each of the historical user perspectives, and a theoretical bandwidth occupied by transmitting the theoretical image; Performing video preprocessing on the panoramic video to obtain a preprocessed panoramic video; wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video; Based on the pre-processed panoramic video and the historical user perspectives, determine transmission information when transmitting the pre-processed panoramic video, actual images seen by each target user perspective, and actual bandwidth occupied by transmitting the actual images; wherein the target user perspective is calculated based on the historical user perspectives; Based on target parameters, an evaluation result of the pre-processed panoramic video is generated; wherein the target parameters include at least one of the transmission information, the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth.

2. The panoramic video evaluation method according to claim 1, characterized in that: The determining, based on the pre-processed panoramic video and the historical user perspective, transmission information when transmitting the pre-processed panoramic video, an actual image seen from the perspective of each target user, and an actual bandwidth occupied by transmitting the actual image includes: Performing perspective prediction based on the historical user perspective to obtain a predicted user perspective, and using the predicted user perspective as a target user perspective; Based on the target user perspective and the pre-processed panoramic video, transmission information when transmitting the pre-processed panoramic video, actual images seen from each of the target user perspectives, and actual bandwidth occupied by transmitting the actual images are determined.

3. The panoramic video evaluation method according to claim 1, characterized in that: The determining, based on the pre-processed panoramic video and the historical user perspective, transmission information when transmitting the pre-processed panoramic video, an actual image seen from the perspective of each target user, and an actual bandwidth occupied by transmitting the actual image includes: Based on the preprocessed panoramic video, determining a video frame to be transmitted in a next cycle; Based on the video frames and the historical user perspectives of the next period, determining an actual image seen by the user at a target user perspective corresponding to each of the video frames, and an actual bandwidth occupied by transmitting the actual image; Simulate the transmission of the video frame to determine the total transmission data volume and the number of freezes in the next transmission cycle; Based on the total amount of transmitted data and the number of freezes, transmission information when transmitting the pre-processed panoramic video is determined.

4. The panoramic video evaluation method according to claim 1, characterized in that: The target parameters include the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth, and the theoretical image and the actual image correspond one to one; The step of generating an evaluation result of the preprocessed panoramic video based on the target parameter includes: Based on the theoretical image and the actual image, determining a first total number of pixels in the theoretical image and a second total number of pixels in the actual image; An evaluation result of the panoramic video is determined based on the first total, the second total, the theoretical bandwidth, and the actual bandwidth.

5. The panoramic video evaluation method according to claim 1, characterized in that: The target parameters include the theoretical image and the actual image; The step of generating an evaluation result of the preprocessed panoramic video based on the target parameter includes: Based on the theoretical image and the actual image, determining a peak signal-to-noise ratio and a structural similarity between the theoretical image and the actual image; Based on one or more of the peak signal-to-noise ratio and the structural similarity, an evaluation result of the panoramic video is obtained.

6. The panoramic video evaluation method according to claim 1, characterized in that: The target parameters include transmission information, and the transmission information includes the total amount of transmitted data and the number of freezes; The step of generating an evaluation result of the preprocessed panoramic video based on the target parameter includes: Based on the total amount of transmitted data and the number of freezes, an evaluation result of the panoramic video is determined.

7. The panoramic video evaluation method according to claim 1, characterized in that: The performing video preprocessing on the panoramic video to obtain a preprocessed panoramic video includes: Converting the panoramic video into a format to generate a coded panoramic video encoded according to a preset projection format; The encoded panoramic video is encoded according to preset encoding parameters to obtain a preprocessed panoramic video.

8. The panoramic video evaluation method according to claim 7, characterized in that: The projection method includes: any one of rectangular spherical projection, cube map projection, equiangular cube map projection, pyramid prism projection, and barrel distortion projection.

9. The panoramic video evaluation method according to claim 7, characterized in that: The encoding parameters include: rectangular block size, quantization parameter, preset configuration parameters of video encoding, picture group size, intra-frame coded frame, forward and backward predicted coded frame, bidirectional predicted interpolated coded frame, one or more of the video encoding standards.

10. A panoramic video evaluation device, characterized in that: include: An acquisition unit, used to acquire a panoramic video and a historical user perspective corresponding to each moment when a user watches the panoramic video; a processing unit, configured to determine, based on the historical user perspectives acquired by the acquisition unit and the panoramic video acquired by the acquisition unit, a theoretical image seen by the user at each of the historical user perspectives, and a theoretical bandwidth occupied by transmitting the theoretical image; The processing unit is further configured to perform video preprocessing on the panoramic video acquired by the acquisition unit to obtain a preprocessed panoramic video; wherein the configuration parameters of the preprocessed panoramic video are different from the configuration parameters of the panoramic video; The processing unit is further configured to determine, based on the pre-processed panoramic video acquired by the acquisition unit and the historical user perspectives acquired by the acquisition unit, transmission information when transmitting the pre-processed panoramic video, an actual image seen by each target user perspective, and an actual bandwidth occupied by transmitting the actual image; wherein the target user perspective is calculated based on the historical user perspectives; The processing unit is further used to generate an evaluation result of the pre-processed panoramic video based on a target parameter; wherein the target parameter includes at least one of the transmission information, the theoretical image, the actual image, the theoretical bandwidth and the actual bandwidth.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the panoramic video evaluation method according to any one of claims 1 to 9.

12. A computer program product, characterized in that When the computer program product runs on a computer, the computer is enabled to implement the panoramic video evaluation method according to any one of claims 1 to 9.

Images