Video processing method, video playing method and device

Abstract

The present invention provides a video processing method, a video playback method and a device, belonging to the technical field of online video playback, wherein a video processing method comprises: obtaining a first determination result; the first determination result is determined according to the transmission delay between the video playback terminal and the edge server, the first determination result is used to indicate that the video image corresponding to the first perspective is expanded, the first perspective is the current perspective or the default main perspective of the user of the video playback terminal; according to the first determination result, the first video image in the panoramic video image is sent to the video playback terminal; the first video image is the expanded video image determined according to the first determination result. The present invention can reduce the bandwidth occupied by the transmission as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of online video playback, and in particular to a video processing method, a video playback method and a device. Background Art

[0002] Panoramic video or VR video transmission usually involves the transmission of stitched panoramic videos, which requires a large transmission bandwidth and a low effective video ratio. To solve this problem, the following technical solutions have been proposed: only transmitting part of the video content in the panoramic video, for example, only transmitting the video image content corresponding to the user's current viewing angle. However, this solution requires real-time requesting the video image content corresponding to the changed viewing angle according to the user's viewing angle change, which can easily cause a large motion to photon (MTP) delay. That is, after the user turns the viewing angle, it takes a long time to play the video content corresponding to the turned viewing angle, affecting the user experience. Summary of the invention

[0003] In view of this, the present invention provides a video processing method, a video playback method and a device, which are used to solve the problem that the current panoramic video or VR video transmission schemes either need to occupy a large bandwidth or easily cause a large MTP delay.

[0004] In order to solve the above technical problems, in a first aspect, the present invention provides a video processing method, which is applied to an edge server, comprising:

[0005] Obtaining a first determination result; wherein the first determination result is determined based on a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that a video image corresponding to a first perspective is expanded, and the first perspective is a current perspective or a default main perspective of a user of the video playback terminal;

[0006] According to the first determination result, a first video image in the panoramic video image is sent to the video playback terminal; wherein the first video image is an expanded video image determined according to the first determination result.

[0007] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0008] Optionally, obtaining the first determination result includes:

[0009] In a case where the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter, and using the first image expansion parameter as the first determination result;

[0010] In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result;

[0011] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0012] Optionally, when the transmission delay is less than a fourth preset value, after determining that the image expansion related parameter is a second image expansion parameter, the method further includes:

[0013] When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result;

[0014] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0015] Optionally, the sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes:

[0016] Sending the first video image and the panoramic video image to the video playback terminal in two paths respectively;

[0017] or,

[0018] splicing the first video image with the panoramic video image and sending the spliced ​​image to the video playback terminal;

[0019] Among them, the first video image is transmitted using a first bit rate and / or a first resolution; the panoramic video image is transmitted using a second bit rate and / or a second resolution; the second bit rate is smaller than the first bit rate, and the second resolution is smaller than the first resolution.

[0020] Optionally, the sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes:

[0021] Sending the first video image and the third video image to the video playback terminal;

[0022] The third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0023] Optionally, the user potential area of ​​interest is predicted in advance based on video content in the panoramic video image, or is determined based on a mark in a video source of the panoramic video image.

[0024] Optionally, the third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion-related parameters and / or the expanded image size-related parameters.

[0025] In a second aspect, the present invention further provides a video playback method, applied to an edge server, comprising:

[0026] Obtain a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded, and the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0027] According to the second determination result, a second video image in the panoramic video image is sent to the video playback terminal; wherein the second video image is an expanded video image determined according to the second determination result.

[0028] Optionally, the second determination result is also used to indicate at least one of a perspective expansion-related parameter when the first perspective is expanded and an image size-related parameter corresponding to the expanded perspective.

[0029] In a third aspect, the present invention further provides a video playback method, applied to a video playback terminal, comprising:

[0030] Receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0031] Play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0032] Optionally, before receiving the first video image in the panoramic video image sent by the edge server using the first bit rate and / or the first resolution, the method further includes:

[0033] Acquire the first viewing angle and the transmission delay;

[0034] Obtaining the first determination result according to the transmission delay;

[0035] The first determination result is sent to the edge server.

[0036] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0037] Optionally, obtaining a first determination result according to the transmission delay includes:

[0038] In a case where the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter, and using the first image expansion parameter as the first determination result;

[0039] In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result;

[0040] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0041] Optionally, when the transmission delay is less than a fourth preset value, after determining that the image expansion related parameter is a second image expansion parameter, the method further includes:

[0042] When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result;

[0043] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0044] Optionally, the receiving a first video image sent by the edge server includes:

[0045] Receiving the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image;

[0046] After playing the first video image or playing the video image corresponding to the first viewing angle in the first video image, the method further includes:

[0047] The user's latest viewing angle is obtained, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, the video image corresponding to the user's latest viewing angle in the third video image is played.

[0048] Optionally, after playing the video image corresponding to the first perspective in the first video image, the method further includes:

[0049] Get the user's viewing angle;

[0050] When the viewing angle movement angle is less than or equal to a fifth preset value, playing a video image corresponding to a new viewing angle of the user in the first video image;

[0051] When the viewing angle is greater than a sixth preset value, the user's new viewing angle is sent to the edge server to request the edge server to send a video image including a fourth video image, where the fourth video image is a video image corresponding to the user's new viewing angle;

[0052] The values ​​of the fifth preset value and the sixth preset value are adapted to the size of the expansion area in the first video image.

[0053] In a fourth aspect, the present invention further provides a video playback method, applied to a video playback terminal, comprising:

[0054] Receiving a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that a first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0055] Play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0056] Optionally, the second determination result is used to indicate at least one of a parameter related to the viewing angle expansion when the first viewing angle is expanded and a parameter related to the image size corresponding to the expanded viewing angle.

[0057] In a fifth aspect, the present invention further provides an edge server, comprising:

[0058] A first acquisition module is used to acquire a first determination result; wherein the first determination result is determined according to a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate to expand a video image corresponding to a first perspective, the first perspective is a current perspective or a default main perspective of a user of the video playback terminal;

[0059] The first transmission module is used to send the first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is the expanded video image determined according to the first determination result.

[0060] In a sixth aspect, the present invention further provides an edge server, comprising:

[0061] A second acquisition module is used to acquire a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate to expand the first viewing angle, the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0062] The second transmission module is used to send the second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is the expanded video image determined according to the second determination result.

[0063] In a seventh aspect, the present invention further provides a video playback terminal, comprising:

[0064] A first receiving module is configured to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to a first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0065] The first playing module is used to play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0066] In an eighth aspect, the present invention further provides a video playback terminal, comprising:

[0067] A second receiving module is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0068] The second playing module is used to play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0069] In a ninth aspect, the present invention further provides an edge server, comprising: a transceiver and a processor;

[0070] The processor is used to obtain a first determination result; wherein the first determination result is determined according to a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate to expand a video image corresponding to a first perspective, the first perspective being a current perspective or a default main perspective of a user of the video playback terminal;

[0071] The transceiver is used to send a first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is an expanded video image determined according to the first determination result.

[0072] In a tenth aspect, the present invention further provides an edge server, comprising: a transceiver and a processor;

[0073] The processor is used to obtain a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate to expand the first perspective, the first perspective is the current perspective of the user of the video playback terminal or the default main perspective;

[0074] The transceiver is used to send the second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is an expanded video image determined according to the second determination result.

[0075] In an eleventh aspect, the present invention further provides a video playback terminal, comprising: a transceiver and a processor;

[0076] The transceiver is used to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0077] The processor is used to play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0078] In a twelfth aspect, the present invention further provides a video playback terminal, comprising: a transceiver and a processor;

[0079] The transceiver is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0080] The processor is used to play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0081] In a thirteenth aspect, the present invention further provides an edge server, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the steps in any one of the above-mentioned video processing methods are implemented.

[0082] In a fourteenth aspect, the present invention further provides a video playback terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the steps in any one of the above-mentioned video playback methods are implemented.

[0083] In a fifteenth aspect, the present invention further provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in any one of the above-mentioned video processing methods or the steps in any one of the above-mentioned video playback methods.

[0084] The beneficial effects of the above technical solution of the present invention are as follows:

[0085] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 A schematic diagram of a video processing method in Embodiment 1 of the present invention;

[0087] Figure 2 A schematic diagram of a video processing method in Embodiment 2 of the present invention;

[0088] Figure 3 A schematic diagram of a stitched distorted image transmitted during the transmission of a panoramic video or a VR video;

[0089] Figure 4 A schematic diagram of a video processing method in Embodiment 3 of the present invention;

[0090] Figure 5 A schematic diagram of a flow chart of a video processing method in Embodiment 4 of the present invention;

[0091] Figure 6 A schematic diagram of a flow chart of a video processing method in Embodiment 5 of the present invention;

[0092] Figure 7 Schematic diagram of a video processing method in Embodiment 6 of the present invention;

[0093] Figure 8 This is a flow chart of a video playback method in Embodiment 7 of the present invention;

[0094] Fig. 9 This is a flow chart of a video playback method in Embodiment 8 of the present invention;

[0095] Fig.10 This is a flowchart of a video playback method in Embodiment 9 of the present invention;

[0096] Fig.11 This is a flow chart of a video playback method in Embodiment 10 of the present invention;

[0097] Fig.12 A schematic diagram of a video playback process in an embodiment of the present invention;

[0098] Fig.13 This is a schematic diagram of the structure of an edge server in Embodiment 11 of the present invention;

[0099] Fig.14 This is a schematic diagram of the structure of an edge server in Embodiment 12 of the present invention;

[0100] Fig.15 This is a schematic diagram of the structure of a video playback terminal in Embodiment 13 of the present invention;

[0101] Fig.16 This is a schematic diagram of the structure of a video playback terminal in Embodiment 14 of the present invention;

[0102] Fig.17 This is a schematic diagram of the structure of an edge server in Embodiment 15 of the present invention;

[0103] Fig.18 This is a schematic diagram of the structure of an edge server in Embodiment 16 of the present invention;

[0104] Fig.19 This is a structural diagram of a video playback terminal in Embodiment 17 of the present invention;

[0105] Fig. 20 This is a schematic diagram of the structure of a video playback terminal in Embodiment 18 of the present invention;

[0106] Fig.21 This is a schematic diagram of the structure of an edge server in Embodiment 19 of the present invention;

[0107] Fig. 22 This is a schematic diagram of the structure of a video playback terminal in Embodiment 20 of the present invention. DETAILED DESCRIPTION

[0108] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0109] In the embodiment of the present invention, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In the embodiment of the present invention, the term "at least one of" describes the association relationship of the associated objects, indicating that one of the objects may exist alone, multiple objects may exist at the same time, or all the objects may exist.

[0110] See also Figure 1 , Figure 1 A schematic flow chart of a video processing method provided in Embodiment 1 of the present invention, the method is applied to an edge server, and includes the following steps:

[0111] Step 11: Obtain a first determination result; wherein the first determination result is determined based on a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that a video image corresponding to a first perspective is expanded, and the first perspective is a current perspective or a default main perspective of a user of the video playback terminal;

[0112] Specifically, the first determination result may be determined by the edge server according to the transmission delay, or may be determined by the video playback terminal according to the transmission delay and then sent to the edge server.

[0113] Step 12: According to the first determination result, the first video image in the panoramic video image is sent to the video playback terminal; wherein the first video image is the expanded video image determined according to the first determination result. Specifically, the first video image in the panoramic video image can be sent to the video playback terminal using the first bit rate and / or the first resolution, that is, the edge server can send the first video image in the panoramic video image to the video playback terminal using the first bit rate and the first resolution, or can send the first video image in the panoramic video image to the video playback terminal using the first bit rate and other resolutions, or can send the first video image in the panoramic video image to the video playback terminal using other bit rates and the first resolution. wherein the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0114] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0115] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0116] That is to say, the first determination result is also used to indicate image expansion-related parameters when the video image corresponding to the first perspective is expanded, or the first determination result is also used to indicate image size-related parameters after the video image corresponding to the first perspective is expanded, or the first determination result is also used to indicate image expansion-related parameters when the video image corresponding to the first perspective is expanded and image size-related parameters after expansion.

[0117] In the embodiments of the present invention, not only can it be determined whether to expand the video image corresponding to the first perspective based on the transmission delay between the video playback terminal and the edge server, but the expansion-related parameters (including the size of the expansion) can also be determined based on the transmission delay between the video playback terminal and the edge server. That is, different transmission delays result in different sizes of expansion of the video image corresponding to the first perspective, and the size of the expansion corresponds to the transmission delay, so as to reduce the bandwidth occupied by the transmission as much as possible while ensuring a low MTP delay.

[0118] Optionally, obtaining the first determination result includes:

[0119] In the case where the transmission delay is greater than or equal to the third preset value, determining the image expansion related parameter as a first image expansion parameter, and using the first image expansion parameter as the first determination result; of course, the first determination result may also include other information;

[0120] In the case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; of course, the first determination result may also include other information;

[0121] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0122] Optionally, when the transmission delay is less than a fourth preset value, after determining that the image expansion related parameter is a second image expansion parameter, the method further includes:

[0123] When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter as a third image expansion parameter, and taking the third image expansion parameter as the first determination result; of course, the first determination result may also include other information;

[0124] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0125] Optionally, the sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes:

[0126] Sending the first video image and the panoramic video image to the video playback terminal in two paths respectively;

[0127] or,

[0128] splicing the first video image with the panoramic video image and sending the spliced ​​image to the video playback terminal;

[0129] Among them, the first video image is transmitted using a first bit rate and / or a first resolution; the panoramic video image is transmitted using a second bit rate and / or a second resolution; the second bit rate is smaller than the first bit rate, and the second resolution is smaller than the first resolution.

[0130] Optionally, the sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes:

[0131] Sending the first video image and the third video image to the video playback terminal;

[0132] The third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0133] Optionally, the user potential area of ​​interest is predicted in advance based on video content in the panoramic video image, or is determined based on a mark in a video source of the panoramic video image.

[0134] Optionally, the third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion-related parameters and / or the expanded image size-related parameters.

[0135] See also Figure 2 , Figure 2A schematic flow chart of a video processing method provided in Embodiment 2 of the present invention, the method is applied to an edge server, and includes the following steps:

[0136] Step 21: Obtain a first viewing angle and a transmission delay between the video playback terminal and the edge server, where the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0137] The current viewing angle of the user of the video playback terminal may be sent by the video playback terminal, and the current viewing angle of the user may be obtained according to the current viewpoint of the user. The edge server may obtain the current viewing angle of the user of the video playback terminal sent by the video playback terminal, and if not, the default (ie, default) main viewing angle may be used instead.

[0138] Step 22: obtaining a first determination result according to the transmission delay, wherein the first determination result is used to indicate at least one of expansion of the video image corresponding to the first viewing angle, image expansion-related parameters, and parameters related to the size of the expanded image;

[0139] The image expansion-related parameters may be, for example, an expansion ratio and / or an expansion size, etc., and the image size-related parameters after expansion may be, for example, coordinates of four points of a rectangular image, etc.

[0140] Step 23: Based on the first determination result, a first bit rate (bit rate, also called bit stream) and / or a first resolution are used to send the first video image in the panoramic video image to the video playback terminal; wherein the first video image is an expanded video image determined based on the first determination result; the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0141] Of course, the edge server may also obtain a determination result indicating that the video image corresponding to the first viewing angle should not be expanded according to the transmission delay. For example, if the transmission delay is very small, the video image corresponding to the first viewing angle may not need to be expanded.

[0142] When the edge server determines, based on the transmission delay, that there is no need to expand the video image corresponding to the first perspective, it only needs to transmit the video image corresponding to the first perspective at the first bit rate and / or first resolution.

[0143] The first video image in the panoramic video image may specifically be a video image composed of a video image of a region corresponding to the first viewing angle and a video image of an expanded region in the panoramic video image.

[0144] Expansion refers to expanding the area of ​​a video image, whereby the expanded video image includes more video content, specifically, video content of some areas surrounding the area before expansion.

[0145] The edge server may be other devices that implement the functions of the edge server. In the scenario of panoramic video playback or virtual reality (VR) video playback, one implementation process may be:

[0146] First, images captured by two or more lenses are stitched together into a panoramic video or VR video;

[0147] Then, the remote server transmits the panoramic video or VR video to the edge server near the user's video playback terminal. The transmitted image here is usually a spliced ​​distorted image (the aspect ratio may be 2:1, for example). Figure 3 What is shown is a spliced ​​distorted image;

[0148] Finally, the edge server performs distortion correction, analysis, transcoding, decoding, rendering and / or compression on the spliced ​​distorted image and sends it to the video playback terminal for video playback.

[0149] In an embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle and / or the related parameters of the expansion (including the size of the expansion) is determined based on the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not required, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience. Moreover, when the user's viewing angle moves, it can provide high-definition video images better, can effectively follow up the changes in the user's viewing angle, and can respond in time in motion videos or when the user's viewing angle changes.

[0150] The specific process of sending the first video image in the panoramic video image to the video playback terminal using the first bit rate and / or the first resolution according to the first determination result may be:

[0151] According to the first determination result, the expansion area mapped on the original image (stitched distorted image) is determined, and then the maximum circumscribed rectangular area of ​​the expansion area is obtained, and the video image after the distorted image in the rectangular area is processed by distortion correction is sent to the video playback terminal. For example, the video image after the distorted image in the rectangular area is processed by distortion correction can be compressed with high resolution and high bit rate and then sent to the video playback terminal.

[0152] The above video processing method is described below with examples.

[0153] Optionally, obtaining a first determination result according to the transmission delay includes:

[0154] When the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter;

[0155] When the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter;

[0156] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0157] In the embodiment of the present invention, the expansion-related parameters when expanding the image corresponding to the user's current viewing angle or the default main viewing angle are adapted to the transmission delay between the video playback terminal and the edge server. In other words, the size of the expanded area when expanding the image corresponding to the user's current viewing angle or the default main viewing angle is adapted to the transmission delay between the video playback terminal and the edge server. In other words, the image expansion-related parameters can be adjusted according to the delay between the video playback terminal and the edge server.

[0158] Specifically, if the transmission delay between the video playback terminal and the edge server is large, the video image corresponding to the current viewing angle or the default main viewing angle of the user of the video playback terminal is greatly expanded, for example, 1 / 3 is expanded up, down, left, and right, to cope with some smaller changes in the user's viewing angle. If the transmission delay between the video playback terminal and the edge server is small, the video image corresponding to the current viewing angle or the default main viewing angle of the user of the video playback terminal is slightly expanded, for example, 1 / 10 is expanded up, down, left, and right, or even not expanded. This is because, if the transmission delay between the video playback terminal and the edge server is large, when the user's viewing angle changes, the terminal uploads the changed viewing angle to the edge server, and the edge server renders the video image corresponding to the new viewing angle according to the new viewing angle, and finally sends it to the video playback terminal. It takes a long time in the middle, resulting in the terminal playing the video image corresponding to the changed viewing angle after a long time of user viewing angle change, affecting the user experience. Therefore, it is necessary to expand the video image corresponding to the current viewing angle. The expanded video image can meet the video image content required by the user's small-angle mobile viewing angle, and there is no need to re-request the edge server to send it, thereby improving the user experience. If the transmission delay between the video playback terminal and the edge server is small, then even if the video playback terminal re-requests the edge server to send the video image content corresponding to the changed viewing angle, it does not need to spend much time. Therefore, there is no need to expand the video image corresponding to the user's current viewing angle or only a small expansion is performed to reduce the bandwidth occupied by the video content transmission.

[0159] Optionally, when the transmission delay is less than a fourth preset value, after determining that the image expansion related parameter is a second image expansion parameter, the method further includes:

[0160] When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter;

[0161] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0162] That is, if the transmission delay between the video playback terminal and the edge server is always small, the expansion of the video image corresponding to the current viewing angle or the default main viewing angle of the user of the video playback terminal can be gradually reduced.

[0163] Usually, the clear field of vision of human eyes is 35 degrees horizontally and 20 degrees vertically; the field of vision of peripheral vision is 120 degrees horizontally and 60 degrees vertically; the delay of VR panoramic video is more suitable within 20ms, and the bandwidth is about 40M. However, in actual applications, the end-to-end delay and bandwidth will change dynamically. Here we perform corresponding processing on the wireless edge side to cope with the change of delay.

[0164] The following example illustrates how to determine the image expansion related parameters based on the transmission delay between the video playback terminal and the edge server.

[0165] Generally, VR experience requires that the motion to photon (MTP) delay does not exceed a certain threshold (Thresh_MTP), such as 20ms. The MTP delay includes the round-trip transmission delay t between the video playback terminal to the edge server and the edge server to the video playback terminal, and also includes the rendering delay. For example, when the rendering frame rate is f, the rendering delay is 1 / f. One method for determining the expansion-related parameters can be:

[0166] When the MTP delay is greater than Thresh_MTP, that is, when the transmission delay between the video playback terminal and the edge server is greater than (Thresh_MTP-1 / f) / 2, the expansion-related parameter N=3, that is, the video image area corresponding to the current viewing angle or the default main viewing angle of the video terminal user is expanded by 1 / 3 on all sides, for example, 1 / 3 on the top, bottom, left and right sides. The video image viewing angle after expansion based on the expansion-related parameter is larger and can cover general viewing angle changes.

[0167] When the network delay is unstable or the jitter is large, that is, in a certain time window (T-window), the MTP delay is greater than Thresh_MTP, in other words, there is a moment when the transmission delay between the video playback terminal and the edge server is greater than (Thresh_MTP-1 / f) / 2, the expansion-related parameters can also take a value of 3.

[0168] When the network delay is small and relatively stable, that is, within a certain time window (T-window), the MTP delay remains less than Thresh_MTP, that is, the transmission delay between the video playback terminal and the edge server is continuously less than (Thresh_MTP-1 / f) / 2, the expansion angle can be gradually reduced, that is, the expansion parameter N is increased until N=N max , N max You can take 10, or take the minimum value of the expansion angle, that is, N = N max , or the video image corresponding to the current viewing angle or the default main viewing angle of the user of the video playback terminal is not expanded, that is, the expanded viewing angle is equal to the current viewing angle or the default main viewing angle of the user of the video playback terminal.

[0169] In other optional specific implementations, when the height of the video image corresponding to the first perspective is less than a preset threshold (for example, 1 / 3 of the resolution height), it is determined based on the transmission delay whether to expand the video image corresponding to the first perspective, and at least one of the image expansion-related parameters and the parameters related to the size of the expanded image.

[0170] Optionally, the sending, according to the first determination result, a first bit rate and / or a first resolution of the first video image in the panoramic video image to the video playback terminal includes:

[0171] Sending the first video image and the panoramic video image to the video playback terminal in two paths respectively;

[0172] or,

[0173] splicing the first video image with the panoramic video image and sending the spliced ​​image to the video playback terminal;

[0174] The panoramic video image is transmitted using a second bit rate and / or a second resolution, the second bit rate is lower than the first bit rate, and the second resolution is lower than the first resolution.

[0175] In an embodiment of the present invention, when the first video image after expanding the video image corresponding to the user's current perspective or the default main perspective is transmitted with high resolution and / or high bit rate, the other video images in the panoramic video image are transmitted with reduced resolution and / or reduced bit rate.

[0176] That is to say, in the embodiment of the present invention, the data within the user's viewing angle is compressed at a high bit rate, and the data outside the viewing angle is compressed and transmitted at a low bit rate. Thus, the transmission bandwidth can be reduced, and when the user's viewing angle changes but the terminal does not receive the video image within the new viewing angle range transmitted at a high bit rate in time, the video image corresponding to the new viewing angle in the panoramic video image transmitted at a low bit rate can be played. In other words, the change of the user's viewing angle can be effectively followed up, and timely response can be given in motion videos or when the user's viewing angle changes.

[0177] Specifically, the expanded video image corresponding to the first viewing angle and the panoramic video image may be transmitted in two paths, or may be spliced ​​and then transmitted.

[0178] Optionally, the sending, according to the first determination result, a first bit rate and / or a first resolution of the first video image in the panoramic video image to the video playback terminal includes:

[0179] Sending the first video image and the third video image to the video playback terminal using the first bit rate and / or the first resolution;

[0180] The third video image includes a video image of a user's potential region of interest (RIO) in the panoramic video image.

[0181] In the embodiment of the present invention, the edge server not only uses the first bit rate and / or the first resolution to send the first video image after expanding the video image corresponding to the user's current viewing angle or the default main viewing angle to the video playback terminal, but also uses the first bit rate and / or the first resolution to send the video image of the user's potential area of ​​interest to the terminal. In other words, the video image of the area that is expected to be of interest to the user is sent to the terminal, so that during the video playback process, if the user is really interested in the video image of this area, the user can immediately see the video image of this area after moving the viewing angle, without the terminal having to upload the user's moving viewing angle to request the video image of this area from the edge server.

[0182] Further optionally, the user potential area of ​​interest is predicted in advance based on the video content in the panoramic video image, or is determined based on a mark in the video source of the panoramic video image.

[0183] When the edge server predicts the user's potential area of ​​interest based on the video content in advance, it can specifically detect the subsequent video images and audio in advance to predict the user's potential area of ​​interest in the subsequent video, for example, the area corresponding to the sound source direction of the stereo audio in the currently playing video content. When the edge server determines the user's potential area of ​​interest based on the mark of the video source, the video producer can mark the user's potential area of ​​interest when making a panoramic video. In addition, the video producer can also mark the current main perspective when making a panoramic video. During the subsequent panoramic video playback process, if the user's actual current perspective is not obtained, the current main perspective (that is, the default main perspective) is used as the above-mentioned first perspective. The current main perspective is the main area of ​​interest.

[0184] Optionally, the third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion-related parameters and / or the expanded image size-related parameters.

[0185] For example, when rendering, the edge server decodes the video image corresponding to the current perspective of the user of the video playback terminal or the default main perspective and the video image of its expanded area, as well as the video image of the user's potential area of ​​interest and the video image of its expanded area, and transmits them to the video playback terminal, such as a VR terminal, at the original resolution (of course, they can also be simply compressed).

[0186] In the embodiment of the present invention, subsequent potential regions of interest may be predicted as candidate expanded viewing angle regions and transcoded at high resolution and high bit rate.

[0187] Similar to the first video image obtained by expanding the video image corresponding to the user's current viewing angle or the default main viewing angle, the third video image can be transmitted separately or spliced ​​with the panoramic video image for transmission.

[0188] See also Figure 4 , Figure 4 A schematic flow chart of a video processing method provided in Embodiment 3 of the present invention, the method is applied to an edge server, and includes the following steps:

[0189] Step 41: receiving a first determination result sent by a video playback terminal, wherein the first determination result is determined based on a transmission delay between the video playback terminal and the edge server; wherein the first determination result is used to indicate at least one of expansion of a video image corresponding to a first viewing angle, image expansion-related parameters, and expanded image size-related parameters, wherein the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0190] The image expansion related parameters may be, for example, an expansion ratio and / or an expansion size, etc., and the image size related parameters after expansion may be, for example, coordinates of four points of a rectangular image, etc. If the first determination result does not indicate the image size related parameters after expansion, the edge server also needs to obtain the first viewing angle, determine the video image area corresponding to the first viewing angle, and expand the video image area corresponding to the first viewing angle, for example, expand the video image area corresponding to the first viewing angle according to the image expansion related parameters;

[0191] Step 42: Based on the first determination result, the first video image in the panoramic video image is sent to the video playback terminal using a first bit rate and / or a first resolution; wherein the first video image is an expanded video image determined based on the first determination result; the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0192] Of course, the first determination result may also be used to indicate that the video image corresponding to the first viewing angle is not to be expanded. For example, if the transmission delay is small, the terminal may obtain the first determination result indicating that the video image corresponding to the first viewing angle is not to be expanded.

[0193] When the first determination result indicates not to expand the video image corresponding to the first viewing angle, the edge server sends only the video image corresponding to the first viewing angle to the video playback terminal at the first bit rate and / or first resolution according to the first determination result.

[0194] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle and / or the related parameters of the expansion (including the size of the expansion) is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that the expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0195] In an embodiment of the present invention, when the first video image after expanding the video image corresponding to the user's current perspective or the default main perspective is transmitted with high resolution and / or high bit rate, the other video images in the panoramic video image are transmitted with reduced resolution and / or reduced bit rate.

[0196] Specifically, the expanded video image and the panoramic video image can be transmitted in two ways, or they can be spliced ​​and then transmitted.

[0197] In an embodiment of the present invention, the edge server not only sends the first video image after expanding the video image corresponding to the user's current perspective or the default main perspective to the video playback terminal, but also can send the video image of the user's potential area of ​​interest to the terminal, or send the video image of the expanded video area when the video image of the user's potential area of ​​interest is expanded and the video image of the user's potential area of ​​interest to the terminal.

[0198] For details, please refer to the above-mentioned embodiment 2, which will not be described in detail here.

[0199] See also Figure 5 , Figure 5 A schematic flow chart of a video processing method provided in Embodiment 4 of the present invention, the method is applied to an edge server, and includes the following steps:

[0200] Step 51: Obtain a second determination result; wherein the second determination result is a transmission delay determination, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate to expand the first viewing angle, the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0201] Step 52: According to the second determination result, the second video image in the panoramic video image is sent to the video playback terminal; wherein the second video image is the expanded video image determined according to the second determination result. Specifically, the edge server may use a first bit rate and / or a first resolution to send the second video image in the panoramic video image to the video playback terminal, wherein the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0202] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0203] Optionally, the second determination result is also used to indicate at least one of a perspective expansion-related parameter when the first perspective is expanded and an image size-related parameter corresponding to the expanded perspective.

[0204] See also Figure 6 , Figure 6 A schematic flow chart of a video processing method provided in Embodiment 5 of the present invention, the method is applied to an edge server, and includes the following steps:

[0205] Step 61: Acquire a first viewing angle and a transmission delay between the video playback terminal and the edge server, where the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0206] The current viewing angle of the user of the video playback terminal may be sent by the video playback terminal, and the current viewing angle of the user may be obtained according to the current viewpoint of the user. The edge server may obtain the current viewing angle of the user of the video playback terminal sent by the video playback terminal, and if not, the default (ie, default) main viewing angle may be used instead.

[0207] Step 62: obtaining a second determination result according to the transmission delay, wherein the second determination result is used to indicate at least one of expanding the first viewing angle, viewing angle expansion related parameters, and image size related parameters corresponding to the expanded viewing angle;

[0208] Step 63: Based on the second determination result, the second video image in the panoramic video image is sent to the video playback terminal using a first bit rate (bit rate, also referred to as bit stream) and / or a first resolution; wherein the second video image is an expanded video image determined based on the second determination result; the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0209] Of course, the edge server may also obtain a determination result indicating that the first viewing angle should not be expanded according to the transmission delay. For example, if the transmission delay is very small, the first viewing angle may not need to be expanded.

[0210] When the edge server determines that the first viewing angle does not need to be expanded according to the transmission delay, it only needs to transmit the video image corresponding to the first viewing angle according to the first bit rate and / or the first resolution.

[0211] The second video image in the panoramic video image may specifically be a video image composed of a video image of a region corresponding to the first viewing angle and a video image of a region corresponding to the expanded viewing angle in the panoramic video image.

[0212] Expansion refers to expanding the viewing angle. The video image corresponding to the expanded viewing angle includes more video content, specifically including video content of some areas around the viewing angle area before expansion.

[0213] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle and / or the related parameters of the expansion (including the size of the expansion) is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that the expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0214] The above video processing method is described below with examples.

[0215] Optionally, obtaining a second determination result according to the transmission delay includes:

[0216] When the transmission delay is greater than or equal to a third preset value, determining the viewing angle expansion related parameter as a first viewing angle expansion parameter;

[0217] When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter;

[0218] Among them, when the first perspective is expanded according to the first perspective expansion parameter, the video area corresponding to the expanded perspective is the first video area, and when the first perspective is expanded according to the second perspective expansion parameter, the video area corresponding to the expanded perspective is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0219] In the embodiment of the present invention, the expansion-related parameters when expanding the user's current viewing angle or the default main viewing angle are adapted to the transmission delay between the video playback terminal and the edge server. In other words, the expansion size when expanding the user's current viewing angle or the default main viewing angle is adapted to the transmission delay between the video playback terminal and the edge server.

[0220] Optionally, when the transmission delay is less than a fourth preset value, after determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter, the method further includes:

[0221] When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter;

[0222] When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

[0223] That is, if the transmission delay between the video playback terminal and the edge server is always small, the expansion of the current viewing angle or the default main viewing angle of the user of the video playback terminal can be gradually reduced.

[0224] Optionally, the sending, according to the second determination result, the second video image in the panoramic video image to the video playback terminal using the first bit rate and / or the first resolution includes:

[0225] Sending the second video image and the panoramic video image to the video playback terminal in two separate paths;

[0226] or,

[0227] splicing the second video image with the panoramic video image and sending the spliced ​​image to the video playback terminal;

[0228] The panoramic video image is transmitted using a second bit rate and / or a second resolution, the second bit rate is lower than the first bit rate, and the second resolution is lower than the first resolution.

[0229] In an embodiment of the present invention, when the second video image of the video image corresponding to the expanded perspective is transmitted at high resolution and / or high bit rate after the user's current perspective or the default main perspective is expanded, the other part of the video image in the panoramic video image is transmitted at reduced resolution and / or reduced bit rate.

[0230] Specifically, the expanded video image and the panoramic video image can be transmitted in two ways, or they can be spliced ​​and then transmitted.

[0231] Optionally, the sending, according to the second determination result, the second video image in the panoramic video image to the video playback terminal using the first bit rate and / or the first resolution includes:

[0232] Sending the second video image and the third video image to the video playback terminal using the first bit rate and / or the first resolution;

[0233] The third video image includes a video image of a user's potential region of interest (RIO) in the panoramic video image.

[0234] In the embodiment of the present invention, the edge server not only sends the corresponding second video image after expanding the user's current viewing angle or the default main viewing angle to the video playback terminal, but also sends the video image of the user's potential area of ​​interest to the terminal. In other words, the video image of the area that the user is expected to be interested in is sent to the terminal, so that during the video playback process, if the user is really interested in the video image of this area, the user can immediately see the video image of this area after moving the viewing angle, without the terminal having to upload the user's moved viewing angle to request the video image of this area from the edge server.

[0235] See also Figure 7 , Figure 7 A flowchart of a video processing method provided in Embodiment 6 of the present invention is provided. The method is applied to an edge server and includes the following steps:

[0236] Step 71: receiving a second determination result sent by the video playback terminal, wherein the second determination result is determined according to a transmission delay between the video playback terminal and the edge server; wherein the second determination result is used to indicate at least one of expansion of a first viewing angle, viewing angle expansion-related parameters, and image size-related parameters corresponding to the expanded viewing angle, wherein the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0237] The parameters related to the expansion of the viewing angle may be, for example, an expansion ratio and / or an expansion angle, and the parameters related to the image size corresponding to the expanded viewing angle may be, for example, the coordinates of four points of a rectangular image. If the first determination result does not indicate the parameters related to the image size corresponding to the expanded viewing angle, the edge server also needs to obtain the first viewing angle and determine the size of the expanded viewing angle, for example, by expanding the first viewing angle according to the parameters related to the expansion of the viewing angle to determine the size of the expanded viewing angle, and determining the image area corresponding to the expanded viewing angle.

[0238] Step 72: Based on the second determination result, the second video image in the panoramic video image is sent to the video playback terminal using a first bit rate and / or a first resolution; wherein the second video image is an expanded video image determined based on the second determination result; the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value.

[0239] Of course, the second determination result can also be used to indicate that the first viewing angle is not to be expanded. For example, if the transmission delay is small, the terminal may obtain a second determination result indicating that the first viewing angle is not to be expanded. In the case where the second determination result is used to indicate that the first viewing angle is not to be expanded, the edge server only sends the video image corresponding to the first viewing angle to the video playback terminal according to the first bit rate and / or the first resolution based on the second determination result. Alternatively, the image size-related parameters indicated by the second determination result are size-related parameters of the video image corresponding to the first viewing angle. For example, if the transmission delay is small, the terminal can determine that the first viewing angle is not to be expanded, and then the size-related parameters of the video image corresponding to the first viewing angle are sent to the edge server.

[0240] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle and / or the related parameters of the expansion (including the size of the expansion) is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that the expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0241] Other technical details are the same or similar to those of the above-mentioned embodiment 5. Please refer to the above-mentioned embodiment 5 for details and will not be repeated here.

[0242] See also Figure 8 , Figure 8: is a flowchart of a video playback method provided by Embodiment 7 of the present invention, the method is applied to a video playback terminal, and includes the following steps:

[0243] Step 81: Receive the first video image in the panoramic video image sent by the edge server, the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal. Optionally, the edge server can send the first video image in the panoramic video image using a first bit rate and / or a first resolution, the first bit rate is greater than or equal to a first preset value, and the first resolution is greater than or equal to a second preset value;

[0244] Step 82: Play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0245] Of course, the first determination result may also be used to indicate that the video image corresponding to the first perspective is not to be expanded. In this case, the edge server may only send the video image corresponding to the first perspective in the panoramic video image to the terminal at the first bit rate and / or the first resolution.

[0246] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0247] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0248] Optionally, after playing the video image corresponding to the first perspective in the first video image, the method further includes:

[0249] Get the user's viewing angle;

[0250] When the viewing angle movement angle is less than or equal to a fifth preset value, playing a video image corresponding to a new viewing angle of the user in the first video image;

[0251] When the viewing angle is greater than a sixth preset value, the user's new viewing angle is sent to the edge server to request the edge server to send a video image including a fourth video image, where the fourth video image is a video image corresponding to the user's new viewing angle;

[0252] The values ​​of the fifth preset value and the sixth preset value are adapted to the size of the expansion area in the first video image.

[0253] The fifth preset value is less than or equal to the sixth preset value.

[0254] Specifically, the fifth preset value and the sixth preset value can be determined according to the size of the expanded area in the first video image. The larger the expanded area, the larger the fifth preset value and the sixth preset value can be, and vice versa.

[0255] That is, when the user's viewing angle is less than a certain threshold, the terminal obtains the video image in the expanded viewing area. When the movement angle is greater than the threshold, the terminal needs to transmit the new viewing angle to the edge server.

[0256] In other optional specific implementations, the user's new perspective can be obtained, and it is determined whether the video area corresponding to the user's new perspective is within the video area corresponding to the first video image. If so, the video image corresponding to the user's new perspective in the first video image is played; otherwise, the new perspective is sent to the edge server to request the edge server to send a video image including the video image corresponding to the user's new perspective (i.e., the fourth video image).

[0257] See also Fig. 9 , Fig. 9 : is a flow chart of a video playback method provided in Embodiment 8 of the present invention, the method is applied to a video playback terminal, and comprises the following steps:

[0258] Step 91: Obtain the first viewing angle and the transmission delay between the video playback terminal and the edge server;

[0259] Step 92: Obtain a first determination result according to the transmission delay, and send the first determination result to the edge server;

[0260] Step 93: receiving a first video image sent by the edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0261] Step 94: Play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0262] Of course, the first determination result may also be used to indicate that the video image corresponding to the first viewing angle is not to be expanded. For example, if the transmission delay is small, the terminal may obtain the first determination result indicating that the video image corresponding to the first viewing angle is not to be expanded.

[0263] When the first determination result indicates that the video image corresponding to the first viewing angle is not to be expanded, the edge server sends only the video image corresponding to the first viewing angle to the video playback terminal according to the first determination result at the first bit rate and / or the first resolution. That is, the video playback terminal only receives the video image corresponding to the first viewing angle.

[0264] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0265] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0266] Optionally, obtaining a first determination result according to the transmission delay includes:

[0267] In a case where the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter, and using the first image expansion parameter as the first determination result;

[0268] In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result;

[0269] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0270] Further optionally, when the transmission delay is less than a fourth preset value, after determining that the image expansion related parameter is a second image expansion parameter, the method further includes:

[0271] When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result;

[0272] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0273] The following example illustrates how to determine the expansion-related parameters based on the transmission delay between the video playback terminal and the edge server.

[0274] Generally, VR experience requires that the motion to photon (MTP) delay does not exceed a certain threshold (Thresh_MTP), such as 20ms. The MTP delay includes the round-trip transmission delay t between the video playback terminal to the edge server and the edge server to the video playback terminal, and also includes the rendering delay. For example, when the rendering frame rate is f, the rendering delay is 1 / f. One method for determining the expansion-related parameters can be:

[0275] When the MTP delay is greater than Thresh_MTP, that is, when the transmission delay between the video playback terminal and the edge server is greater than (Thresh_MTP-1 / f) / 2, the expansion-related parameter N=3, that is, the video image area corresponding to the current viewing angle or the default main viewing angle of the video terminal user is expanded by 1 / 3 on all sides, for example, 1 / 3 on the top, bottom, left and right sides. The video image viewing angle after expansion based on the expansion-related parameter is larger and can cover general viewing angle changes.

[0276] When the network delay is unstable or the jitter is large, that is, in a certain time window (T-window), the MTP delay is greater than Thresh_MTP, in other words, there is a moment when the transmission delay between the video playback terminal and the edge server is greater than (Thresh_MTP-1 / f) / 2, the expansion-related parameters can also take a value of 3.

[0277] When the network delay is small and relatively stable, that is, within a certain time window (T-window), the MTP delay remains less than Thresh_MTP, that is, the transmission delay between the video playback terminal and the edge server is continuously less than (Thresh_MTP-1 / f) / 2, the expansion angle can be gradually reduced, that is, the expansion parameter N is increased until N=N max , N max You can take 10, or take the minimum value of the expansion angle, that is, N = N max , or the video image corresponding to the current viewing angle or the default main viewing angle of the user of the video playback terminal is not expanded, that is, the expanded viewing angle is equal to the current viewing angle or the default main viewing angle of the user of the video playback terminal.

[0278] In addition, when receiving the first video image, the terminal can also receive a panoramic video image sent by the edge server using a second bit rate and / or a second resolution. The panoramic video image can be transmitted in two ways with the second video image, or can be spliced ​​with the second video image and transmitted.

[0279] Optionally, the receiving a first video image sent by the edge server includes:

[0280] Receiving the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image;

[0281] After playing the first video image or playing the video image corresponding to the first viewing angle in the first video image, the method further includes:

[0282] The user's latest viewing angle is obtained, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, the video image corresponding to the user's latest viewing angle in the third video image is played.

[0283] In the embodiment of the present invention, when receiving the second video image, the terminal may also receive a third video image sent by the edge server using the first bit rate and / or the first resolution, where the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0284] In addition, the third video image may also include a video image of an expanded area around the user's potential area of ​​interest obtained in the manner of expanding the first video image.

[0285] See also Fig.10 , Fig.10 1 is a flow chart of a video playback method provided in Embodiment 9 of the present invention. The method is applied to a video playback terminal and includes the following steps:

[0286] Step 101: receiving a second video image in a panoramic video image sent by an edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that a first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0287] Step 102: Play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0288] Of course, the edge server may also obtain a determination result indicating that the first viewing angle should not be expanded according to the transmission delay. For example, if the transmission delay is very small, the first viewing angle may not need to be expanded.

[0289] When the edge server determines that the first viewing angle does not need to be expanded according to the transmission delay, it only needs to transmit the video image corresponding to the first viewing angle according to the first bit rate and / or the first resolution.

[0290] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0291] Optionally, the second determination result is used to indicate at least one of a parameter related to the viewing angle expansion when the first viewing angle is expanded and a parameter related to the image size corresponding to the expanded viewing angle.

[0292] In addition, when receiving the second video image, the terminal can also receive a panoramic video image sent by the edge server using a second bit rate and / or a second resolution. The panoramic video image can be transmitted in two ways with the second video image, or can be spliced ​​with the second video image and transmitted.

[0293] When receiving the second video image, the terminal can also receive a third video image sent by the edge server using the first bit rate and / or the first resolution. The third video image includes a video image of the user's potential area of ​​interest in the panoramic video image, and may also include a video image of an expanded area around the user's potential area of ​​interest obtained by expanding the second video image.

[0294] See also Fig.11 , Fig.11 1 is a flow chart of a video playback method provided in Embodiment 10 of the present invention. The method is applied to a video playback terminal and includes the following steps:

[0295] Step 111: Obtain the first viewing angle and the transmission delay between the video playback terminal and the edge server;

[0296] Step 112: Obtain a second determination result according to the transmission delay, and send the second determination result to the edge server;

[0297] Step 113: receiving a second video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0298] The second video image is the video image in the video area corresponding to the expanded viewing angle.

[0299] Step 114: Play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0300] Of course, the edge server may also obtain a determination result indicating that the first viewing angle should not be expanded according to the transmission delay. For example, if the transmission delay is very small, the first viewing angle may not need to be expanded.

[0301] When the edge server determines that the first viewing angle does not need to be expanded according to the transmission delay, it only needs to transmit the video image corresponding to the first viewing angle according to the first bit rate and / or the first resolution. In other words, the video playback terminal can only receive the video image corresponding to the first viewing angle.

[0302] In the embodiment of the present invention, whether it is necessary to expand the video image area corresponding to the user's current viewing angle or the default main viewing angle is determined according to the transmission delay between the video playback terminal and the edge server, and the video image content corresponding to the expanded video image area is sent to the terminal for playback, or when it is determined that expansion is not necessary, only the video image content corresponding to the video image area corresponding to the current viewing angle or the default main viewing angle is sent to the terminal for playback. Thus, the bandwidth occupied by the transmission can be reduced as much as possible on the basis of ensuring that the MTP delay is low and does not affect the user experience.

[0303] Optionally, the second determination result is used to indicate at least one of a parameter related to the viewing angle expansion when the first viewing angle is expanded and a parameter related to the image size corresponding to the expanded viewing angle.

[0304] Optionally, obtaining a second determination result according to the transmission delay includes:

[0305] When the transmission delay is greater than or equal to a third preset value, determining the viewing angle expansion related parameter as a first viewing angle expansion parameter;

[0306] When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter;

[0307] Among them, when the first perspective is expanded according to the first perspective expansion parameter, the video area corresponding to the expanded perspective is the first video area, and when the first perspective is expanded according to the second perspective expansion parameter, the video area corresponding to the expanded perspective is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0308] Further optionally, when the transmission delay is less than a fourth preset value, after determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter, the method further includes:

[0309] When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter;

[0310] When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

[0311] In addition, when receiving the second video image, the terminal may also receive a panoramic video image sent by the edge server using a second bit rate and / or a second resolution. The panoramic video image may be transmitted in two ways with the second video image, or may be spliced ​​with the second video image and transmitted.

[0312] Optionally, the receiving a first video image sent by the edge server includes:

[0313] Receiving the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image;

[0314] After playing the first video image or playing the video image corresponding to the first viewing angle in the first video image, the method further includes:

[0315] The user's latest viewing angle is obtained, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, the video image corresponding to the user's latest viewing angle in the third video image is played.

[0316] In the embodiment of the present invention, when receiving the second video image, the terminal may also receive a third video image sent by the edge server using the first bit rate and / or the first resolution, where the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0317] In addition, the third video image may also include a video image of an expanded area around the user's potential area of ​​interest obtained in the manner of expanding the first video image.

[0318] See also Fig.12 , Fig.12 It is a schematic diagram of the video playback process. The specific processing process includes:

[0319] The VR terminal determines whether to expand the video image corresponding to the user's current viewing angle according to the transmission delay between the VR terminal and the edge server, calculates the size-related parameters of the expanded video image area, and then sends the calculated size-related parameters of the expanded video image area to the edge server;

[0320] The edge server selects an alternative expanded viewing angle, renders the video image of the alternative expanded viewing angle and the expanded video image area, and sends the rendered video image to the VR terminal.

[0321] See also Fig.13 , Fig.13 1 is a schematic diagram of the structure of an edge server provided in the eleventh embodiment of the present invention. The edge server 130 includes:

[0322] A first acquisition module 131 is used to acquire a first determination result; wherein the first determination result is determined according to a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate to expand a video image corresponding to a first perspective, the first perspective being a current perspective or a default main perspective of a user of the video playback terminal;

[0323] The first transmission module 132 is used to send a first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is an expanded video image determined according to the first determination result.

[0324] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0325] Optionally, the first acquisition module 131 is used to determine that the image expansion related parameter is a first image expansion parameter when the transmission delay is greater than or equal to a third preset value, and use the first image expansion parameter as the first determination result; when the transmission delay is less than a fourth preset value, determine that the image expansion related parameter is a second image expansion parameter, and use the second image expansion parameter as the first determination result;

[0326] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0327] Optionally, the first acquisition module 131 is further configured to, when the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determine that the image expansion related parameter is a third image expansion parameter, and use the third image expansion parameter as the first determination result;

[0328] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0329] Optionally, the first transmission module 132 is used to send the first video image and the panoramic video image to the video playback terminal in two paths respectively; or, splice the first video image and the panoramic video image and send them to the video playback terminal;

[0330] Among them, the first video image is transmitted using a first bit rate and / or a first resolution; the panoramic video image is transmitted using a second bit rate and / or a second resolution; the second bit rate is smaller than the first bit rate, and the second resolution is smaller than the first resolution.

[0331] Optionally, the first transmission module 132 is used to send the first video image and the third video image to the video playback terminal;

[0332] The third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0333] Optionally, the user potential area of ​​interest is predicted in advance based on video content in the panoramic video image, or is determined based on a mark in a video source of the panoramic video image.

[0334] Optionally, the third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion-related parameters and / or the expanded image size-related parameters.

[0335] See also Fig.14 , Fig.14 1 is a schematic diagram of the structure of an edge server provided in Embodiment 12 of the present invention. The edge server 140 includes:

[0336] A second acquisition module 141 is used to acquire a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate to expand the first viewing angle, the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0337] The second transmission module 142 is used to send the second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is the expanded video image determined according to the second determination result.

[0338] Optionally, the second determination result is also used to indicate at least one of a perspective expansion-related parameter when the first perspective is expanded and an image size-related parameter corresponding to the expanded perspective.

[0339] See also Fig.15 , Fig.15 1 is a schematic diagram of the structure of a video playback terminal provided in Embodiment 13 of the present invention. The video playback terminal 150 includes:

[0340] A first receiving module 151 is configured to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0341] The first playing module 152 is configured to play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0342] Optionally, the video playback terminal 150 further includes:

[0343] A third acquisition module, used to acquire the first viewing angle and the transmission delay;

[0344] A determination module, configured to obtain the first determination result according to the transmission delay;

[0345] A sending module is used to send the first determination result to the edge server.

[0346] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0347] Optionally, the determination module is used to determine, when the transmission delay is greater than or equal to a third preset value, that the image expansion related parameter is a first image expansion parameter, and use the first image expansion parameter as the first determination result; and when the transmission delay is less than a fourth preset value, determine that the image expansion related parameter is a second image expansion parameter, and use the second image expansion parameter as the first determination result;

[0348] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0349] Optionally, the determination module is further configured to, when the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determine that the image expansion-related parameter is a third image expansion parameter, and use the third image expansion parameter as the first determination result;

[0350] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0351] Optionally, the first receiving module is used to receive the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image;

[0352] The video playback terminal 150 also includes:

[0353] The third playback module is used to obtain the user's latest viewing angle, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, the video image corresponding to the user's latest viewing angle in the third video image is played.

[0354] Optionally, the video playback terminal 150 further includes:

[0355] A viewing angle acquisition module is used to acquire the viewing angle of the user;

[0356] A fourth playback module, configured to play a video image corresponding to a new viewing angle of the user in the first video image when the viewing angle movement angle is less than or equal to a fifth preset value;

[0357] a fifth playback module, configured to send the user's new viewing angle to the edge server when the viewing angle movement angle is greater than a sixth preset value, so as to request the edge server to send a video image including a fourth video image, where the fourth video image is a video image corresponding to the user's new viewing angle;

[0358] The values ​​of the fifth preset value and the sixth preset value are adapted to the size of the expansion area in the first video image.

[0359] See also Fig.16 , Fig.16 1 is a schematic diagram of the structure of a video playback terminal provided in Embodiment 14 of the present invention. The video playback terminal 160 includes:

[0360] The second receiving module 161 is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0361] The second playing module 162 is used to play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0362] Optionally, the second determination result is used to indicate at least one of a parameter related to the viewing angle expansion when the first viewing angle is expanded and a parameter related to the image size corresponding to the expanded viewing angle.

[0363] See also Fig.17 , Fig.17 1 is a schematic diagram of the structure of an edge server provided in Embodiment 15 of the present invention. The edge server 170 includes: a transceiver 171 and a processor 172;

[0364] The processor 172 is used to obtain a first determination result; wherein the first determination result is determined according to a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate to expand a video image corresponding to a first perspective, the first perspective being a current perspective or a default main perspective of a user of the video playback terminal;

[0365] The transceiver 171 is used to send the first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is an expanded video image determined according to the first determination result.

[0366] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0367] Optionally, the processor 172 is configured to, when the transmission delay is greater than or equal to a third preset value, determine that the image expansion related parameter is a first image expansion parameter, and use the first image expansion parameter as the first determination result; when the transmission delay is less than a fourth preset value, determine that the image expansion related parameter is a second image expansion parameter, and use the second image expansion parameter as the first determination result;

[0368] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0369] Optionally, the processor 172 is further configured to, when the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determine that the image expansion related parameter is a third image expansion parameter, and use the third image expansion parameter as the first determination result;

[0370] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0371] Optionally, the transceiver 171 is used to send the first video image and the panoramic video image to the video playback terminal in two paths respectively; or, splice the first video image and the panoramic video image and send them to the video playback terminal;

[0372] Among them, the first video image is transmitted using a first bit rate and / or a first resolution; the panoramic video image is transmitted using a second bit rate and / or a second resolution; the second bit rate is smaller than the first bit rate, and the second resolution is smaller than the first resolution.

[0373] Optionally, the transceiver 171 is used to send the first video image and the third video image to the video playback terminal;

[0374] The third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

[0375] Optionally, the user potential area of ​​interest is predicted in advance based on video content in the panoramic video image, or is determined based on a mark in a video source of the panoramic video image.

[0376] Optionally, the third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion-related parameters and / or the expanded image size-related parameters.

[0377] See also Fig.18 , Fig.181 is a schematic diagram of the structure of an edge server provided in Embodiment 16 of the present invention. The edge server 180 includes: a transceiver 181 and a processor 182;

[0378] The processor 182 is used to obtain a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate to expand the first perspective, the first perspective is the current perspective of the user of the video playback terminal or the default main perspective;

[0379] The transceiver 181 is used to send the second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is an expanded video image determined according to the second determination result.

[0380] Optionally, the second determination result is also used to indicate at least one of a perspective expansion-related parameter when the first perspective is expanded and an image size-related parameter corresponding to the expanded perspective.

[0381] See also Fig.19 , Fig.19 1 is a schematic diagram of the structure of a video playback terminal provided in Embodiment 17 of the present invention. The video playback terminal 190 includes: a transceiver 191 and a processor 192;

[0382] The transceiver 191 is used to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal;

[0383] The processor 192 is configured to play the first video image or play the video image corresponding to the first viewing angle in the first video image.

[0384] Optionally, the processor 192 is configured to obtain the first viewing angle and the transmission delay;

[0385] The processor 192 is further configured to obtain the first determination result according to the transmission delay;

[0386] The transceiver 191 is further configured to send the first determination result to the edge server.

[0387] Optionally, the first determination result is also used to indicate at least one of image expansion-related parameters when expanding the video image corresponding to the first viewing angle and parameters related to the size of the expanded image.

[0388] Optionally, the processor 192 is configured to, when the transmission delay is greater than or equal to a third preset value, determine that the image expansion related parameter is a first image expansion parameter, and use the first image expansion parameter as the first determination result; when the transmission delay is less than a fourth preset value, determine that the image expansion related parameter is a second image expansion parameter, and use the second image expansion parameter as the first determination result;

[0389] Among them, when the video image corresponding to the first perspective is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

[0390] Optionally, the processor 192 is further configured to, when the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determine that the image expansion related parameter is a third image expansion parameter, and use the third image expansion parameter as the first determination result;

[0391] When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

[0392] Optionally, the transceiver 191 is configured to receive the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image;

[0393] The processor 192 is further configured to obtain the user's latest viewing angle, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, play the video image in the third video image corresponding to the user's latest viewing angle.

[0394] Optionally, the transceiver 191 is also used to obtain the user's viewing angle;

[0395] The processor 192 is further configured to play a video image corresponding to a new perspective of the user in the first video image when the perspective movement angle is less than or equal to a fifth preset value;

[0396] The transceiver 192 is further configured to, when the viewing angle movement angle is greater than a sixth preset value, send the user's new viewing angle to the edge server to request the edge server to send a video image including a fourth video image, where the fourth video image is a video image corresponding to the user's new viewing angle;

[0397] The values ​​of the fifth preset value and the sixth preset value are adapted to the size of the expansion area in the first video image.

[0398] See also Fig. 20 , Fig. 20 2 is a schematic diagram of a video playback terminal according to an eighteenth embodiment of the present invention. The video playback terminal 200 includes: a transceiver 201 and a processor 202;

[0399] The transceiver 201 is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal;

[0400] The processor 202 is configured to play the second video image or play the video image corresponding to the first viewing angle in the second video image.

[0401] Optionally, the second determination result is used to indicate at least one of a parameter related to the viewing angle expansion when the first viewing angle is expanded and a parameter related to the image size corresponding to the expanded viewing angle.

[0402] See also Fig.21 , Fig.21 1 is a schematic diagram of the structure of an edge server provided in Embodiment 19 of the present invention. The edge server 210 includes a processor 211, a memory 212, and a program stored in the memory 212 and executable on the processor 211; when the processor 211 executes the program, any of the steps in the above-mentioned video processing method applied to the edge server is implemented.

[0403] See also Fig. 22 , Fig. 22 It is a structural schematic diagram of a video playback terminal provided in Embodiment 20 of the present invention. The video playback terminal 220 includes a processor 221, a memory 222, and a program stored in the memory 222 and executable on the processor 221; when the processor 221 executes the program, any of the steps in the above-mentioned video playback method applied to the video playback terminal is implemented.

[0404] Embodiment 21 of the present invention provides a readable storage medium on which a program is stored. When the program is executed by a processor, the steps in any one of the video processing methods in Embodiments 1 to 6 or the steps in any one of the video playing methods in Embodiments 7 to 10 are implemented. For details, please refer to the description of the method steps in the corresponding embodiments above.

[0405] The above-mentioned readable storage medium includes computer-readable storage medium. Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0406] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video processing method, applied to an edge server, characterized in that: include: Obtaining a first determination result; wherein the first determination result is determined based on a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, the first determination result is used to indicate expansion of a video image corresponding to a first perspective, the first perspective being a current perspective or a default main perspective of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and a post-expansion image size-related parameter when expanding the video image corresponding to the first perspective; According to the first determination result, sending a first video image in the panoramic video image to the video playback terminal; wherein the first video image is an expanded video image determined according to the first determination result; The obtaining of the first determination result includes: In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; Among them, when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the expanded video area is the second video area; when the video image corresponding to the first perspective is expanded according to the third image expansion parameter, the expanded video area is the third video area, and the third video area is smaller than the second video area.

2. The method according to claim 1, characterized in that The obtaining of the first determination result further includes: In a case where the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter, and using the first image expansion parameter as the first determination result; Among them, when the video image corresponding to the first viewing angle is expanded according to the first image expansion parameter, the expanded video area is the first video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value.

3. The method according to claim 1, characterized in that The sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes: Sending the first video image and the panoramic video image to the video playback terminal in two paths respectively; or, splicing the first video image with the panoramic video image and sending the spliced ​​image to the video playback terminal; Among them, the first video image is transmitted using a first bit rate and / or a first resolution; the panoramic video image is transmitted using a second bit rate and / or a second resolution; the second bit rate is smaller than the first bit rate, and the second resolution is smaller than the first resolution.

4. The method according to claim 1, characterized in that: The sending, according to the first determination result, the first video image in the panoramic video image to the video playback terminal includes: Sending the first video image and the third video image to the video playback terminal; The third video image includes a video image of a user's potential area of ​​interest in the panoramic video image.

5. The method according to claim 4, characterized in that The user's potential area of ​​interest is predicted in advance based on the video content in the panoramic video image, or is determined based on a mark in a video source of the panoramic video image.

6. The method according to claim 4, characterized in that The third video image also includes a video image of a video area that is expanded when the video image of the user's potential area of ​​interest is expanded according to the image expansion related parameters and / or the expanded image size related parameters.

7. A video processing method, applied to an edge server, characterized in that: include: Obtaining a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate expansion of a first viewing angle, the first viewing angle being the current viewing angle or the default main viewing angle of a user of the video playback terminal; the second determination result is also used to indicate at least one of viewing angle expansion-related parameters when the first viewing angle is expanded and image size-related parameters corresponding to the expanded viewing angle; According to the second determination result, sending a second video image in the panoramic video image to the video playback terminal; wherein the second video image is an expanded video image determined according to the second determination result; Obtaining the second determination result includes: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

8. A video playing method, applied to a video playing terminal, characterized in that: include: Receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and an image size-related parameter after expansion when the video image corresponding to the first viewing angle is expanded; Playing the first video image or playing the video image corresponding to the first viewing angle in the first video image; Before receiving the first video image in the panoramic video image sent by the edge server using the first bit rate and / or the first resolution, the method further includes: Acquire the first viewing angle and the transmission delay; Obtaining the first determination result according to the transmission delay; sending the first determination result to the edge server; The obtaining a first determination result according to the transmission delay includes: In a case where the transmission delay is greater than or equal to a third preset value, determining the image expansion related parameter to be a first image expansion parameter, and using the first image expansion parameter as the first determination result; In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; Wherein, when the video image corresponding to the first viewing angle is expanded according to the first image expansion parameter, the video area expanded is the first video area, and when the video image corresponding to the first viewing angle is expanded according to the second image expansion parameter, the video area expanded is the second video area, and the first video area is larger than the second video area; the third preset value is greater than or equal to the fourth preset value; After determining that the image expansion related parameter is a second image expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; When the video image corresponding to the first viewing angle is expanded according to the third image expansion parameter, the expanded video area is a third video area, and the third video area is smaller than the second video area.

9. The method according to claim 8, characterized in that The receiving a first video image sent by the edge server includes: Receiving the first video image and the third video image sent by the edge server; wherein the third video image includes a video image of a user's potential area of ​​interest in the panoramic video image; After playing the first video image or playing the video image corresponding to the first viewing angle in the first video image, the method further includes: The user's latest viewing angle is obtained, and if the video area corresponding to the user's latest viewing angle is within the range of the third video image, the video image corresponding to the user's latest viewing angle in the third video image is played.

10. The method according to claim 8, characterized in that After playing the video image corresponding to the first viewing angle in the first video image, the method further includes: Get the user's viewing angle; When the viewing angle movement angle is less than or equal to a fifth preset value, playing a video image corresponding to a new viewing angle of the user in the first video image; When the viewing angle is greater than a sixth preset value, the user's new viewing angle is sent to the edge server to request the edge server to send a video image including a fourth video image, where the fourth video image is a video image corresponding to the user's new viewing angle; The values ​​of the fifth preset value and the sixth preset value are adapted to the size of the expansion area in the first video image.

11. A video playing method, applied to a video playing terminal, characterized in that: include: Receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate expansion of a first viewing angle; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the second determination result is also used to indicate at least one of a viewing angle expansion-related parameter when expanding the first viewing angle and an image size-related parameter corresponding to the expanded viewing angle; Playing the second video image or playing the video image corresponding to the first viewing angle in the second video image; The second determination result is determined in the following manner: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

12. An edge server, characterized in that: include: A first acquisition module is used to acquire a first determination result; wherein the first determination result is determined based on a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, the first determination result is used to indicate expansion of a video image corresponding to a first perspective, the first perspective is a current perspective or a default main perspective of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and a parameter related to the image size after expansion when the video image corresponding to the first perspective is expanded; A first transmission module, configured to send a first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is an expanded video image determined according to the first determination result; Wherein, obtaining the first determination result includes: In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; Among them, when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the expanded video area is the second video area; when the video image corresponding to the first perspective is expanded according to the third image expansion parameter, the expanded video area is the third video area, and the third video area is smaller than the second video area.

13. An edge server, characterized in that: include: A second acquisition module is used to acquire a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate expansion of a first viewing angle, the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the second determination result is also used to indicate at least one of a viewing angle expansion-related parameter when the first viewing angle is expanded and an image size-related parameter corresponding to the expanded viewing angle; A second transmission module, configured to send a second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is an expanded video image determined according to the second determination result; Wherein, obtaining the second determination result includes: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

14. A video playback terminal, characterized in that: include: A first receiving module is configured to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to a first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and an image size-related parameter after expansion when the video image corresponding to the first viewing angle is expanded; A first playback module, configured to play the first video image or a video image corresponding to the first viewing angle in the first video image; The first determination result is determined in the following manner: In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; Among them, when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the expanded video area is the second video area; when the video image corresponding to the first perspective is expanded according to the third image expansion parameter, the expanded video area is the third video area, and the third video area is smaller than the second video area.

15. A video playback terminal, characterized in that: include: A second receiving module is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that the first viewing angle is expanded; the first viewing angle is the current viewing angle or the default main viewing angle of the user of the video playback terminal; the second determination result is also used to indicate at least one of the viewing angle expansion-related parameters when the first viewing angle is expanded and the image size-related parameters corresponding to the expanded viewing angle; A second playing module, used for playing the second video image or playing the video image corresponding to the first viewing angle in the second video image; The second determination result is determined in the following manner: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

16. An edge server, characterized in that: include: transceivers and processors; The processor is used to obtain a first determination result; wherein the first determination result is determined according to a transmission delay, the transmission delay is a transmission delay between the video playback terminal and the edge server, the first determination result is used to indicate that a video image corresponding to a first perspective is expanded, the first perspective is a current perspective or a default main perspective of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and a post-expansion image size-related parameter when the video image corresponding to the first perspective is expanded; The transceiver is used to send a first video image in the panoramic video image to the video playback terminal according to the first determination result; wherein the first video image is an expanded video image determined according to the first determination result; Wherein, obtaining the first determination result includes: In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; Among them, when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the expanded video area is the second video area; when the video image corresponding to the first perspective is expanded according to the third image expansion parameter, the expanded video area is the third video area, and the third video area is smaller than the second video area.

17. An edge server, characterized in that: include: transceivers and processors; The processor is used to obtain a second determination result; wherein the second determination result is determined by transmission delay, the transmission delay is the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate expansion of a first viewing angle, the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the second determination result is also used to indicate at least one of a viewing angle expansion-related parameter when the first viewing angle is expanded and an image size-related parameter corresponding to the expanded viewing angle; The transceiver is used to send a second video image in the panoramic video image to the video playback terminal according to the second determination result; wherein the second video image is an expanded video image determined according to the second determination result; Wherein, obtaining the second determination result includes: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

18. A video playback terminal, characterized in that: include: transceivers and processors; The transceiver is used to receive a first video image in a panoramic video image sent by an edge server; wherein the first video image is an expanded video image determined according to a first determination result, the first determination result is obtained according to a transmission delay between the video playback terminal and the edge server, and the first determination result is used to indicate that the video image corresponding to the first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the first determination result is also used to indicate at least one of an image expansion-related parameter and an image size-related parameter after expansion when the video image corresponding to the first viewing angle is expanded; The processor is configured to play the first video image or play a video image corresponding to the first viewing angle in the first video image; The first determination result is determined in the following manner: In a case where the transmission delay is less than a fourth preset value, determining the image expansion related parameter as a second image expansion parameter, and using the second image expansion parameter as the first determination result; When the first determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the image expansion related parameter to be a third image expansion parameter, and using the third image expansion parameter as the first determination result; Among them, when the video image corresponding to the first perspective is expanded according to the second image expansion parameter, the expanded video area is the second video area; when the video image corresponding to the first perspective is expanded according to the third image expansion parameter, the expanded video area is the third video area, and the third video area is smaller than the second video area.

19. A video playback terminal, characterized in that: include: transceivers and processors; The transceiver is used to receive a second video image in the panoramic video image sent by the edge server; wherein the second video image is an expanded video image determined according to a second determination result, the second determination result is obtained according to the transmission delay between the video playback terminal and the edge server, and the second determination result is used to indicate that a first viewing angle is expanded; the first viewing angle is a current viewing angle or a default main viewing angle of a user of the video playback terminal; the second determination result is also used to indicate at least one of a viewing angle expansion-related parameter when the first viewing angle is expanded and an image size-related parameter corresponding to the expanded viewing angle; The processor is configured to play the second video image or play a video image corresponding to the first viewing angle in the second video image; The second determination result is determined in the following manner: When the transmission delay is less than a fourth preset value, determining the viewing angle expansion related parameter as a second viewing angle expansion parameter; Wherein, when the first viewing angle is expanded according to the second viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is the second video area; After determining that the viewing angle expansion related parameter is a second viewing angle expansion parameter when the transmission delay is less than a fourth preset value, the method further includes: When the second determination result is obtained again according to the transmission delay, if the transmission delay remains less than the fourth preset value, determining the viewing angle expansion related parameter to be a third viewing angle expansion parameter; When the first viewing angle is expanded according to the third viewing angle expansion parameter, the video area corresponding to the expanded viewing angle is a third video area, and the third video area is smaller than the second video area.

20. An edge server, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the processor implements the steps of the video processing method according to any one of claims 1 to 6 or the steps of the video processing method according to claim 7.

21. A video playback terminal, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the processor implements the steps of the video playback method according to any one of claims 8 to 10 or the steps of the video playback method according to claim 11.

22. A readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it implements the steps of the video processing method as described in any one of claims 1 to 6, or implements the steps of the video processing method as described in claim 7, or implements the steps of the video playback method as described in any one of claims 8 to 10, or implements the steps of the video playback method as described in claim 11.

Images