Immersive panoramic video wireless transmission method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an immersive panoramic video wireless transmission method and device, equipment and a storage medium, and relates to the technical field of display. The method comprises the steps of determining a target transmission video from a plurality of videos to be transmitted according to a transmission demand, obtaining a visual quality evaluation parameter of the target transmission video, and generating a channel constraint condition based on the target transmission video and a channel matching variable, and constructing an optimization target model based on the visual quality evaluation parameters and the channel constraint conditions, and solving to obtain channel allocation data. And selecting a target code rate video from the code rate sub-videos corresponding to each target transmission video according to the channel allocation data, determining a target transmission channel of the target code rate video, and transmitting the corresponding code rate sub-videos based on the target transmission channel. In the channel allocation process, the visual quality evaluation parameters are generated for each target transmission video to participate in the optimization process, the uniform allocation process is carried out, the content association between the target transmission videos is fully considered, and the receiving quality of the panoramic video is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an immersive panoramic video wireless transmission method, device, equipment and storage medium. Background Art

[0002] Immersive panoramic video adaptive transmission technology is developing rapidly in response to the development of virtual reality applications. Due to the limited bandwidth resources of wireless transmission, in the design of immersive panoramic video wireless transmission system, the immersive panoramic video is usually divided into multiple video slices of equal size. During transmission, usually not all video slices are transmitted, but only the relevant video slices within the user's field of view (FoV) are transmitted to effectively save the bandwidth required for transmission.

[0003] In the related art, the video clips are encoded independently on the server, and when the user requests, available channel resources are independently allocated for each video clip for transmission. This method does not fully consider the content association between video clips when allocating channel resources, resulting in difficulty in improving the reception quality of panoramic videos. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an immersive panoramic video wireless transmission method, device, equipment and storage medium to improve the data quality of video transmission.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes an immersive panoramic video wireless transmission method, comprising:

[0006] Determine at least one target transmission video from a plurality of videos to be transmitted according to a transmission requirement, wherein the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate;

[0007] Acquire a visual quality assessment parameter of each of the target transmission videos, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel;

[0008] Building an optimization target model based on the visual quality assessment parameters and the channel constraint conditions, and solving the optimization target model to obtain channel allocation data;

[0009] Selecting a target bit rate video from the bit rate sub-videos corresponding to each of the target transmission videos according to the channel allocation data, and determining a target transmission channel for the target bit rate video;

[0010] The corresponding bit rate sub-videos are transmitted respectively based on the target transmission channel.

[0011] In some embodiments, obtaining a visual quality assessment parameter of each target transmission video includes:

[0012] Acquire a first fitting parameter and a second fitting parameter of the target transmission video;

[0013] Obtain the encoding bit rate corresponding to each bit rate sub-video;

[0014] The visual quality assessment parameter corresponding to each of the bit rate sub-videos is calculated based on the encoding bit rate, the first fitting parameter and the second fitting parameter, and the visual quality assessment parameter is used to indicate the transmission quality of the video content of the bit rate sub-video.

[0015] In some embodiments, obtaining the first fitting parameter and the second fitting parameter of the target transmission video includes:

[0016] Encode multiple continuous image frames corresponding to the target transmission video in parallel at multiple different encoding rates to obtain corresponding encoding results, and obtain a peak signal-to-noise ratio corresponding to each encoding result;

[0017] A plurality of groups of fitting data are generated according to the encoding bit rate and the corresponding peak signal-to-noise ratio, and an exponential function fitting process is performed using the fitting data to obtain the first fitting parameter and the second fitting parameter.

[0018] In some embodiments, generating a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel includes:

[0019] Generate the channel matching variable corresponding to each bit rate sub video and each transmission channel, if the bit rate sub video is assigned to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value;

[0020] The channel constraint condition is generated based on the channel matching variable, the number of the encoding code rates, and the encoding code rates.

[0021] In some embodiments, generating the channel constraint condition based on the channel matching variable, the number of the encoding code rates, and the encoding code rates includes:

[0022] Generate a first constraint condition based on the channel matching variable, where the first constraint condition is used to constrain the sum of the number of the transmission channels allocated to the bit rate sub-videos corresponding to all the target transmission videos to be less than or equal to the total number of the transmission channels;

[0023] Generate a second constraint condition based on the channel matching variable, where the second constraint condition is used to constrain each of the transmission channels to be allocated to only one of the bitrate sub-videos of the target transmission video;

[0024] generating a third constraint condition for constraining the number of the encoding bit rates to be a preset number;

[0025] When the encoding bit rate is greater than the channel capacity of a single transmission channel, more than one transmission channel is allocated to the bit rate sub-video corresponding to the encoding bit rate, and a fourth constraint condition is generated;

[0026] The channel constraint condition is obtained according to the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.

[0027] In some embodiments, constructing an optimization target model based on the visual quality assessment parameter and the channel constraint condition includes:

[0028] Obtaining a preset weight of the target transmission video and the corresponding visual quality assessment parameter of each bitrate sub-video;

[0029] generating an overall transmission quality parameter based on the preset weight and the visual quality assessment parameter;

[0030] Maximizing the total transmission quality parameter to generate a resource optimization objective function;

[0031] The optimization target model is constructed based on the resource optimization objective function and the channel constraint condition.

[0032] In some embodiments, generating the total transmission quality parameter based on the preset weight and the visual quality assessment parameter includes:

[0033] Obtaining the sum of all the visual quality assessment parameters corresponding to each of the target transmission videos to obtain a first intermediate value;

[0034] Multiplying the preset weight by the corresponding first intermediate value to obtain a second intermediate value;

[0035] The total transmission quality parameter is obtained by accumulating all the second intermediate values.

[0036] In some embodiments, solving the optimization target model to obtain channel allocation data includes:

[0037] Initializing a channel set and a cumulative channel transmission capacity corresponding to each of the target transmission videos;

[0038] Allocating the transmission channel to each of the target transmission videos in sequence, and in each allocation process, allocating one of the target transmission videos to each of the transmission channels;

[0039] Calculate, according to the allocation result, the quality gain corresponding to each transmission channel being allocated to different target transmission videos;

[0040] Allocating the transmission channel as a target transmission channel for the target transmission video that maximizes the quality gain;

[0041] Solving the optimization target model based on the target transmission channel to obtain the target bit rate video corresponding to each target transmission video;

[0042] The channel allocation data is obtained based on the target bit rate video and the target transmission channel.

[0043] In some embodiments, solving the optimization target model based on the target transmission channel to obtain the target bit rate video corresponding to each target transmission video includes:

[0044] Obtaining the transmission bandwidth of the target transmission channel corresponding to each target transmission video;

[0045] Determine at least one candidate sub-video from the corresponding bitrate sub-videos based on the transmission bandwidth;

[0046] Select the candidate sub-videos one by one, set the corresponding channel matching variable to the first value, and set the channel matching variables of other bitrate sub-videos to the second value;

[0047] Based on the updated channel matching variable, while satisfying the channel constraint condition, the resource optimization objective function is calculated, and the candidate sub-video corresponding to each target transmission channel when the total transmission quality parameter is maximized is selected as the corresponding target bit rate video.

[0048] In some embodiments, determining at least one target transmission video from a plurality of videos to be transmitted according to the transmission requirement includes:

[0049] determining a target viewing angle according to the transmission requirements;

[0050] A video that meets the target viewing angle is screened from the multiple videos to be transmitted as the target transmission video.

[0051] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes an immersive panoramic video wireless transmission device, comprising:

[0052] Video selection module: used to determine at least one target transmission video from multiple videos to be transmitted according to transmission requirements, wherein the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate;

[0053] A quality parameter calculation module: used to obtain a visual quality assessment parameter of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel;

[0054] An optimization model building module is used to build an optimization target model based on the visual quality evaluation parameters and the channel constraint conditions, and solve the optimization target model to obtain channel allocation data;

[0055] A channel allocation module: configured to select a target bit rate video from the bit rate sub-videos corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel for the target bit rate video;

[0056] Video transmission module: used to transmit the corresponding bit rate sub-videos based on the target transmission channel.

[0057] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0058] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0059] The immersive panoramic video wireless transmission method, device, equipment and storage medium proposed in the embodiment of the present application determine at least one target transmission video from multiple videos to be transmitted according to the transmission requirements, wherein the target transmission video includes at least one rate sub-video corresponding to the encoding rate. Then, the visual quality assessment parameters of each target transmission video are obtained, and the channel constraint conditions are generated based on the channel matching variables corresponding to the target transmission video and at least one transmission channel, and then the optimization target model is constructed based on the visual quality assessment parameters and the channel constraint conditions, and the optimization target model is solved to obtain the channel allocation data. Then, according to the channel allocation data, the target rate video is selected from the rate sub-video corresponding to each target transmission video, and the target transmission channel of the target rate video is determined, and finally the corresponding rate sub-videos are transmitted respectively based on the target transmission channel. In the process of channel allocation, the embodiment of the present application generates a visual quality assessment parameter for each target transmission video to participate in the optimization process, and a unified allocation process is performed for all target transmission videos, fully considering the content association between the target transmission videos, and improving the reception quality of the panoramic video. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flow chart of an immersive panoramic video wireless transmission method provided in an embodiment of the present application.

[0061] Figure 2 This is a flowchart of obtaining visual quality assessment parameters of each target transmission video provided by an embodiment of the present application.

[0062] Figure 3 This is a flowchart of obtaining the first fitting parameter and the second fitting parameter of the target transmission video provided by an embodiment of the present application.

[0063] Figure 4 It is a flowchart of generating channel constraints based on channel matching variables corresponding to a target transmission video and at least one transmission channel provided by an embodiment of the present application.

[0064] Figure 5 It is a flowchart of generating channel constraints based on channel matching variables, the number of encoding rates, and the encoding rate provided in an embodiment of the present application.

[0065] Figure 6 It is a flowchart of constructing an optimization target model based on visual quality assessment parameters and channel constraints provided in an embodiment of the present application.

[0066] Figure 7 It is a flow chart for solving the optimization target model to obtain channel allocation data provided by an embodiment of the present application.

[0067] Figure 8It is a flowchart of solving and optimizing a target model based on a target transmission channel to obtain a target bit rate video corresponding to each target transmission video provided by an embodiment of the present application.

[0068] Fig. 9 It is a schematic diagram for verifying the effect of the immersive panoramic video wireless transmission method provided in an embodiment of the present application.

[0069] Fig.10 This is a structural block diagram of an immersive panoramic video wireless transmission device provided in another embodiment of the present application.

[0070] Fig.11 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0074] Immersive panoramic video adaptive transmission technology is developing rapidly in response to the development of virtual reality applications. Due to the limited bandwidth resources of wireless transmission, in the design of immersive panoramic video wireless transmission system, the immersive panoramic video is usually divided into multiple video slices of equal size. During transmission, usually not all video slices are transmitted, but only the relevant video slices within the user's field of view (FoV) are transmitted to effectively save the bandwidth required for transmission.

[0075] In the related art, the video clips are encoded independently on the server, and when the user requests, available channel resources are independently allocated for each video clip for transmission. This method does not fully consider the content association between video clips when allocating channel resources, resulting in difficulty in improving the reception quality of panoramic videos.

[0076] Based on this, the embodiments of the present application provide an immersive panoramic video wireless transmission method, device, equipment and storage medium. During the channel allocation process, a visual quality assessment parameter is generated for each target transmission video to participate in the optimization process. A unified allocation process is performed for all target transmission videos, fully considering the content correlation between the target transmission videos, thereby improving the reception quality of the panoramic video.

[0077] The embodiments of the present application provide an immersive panoramic video wireless transmission method, apparatus, device and storage medium, which are specifically illustrated by the following embodiments. First, the immersive panoramic video wireless transmission method in the embodiments of the present application is described.

[0078] The immersive panoramic video wireless transmission method provided in the embodiment of the present application relates to the field of communication technology. The immersive panoramic video wireless transmission method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, and can also be a computer program running in a terminal or a server side. For example, a computer program can be a native program or software module in an operating system; it can be a local (Native) application (Application, APP), that is, a program that needs to be installed in the operating system to run, such as a client that supports video data transmission, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in. Among them, the terminal communicates with the server through a network. The immersive panoramic video wireless transmission method can be executed by a terminal or a server, or by a terminal and a server in collaboration.

[0079] In some embodiments, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer or a smart watch, etc. In addition, the terminal may also be an intelligent vehicle-mounted device. The intelligent vehicle-mounted device applies the immersive panoramic video wireless transmission method of this embodiment to provide related services and enhance the driving experience. The server may be an independent server, or it may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; it may also be a service node in a blockchain system, and each service node in the blockchain system forms a peer-to-peer (Peer To Peer, P2P) network, and the P2P protocol is an application layer protocol running on the Transmission Control Protocol (Transmission Control Protocol, TCP) protocol. The terminal and the server may be connected via Bluetooth, Universal Serial Bus (Universal Serial Bus, USB) or network communication connection methods, which are not limited in this embodiment.

[0080] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0081] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0082] The following describes an immersive panoramic video wireless transmission method in an embodiment of the present application.

[0083] Figure 1 is an optional flowchart of the immersive panoramic video wireless transmission method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps 110 to 150. It can also be understood that this embodiment Figure 1 The order of step 110 to step 150 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0084] Step 110: Determine at least one target transmission video from a plurality of videos to be transmitted according to a transmission requirement.

[0085] In one embodiment, in a panoramic video scene, the entire panoramic video can be pre-divided into different video slices, each of which is used as a video to be transmitted, wherein the slices can be divided into equal sizes or different sizes, and the following is explained by taking panoramic video transmission as an example. It can be understood that in other video scenes, the video to be transmitted can also be a complete undivided video. This embodiment does not limit this.

[0086] In one embodiment, when a user needs to transmit a panoramic video, a corresponding transmission demand will be generated. At this time, the user's viewing angle can be obtained according to the transmission demand, and it is used as the target viewing angle. Then, the relevant video that meets the target viewing angle is selected from multiple videos to be transmitted as the target transmission video. In other words, the target transmission video can cover the target viewing angle, so that the user can independently decode and render the received video and play it in real time on his head-mounted display, thereby realizing immersive panoramic viewing.

[0087] In one embodiment, since the target transmission video needs to be transmitted after encoding to remove redundant data, reduce the amount of data transmitted, save storage space, and reduce the demand for network bandwidth, the target transmission video can be a video stored after being encoded in advance. In addition, different encoding bit rates will affect the quality of the video after encoding. The higher the encoding bit rate, the more bits are transmitted per second, so that more image details and color information can be retained, but the high encoding bit rate will also increase the file size of the encoded video, affecting the transmission efficiency. Therefore, in the embodiment of the present application, the bit rate sub-videos corresponding to different encoding bit rates corresponding to each target transmission video are stored, that is, the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate. When transmitting, each target transmission video will select one of the bit rate sub-videos, and the encoding bit rates of the bit rate sub-videos selected by different target transmission videos can be the same or different, which is determined according to the subsequent channel allocation process.

[0088] Step 120: Obtain visual quality assessment parameters of each target transmission video, and generate channel constraint conditions based on channel matching variables corresponding to the target transmission video and at least one transmission channel.

[0089] In one embodiment, the visual quality evaluation parameter is used to evaluate the transmission quality of the video content of the bit rate sub-video of the target transmission video at different encoding bit rates, wherein the visual quality will be fed back to the final received visual quality. Figure 2 , Figure 2 : is a flowchart of obtaining visual quality assessment parameters of each target transmission video provided by an embodiment of the present application, which specifically includes the following steps:

[0090] Step 210: Obtain a first fitting parameter and a second fitting parameter of a target transmission video.

[0091] In one embodiment, for the i-th target transmission video, it is assumed that the corresponding bit rate sub-video has L encoding bit rates. In this case, the encoding bit rate is expressed as: R i = {r i1 ,r i2 ,L,r iL}, where r iL represents the Lth encoding rate, R i Represents all encoding bit rates corresponding to the i-th target transmission video.

[0092] At this time, refer to Figure 3 , Figure 3 The flowchart of obtaining the first fitting parameter and the second fitting parameter of the target transmission video provided by the embodiment of the present application specifically includes the following steps:

[0093] Step 310: encode multiple continuous image frames corresponding to the target transmission video in parallel at multiple different encoding rates to obtain corresponding encoding results, and obtain the peak signal-to-noise ratio corresponding to each encoding result.

[0094] In one embodiment, for a target transmission video, it includes multiple continuous image frames, which are regarded as a unit (Group of Pictures, GOP) during the encoding process. The unit includes a key frame (I frame) and multiple dependent frames (P frames or B frames). The key frame is a complete image, and the dependent frame contains the difference information from the previous frame. Then, different encoding bit rates are used to encode the GOP in parallel. The encoder compresses the frames in the GOP according to the encoding bit rate and outputs a video stream or video file as the encoding result. The encoding result contains all compressed frame data and necessary metadata so that the decoder can correctly decode and play the video.

[0095] Next, calculate the peak signal-to-noise ratio corresponding to each encoding result. The calculation process is described as: calculate the square of the difference between each corresponding pixel between the key frame of the target transmission video and the compressed frame of the encoding result, and then calculate the average value to get the MSE. Then determine the MAX according to the color depth of the video. I For example, for an 8-bit RGB image, the maximum value of each color channel is usually 255. Then the calculated MSE and the determined MAX I Substitute the value into the PSNR formula to calculate the final PSNR value, which is expressed as:

[0096]

[0097] Step 320: Generate multiple groups of fitting data according to the encoding bit rate and the corresponding peak signal-to-noise ratio, and use the fitting data to perform an exponential function fitting process to obtain a first fitting parameter and a second fitting parameter.

[0098] In one embodiment, the coding rate and the corresponding peak signal-to-noise ratio are associated, and each coding rate corresponds to a peak signal-to-noise ratio as a fitting data. The number of fitting data is the same as the number of coding rates. Then, the fitting data is used to perform an exponential function fitting process, and the first fitting parameter and the second fitting parameter are obtained by using the nonlinear least squares method, which are expressed as:

[0099]

[0100] Among them, F i represents the fitting function corresponding to the i-th target transmission video, x represents the encoding bit rate in the fitting data, y represents the peak signal-to-noise ratio, and a i represents the first fitting parameter corresponding to the i-th target transmission video, b irepresents the second fitting parameter corresponding to the i-th target transmission video. It can be understood that only part of the encoding bit rates can be selected to participate in the fitting process to reduce the time consumption of the fitting process and improve the calculation efficiency.

[0101] Step 220: Obtain the encoding bit rate corresponding to each bit rate sub-video.

[0102] Step 230: Calculate the visual quality assessment parameter corresponding to each bit rate sub-video based on the encoding bit rate, the first fitting parameter and the second fitting parameter.

[0103] In one embodiment, for the l-th bitrate sub-video of the i-th target transmission video, its visual quality assessment parameter is expressed as:

[0104]

[0105] Among them, q il represents the visual quality assessment parameter of the l-th bitrate sub-video of the i-th target transmission video, r il represents the i-th encoding bit rate.

[0106] According to the above process, the visual quality assessment parameters of each bit rate sub-video of each target transmission video are obtained to participate in the subsequent channel allocation process.

[0107] In one embodiment, in a video transmission system, a base station collects the user's target viewing angle in real time through a channel, and then determines the target transmission video required by the user according to the target viewing angle, selects the bit rate sub-video that is most suitable for transmission for each target transmission video, and allocates wireless channels with different data for each bit rate sub-video, and uses the channel to transmit the related bit rate sub-video. Therefore, the embodiment of the present application generates channel constraints based on the channel matching variables corresponding to the target transmission video and at least one transmission channel.

[0108] In one embodiment, referring to Figure 4 , Figure 4 This is a flow chart of generating channel constraints based on channel matching variables corresponding to a target transmission video and at least one transmission channel provided by an embodiment of the present application, which specifically includes the following steps:

[0109] Step 410: Generate a channel matching variable corresponding to each bit rate sub-video and each transmission channel. If the bit rate sub-video is assigned to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value.

[0110] In one embodiment, a channel matching variable is defined to determine whether a bit rate sub video is assigned to a transmission channel. If the bit rate sub video is assigned to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value, for example, the first value is 1 and the second value is 0. For example, there are K transmission channels in total, and the lth bit rate sub video of the i-th target transmission video is transmitted using the kth transmission channel. At this time, the corresponding channel matching variable is 1, otherwise is 0.

[0111] Step 420: Generate channel constraints based on the channel matching variable, the number of encoding bit rates, and the encoding bit rate.

[0112] In one embodiment, there are four channel constraints, namely, a first constraint, a second constraint, a third constraint, and a fourth constraint. Figure 5 , Figure 5 This is a flow chart of generating channel constraints based on channel matching variables, the number of encoding rates, and the encoding rate provided in an embodiment of the present application, which specifically includes the following steps:

[0113] Step 510: Generate a first constraint condition based on the channel matching variable.

[0114] In one embodiment, the first constraint condition is used to constrain the sum of the number of transmission channels allocated to the bit rate sub-videos corresponding to all target transmission videos to be less than or equal to the total number of transmission channels. Assuming that the number of target transmission videos is N and the number of transmission channels is K, the first constraint condition is expressed as:

[0115]

[0116] It can be seen that the purpose of the first constraint is to constrain the total number of transmission channels, that is, the total number of transmission channels allocated to the bit rate sub-videos corresponding to all target transmission videos cannot exceed the total number of transmission channels available in the wireless channel.

[0117] Step 520: Generate a second constraint condition based on the channel matching variable.

[0118] In one embodiment, the second constraint condition is used to constrain each transmission channel to be allocated to only one bit rate sub-video of a target transmission video, which is expressed as:

[0119]

[0120] That is to say, each transmission channel can only be allocated to a bit rate sub-video corresponding to a target transmission channel.

[0121] Step 530: Generate a third constraint condition for constraining the number of encoding bit rates to a preset number.

[0122] In one embodiment, assuming that the number of encoding bit rates is a preset number L, the third constraint condition is expressed as:

[0123] r il ∈{r i1 ,r i2 ,L,r iL}

[0124] That is, for each target transmission video, the encoding bit rate of its bit rate sub-video is one of L encoding bit rates.

[0125] Step 540: When the encoding bit rate is greater than the channel capacity of a single transmission channel, more than one transmission channel is allocated to the bit rate sub-video corresponding to the encoding bit rate, and a fourth constraint condition is generated.

[0126] In one embodiment, the fourth constraint is expressed as:

[0127]

[0128] Among them, |C k | represents the channel capacity of the kth transmission channel.

[0129] That is to say, if the transmission rate required by the encoding bit rate corresponding to a certain bit rate sub-video exceeds the capacity of a single transmission channel, multiple transmission channels can be used to meet the transmission requirements, ensuring that the bit rate sub-video can support its transmission rate requirements by allocating a sufficient number of transmission channels.

[0130] Step 550: Obtain channel constraints according to the first constraint, the second constraint, the third constraint and the fourth constraint.

[0131] In one embodiment, the channel constraint is expressed as:

[0132]

[0133] Step 130: construct an optimization target model based on the visual quality assessment parameters and the channel constraint conditions, and solve the optimization target model to obtain channel allocation data.

[0134] In one embodiment, the purpose of video transmission is to ensure the highest visual quality, so a corresponding optimization target model is generated based on this purpose. Figure 6 , Figure 6 This is a flowchart of constructing an optimization target model based on visual quality assessment parameters and channel constraints provided by an embodiment of the present application, which specifically includes the following steps:

[0135] Step 610: Obtain a preset weight of the target transmission video and a corresponding visual quality assessment parameter of each bitrate sub-video.

[0136] In one embodiment, for a panoramic video, the corresponding importance of target transmission videos at different perspectives is different, so different preset weights are assigned according to their importance. The preset weights here are set according to actual conditions. Each target transmission video includes a corresponding preset weight. For example, the preset weight of the i-th target transmission video is expressed as w i .

[0137] At this time, the preset weight of each target transmission video and the corresponding visual quality assessment parameter of each bit rate sub-video are obtained.

[0138] Step 620: Generate an overall transmission quality parameter based on the preset weight and the visual quality assessment parameter.

[0139] In one embodiment, firstly, the sum of all visual quality assessment parameters corresponding to each target transmission video is obtained to obtain a first intermediate value.

[0140] Multiplying the preset weight by the corresponding first intermediate value to obtain a second intermediate value;

[0141] All second intermediate values ​​are accumulated to obtain the overall transmission quality parameter.

[0142] The overall transmission quality parameter is expressed as:

[0143]

[0144] Where Q(R) represents the total transmission quality parameter, R represents the bit rate sub-video to be transmitted selected from all target transmission videos, represents the first intermediate value, Indicates the second intermediate value.

[0145] Step 630: Maximize the total transmission quality parameter to generate a resource optimization objective function.

[0146] In one embodiment, the resource optimization objective function is expressed as:

[0147]

[0148] Step 640: construct an optimization target model based on the resource optimization objective function and the channel constraint conditions.

[0149] In one embodiment, the optimization target model is expressed as:

[0150]

[0151] It can be seen that the optimization target model is a mixed integer nonlinear programming problem, and it is difficult to obtain the solution of the bit rate sub-video and the corresponding channel matching variables of each target transmission video in polynomial time. Therefore, the embodiment of the present application solves it through the following iterative method.

[0152] In one embodiment, referring to Figure 7 , Figure 7 This is a flow chart of solving the optimization target model to obtain channel allocation data provided by an embodiment of the present application, which specifically includes the following steps:

[0153] Step 710: Initialize the channel set and accumulated channel transmission capacity corresponding to each target transmission video.

[0154] In one embodiment, for the i-th target transmission video, its channel set is represented as {H i}, the channel capacity of all transmission channels in the channel set is added up to get the cumulative channel transmission capacity, which is expressed as: {B i}.

[0155] Step 720: Allocate a transmission channel to each target transmission video in turn. In each allocation process, each target transmission video is only allocated to one transmission channel.

[0156] In one embodiment, if there are K transmission channels, they need to be allocated K times. In each allocation process, each transmission channel is allocated one target transmission video, that is, each transmission channel will not be allocated to two or more target transmission videos.

[0157] Step 730: Calculate the quality gain corresponding to each transmission channel being allocated to different target transmission videos according to the allocation result.

[0158] In one embodiment, taking the allocation of the kth transmission channel as an example, assuming that the kth channel is allocated to the i-th target transmission video, the quality gain at this time is expressed as:

[0159]

[0160] in, represents the quality gain of the kth transmission channel allocated to the i-th target transmission video, F i represents the fitting function of the i-th target transmission video, |C k | represents the channel capacity of the kth transmission channel, and B i and B i +|C k |Substitute x as the independent variable into the fitting function F i , we can get F i (B i ) and Fi (B i +|C k |).

[0161] Step 740: Allocate a transmission channel for the target transmission video that maximizes the quality gain as the target transmission channel.

[0162] In one embodiment, the quality gain corresponding to the k-th transmission channel allocated to each target transmission video is calculated, and the target transmission video corresponding to the maximum value of the quality gain is selected, for example, the p-th target transmission video, and the k-th transmission channel is used as the target transmission channel of the p-th target transmission video. At this time, the channel set corresponding to the p-th target transmission video {H p} and the cumulative channel transmission capacity {B p} is updated accordingly, expressed as:

[0163] {H p}={H p}∪{k}

[0164] B p =B p +|C k |

[0165] It is understandable that, for a target transmission video, the corresponding target transmission channel may be more than one. The above process is continuously performed until all available transmission channels are allocated.

[0166] Step 750: Solve and optimize the target model based on the target transmission channel to obtain the target bit rate video corresponding to each target transmission video.

[0167] In one embodiment, the above iterative process is performed until each transmission channel is allocated, and the target transmission channel corresponding to each target transmission video is obtained, and then the optimization target model is solved. Figure 8 , Figure 8 The present invention provides a flowchart of solving an optimization target model based on a target transmission channel to obtain a target bit rate video corresponding to each target transmission video, which specifically includes the following steps:

[0168] Step 810: Obtain the transmission bandwidth of the target transmission channel corresponding to each target transmission video.

[0169] In one embodiment, for each target transmission video, the transmission bandwidth of the target transmission channel can be obtained by accumulating the channel capacity of at least one target transmission channel acquired.

[0170] Step 820: Determine at least one candidate sub-video from the corresponding bitrate sub-videos based on the transmission bandwidth.

[0171] In one embodiment, since the bit rate sub-videos corresponding to different encoding bit rates require different transmission bandwidths, in order to ensure that the transmission channel can meet the transmission requirements, the bit rate sub-video corresponding to the encoding bit rate smaller than the transmission bandwidth is selected as the candidate sub-video.

[0172] Step 830: Select candidate sub-videos one by one, set the corresponding channel matching variables to the first value, and set the channel matching variables of other bitrate sub-videos to the second value.

[0173] In one embodiment, candidate sub-videos are selected one by one, and the corresponding channel matching variables are set to the first value, and the channel matching variables of other bit rate sub-videos are set to the second value. For example, there are 4 candidate sub-videos for the i-th target transmission video, and its target transmission channel is the k-th transmission channel. When the first candidate sub-video is selected, its channel matching variable The channel matching variables of other candidate word videos are 0, expressed as:

[0174] Step 840: Based on the updated channel matching variables, while satisfying the channel constraints, the resource optimization objective function is calculated, and the candidate sub-video of each target transmission channel corresponding to the maximum total transmission quality parameter is selected as the corresponding target bit rate video.

[0175] In one embodiment, according to the above example, the channel matching variables of the candidate sub-videos corresponding to the target transmission video are updated, and then, under the premise of satisfying the channel constraint conditions, they are substituted into the resource optimization objective function, the corresponding total transmission quality parameter is calculated, and the candidate sub-videos corresponding to each target transmission channel when the total transmission quality parameter is maximized are selected as the corresponding target bit rate video. In other words, when solving the optimization target model, the target transmission channel corresponding to the target transmission video is first determined, that is, the association between the parameter i and the parameter k is determined, and then the number of bit rate sub-videos of each target transmission video participating in the solution process is reduced, and a smaller number of candidate sub-videos are used to replace all bit rate sub-videos, thereby reducing the complexity of the solution and improving the efficiency of the solution.

[0176] The above process can determine to select a target bit rate video from its corresponding candidate sub-videos for each target transmission video by solving the optimization target model.

[0177] Step 760: Obtain channel allocation data based on the target bit rate video and the target transmission channel.

[0178] In one embodiment, each target transmission video has a corresponding target bit rate video and a corresponding target transmission channel, that is, the target bit rate video and the target transmission channel can be summarized as channel allocation data.

[0179] Step 140: Select a target bit rate video from the bit rate sub-videos corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel for the target bit rate video.

[0180] In one embodiment, after the channel allocation data is obtained, the target bit rate video corresponding to each target transmission video can be determined therefrom, and the target transmission channel of the target bit rate video can be determined.

[0181] Step 150: Transmit the corresponding bit rate sub-videos respectively based on the target transmission channel.

[0182] In one embodiment, the corresponding bit rate sub-videos are sent to the user via the target transmission channel, and the user can perform operations such as decoding and splicing on all received bit rate sub-videos as required.

[0183] The embodiment of the present application uses a function fitting method to construct visual quality assessment parameters for each target transmission video in an immersive panoramic video, and uses the visual quality assessment parameters to evaluate the correlation between the visual quality of each target transmission video and the adopted encoding bit rate. And based on the visual quality assessment parameters, an optimization target model is constructed during the transmission process to maximize the visual quality of the video received by the user when the transmission resources are optimal. In addition, when allocating a target transmission channel for each target transmission video, the quality gain is used to select a suitable target transmission channel for each target transmission video to improve the utilization rate of transmission resources.

[0184] In one embodiment, referring to Fig. 9 , Fig. 9 It is a schematic diagram for verifying the effect of the immersive panoramic video wireless transmission method provided in an embodiment of the present application.

[0185] First, two channel allocation methods in the related art are selected, denoted as CA1 and CA2. Specifically, the CA1 method allocates all transmission channels equally to each video segment to be transmitted (equivalent to the video to be transmitted in the embodiment of the present application). The CA2 method arranges all transmission channels in descending order according to the channel capacity and divides them into Then, the video segments to be transmitted are arranged in descending order of importance, and then each group of transmission channels is matched one by one with each video segment to be transmitted.

[0186] The simulation experiment configuration parameters are as follows: Use the test videos in the PVS-HMEM immersive panoramic video dataset, including Camping, Graffiti, Stratosphere, TalkingInCar and Waterskiing. The resolution of the test immersive panoramic video is 3840x1920, and the frame rate is 30fps. Here, the GOP length of the test immersive panoramic video is set to 1 second, so it contains 30 frames of images. In addition, the test immersive panoramic video is set to use three video division modes, including {6×4, 8×6, 8×8}. And the total number of target transmission videos covered by the user's target perspective in the above three division modes is recorded as N={9,18,24} respectively. The encoding bit rate can be 7. In addition, the immersive panoramic video wireless transmission system uses K=32 OFDM channels as transmission channels, and the gain of each transmission channel obeys the Rayleigh distribution.

[0187] Fig. 9 The experimental results show the average received video quality obtained by users using different wireless channel allocation methods to test immersive panoramic videos in various division modes, with the peak signal-to-noise ratio WS-PSNR as the evaluation parameter. It can be seen that under various division modes, the CA1 method obtains the worst user reception WS-PSNR performance. When considering the differences in content characteristics of different video slices and adopting unequal channel allocation, although the CA2 method can improve the user reception WS-PSNR performance compared to CA1. However, the immersive panoramic video wireless transmission method of the embodiment of the present application has the best reception WS-PSNR performance.

[0188] The technical solution provided by the embodiment of the present application determines at least one target transmission video from multiple videos to be transmitted according to the transmission requirements, wherein the target transmission video includes at least one rate sub-video corresponding to the encoding rate. Then, the visual quality assessment parameters of each target transmission video are obtained, and the channel constraint conditions are generated based on the channel matching variables corresponding to the target transmission video and at least one transmission channel, and then the optimization target model is constructed based on the visual quality assessment parameters and the channel constraint conditions, and the optimization target model is solved to obtain the channel allocation data. Then, according to the channel allocation data, the target rate video is selected from the rate sub-video corresponding to each target transmission video, and the target transmission channel of the target rate video is determined, and finally the corresponding rate sub-videos are transmitted respectively based on the target transmission channel. In the process of channel allocation, the embodiment of the present application generates a visual quality assessment parameter for each target transmission video to participate in the optimization process, and a unified allocation process is performed for all target transmission videos, fully considering the content association between the target transmission videos, and improving the reception quality of the panoramic video.

[0189] The present application also provides an immersive panoramic video wireless transmission device, which can implement the above-mentioned immersive panoramic video wireless transmission method. Fig.10 , the device comprises:

[0190] The video selection module 1010 is used to determine at least one target transmission video from a plurality of videos to be transmitted according to transmission requirements, wherein the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate.

[0191] The quality parameter calculation module 1020 is used to obtain the visual quality assessment parameters of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel.

[0192] The optimization model construction module 1030 is used to construct an optimization target model based on visual quality assessment parameters and channel constraints, and solve the optimization target model to obtain channel allocation data.

[0193] Channel allocation module 1040: used to select a target bit rate video from the bit rate sub-videos corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel for the target bit rate video.

[0194] Video transmission module 1050: used to transmit corresponding bit rate sub-videos based on target transmission channels.

[0195] The specific implementation of the immersive panoramic video wireless transmission device of this embodiment is basically the same as the specific implementation of the immersive panoramic video wireless transmission method described above, and will not be repeated here.

[0196] The present application also provides an electronic device, including:

[0197] at least one memory;

[0198] at least one processor;

[0199] at least one program;

[0200] The program is stored in the memory, and the processor executes the at least one program to implement the above-mentioned immersive panoramic video wireless transmission method implemented in this application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0201] See also Fig.11 , Fig.11 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0202] The processor 1101 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0203] The memory 1102 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 may store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the immersive panoramic video wireless transmission method of the embodiments of this application;

[0204] Input / output interface 1103, used to implement information input and output;

[0205] Communication interface 1104, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and

[0206] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0207] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0208] An embodiment of the present application also provides a storage medium, which is a storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned immersive panoramic video wireless transmission method is implemented.

[0209] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0210] The immersive panoramic video wireless transmission method, device, equipment and storage medium proposed in the embodiment of the present application determine at least one target transmission video from multiple videos to be transmitted according to the transmission requirements, wherein the target transmission video includes at least one rate sub-video corresponding to the encoding rate. Then, the visual quality assessment parameters of each target transmission video are obtained, and the channel constraint conditions are generated based on the channel matching variables corresponding to the target transmission video and at least one transmission channel, and then the optimization target model is constructed based on the visual quality assessment parameters and the channel constraint conditions, and the optimization target model is solved to obtain the channel allocation data. Then, according to the channel allocation data, the target rate video is selected from the rate sub-video corresponding to each target transmission video, and the target transmission channel of the target rate video is determined, and finally the corresponding rate sub-videos are transmitted respectively based on the target transmission channel. In the process of channel allocation, the embodiment of the present application generates a visual quality assessment parameter for each target transmission video to participate in the optimization process, and a unified allocation process is performed for all target transmission videos, fully considering the content association between the target transmission videos, and improving the reception quality of the panoramic video.

[0211] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0212] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0213] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0215] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0216] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0217] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0218] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0219] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0220] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store programs.

[0221] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. An immersive panoramic video wireless transmission method, characterized in that: include: Determine at least one target transmission video from a plurality of videos to be transmitted according to a transmission requirement, wherein the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate; Acquire a visual quality assessment parameter of each of the target transmission videos, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel; Building an optimization target model based on the visual quality assessment parameters and the channel constraint conditions, and solving the optimization target model to obtain channel allocation data; Selecting a target bit rate video from the bit rate sub-videos corresponding to each of the target transmission videos according to the channel allocation data, and determining a target transmission channel for the target bit rate video; The corresponding bit rate sub-videos are transmitted respectively based on the target transmission channel.

2. The immersive panoramic video wireless transmission method according to claim 1, characterized in that: The obtaining of the visual quality assessment parameter of each target transmission video comprises: Acquire a first fitting parameter and a second fitting parameter of the target transmission video; Obtain the encoding bit rate corresponding to each bit rate sub-video; The visual quality assessment parameter corresponding to each of the bit rate sub-videos is calculated based on the encoding bit rate, the first fitting parameter and the second fitting parameter, and the visual quality assessment parameter is used to indicate the transmission quality of the video content of the bit rate sub-video.

3. The immersive panoramic video wireless transmission method according to claim 2, characterized in that: The obtaining of the first fitting parameter and the second fitting parameter of the target transmission video comprises: Encode multiple continuous image frames corresponding to the target transmission video in parallel at multiple different encoding rates to obtain corresponding encoding results, and obtain a peak signal-to-noise ratio corresponding to each encoding result; A plurality of groups of fitting data are generated according to the encoding bit rate and the corresponding peak signal-to-noise ratio, and an exponential function fitting process is performed using the fitting data to obtain the first fitting parameter and the second fitting parameter.

4. The immersive panoramic video wireless transmission method according to claim 1, characterized in that: The generating of channel constraint conditions based on the channel matching variables corresponding to the target transmission video and at least one transmission channel comprises: Generate the channel matching variable corresponding to each bit rate sub video and each transmission channel, if the bit rate sub video is assigned to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value; The channel constraint condition is generated based on the channel matching variable, the number of the encoding code rates, and the encoding code rates.

5. The immersive panoramic video wireless transmission method according to claim 3, characterized in that: The generating the channel constraint condition based on the channel matching variable, the number of the encoding code rates, and the encoding code rates comprises: Generate a first constraint condition based on the channel matching variable, where the first constraint condition is used to constrain the sum of the number of the transmission channels allocated to the bit rate sub-videos corresponding to all the target transmission videos to be less than or equal to the total number of the transmission channels; Generate a second constraint condition based on the channel matching variable, where the second constraint condition is used to constrain each of the transmission channels to be allocated to only one of the bitrate sub-videos of the target transmission video; generating a third constraint condition for constraining the number of the encoding bit rates to be a preset number; When the encoding bit rate is greater than the channel capacity of a single transmission channel, more than one transmission channel is allocated to the bit rate sub-video corresponding to the encoding bit rate, and a fourth constraint condition is generated; The channel constraint condition is obtained according to the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.

6. The immersive panoramic video wireless transmission method according to claim 4, characterized in that: The constructing of the optimization target model based on the visual quality assessment parameter and the channel constraint condition comprises: Obtaining a preset weight of the target transmission video and the corresponding visual quality assessment parameter of each bitrate sub-video; generating an overall transmission quality parameter based on the preset weight and the visual quality assessment parameter; Maximizing the total transmission quality parameter to generate a resource optimization objective function; The optimization target model is constructed based on the resource optimization objective function and the channel constraint condition.

7. The immersive panoramic video wireless transmission method according to claim 6, characterized in that: The generating of the total transmission quality parameter based on the preset weight and the visual quality assessment parameter comprises: Obtaining the sum of all the visual quality assessment parameters corresponding to each of the target transmission videos to obtain a first intermediate value; Multiplying the preset weight by the corresponding first intermediate value to obtain a second intermediate value; The total transmission quality parameter is obtained by accumulating all the second intermediate values.

8. The immersive panoramic video wireless transmission method according to claim 6, characterized in that: The step of solving the optimization target model to obtain channel allocation data includes: Initializing a channel set and a cumulative channel transmission capacity corresponding to each of the target transmission videos; Allocating the transmission channel to each of the target transmission videos in sequence, and in each allocation process, allocating one of the target transmission videos to each of the transmission channels; Calculate, according to the allocation result, the quality gain corresponding to each transmission channel being allocated to different target transmission videos; Allocating the transmission channel as a target transmission channel for the target transmission video that maximizes the quality gain; Solving the optimization target model based on the target transmission channel to obtain the target bit rate video corresponding to each target transmission video; The channel allocation data is obtained based on the target bit rate video and the target transmission channel.

9. The immersive panoramic video wireless transmission method according to claim 8, characterized in that: The step of solving the optimization target model based on the target transmission channel to obtain the target bit rate video corresponding to each target transmission video includes: Obtaining the transmission bandwidth of the target transmission channel corresponding to each target transmission video; Determine at least one candidate sub-video from the corresponding bitrate sub-videos based on the transmission bandwidth; Select the candidate sub-videos one by one, set the corresponding channel matching variable to the first value, and set the channel matching variables of other bitrate sub-videos to the second value; Based on the updated channel matching variable, while satisfying the channel constraint condition, the resource optimization objective function is calculated, and the candidate sub-video corresponding to each target transmission channel when the total transmission quality parameter is maximized is selected as the corresponding target bit rate video.

10. The immersive panoramic video wireless transmission method according to any one of claims 1 to 9, characterized in that: The step of determining at least one target transmission video from a plurality of videos to be transmitted according to the transmission requirement comprises: determining a target viewing angle according to the transmission requirements; A video that meets the target viewing angle is screened from the multiple videos to be transmitted as the target transmission video.

11. An immersive panoramic video wireless transmission device, characterized in that: include: Video selection module: used to determine at least one target transmission video from multiple videos to be transmitted according to transmission requirements, wherein the target transmission video includes at least one bit rate sub-video corresponding to the encoding bit rate; A quality parameter calculation module: used to obtain a visual quality assessment parameter of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel; An optimization model building module is used to build an optimization target model based on the visual quality evaluation parameters and the channel constraint conditions, and solve the optimization target model to obtain channel allocation data; A channel allocation module: configured to select a target bit rate video from the bit rate sub-videos corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel for the target bit rate video; Video transmission module: used to transmit the corresponding bit rate sub-videos based on the target transmission channel.

12. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the immersive panoramic video wireless transmission method according to any one of claims 1 to 10 when executing the computer program.

13. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the immersive panoramic video wireless transmission method according to any one of claims 1 to 10 is implemented.

Images