Immersive panoramic video wireless transmission method and device, equipment and storage medium

Abstract

Embodiments of the present application provide an immersive panoramic video wireless transmission method, device, equipment and storage medium, relating to the display technical field. The method determines a target transmission video from a plurality of to-be-transmitted videos according to transmission requirements, obtains a visual quality evaluation parameter of the target transmission video, generates a channel constraint condition based on the target transmission video and a channel matching variable, constructs an optimization target model based on the visual quality evaluation parameter and the channel constraint condition, and solves the optimization target model to obtain channel allocation data. According to the channel allocation data, a target code rate video is selected from a code rate sub-video corresponding to each target transmission video, and a target transmission channel of the target code rate video is determined, and the corresponding code rate sub-video is transmitted based on the target transmission channel. In the channel allocation process, the visual quality evaluation parameter of each target transmission video is generated to participate in the optimization process, and a unified allocation process is performed, the content correlation between the target transmission videos is fully considered, and the reception quality of the panoramic video is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to an immersive panoramic video wireless transmission method and device, equipment and storage medium. BACKGROUND

[0002] The immersive panoramic video adaptive transmission technology develops rapidly with the development of virtual reality application. Due to the limited bandwidth resources of wireless transmission, in the design of the immersive panoramic video wireless transmission system, the immersive panoramic video is usually divided into multiple video slices of equal size, and when transmitting, not all video slices are transmitted, but only the relevant video slices within the field of view (FoV) of the user are transmitted to effectively save the required bandwidth for transmission.

[0003] In the related art, the video slices are independently encoded on the server, and when the user requests, the available channel resources are independently allocated for transmission for each video slice. This way does not fully consider the content association between the video slices when allocating channel resources, which makes it difficult to improve the reception quality of panoramic video. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose an immersive panoramic video wireless transmission method, device, equipment and storage medium to improve the data quality of video transmission.

[0005] To achieve the above purpose, the first aspect of the embodiments of the present application proposes an immersive panoramic video wireless transmission method, comprising:

[0006] According to the transmission requirement, at least one target transmission video is determined from a plurality of to-be-transmitted videos, and the target transmission video includes a code rate sub-video corresponding to at least one encoding code rate;

[0007] Obtain the visual quality evaluation parameter of each target transmission video, and generate a channel constraint condition based on the channel matching variable corresponding to the target transmission video and at least one transmission channel;

[0008] Based on the visual quality evaluation parameter and the channel constraint condition, an optimization target model is constructed, and the channel allocation data is obtained by solving the optimization target model;

[0009] According to the channel allocation data, a target code rate video is selected from the code rate sub-video corresponding to each target transmission video, and a target transmission channel of the target code rate video is determined;

[0010] Based on the target transmission channel, the corresponding code rate sub-video is transmitted respectively.

[0011] In some embodiments, the visual quality evaluation parameter of each target transmission video is obtained by:

[0012] The first fitting parameter and the second fitting parameter of the target transmission video are obtained.

[0013] The encoding rate corresponding to each rate sub-video is obtained.

[0014] The visual quality evaluation parameter corresponding to each rate sub-video is calculated based on the encoding rate, the first fitting parameter and the second fitting parameter, and the visual quality evaluation parameter is used to indicate the transmission quality of the video content of the rate sub-video.

[0015] In some embodiments, the first fitting parameter and the second fitting parameter of the target transmission video are obtained by:

[0016] A plurality of continuous image frames corresponding to the target transmission video are encoded in parallel at a plurality of different encoding rates to obtain corresponding encoding results, and the peak signal-to-noise ratio corresponding to each encoding result is obtained.

[0017] A plurality of sets of fitting data are generated according to the encoding rate and the corresponding peak signal-to-noise ratio, and an exponential function fitting process is performed on the fitting data to obtain the first fitting parameter and the second fitting parameter.

[0018] In some embodiments, the channel constraint condition is generated based on the channel matching variable corresponding to the target transmission video and at least one transmission channel, comprising:

[0019] The channel matching variable corresponding to each rate sub-video and each transmission channel is generated, and if the rate sub-video is assigned to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value.

[0020] The channel constraint condition is generated based on the channel matching variable, the number of encoding rates and the encoding rates.

[0021] In some embodiments, the channel constraint condition is generated based on the channel matching variable, the number of encoding rates and the encoding rates, comprising:

[0022] A first constraint condition is generated based on the channel matching variable, and the first constraint condition is used to constrain the sum of the number of transmission channels to which all rate sub-videos corresponding to the target transmission video are assigned to be less than or equal to the total number of transmission channels.

[0023] generating a second constraint condition based on the channel matching variable, the second constraint condition being used to constrain each of the transmission channels to be allocated to only one of the code rate sub-videos of the target transmission video;

[0024] generating a third constraint condition used to constrain the number of the encoding code rates to be a preset number;

[0025] when the encoding code rate is greater than the channel capacity of a single transmission channel, allocating more than one of the transmission channels to the code rate sub-video corresponding to the encoding code rate, and generating a fourth constraint condition;

[0026] obtaining the channel constraint condition according to the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.

[0027] In some embodiments, the constructing an optimization target model based on the visual quality evaluation parameter and the channel constraint condition comprises:

[0028] obtaining a preset weight of the target transmission video and the visual quality evaluation parameter of each of the corresponding code rate sub-videos;

[0029] generating a total transmission quality parameter based on the preset weight and the visual quality evaluation parameter;

[0030] maximizing the total transmission quality parameter to generate a resource optimization target function;

[0031] constructing the optimization target model based on the resource optimization target function and the channel constraint condition.

[0032] In some embodiments, the generating a total transmission quality parameter based on the preset weight and the visual quality evaluation parameter comprises:

[0033] obtaining a sum of all the visual quality evaluation parameters corresponding to each of the target transmission videos to obtain a first intermediate value;

[0034] multiplying the preset weight by the corresponding first intermediate value to obtain a second intermediate value;

[0035] accumulating all the second intermediate values to obtain the total transmission quality parameter.

[0036] In some embodiments, the solving the optimization target model to obtain channel allocation data comprises:

[0037] initializing a channel set corresponding to each of the target transmission videos and a cumulative channel transmission capacity;

[0038] allocating the transmission channels to each of the target transmission videos in sequence, and in each allocation process, allocating one of the target transmission videos to each of the transmission channels;

[0039] calculating a quality gain corresponding to each of the transmission channels being allocated to different target transmission videos according to the allocation result;

[0040] allocating the transmission channel with the maximum quality gain as a target transmission channel to the target transmission video with the maximum quality gain;

[0041] solving the optimization target model based on the target transmission channel to obtain the target code rate video corresponding to each of the target transmission videos;

[0042] obtaining the channel allocation data based on the target code rate video and the target transmission channel.

[0043] In some embodiments, the solving of the optimization target model based on the target transmission channel to obtain the target code rate video corresponding to each of the target transmission videos comprises:

[0044] obtaining a transmission bandwidth of the target transmission channel corresponding to each of the target transmission videos;

[0045] determining at least one candidate sub-video from the code rate sub-video corresponding to the transmission bandwidth;

[0046] selecting the candidate sub-video one by one, setting the channel matching variable corresponding to the candidate sub-video as the first value, and setting the channel matching variable of other code rate sub-videos as the second value;

[0047] based on the updated channel matching variable, calculating the resource optimization target function under the condition of satisfying the channel constraint condition, and selecting the candidate sub-video corresponding to each of the target transmission channels when the total transmission quality parameter is maximized as the target code rate video corresponding thereto.

[0048] In some embodiments, the determining of at least one target transmission video from a plurality of to-be-transmitted videos according to a transmission requirement comprises:

[0049] determining a target view angle according to the transmission requirement;

[0050] filtering a video satisfying the target view angle from the plurality of to-be-transmitted videos as the target transmission video.

[0051] To achieve the above object, a second aspect of the embodiments of the present application proposes an immersive panoramic video wireless transmission device, comprising:

[0052] The video selection module is configured to determine at least one target transmission video from a plurality of to-be-transmitted videos according to transmission requirements, wherein the target transmission video comprises at least one code rate sub-video corresponding to an encoding code rate;

[0053] The quality parameter calculation module is configured to obtain a visual quality evaluation parameter of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel;

[0054] The optimization model construction module is configured to construct an optimization target model based on the visual quality evaluation parameter and the channel constraint condition, and solve the optimization target model to obtain channel allocation data;

[0055] The channel allocation module is configured to select a target code rate video from the code rate sub-video corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel of the target code rate video;

[0056] The video transmission module is configured to transmit the code rate sub-video corresponding to the target transmission channel based on the target transmission channel.

[0057] To achieve the above object, a third aspect of embodiments of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method of the first aspect when executing the computer program.

[0058] To achieve the above object, a fourth aspect of embodiments of the present application provides a storage medium, the storage medium being a storage medium, the storage medium storing a computer program, and the computer program being executed by a processor to implement the method of the first aspect.

[0059] The immersive panoramic video wireless transmission method, device, equipment and storage medium provided by the embodiments of the present application determine at least one target transmission video from a plurality of to-be-transmitted videos according to transmission requirements, wherein the target transmission video includes a code rate sub-video corresponding to at least one encoding code rate. Then, the visual quality evaluation parameter of each target transmission video is obtained, and the channel constraint condition is generated based on the channel matching variable corresponding to the target transmission video and at least one transmission channel. Then, the optimization target model is constructed based on the visual quality evaluation parameter and the channel constraint condition, and the channel allocation data is obtained by solving the optimization target model. Then, the target code rate video is selected from the code rate sub-video corresponding to each target transmission video according to the channel allocation data, and the target transmission channel of the target code rate video is determined. Finally, the corresponding code rate sub-video is transmitted based on the target transmission channel. In the channel allocation process, the embodiments of the present application generate a visual quality evaluation parameter for each target transmission video to participate in the optimization process, and perform a unified allocation process for all target transmission videos, fully consider the content correlation between the target transmission videos, and improve the reception quality of the panoramic video. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 FIG. 1 is a flowchart of an immersive panoramic video wireless transmission method provided by the embodiments of the present application.

[0061] Figure 2 FIG. 2 is a flowchart of obtaining a visual quality evaluation parameter of each target transmission video provided by the embodiments of the present application.

[0062] Figure 3 FIG. 3 is a flowchart of obtaining a first fitting parameter and a second fitting parameter of a target transmission video provided by the embodiments of the present application.

[0063] Figure 4 FIG. 4 is a flowchart of generating a channel constraint condition based on a channel matching variable corresponding to a target transmission video and at least one transmission channel provided by the embodiments of the present application.

[0064] Figure 5 FIG. 5 is a flowchart of generating a channel constraint condition based on a channel matching variable, a number of encoding code rates and an encoding code rate provided by the embodiments of the present application.

[0065] Figure 6 FIG. 6 is a flowchart of constructing an optimization target model based on a visual quality evaluation parameter and a channel constraint condition provided by the embodiments of the present application.

[0066] Figure 7 FIG. 7 is a flowchart of solving an optimization target model to obtain channel allocation data provided by the embodiments of the present application.

[0067] Figure 8FIG. 1 is a flowchart of a method for solving an optimization target model based on a target transmission channel according to an embodiment of the present application.

[0068] Figure 9 FIG. 6 is a schematic diagram of effect verification of an immersive panoramic video wireless transmission method according to an embodiment of the present application.

[0069] Figure 10 FIG. 7 is a structural block diagram of an immersive panoramic video wireless transmission device according to another embodiment of the present application.

[0070] Figure 11 FIG. 8 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0072] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0074] The immersive panoramic video adaptive transmission technology develops rapidly with the development of virtual reality applications. Due to the limited bandwidth resources of wireless transmission, in the design of the immersive panoramic video wireless transmission system, the immersive panoramic video is usually divided into multiple video slices of equal size, and when transmitting, not all video slices are transmitted, but only the relevant video slices within the user's field of view (FoV) are transmitted to effectively save the required bandwidth for transmission.

[0075] In the related art, the video slices are independently encoded on the server, and when the user requests, the available channel resources are independently allocated for each video slice for transmission. This way does not fully consider the content association between the video slices when allocating channel resources, which makes it difficult to improve the reception quality of panoramic video.

[0076] Based on this, the embodiment of the application provides a kind of immersive panoramic video wireless transmission method, device, equipment and storage medium, in the channel allocation process, for each target transmission video, a visual quality evaluation parameter is generated to participate in optimization process, for all target transmission video, uniform distribution process is carried out, the content association between target transmission video is fully considered, and the receiving quality of panoramic video is improved.

[0077] The embodiment of the application provides an immersive panoramic video wireless transmission method, device, equipment and storage medium, which is specifically described by the following embodiment, first, the immersive panoramic video wireless transmission method in the embodiment of the application is described.

[0078] The immersive panoramic video wireless transmission method provided by the embodiment of the application relates to the technical field of communication. The immersive panoramic video wireless transmission method provided by the embodiment of the application can be applied in a terminal, can be applied in a server, and can also be a computer program running in the terminal or the server. For example, the computer program can be a native program or a software module in an operating system; it can be a native application program (APP), i.e., a program that needs to be installed in an operating system to run, such as a client supporting video data transmission, i.e., a program that only needs to be downloaded into a browser environment to run; it can also be an applet that can be embedded into any APP. In summary, the above computer program can be any form of application program, module or plug-in. The terminal communicates with the server through a network. The immersive panoramic video wireless transmission method can be executed by the terminal or the server, or cooperatively executed by the terminal and the server.

[0079] In some embodiments, the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart watch, or the like. In addition, the terminal can also be a smart vehicle-mounted device. The smart vehicle-mounted device applies the immersive panoramic video wireless transmission method of the embodiments to provide related services and improve the driving experience. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms; or a service node in a blockchain system, the service nodes in the blockchain system form a peer-to-peer (P2P) network, and the P2P protocol is an application layer protocol running on the transmission control protocol (TCP) protocol. The terminal and the server can be connected through a communication connection mode such as Bluetooth, a universal serial bus (USB), or a network, and the embodiments are not limited herein.

[0080] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0081] It should be noted that in various specific embodiments of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the user, such as user information, user behavior data, user history data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the user's separate permission or separate consent will be obtained through a pop-up window or by jumping to a confirmation page, and after obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of the embodiments of the present application will be obtained.

[0082] The immersive panoramic video wireless transmission method in the embodiments of the present application is described below.

[0083] Figure 1 is an optional flowchart of the immersive panoramic video wireless transmission method provided by the embodiments of the present application, Figure 1 The method in can include but is not limited to steps 110-150. It can be understood that the embodiments of the present application do not make specific limitations on the order of steps 110-150 in Figure 1 The order of steps 110-150 in can be adjusted according to actual needs, or some steps can be reduced or added.

[0084] Step 110: Determine at least one target transmission video from a plurality of to-be-transmitted videos according to transmission requirements.

[0085] In an embodiment, in a panoramic video scene, the entire panoramic video can be divided into different video clips in advance, and each video clip is a to-be-transmitted video. The division can be equal size or different size, and the following examples are based on panoramic video transmission. It can be understood that in other video scenes, the to-be-transmitted video can also be a complete video that has not been divided. The embodiments of the present application do not make specific limitations on this.

[0086] In an embodiment, when a user needs to transmit a panoramic video, a corresponding transmission requirement will be generated. At this time, the target perspective of the user can be obtained according to the transmission requirement, and then relevant videos that meet the target perspective are selected from a plurality of to-be-transmitted videos as target transmission videos. That is, the target transmission video can cover the target perspective, so that the user can independently decode and render the received video and play it on the head-mounted display in real time, thereby realizing immersive panoramic viewing.

[0087] In an embodiment, since the target transmission video needs to be encoded before transmission to remove redundant data, reduce the amount of data transmitted, save storage space, and reduce the demand for network bandwidth, the target transmission video can be a video that has been pre-encoded and stored. In addition, different encoding rates will affect the quality of the video after encoding. The higher the encoding rate, the more bits per second will be transmitted, thereby retaining more image details and color information. However, a high encoding rate will also increase the file size of the encoded video, affecting transmission efficiency. Therefore, in an embodiment of the present application, a rate sub-video corresponding to each target transmission video is stored for each encoding rate, that is, the target transmission video includes at least one rate sub-video corresponding to an encoding rate. When transmitting, each target transmission video will select one rate sub-video. The encoding rate of the rate sub-video selected by different target transmission videos can be the same or different, and is determined according to the subsequent channel allocation process.

[0088] Step 120: Obtain a visual quality evaluation parameter of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel.

[0089] In an embodiment, the visual quality evaluation parameter is used to evaluate the transmission quality of the video content of the rate sub-video of the target transmission video at different encoding rates, wherein the visual quality is fed back to the final reception visual quality. Referring to Figure 2 , Figure 2 is a flowchart provided by an embodiment of the present application for obtaining a visual quality evaluation parameter of each target transmission video, and specifically includes the following steps:

[0090] Step 210: Obtain a first fitting parameter and a second fitting parameter of the target transmission video.

[0091] In an embodiment, for the ith target transmission video, assuming that the corresponding rate sub-video has L encoding rates, the encoding rate is represented as: R i ={r i1 ,r i2 ,L,r iL}, wherein r iL represents the Lth encoding rate, and R i represents all encoding rates corresponding to the ith target transmission video.

[0092] At this time, referring to Figure 3 , Figure 3 is a flowchart provided by an embodiment of the present application for obtaining a first fitting parameter and a second fitting parameter of a target transmission video, and specifically includes the following steps:

[0093] Step 310: Parallel encoding of multiple continuous image frames corresponding to the target transmission video at multiple different encoding rates to obtain corresponding encoding results, and obtaining the peak signal-to-noise ratio corresponding to each encoding result.

[0094] In an embodiment, for a target transmission video, it includes multiple continuous image frames, which are regarded as a unit (Group of Pictures, GOP) in the encoding process, the unit includes a key frame (I frame) and multiple dependent frames (P frame or B frame), the key frame is a complete image, and the dependent frame contains difference information with the previous frame. Then the GOP is parallel encoded using different encoding rates, and the encoder compresses the frames in the GOP according to the encoding rate, and outputs a video stream or a video file as the encoding result, which contains all the compressed frame data and necessary metadata, so that the decoder can correctly decode and play the video.

[0095] Next, the peak signal-to-noise ratio corresponding to each encoding result is calculated. The calculation process is described as follows: the difference square of each corresponding pixel point between the key frame of the target transmission video and the compressed frame in the encoding result is calculated, and then the average value is obtained to obtain the MSE. Then the value of MAX I is determined according to the color depth of the video. For example, for an 8-bit deep RGB image, the maximum value of each color channel is usually 255. Then the calculated MSE and the determined MAX I value are substituted into the PSNR formula to calculate the final PSNR value, which is expressed as:

[0096]

[0097] Step 320: Generating multiple sets of fitting data according to the encoding rate and the corresponding peak signal-to-noise ratio, and performing the fitting process of the exponential function using the fitting data to obtain the first fitting parameter and the second fitting parameter.

[0098] In an embodiment, the encoding rate and the corresponding peak signal-to-noise ratio are associated, each encoding rate corresponds to a peak signal-to-noise ratio as a fitting data, and the number of fitting data is obtained according to the number of encoding rates. Then the fitting process of the exponential function is performed using the fitting data, and the first fitting parameter and the second fitting parameter are obtained by fitting using the nonlinear least squares method, which is expressed as:

[0099]

[0100] wherein F i represents the fitting function corresponding to the i-th target transmission video, x represents the encoding rate in the fitting data, y represents the peak signal-to-noise ratio, a i represents the first fitting parameter corresponding to the i-th target transmission video, and b iThe second fitting parameter corresponding to the ith target transmission video is represented. It can be understood that only part of the encoding code rate is selected to participate in the fitting process to reduce the time consumption of the fitting process and improve the calculation efficiency.

[0101] Step 220: Obtain the encoding code rate corresponding to each code rate sub-video.

[0102] Step 230: Calculate the visual quality evaluation parameter corresponding to each code rate sub-video based on the encoding code rate, the first fitting parameter and the second fitting parameter.

[0103] In an embodiment, for the lth code rate sub-video of the ith target transmission video, the visual quality evaluation parameter is represented as:

[0104]

[0105] Wherein, q il The visual quality evaluation parameter of the lth code rate sub-video of the ith target transmission video is represented, r il The ith encoding code rate is represented.

[0106] According to the above process, the visual quality evaluation parameter of each code rate sub-video of each target transmission video is obtained, which participates in the subsequent channel allocation process.

[0107] In an embodiment, in a video transmission system, the base station collects the target view angle of the user in real time through the channel, and then determines the target transmission video required by the user according to the target view angle, selects the code rate sub-video most suitable for transmission of each target transmission video, and allocates different data wireless channels to each code rate sub-video, and transmits the relevant code rate sub-video by using the channel. Therefore, the embodiment of the present application generates a channel constraint condition based on the channel matching variable corresponding to the target transmission video and at least one transmission channel.

[0108] In an embodiment, referring to Figure 4 , Figure 4 The flowchart provided by the embodiment of the present application for generating a channel constraint condition based on the channel matching variable corresponding to the target transmission video and at least one transmission channel, specifically includes the following steps:

[0109] Step 410: Generate a channel matching variable corresponding to each code rate sub-video and each transmission channel. If the code rate sub-video is allocated to the transmission channel for transmission, the channel matching variable is a first value, otherwise it is a second value.

[0110] In an embodiment, a channel matching variable is defined to determine whether a rate sub-video is allocated to a transmission channel, if the rate sub-video is allocated to the transmission channel for transmission, the channel matching variable is a first value, otherwise, the channel matching variable is a second value, for example, the first value is 1 and the second value is 0. For example, there are K transmission channels, the lth rate sub-video of the ith target transmission video is transmitted by the kth transmission channel, at this time, the corresponding channel matching variable is 1, otherwise, it is 0.

[0111] Step 420: generating a channel constraint condition based on the channel matching variable, the number of encoding rates and the encoding rates.

[0112] In an embodiment, the channel constraint condition has four, which are a first constraint condition, a second constraint condition, a third constraint condition and a fourth constraint condition. Referring to Figure 5 , Figure 5 is a flowchart provided by the embodiment of the application for generating a channel constraint condition based on a channel matching variable, a number of encoding rates and encoding rates, and specifically includes the following steps:

[0113] Step 510: generating a first constraint condition based on the channel matching variable.

[0114] In an embodiment, the first constraint condition is used to constrain the sum of the number of transmission channels allocated to the rate sub-video corresponding to all target transmission videos to be less than or equal to the total number of transmission channels. Assuming that the number of target transmission videos is N and the number of transmission channels is K, the first constraint condition is expressed as:

[0115]

[0116] It can be seen that the purpose of the first constraint condition is to constrain the total number of transmission channels, that is, the number of transmission channels to which the rate sub-video corresponding to all target transmission videos is allocated cannot exceed the total number of available transmission channels in the wireless channel.

[0117] Step 520: generating a second constraint condition based on the channel matching variable.

[0118] In an embodiment, the second constraint condition is used to constrain each transmission channel to be allocated to only one rate sub-video of one target transmission video, which is expressed as:

[0119]

[0120] That is, each transmission channel can only be allocated to one rate sub-video corresponding to one target transmission channel.

[0121] ​​Step 530: generating a third constraint condition for limiting the number of encoding code rates to a preset number.

[0122] In an embodiment, assuming that the number of encoding code rates is a preset number L, the third constraint condition is expressed as:

[0123] r il ∈{r i1 ,r i2 ,L,r iL}

[0124] That is, for each target transmission video, the encoding code rate of the code rate sub-video is one of the L encoding code rates.

[0125] Step 540: when the encoding code rate is greater than the channel capacity of a single transmission channel, allocating more than one transmission channel for the code rate sub-video corresponding to the encoding code rate, and generating a fourth constraint condition.

[0126] In an embodiment, the fourth constraint condition is expressed as:

[0127]

[0128] Wherein, |C k | represents the channel capacity of the kth transmission channel.

[0129] That is, if the transmission rate required by the encoding code rate corresponding to a certain code rate sub-video exceeds the capacity of a single transmission channel, multiple transmission channels can be used to meet the transmission demand, ensuring that the code rate sub-video can support its transmission rate demand by allocating a sufficient number of transmission channels.

[0130] Step 550: obtaining a channel constraint condition according to the first constraint condition, the second constraint condition, the third constraint condition and the fourth constraint condition.

[0131] In an embodiment, the channel constraint condition is expressed as:

[0132]

[0133] Step 130: constructing an optimization target model based on the visual quality evaluation parameter and the channel constraint condition, and solving the optimization target model to obtain channel allocation data.

[0134] In an embodiment, the purpose of video transmission is to ensure the highest visual quality, so the corresponding optimization target model is generated based on this purpose. Referring to Figure 6 , Figure 6 is a flowchart provided by an embodiment of the application for constructing an optimization target model based on a visual quality evaluation parameter and a channel constraint condition, and specifically includes the following steps:

[0135] Step 610: Obtain the preset weight of the target transmission video and the visual quality evaluation parameter of each corresponding rate sub-video.

[0136] In an embodiment, for panoramic video, different importance of different view target transmission videos correspond, so different preset weights are allocated according to the importance, and the preset weight is set according to the actual situation, and each target transmission video contains a corresponding preset weight, for example, the preset weight of the ith target transmission video is represented as w i .

[0137] At this time, the preset weight of each target transmission video and the visual quality evaluation parameter of each corresponding rate sub-video are obtained.

[0138] Step 620: Generate the total transmission quality parameter based on the preset weight and the visual quality evaluation parameter.

[0139] In an embodiment, first, the sum of all visual quality evaluation parameters corresponding to each target transmission video is obtained, and a first intermediate value is obtained,

[0140] The preset weight is multiplied by the corresponding first intermediate value to obtain a second intermediate value.

[0141] All second intermediate values are accumulated to obtain the total transmission quality parameter.

[0142] The total transmission quality parameter is represented as:

[0143]

[0144] Wherein, Q(R) represents the total transmission quality parameter, R represents all target transmission video selected rate sub-video to be transmitted, represents the first intermediate value, represents the second intermediate value.

[0145] Step 630: Maximize the total transmission quality parameter to generate the resource optimization target function.

[0146] In an embodiment, the resource optimization target function is represented as:

[0147]

[0148] Step 640: Construct the optimization target model based on the resource optimization target function and the channel constraint condition.

[0149] In an embodiment, the optimization target model is represented as:

[0150]

[0151] It can be seen that the optimization target model is a mixed integer nonlinear programming problem, and it is difficult to obtain a solution of the optimization target model with respect to the code rate sub-video of each target transmission video and the corresponding channel matching variable in a polynomial time. Therefore, the embodiment of the present application solves the optimization target model by using the following iterative method.

[0152] In an embodiment, referring to Figure 7 , Figure 7 is a flowchart of a process of solving the optimization target model to obtain channel allocation data provided by the embodiment of the present application, and specifically includes the following steps:

[0153] Step 710: initializing a channel set corresponding to each target transmission video and cumulative channel transmission capacity.

[0154] In an embodiment, for the i th target transmission video, the channel set is represented as {H i}, and the cumulative channel transmission capacity can be obtained by adding up the channel capacities of all transmission channels in the channel set, which is represented as: {B i}.

[0155] Step 720: sequentially allocating transmission channels to each target transmission video, and in each allocation process, each target transmission video is allocated to only one transmission channel.

[0156] In an embodiment, there are K transmission channels, and thus K times of allocation are required. In each allocation process, each transmission channel is allocated to one target transmission video, that is, each transmission channel will not be allocated to two or more target transmission videos.

[0157] Step 730: calculating the quality gain corresponding to each transmission channel allocated to different target transmission videos according to the allocation result.

[0158] In an embodiment, taking the allocation of the k th transmission channel as an example, assuming that the k th channel is allocated to the i th target transmission video, the quality gain at this time is represented as:

[0159]

[0160] wherein, represents the quality gain of the k th transmission channel allocated to the i th target transmission video, F i represents the fitting function of the i th target transmission video, |C k | represents the channel capacity of the k th transmission channel, and B i and B i + |C k | are respectively taken as the independent variable x to be substituted into the fitting function F i , so that F i (B i ) and Fi (B i +|C k |)。

[0161] Step 740: allocating the transmission channel with the maximum quality gain as the target transmission channel for the target transmission video.

[0162] In an embodiment, the quality gain of the kth transmission channel allocated to each target transmission video is calculated, the target transmission video corresponding to the maximum quality gain is selected, for example, the pth target transmission video, and the kth transmission channel is allocated as the target transmission channel of the pth target transmission video. At this time, the channel set {H p} and the cumulative channel transmission capacity {B p} corresponding to the pth target transmission video are updated, and are expressed as:

[0163] {H p}={H p}∪{k}

[0164] B p =B p +|C k |

[0165] It can be understood that for a target transmission video, the target transmission channel corresponding thereto can be more than one. The above process is continuously executed until all the available transmission channels are allocated.

[0166] Step 750: solving the optimization target model based on the target transmission channel to obtain the target code rate video corresponding to each target transmission video.

[0167] In an embodiment, the above iterative process is executed until each transmission channel is allocated to obtain the target transmission channel corresponding to each target transmission video, and the optimization target model is solved. Referring to Figure 8 , Figure 8 is a flowchart provided by the embodiment of the present application for solving the optimization target model based on the target transmission channel to obtain the target code rate video corresponding to each target transmission video, and specifically includes the following steps:

[0168] Step 810: obtaining the transmission bandwidth of the target transmission channel corresponding to each target transmission video.

[0169] In an embodiment, for each target transmission video, the channel capacity of at least one target transmission channel obtained by the target transmission video is accumulated to obtain the transmission bandwidth of the target transmission channel.

[0170] Step 820: determining at least one candidate sub-video from the corresponding code rate sub-video based on the transmission bandwidth.

[0171] In an embodiment, since the transmission bandwidths required by the sub-videos corresponding to different encoding rates are different, in order to ensure that the transmission channel can meet the transmission requirement, the sub-video corresponding to the encoding rate smaller than the transmission bandwidth is selected as the candidate sub-video.

[0172] Step 830: The candidate sub-videos are selected one by one, the channel matching variable corresponding to each candidate sub-video is set as a first value, and the channel matching variables of the other sub-videos are set as a second value.

[0173] In an embodiment, the candidate sub-videos are selected one by one, the channel matching variable corresponding to each candidate sub-video is set as a first value, and the channel matching variables of the other sub-videos are set as a second value. For example, when the first candidate sub-video is selected, the channel matching variable of the first candidate sub-video is set as 1, and the channel matching variables of the other candidate sub-videos are set as 0. The channel matching variables of the other candidate sub-videos are 0, which is expressed as:

[0174] Step 840: Based on the updated channel matching variables, the resource optimization objective function is calculated under the condition that the channel constraint condition is met, and the candidate sub-video corresponding to each target transmission channel when the total transmission quality parameter is maximized is selected as the corresponding target rate video.

[0175] In an embodiment, according to the above example, the channel matching variables of the candidate sub-videos corresponding to the target transmission videos are updated, and then the resource optimization objective function is calculated under the premise that the channel constraint condition is met, the corresponding total transmission quality parameter is calculated, and the candidate sub-video corresponding to each target transmission channel when the total transmission quality parameter is maximized is selected as the corresponding target rate video. That is, when solving the optimization objective model, the target transmission channel corresponding to the target transmission video is determined, that is, the association between the parameters i and k is determined, and then the number of rate sub-videos of each target transmission video participating in the solving process is reduced, and the candidate sub-videos with fewer numbers are used to replace all rate sub-videos, thereby reducing the solving complexity and improving the solving efficiency.

[0176] The above process can determine the target rate video selected from the corresponding candidate sub-videos for each target transmission video by solving the optimization objective model.

[0177] Step 760: Channel allocation data is obtained based on the target rate video and the target transmission channel.

[0178] In an embodiment, each target transmission video has a corresponding target rate video, and also has a corresponding target transmission channel, so that the target rate video and the target transmission channel can be summarized as channel allocation data.

[0179] Step 140: selecting a target code rate video from each target transmission video corresponding code rate sub-video according to the channel allocation data, and determining the target transmission channel of the target code rate video.

[0180] In an embodiment, after the channel allocation data is obtained, the target code rate video corresponding to each target transmission video can be determined, and the target transmission channel of the target code rate video can be determined.

[0181] Step 150: transmitting the corresponding code rate sub-video based on the target transmission channel.

[0182] In an embodiment, the corresponding code rate sub-video is sent to the user through the target transmission channel, and the user can perform decoding, splicing and other operations on all received code rate sub-videos according to requirements.

[0183] Embodiments of the present application use function fitting to construct the visual quality evaluation parameter of each target transmission video in the immersive panoramic video, and use the visual quality evaluation parameter to evaluate the correlation between the visual quality of each target transmission video and the encoding code rate. And based on the visual quality evaluation parameter, an optimization target model in the transmission process is constructed, which maximizes the visual quality of the video received by the user under the condition of optimal transmission resources. In addition, when allocating the target transmission channel for each target transmission video, the quality gain is used to select the appropriate target transmission channel for each target transmission video, thereby improving the utilization rate of transmission resources.

[0184] In an embodiment, referring to Figure 9 , Figure 9 is an effect verification schematic diagram of the immersive panoramic video wireless transmission method provided by the embodiments of the present application.

[0185] First, two channel allocation methods in related technologies are selected, denoted as CA1 and CA2. Specifically, the CA1 method equally allocates all transmission channels to each video slice (equivalent to the to-be-transmitted video in the embodiments of the present application) to be transmitted. The CA2 method arranges all transmission channels in descending order according to channel capacity, and divides them into groups, and then arranges each video slice to be transmitted in descending order according to importance, and then matches each group of transmission channels with each video slice to be transmitted.

[0186] The simulation experiment configuration parameters are as follows: the test video in the PVS-HMEM immersive panoramic video dataset is used, including Camping, Graffiti, Stratosphere, TalkingInCar, and Waterskiing. The test immersive panoramic video resolution is 3840x1920, and the frame rate is 30fps. Here, the test immersive panoramic video GOP length is set to 1 second, so it contains 30 frames of images. In addition, the test immersive panoramic video adopts three video division modes, including {6x4, 8x6, 8x8}. And the total number of target transmission videos covered by the user's target view angle in the above three division modes is recorded as N={9, 18, 24}. The encoding code rate can be 7. In addition, the immersive panoramic video wireless transmission system uses K=32 OFDM channels as transmission channels, and the gain of each transmission channel obeys the Rayleigh distribution.

[0187] Figure 9 For the experimental results, the average received video quality of the test immersive panoramic video obtained by the user using different wireless channel allocation methods in various division modes is shown, with the peak signal-to-noise ratio WS-PSNR as the evaluation parameter. It can be seen that in various division modes, the CA1 method obtains the worst user received WS-PSNR performance. When considering the content characteristic differences of different video slices, after using unequal channel allocation, although the CA2 method can improve the user received WS-PSNR performance compared with CA1. But the received WS-PSNR performance of the immersive panoramic video wireless transmission method provided in the embodiment of the application is optimal.

[0188] The technical scheme provided by the embodiment of the application determines at least one target transmission video from a plurality of to-be-transmitted videos according to the transmission requirement, wherein the target transmission video includes a code rate sub-video corresponding to at least one encoding code rate. Then, the visual quality evaluation parameter of each target transmission video is obtained, and the channel constraint condition is generated based on the channel matching variable corresponding to the target transmission video and at least one transmission channel. Then, the optimization target model is constructed based on the visual quality evaluation parameter and the channel constraint condition, and the channel allocation data is obtained by solving the optimization target model. Then, the target code rate video is selected from the code rate sub-video corresponding to each target transmission video according to the channel allocation data, and the target transmission channel of the target code rate video is determined, and finally the corresponding code rate sub-video is transmitted based on the target transmission channel. In the channel allocation process of the embodiment of the application, a visual quality evaluation parameter is generated for each target transmission video to participate in the optimization process, and a unified allocation process is performed for all target transmission videos, fully considering the content correlation between the target transmission videos, and improving the reception quality of the panoramic video.

[0189] The embodiment of the present application further provides an immersive panoramic video wireless transmission device, which can implement the immersive panoramic video wireless transmission method. Figure 10 The device comprises:

[0190] The video selection module 1010 is configured to determine at least one target transmission video from the plurality of to-be-transmitted videos according to transmission requirements, wherein the target transmission video comprises a code rate sub-video corresponding to at least one encoding code rate.

[0191] The quality parameter calculation module 1020 is configured to obtain a visual quality evaluation parameter of each target transmission video, and generate a channel constraint condition based on a channel matching variable corresponding to the target transmission video and at least one transmission channel.

[0192] The optimization model construction module 1030 is configured to construct an optimization target model based on the visual quality evaluation parameter and the channel constraint condition, and solve the optimization target model to obtain channel allocation data.

[0193] The channel allocation module 1040 is configured to select a target code rate video from the code rate sub-video corresponding to each target transmission video according to the channel allocation data, and determine a target transmission channel of the target code rate video.

[0194] The video transmission module 1050 is configured to transmit the corresponding code rate sub-video based on the target transmission channel.

[0195] The specific implementation of the immersive panoramic video wireless transmission device of the embodiment is basically the same as the specific implementation of the immersive panoramic video wireless transmission method, and will not be described here.

[0196] The embodiment of the present application further provides an electronic device, which comprises:

[0197] at least one memory;

[0198] at least one processor;

[0199] at least one program;

[0200] The program is stored in the memory, and the processor executes the at least one program to implement the immersive panoramic video wireless transmission method of the present application. The electronic device can be any intelligent terminal, including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0201] Please refer to Figure 11 , Figure 11 which illustrates the hardware structure of the electronic device of another embodiment, and the electronic device comprises:

[0202] The processor 1101 can be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0203] The memory 1102 can be implemented by a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1102 and are called and executed by the processor 1101 to implement the immersive panoramic video wireless transmission method of the embodiments of the present application.

[0204] The input / output interface 1103 is configured to implement information input and output.

[0205] The communication interface 1104 is configured to implement the communication interaction between the device and other devices. The communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0206] The bus 1105 is configured to transmit information between various components (for example, the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104) of the device.

[0207] The processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104 are connected to each other by the bus 1105 to realize the communication connection between the device.

[0208] The embodiments of the present application also provide a storage medium. The storage medium is a storage medium, and the storage medium stores a computer program. The computer program is executed by the processor to implement the above immersive panoramic video wireless transmission method.

[0209] The memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory disposed remotely relative to the processor, which can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0210] The immersive panoramic video wireless transmission method, device, equipment and storage medium provided by the embodiment of the application determine at least one target transmission video from a plurality of to-be-transmitted videos according to transmission requirements, wherein the target transmission video includes a code rate sub-video corresponding to at least one encoding code rate. Then, a visual quality evaluation parameter of each target transmission video is obtained, and a channel constraint condition is generated based on a channel matching variable corresponding to the target transmission video and at least one transmission channel. Then, an optimization target model is constructed based on the visual quality evaluation parameter and the channel constraint condition, and the optimization target model is solved to obtain channel allocation data. Then, a target code rate video is selected from the code rate sub-video corresponding to each target transmission video according to the channel allocation data, and a target transmission channel of the target code rate video is determined. Finally, the corresponding code rate sub-video is transmitted based on the target transmission channel. In the channel allocation process, the embodiment of the application generates a visual quality evaluation parameter for each target transmission video to participate in the optimization process, and performs a unified allocation process for all target transmission videos, fully considers the content correlation between the target transmission videos, and improves the reception quality of the panoramic video.

[0211] The embodiments described in the embodiments of the application are used to more clearly illustrate the technical solutions of the embodiments of the application, and do not constitute a limitation on the technical solutions provided by the embodiments of the application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0212] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the application, and can include more or fewer steps than the figures, or combine certain steps or different steps.

[0213] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the application.

[0214] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0215] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a general order and / or structure unless otherwise indicated. Furthermore, the terms "comprise", "comprising", "has", "having", "includes", "including", "contain", "containing" or any other similar forms are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains items or components does not include items or components not explicitly recited. The terms "a" or "an", as used herein in the detailed description and in the claims, mean "one or more" or "at least one", unless otherwise indicated.

[0216] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0217] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0218] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0219] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0220] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0221] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and the scope of the rights of the embodiments of the present application is not limited thereto. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A method for wireless transmission of immersive panoramic video, characterized in that, include: Based on transmission requirements, at least one target transmission video is determined from multiple videos to be transmitted, wherein the target transmission video includes at least one bitrate sub-video corresponding to the encoded bitrate. Obtain visual quality evaluation parameters for each target transmitted video, and generate channel constraints based on channel matching variables corresponding to the target transmitted video and at least one transmission channel; An optimization target model is constructed based on the visual quality assessment parameters and the channel constraints, and the channel allocation data is obtained by solving the optimization target model. Based on the channel allocation data, a target bitrate video is selected from the bitrate sub-videos corresponding to each target transmission video, and the target transmission channel of the target bitrate video is determined. The corresponding bitrate sub-videos are transmitted based on the target transmission channel.

2. The immersive panoramic video wireless transmission method according to claim 1, characterized in that, The step of obtaining the visual quality evaluation parameters for each of the target transmitted videos includes: Obtain the first and second fitting parameters of the target transmitted video; Obtain the encoded bitrate corresponding to each of the bitrate sub-videos; The visual quality evaluation parameter corresponding to each bitrate sub-video is calculated based on the encoding bitrate, the first fitting parameter, and the second fitting parameter. The visual quality evaluation parameter is used to indicate the transmission quality of the video content of the bitrate sub-video.

3. The immersive panoramic video wireless transmission method according to claim 2, characterized in that, The step of obtaining the first fitting parameters and the second fitting parameters of the target transmitted video includes: Multiple consecutive image frames corresponding to the target transmitted video are encoded in parallel at multiple different coding bitrates to obtain the corresponding encoding results, and the peak signal-to-noise ratio corresponding to each encoding result is obtained; Multiple sets of fitting data are generated based on the coding bitrate and the corresponding peak signal-to-noise ratio. The fitting data is then used to perform an exponential function fitting process to obtain the first fitting parameter and the second fitting parameter.

4. The immersive panoramic video wireless transmission method according to claim 1, characterized in that, The generation of channel constraint conditions based on the channel matching variables corresponding to the target transmitted video and at least one transmission channel includes: Generate a channel matching variable for each bitrate sub-video and each transmission channel. If the bitrate sub-video is assigned to the transmission channel for transmission, the channel matching variable is a first value; otherwise, it is a second value. The channel constraints are generated based on the channel matching variables, the number of coding rates, and the coding rates.

5. The immersive panoramic video wireless transmission method according to claim 3, characterized in that, The process of generating the channel constraints based on the channel matching variable, the number of coding rates, and the coding rates includes: A first constraint is generated based on the channel matching variable. The first constraint is used to constrain the sum of the number of transmission channels allocated to the bitrate sub-videos corresponding to all the target transmission videos to be less than or equal to the total number of transmission channels. A second constraint is generated based on the channel matching variable. The second constraint is used to constrain each of the transmission channels to be allocated only to one of the bitrate sub-videos of the target transmission video. Generate a third constraint condition to constrain the number of encoded code rates to a preset number; When the coding bitrate is greater than the channel capacity of a single transmission channel, more than one transmission channel is allocated to the bitrate sub-video corresponding to the coding bitrate, and a fourth constraint condition is generated. The channel constraint is obtained based on the first constraint, the second constraint, the third constraint, and the fourth constraint.

6. The immersive panoramic video wireless transmission method according to claim 4, characterized in that, The construction of the optimization target model based on the visual quality assessment parameters and the channel constraints includes: Obtain the preset weight of the target transmitted video and the visual quality evaluation parameters of each corresponding bitrate sub-video; Generate total transmission quality parameters based on the preset weights and the visual quality assessment parameters; Maximize the total transmission quality parameters to generate a resource optimization objective function; The optimization objective model is constructed based on the resource optimization objective function and the channel constraints.

7. The immersive panoramic video wireless transmission method according to claim 6, characterized in that, The generation of total transmission quality parameters based on the preset weights and the visual quality assessment parameters includes: The sum of all the visual quality evaluation parameters corresponding to each target transmitted video is obtained to get a first intermediate value; Multiply the preset weight by the corresponding first intermediate value to obtain the second intermediate value; The total transmission quality parameter is obtained by summing all the second intermediate values.

8. The immersive panoramic video wireless transmission method according to claim 6, characterized in that, The process of solving the optimization target model to obtain channel allocation data includes: Initialize the channel set and cumulative channel transmission capacity corresponding to each of the target transmitted videos; The transmission channels are sequentially assigned to each of the target transmission videos, and in each assignment process, each transmission channel is assigned to one of the target transmission videos; Calculate the quality gain of each transmission channel assigned to the different target transmission videos based on the allocation results; To maximize the quality gain of the target transmitted video, the transmission channel is assigned as the target transmission channel; Based on the target transmission channel, the optimized target model is solved to obtain the target bitrate video corresponding to each target transmission video; The channel allocation data is obtained based on the target bitrate video and the target transmission channel.

9. The immersive panoramic video wireless transmission method according to claim 8, characterized in that, The step of solving the optimized target model based on the target transmission channel to obtain the target bitrate video corresponding to each target transmission video includes: Obtain the transmission bandwidth of the target transmission channel corresponding to each of the target transmitted videos; Based on the transmission bandwidth, at least one candidate sub-video is determined from the corresponding bitrate sub-video; The candidate sub-videos are selected one by one, and their corresponding channel matching variables are set to the first value. The channel matching variables of the other bitrate sub-videos are set to the second value. Based on the updated channel matching variables, and under the condition that the channel constraints are met, the resource optimization objective function is calculated, and the candidate sub-videos of each target transmission channel corresponding to the maximization of the total transmission quality parameters are selected as the corresponding target bitrate videos.

10. The immersive panoramic video wireless transmission method according to any one of claims 1 to 9, characterized in that, The step of determining at least one target video for transmission from multiple videos to be transmitted according to transmission requirements includes: Determine the target perspective based on the transmission requirements; The target transmission video is selected from the plurality of videos to be transmitted and the video that meets the target perspective.

11. An immersive panoramic video wireless transmission device, characterized in that, include: Video selection module: used to determine at least one target transmission video from multiple videos to be transmitted according to transmission requirements, wherein the target transmission video includes at least one bitrate sub-video corresponding to the encoding bitrate; Quality parameter calculation module: used to obtain visual quality evaluation parameters for each target transmitted video, and generate channel constraint conditions based on the channel matching variables corresponding to the target transmitted video and at least one transmission channel; Optimization model construction module: used to construct an optimization target model based on the visual quality assessment parameters and the channel constraints, and solve the optimization target model to obtain channel allocation data; Channel allocation module: used to select a target bitrate video from the bitrate sub-videos corresponding to each target transmission video according to the channel allocation data, and determine the target transmission channel of the target bitrate video; Video transmission module: used to transmit the corresponding bitrate sub-videos based on the target transmission channel.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the immersive panoramic video wireless transmission method according to any one of claims 1 to 10.

13. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the immersive panoramic video wireless transmission method according to any one of claims 1 to 10.

Images