Resource scheduling method and system for multi-user multi-mode flow integrity transmission task

Abstract

The invention discloses a resource scheduling method for a multi-user multi-mode flow integrity transmission task, which comprises the following steps of: acquiring an audiovisual flow transmission rate and a tactile flow transmission time delay for a multi-user multi-mode flow transmission task; according to the audio-visual stream transmission rate and the tactile stream transmission time delay, establishing an optimized mathematical model of resource scheduling by taking maximization of the multi-user multi-mode perception utility as a target, and solving the model to obtain the resource quantity of maximization of the multi-user multi-mode perception utility under the constraint of spectrum and power resources; and allocating power and spectrum resources to multiple users according to the number of resources required by the users. According to the resource scheduling method for the multi-user multi-modal flow integrity transmission task, the real-time requirement of tactile flow transmission and the integrity requirement of audio-visual flow transmission are comprehensively considered, and the multi-modal perception utility of the user in virtual-real fusion service is improved through allocation of frequency spectrum and power resources.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications, and more specifically, relates to a resource scheduling method for multi-user multimodal stream integrity transmission tasks. Background Art

[0002] The emergence and rapid expansion of the metaverse has fueled the rapid development of virtual-reality convergence services, primarily based on technologies such as virtual reality, augmented reality, and mixed reality. In user-centric virtual-reality convergence, multisensory virtual-reality convergence services based on vision, hearing, and touch can provide multiple users with an immersive experience. Because the transmission characteristics and requirements of different sensory signals in immersive virtual-reality convergence services vary significantly, further research is needed to effectively transmit multi-user multimodal streams.

[0003] In multimodal streaming, audiovisual streaming offers the characteristics of enhanced mobile broadband communication, while tactile streaming offers ultra-reliable low-latency communication. The flexible frame structure and resource puncturing scheme in 5G New Radio are effective transmission methods that meet the performance requirements of multimodal services in cellular networks. Traditional wireless resource optimization focuses on successful data packet transmission, but multimodal streaming should consider the integrity of the audiovisual stream while also meeting the requirements of the tactile stream.

[0004] From a network transmission perspective, each audiovisual frame consists of dozens of Internet Protocol (IP) packets. Without considering layering or slicing schemes, the audiovisual frame can only be decoded at the receiving end when all relevant IP packets are successfully received. If the receiver does not receive one or more IP packets, the corresponding audiovisual frame cannot be decoded, affecting the user's immersive experience. For multi-user, multi-sensory virtual and real-reality fusion services, the complete transmission of audiovisual frames and the real-time transmission of tactile streams are both important for the user's immersive multi-sensory experience. How to schedule resources under the real-time constraints of user tactile stream transmission to meet the user's audiovisual stream integrity transmission requirements is a challenging problem. Summary of the Invention

[0005] Purpose of the invention: In response to the shortcomings of the existing technology, the present invention first provides a resource scheduling method for the integrity transmission task of multi-user multimodal streams. By scheduling spectrum and power resources, the multimodal perception utility of multiple users is optimized, and the integrity transmission requirements of multi-user audiovisual streams are met while considering the real-time transmission of multi-user tactile streams.

[0006] Another object of the present invention is to provide a resource scheduling system for multi-user multi-modal stream integrity transmission tasks, so as to implement the resource scheduling method for multi-user multi-modal stream integrity transmission tasks mentioned above.

[0007] Based on the above objectives, the present invention provides, on one hand, a resource scheduling method for a multi-user multimodal stream integrity transmission task, characterized in that it includes the following steps: S1. For multi-user multimodal streaming tasks, obtain the audiovisual stream transmission rate and tactile stream transmission delay; S2. Establishing a mathematical model for optimizing resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solving the model to obtain a quantity of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; S3. Allocate power and spectrum resources to multiple users based on the amount of resources required by the users.

[0008] As a preferred technical solution, step S1 obtains the audiovisual stream transmission rate and tactile stream transmission delay for a multi-user multimodal stream transmission task; including: S11. For multi-user multimodal streaming tasks, establish a transmission model for tactile streaming and audio-visual streaming; S12. Calculate the audiovisual stream transmission rate and the tactile stream transmission delay.

[0009] Preferably, step S11 establishes a transmission model of tactile stream and audiovisual stream for a multi-user multimodal stream transmission task, specifically including: For audio-visual streams, the duration of the user's audio-visual frame is seconds, the time reserved for each audiovisual frame transmission is time slots , each time slot is divided into Mini-slots ,The audio-visual stream is transmitted on a time-slot basis, while the tactile stream is transmitted on a ,microslot basis; The base station provides services to users within the coverage area. The set of base stations is , the user set is , each base station occupies several spectrum resource blocks, and some spectrum resource blocks are reused between base stations. The set of spectrum resource blocks is expressed as ; Step S12 calculates the audiovisual stream transmission rate and the tactile stream transmission delay, specifically including: Base Station Users of the service In the time slot The audiovisual stream transmission rate obtained within is expressed as:

[0010] in, is the bandwidth of the spectrum resource block, Time slot Internal users In spectrum resource blocks The signal-to-interference-noise ratio on Represents a user In the time slot Whether the spectrum resource block is occupied , Indicates that the user is in the time slot Occupied spectrum resource blocks , Indicates that the user is in the time slot Unoccupied spectrum resource blocks ; Base Station Users of the service In mini-slots The tactile flow transmission rate obtained within is expressed as:

[0011] in, Mini-slot Internal users In spectrum resource blocks The signal-to-interference-noise ratio on Represents a user In micro-slots Whether the spectrum resource block is occupied , Represents a user In mini-slots Occupied spectrum resource blocks , Represents a user In mini-slots Unoccupied spectrum resource blocks , is the channel dispersion and , is the block length of the tactile stream, is a function The inverse function of ; The user In mini-slots The tactile data packet size is expressed as , then the user In mini-slots The tactile transmission delay within is expressed as:

[0012] The user The packet arrival rate of the tactile flow is expressed as , the duration of the mini-slot is expressed as , then the user In the time slot The transmission delay of all tactile data packets arriving within is expressed as:

[0013] The user's tactile transmission performance is expressed as the tactile sensitivity coefficient. In the time slot The tactile sensitivity coefficient is expressed as:

[0014] in, Indicates the maximum transmission delay of tactile stream, Indicates the minimum tactile transmission delay.

[0015] Preferably, step S2 establishes an optimization mathematical model for resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solves the model to obtain the number of resources that can maximize the multi-user multimodal perception utility under spectrum and power resource constraints; including: S21. Based on the multi-attribute utility theory, a user multimodal perception utility function is constructed. Spectrum resource allocation strategy and power allocation strategy are used as optimization targets. Maximizing multimodal perception utility of multiple users is used as the objective function. User audiovisual frame size, spectrum resource, and power resource limits are used as constraints. A resource scheduling model for the integrity transmission of multimodal streams is constructed. S22. Solve the resource scheduling model to obtain the number of resources that maximizes the multi-user multimodal sensing utility under spectrum and power resource constraints.

[0016] Preferably, in step S21, a resource scheduling model for the integrity transmission of multimodal streams is constructed, and the specific method is as follows: S211. Constructing user multimodal perception utility function based on multi-attribute utility theory. In the time slot The calculation expression of the multimodal perception utility obtained is:

[0017] in, For users In the time slot The transmission rate of the internal audio and video stream, For users In the time slot The transmission time of the internal audio-visual stream, For users The size of the requested audio-visual chunk, is the minimum value of the audio-visual block, is the maximum value of the tactile stream transmission delay, is the minimum value of tactile stream transmission delay, For users In the time slot Transmission delay of the internal tactile stream; S212. Establish a resource scheduling model for multimodal stream integrity transmission to represent the optimization problem:

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

[0024] in, is the set of base stations, is the number of base stations; is a collection of users, is the number of users; is a collection of spectrum resource blocks, is the total number of spectrum resource blocks; Indicates base station The number of occupied spectrum resource blocks; is the set of time slots, is the total number of time slots prepared for the transmission of each audiovisual frame; Time slot The collection of mini-slots contained within, is the number of mini-slots contained in each time slot; Spectrum resource scheduling strategy for audio-visual streams, Indicates spectrum resource blocks In the time slot Assigned to users ,otherwise ; represents the spectrum resource scheduling strategy for tactile streaming, Indicates spectrum resource blocks In mini-slots Assigned to users ,otherwise ; Indicates the user's power allocation strategy. The total transmission power of each base station in each time slot cannot exceed ; Indicates the duration of each time slot; Represents a user The requested audio-visual frame size. Each audio-visual frame is divided into audio-visual blocks; Step S22 solves the resource scheduling model to obtain the number of resources that maximizes the multi-user multimodal sensing utility under the constraints of spectrum and power resources. The specific method is: First, based on Lyapunov theory, the optimization problem in step S212 is decomposed into a multimodal perception utility maximization problem for each time slot. Then, spectrum resources are allocated based on a one-to-many matching algorithm, and power resources are allocated based on the alternating direction multiplier method. Finally, the spectrum resource and power resource allocation algorithms are combined to obtain stable spectrum resource allocation strategies and power resource allocation strategies.

[0025] Preferably, step S3 allocates power and spectrum resources to multiple users according to the amount of resources required by the users; The transmission of the tactile stream adopts a resource puncture method, that is, the transmission of the tactile stream occupies the spectrum resources allocated to the audio-visual stream.

[0026] As a preference, the user When the current audiovisual frame has been transmitted, the spectrum and power resources it occupies are allocated to the user whose audiovisual frame has not been transmitted yet; at this time, the user The tactile streaming transmission occupies the spectrum of other users' audio-visual frame transmission.

[0027] A second aspect of the present invention provides a resource scheduling system for multi-user multimodal stream integrity transmission tasks, comprising: A transmission task acquisition module is used to obtain the transmission rate of the audiovisual stream and the transmission delay of the tactile stream for a multi-user multimodal stream transmission task; a transmission strategy optimization module, configured to establish a resource scheduling optimization mathematical model based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solve the resource scheduling model to obtain the number of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; The transmission task allocation module is used to allocate power and spectrum resources to multiple users according to the number of resources required by the users.

[0028] The present invention also provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions enable a computer to execute the resource scheduling method for the multi-user multi-modal stream integrity transmission task described above.

[0029] The present invention also provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus; Among them, the processor, communication interface, and memory communicate with each other through a communication bus; The processor is used to call the logic instructions in the memory to execute the resource scheduling method for the multi-user multi-modal stream integrity transmission task.

[0030] Beneficial effects: The resource scheduling method for multi-user multimodal stream integrity transmission tasks proposed in the present invention comprehensively considers the real-time requirements of tactile stream transmission and the integrity requirements of audio-visual stream transmission, and improves the multimodal perception utility of users in virtual-reality fusion services through the allocation of spectrum and power resources. Specifically, the present invention takes maximizing the multimodal perception utility of multiple users as the objective function, and takes the user audio-visual frame size, spectrum resources and power resource restrictions as constraints to construct a resource scheduling model for multimodal stream integrity transmission tasks. The resource scheduling optimization model is solved based on Lyapunov theory, many-to-one matching theory and ADMM algorithm to obtain the spectrum and power resource allocation strategy for multiple users under resource-constrained conditions. Utilizing the obtained resource allocation strategy, resource utilization can be further improved, and the immersive service experience of multiple users can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of a multi-cell multi-modal flow downlink transmission system according to an embodiment of the present invention; Figure 2 This is a flow chart of a resource scheduling method for a multi-user multi-modal stream integrity transmission task provided in Example 1 of the present invention; Figure 3 Schematic diagram of a multimodal streaming video resource scheduling model provided in Example 1 of the present invention; Figure 4 A graph showing the relationship between the average multimodal perceived utility and the number of users and the number of spectrum resource blocks provided by an embodiment of the present invention; Figure 5 This figure shows how the average multimodal perceptual utility changes with the number of audiovisual blocks in each audiovisual frame under different solutions provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0033] The following combination Figure 1-Figure 5The present invention describes a resource scheduling method and system for multi-user multi-modal stream integrity transmission tasks.

[0034] Example 1: This example provides a resource scheduling method for multi-user multi-modal stream integrity transmission tasks. Figure 1 The transmission task of the transmission system shown includes a user layer and a base station layer.

[0035] The user layer consists of several users waiting for service, and users request multimodal content, including audio-visual streams and tactile streams, from the base station through the wireless uplink.

[0036] The base station layer consists of several base stations that provide multimodal content to users. The base stations cache the multimodal content requested by users and send the audio-visual stream and tactile stream to users through wireless downlink.

[0037] Inter-cell interference is caused by multiple base stations sharing the same spectrum resource blocks.

[0038] This embodiment proposes a resource scheduling method for multi-cell multi-user multi-modal stream integrity transmission tasks, which maximizes the user's multi-modal perception utility by utilizing limited spectrum and power resources and enhances the user's immersive experience. Figure 2 As shown, in general, the steps include: S1. For multi-user multimodal streaming tasks, obtain the audiovisual stream transmission rate and tactile stream transmission delay; S2. Establishing a mathematical model for optimizing resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solving the model to obtain a quantity of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; S3. Allocate power and spectrum resources to multiple users based on the amount of resources required by the users.

[0039] Furthermore, step S1 obtains the audiovisual stream transmission rate and the tactile stream transmission delay for the multi-user multimodal stream transmission task; including: S11. For multi-user multimodal streaming tasks, establish a transmission model for tactile streaming and audio-visual streaming: The transmission resource scheduling model of multimodal flow is as follows: Figure 3 As shown in the figure, the multimodal stream mainly includes the audio-visual stream and the tactile stream. Since the tactile data packet size is small and has bursty characteristics, the transmission of the tactile stream can adopt a resource puncture solution, that is, the bursty tactile stream can occupy the transmission resources of the audio-visual stream. For audio-visual streams, the duration of the user's audio-visual frame is seconds, the time reserved for each audiovisual frame transmission is time slots , each time slot is divided into Mini-slots ,The audio-visual stream is transmitted on a time-slot basis, while the tactile stream is transmitted on a ,microslot basis; The base station provides services to users within the coverage area. The set of base stations is , the user set is , each base station occupies several spectrum resource blocks, and some spectrum resource blocks are reused between base stations. The set of spectrum resource blocks is expressed as ; S12. Calculate the audiovisual stream transmission rate and the tactile stream transmission delay: Base Station Users of the service In the time slot The audiovisual stream transmission rate obtained within is expressed as:

[0040] in, is the bandwidth of the spectrum resource block, Time slot Internal users In spectrum resource blocks The signal-to-interference-noise ratio on Represents a user In the time slot Whether the spectrum resource block is occupied , Indicates that the user is in the time slot Occupied spectrum resource blocks , Indicates that the user is in the time slot Unoccupied spectrum resource blocks ; Base Station Users of the service In mini-slots The tactile flow transmission rate obtained within is expressed as:

[0041] in, Mini-slot Internal users In spectrum resource blocks The signal-to-interference-noise ratio on Represents a user In mini-slots Whether the spectrum resource block is occupied , Represents a user In mini-slots Occupied spectrum resource blocks , Represents a user In mini-slots Unoccupied spectrum resource blocks , is the channel dispersion and , is the block length of the tactile stream, is a function The inverse function of ; The user In mini-slots The tactile data packet size is expressed as , then the user In mini-slots The tactile transmission delay within is expressed as:

[0042] The user The packet arrival rate of the tactile flow is expressed as , the duration of the mini-slot is expressed as , then the user In the time slot The transmission delay of all tactile data packets arriving within is expressed as:

[0043] The user's tactile transmission performance is expressed as the tactile sensitivity coefficient. In the time slot The tactile sensitivity coefficient is expressed as:

[0044] in, Indicates the maximum transmission delay of tactile stream, Indicates the minimum tactile transmission delay.

[0045] Furthermore, step S2 establishes an optimization mathematical model for resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solves the model to obtain the number of resources that can maximize the multi-user multimodal perception utility under spectrum and power resource constraints; including: S21. Based on the multi-attribute utility theory, a user multimodal perception utility function is constructed. The spectrum resource allocation strategy and power allocation strategy are used as optimization objects. Maximizing the multimodal perception utility of multiple users is used as the objective function. The user audiovisual frame size, spectrum resource, and power resource limits are used as constraints. A resource scheduling model for the integrity transmission of multimodal streams is constructed. The specific method is as follows: S211. Construct user multimodal perception utility function based on multi-attribute utility theory, given tactile sensitivity coefficient ,user In the time slot The conditional utility of the inner audiovisual experience is expressed as:

[0046] in, For users In the time slot The transmission time of the internal audio-visual stream, For users The size of the requested audio-visual chunk, is the minimum value of the audio-visual block; S212, User In the time slot The calculation expression of the multimodal perception utility obtained is:

[0047] in, For users In the time slot The transmission rate of the internal audio and video stream, For users In the time slot The transmission time of the internal audio-visual stream, For users The size of the requested audio-visual chunk, is the minimum value of the audio-visual block, is the maximum value of the tactile stream transmission delay, is the minimum value of tactile stream transmission delay, For users In the time slot Transmission delay of the internal tactile stream; S213. Establish a resource scheduling model for multimodal stream integrity transmission to represent the optimization problem:

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] in, is the set of base stations, is the number of base stations; is a collection of users, is the number of users; is a collection of spectrum resource blocks, is the total number of spectrum resource blocks; Indicates base station The number of occupied spectrum resource blocks; is the set of time slots, is the total number of time slots prepared for the transmission of each audiovisual frame; Time slot The collection of mini-slots contained within, is the number of mini-slots contained in each time slot; Spectrum resource scheduling strategy for audio-visual streams, Indicates spectrum resource blocks In the time slot Assigned to users ,otherwise ; represents the spectrum resource scheduling strategy for tactile streaming, Indicates spectrum resource blocks In mini-slots Assigned to users ,otherwise ; Indicates the user's power allocation strategy. The total transmission power of each base station in each time slot cannot exceed ; Indicates the duration of each time slot; Represents a user The requested audio-visual frame size. Each audio-visual frame is divided into audio-visual blocks; S22. Solve the resource scheduling model to obtain the amount of resources that maximizes the multi-user multimodal sensing utility under spectrum and power resource constraints; The difficulty in solving this maximization problem lies in the fact that the integrity of the audiovisual streams is coupled with the spectrum and power resource allocation strategies of different time slots. In addition, maximizing the multimodal perceptual utility of each time slot is a complex non-convex integer nonlinear programming problem.

[0055] S221. To solve the long-term resource allocation coupling problem during the transmission of audiovisual stream integrity, a virtual audiovisual stream data surplus queue is constructed for users based on Lyapunov theory to guide the allocation of spectrum and power resources. Virtual audio-visual stream data remaining queue The initial value of is 0 and is updated in each time slot as follows:

[0056] In each time slot, we only need to obtain the current amount of transmitted data and the channel status to formulate a resource allocation strategy. The optimization problem for each time slot is expressed as:

[0057]

[0058]

[0059]

[0060]

[0061] in, It is a control parameter used to dynamically adjust the balance between maximizing the multimodal perception utility and minimizing the amount of remaining data transmission. , , and there are

[0062] in is the maximum transmit power of the base station, is the power spectral density of the noise, For the time slot Internal base station To the user channel gain.

[0063] S222. In order to solve the optimization problem of each time slot, a two-time-scale spectrum resource allocation algorithm based on one-to-many matching theory is first designed to optimize the spectrum resource allocation strategy. and .

[0064] Set the user's preference list to , the user's preference function is:

[0065] The preference list of spectrum resource blocks is , the preference function of spectrum resource blocks is .

[0066] The two-time-scale spectrum resource allocation algorithm based on one-to-many matching theory in step S222 includes the following steps: a. Given the system initialization value: user and resource block matching , the set of users who have audio-visual streaming tasks , the user set where the audio-visual streaming task transmission ends , the user's preference list , preference list of spectrum resource blocks ; b. For base stations Perform user matching for each occupied spectrum resource block; c. For base stations Occupied spectrum resource blocks , based on the existing user and resource block matching results and the user's preference list Update spectrum resource blocks Preference list and from Delete the items that exist in the collection Users in d. When spectrum resource blocks Most preferred users Not the user it is currently matching to When the user If the allocated spectrum resource block does not reach the upper limit, the spectrum resource block Assign to user Otherwise, the user Query the occupied spectrum resource blocks Spectrum resource blocks with low preference values , and accept and refuse If the user Spectrum Resource Block The preference value of is the lowest, then the user Reject spectrum resource blocks ; e. Until it is a spectrum resource block Once a stable matching result is found, the loop ends; f. For the set The user in the audiovisual streaming task is completed, and no spectrum resources and power resources are allocated. The tactile transmission task occupies Spectrum and power resources for medium-sized users; g. The loop ends until a stable matching result is found between the users and spectrum resource blocks of each base station.

[0067] S223. To solve the power optimization problem, the alternating direction multiplier method ADMM (Alternating Direction Method of Multipliers) is used to minimize the remaining queue length of the audiovisual data volume. The power optimization problem is expressed as

[0068]

[0069] in Indicates base station The number of users served.

[0070] In order to obtain the optimal power allocation solution, the power allocation variables of each base station are divided into two parts. The power allocation strategy is expressed as and In the time slot Inside, base station The total length of the data queue for serving users is expressed as , base station The queue length minimization problem is expressed as:

[0071]

[0072] in, , , , , and is a unit vector. For the scaled ADMM algorithm, the augmented Lagrangian associated with the above queue length minimization problem is expressed as:

[0073] in, is the penalty parameter, is the Lagrange multiplier, is the scaled dual variable.

[0074] The iteration of the primal and dual variables is expressed as:

[0075]

[0076]

[0077] in, is the number of iterations.

[0078] The raw residual is , the dual residual is , the termination criterion of the power allocation algorithm based on ADMM is and ,in is a predefined threshold value of the original condition, is a predefined threshold for the dual condition.

[0079] The multi-modal transmission power allocation algorithm based on ADMM is expressed as: a. Spectrum resource allocation strategy for a given user and ; b. For each base station, when as well as When, update , , , as well as ; c. , until the loop ends.

[0080] S224. Combine the two-time-scale spectrum resource allocation algorithm based on one-to-many matching theory and the multimodal transmission power allocation algorithm based on ADMM. Repeat the execution of the spectrum resource allocation algorithm and the transmission power allocation algorithm for each time slot in the transmission process of each video frame until a stable spectrum resource and power resource allocation strategy is obtained.

[0081] Furthermore, in step S3, power and spectrum resources are allocated to multiple users according to the number of resources required by the users. The transmission of the tactile stream adopts a resource puncture method, that is, the transmission of the tactile stream occupies the spectrum resources allocated to the audio-visual stream.

[0082] In the user When the current audiovisual frame has been transmitted, the spectrum and power resources it occupies are allocated to the user whose audiovisual frame has not been transmitted yet; at this time, the user The tactile streaming transmission occupies the spectrum of other users' audio-visual frame transmission.

[0083] Embodiment 2: In order to apply the above-mentioned resource scheduling method for multi-user multimodal stream integrity transmission tasks, this embodiment provides a resource scheduling system for multi-user multimodal stream integrity transmission tasks.

[0084] A transmission task acquisition module is used to obtain the transmission rate of the audiovisual stream and the transmission delay of the tactile stream for a multi-user multimodal stream transmission task; a transmission strategy optimization module, configured to establish a resource scheduling optimization mathematical model based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solve the resource scheduling model to obtain the number of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; The transmission task allocation module is used to allocate power and spectrum resources to multiple users according to the number of resources required by the users.

[0085] The working process and working principle of each module are the same as those in Example 1 and will not be repeated in this embodiment.

[0086] Embodiment 3: This embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon. The computer instructions enable a computer to execute a resource scheduling method for a multi-user multimodal stream integrity transmission task. The method includes the following steps: S1. For multi-user multimodal streaming tasks, obtain the audiovisual stream transmission rate and tactile stream transmission delay; S2. Establishing a mathematical model for optimizing resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solving the model to obtain a quantity of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; S3. Allocate power and spectrum resources to multiple users based on the amount of resources required by the users.

[0087] Embodiment 4: This embodiment provides an electronic device, which may include: a processor, a communications interface, a memory 630, and a communications bus, wherein the processor, the communications interface, and the memory communicate with each other via the communications bus. The processor may invoke logic instructions in the memory to execute a resource scheduling method for a multi-user multimodal stream integrity transmission task, the method comprising the following steps: S1. For multi-user multimodal streaming tasks, obtain the audiovisual stream transmission rate and tactile stream transmission delay; S2. Establishing a mathematical model for optimizing resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solving the model to obtain a quantity of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; S3. Allocate power and spectrum resources to multiple users based on the amount of resources required by the users.

[0088] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0089] Embodiment 5: This embodiment provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform a resource scheduling method for a multi-user multimodal stream integrity transmission task. The method includes the following steps: S1. For multi-user multimodal streaming tasks, obtain the audiovisual stream transmission rate and tactile stream transmission delay; S2. Establishing a mathematical model for optimizing resource scheduling based on the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, and solving the model to obtain a quantity of resources that maximizes the multi-user multimodal perception utility under spectrum and power resource constraints; S3. Allocate power and spectrum resources to multiple users based on the amount of resources required by the users.

[0090] Based on the above embodiments, in order to verify the resource scheduling method and system for the multi-user multi-modal stream integrity transmission task of the present invention, in the face of Figure 1 A simulation experiment is conducted to investigate the effect of resource scheduling in the transmission task of the system shown.

[0091] The simulation parameters are set as follows: Total number of base stations is 3, and the total number of users is is 30, the frame rate of the user audiovisual stream is 60 frames per second, and the user audiovisual frame size set is , the size of each pixel is 12 bits, the duration of the time slot is The duration of a microslot is 1 millisecond, and the duration of a microslot is 1 / 7 millisecond. The proposed resource scheduling method for the integrity transmission of multi-user multimodal streams is compared with an incomplete transmission scheme, a single-slot transmission scheme, and an average resource allocation scheme. The incomplete transmission scheme does not consider the integrity of the user's audiovisual stream transmission. The single-slot transmission scheme schedules resources based solely on the time slot without considering microslots. In the average resource allocation transmission scheme, the base station evenly allocates spectrum and power resources to users.

[0092] like Figure 4 The figure shows the relationship between the total number of users and the total number of spectrum resource blocks and the average multimodal perception utility of users. The average multimodal perception utility of users decreases as the total number of users increases, because the increase in the number of users also increases the reuse rate of spectrum resource blocks between cells, which increases the interference between cells and thus reduces the multimodal perception utility. The average multimodal perception utility of users increases with the increase in spectrum resource blocks. The increase in spectrum resource blocks promotes the improvement of multimodal streaming transmission performance, thereby improving the multimodal perception utility. Therefore, the service requirements of users' multimodal services can be met by adjusting the number of spectrum resource blocks and the number of users served by the base station.

[0093] like Figure 5 As shown, under all schemes, the user's average multimodal perceptual utility increases with the increase in the number of audiovisual blocks contained in each audiovisual frame, because the increase in the number of audiovisual blocks contained in the audiovisual frame can reduce the impact of the integrity transmission requirement on the audiovisual stream transmission. The resource scheduling scheme proposed in the present invention is superior to other benchmark algorithms in terms of the user's average multimodal perceptual utility, because the present invention takes into account the real-time transmission requirements of the tactile stream and the integrity transmission characteristics of the audiovisual stream, further improving the utilization rate of spectrum resources and power resources and promoting the efficient transmission of multimodal streams, thereby improving the user's multimodal perceptual utility. Therefore, considering the integrity transmission of the audiovisual stream, the number of audiovisual blocks contained in the audiovisual frame should be appropriately set, so as to ensure the user's multimodal immersive service experience with a lower audiovisual frame processing complexity.

[0094] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A resource scheduling method for multi-user multi-modal stream integrity transmission tasks, characterized in that: The steps include: S1. For multi-user multi-modal streaming tasks, obtain the audio-visual streaming transmission rate and the tactile streaming transmission delay; S2. According to the audiovisual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multimodal perception utility, establish an optimization mathematical model for resource scheduling, and solve the model to obtain the number of resources that maximize the multi-user multimodal perception utility under the constraints of spectrum and power resources; S3. Allocate power and spectrum resources to multiple users according to the number of resources required by the users.

2. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 1 is characterized in that: Step S1 obtains the audiovisual stream transmission rate and the tactile stream transmission delay for a multi-user multimodal stream transmission task; including: S11. For multi-user multi-modal stream transmission tasks, a transmission model of tactile stream and audio-visual stream is established; S12. Calculate the transmission rate of the audiovisual stream and the transmission delay of the tactile stream.

3. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 2 is characterized in that: Step S11 establishes a transmission model of tactile stream and audio-visual stream for the multi-user multi-modal stream transmission task, specifically including: For audio-visual streams, the duration of the user's audio-visual frame is seconds, the time reserved for each audio-visual frame transmission is Time slots , each time slot is divided into Mini-slots ,The audio-visual stream is transmitted on a time-slot basis, while the tactile stream is transmitted on a micro-time-slot basis; The base station provides services to users within the coverage area. The set of base stations is , the user set is , each base station occupies a number of spectrum resource blocks, and some spectrum resource blocks are reused between base stations. The set of spectrum resource blocks is expressed as ; Step S12 calculates the audiovisual stream transmission rate and the tactile stream transmission delay, specifically including: Base Station Users of the service In time slot The audiovisual stream transmission rate obtained within is expressed as: ； in, is the bandwidth of the spectrum resource block, For time slot Internal User In spectrum resource blocks The signal-to-interference-noise ratio on Indicates user In time slot Whether the spectrum resource block is occupied Indicates that the user is in the time slot Occupied spectrum resource blocks Indicates that the user is in the time slot Unoccupied spectrum resource blocks ; Base Station Users of the service In micro-slot The tactile flow transmission rate obtained within is expressed as: ； in, Mini-slot Internal User In spectrum resource blocks The signal-to-interference-noise ratio on Indicates user In micro-slot Whether the spectrum resource block is occupied , Indicates user In micro-slot Occupied spectrum resource blocks , Indicates user In micro-slot Unoccupied spectrum resource blocks , is the channel dispersion and , is the block length of the tactile stream, is a function The inverse function of ; The user In micro-slot The haptic data packet size is expressed as , then the user In micro-slot The tactile transmission delay within is expressed as: ； The user The packet arrival rate of the tactile flow is expressed as , the duration of the mini-slot is expressed as , then the user In time slot The transmission delay of all tactile data packets arriving within is expressed as: ； The user's tactile transmission performance is expressed by the tactile sensitivity coefficient. In time slot The tactile sensitivity coefficient is expressed as: ； in, Indicates the maximum transmission delay of tactile stream, Indicates the minimum tactile transmission delay.

4. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 1, characterized in that: Step S2 establishes an optimization mathematical model for resource scheduling based on the transmission rate of the audiovisual stream and the transmission delay of the tactile stream, with the goal of maximizing the multi-user multimodal perception utility, and solves the model to obtain the number of resources that can maximize the multi-user multimodal perception utility under the constraints of spectrum and power resources; including: S21. Based on the multi-attribute utility theory, a user multimodal perception utility function is constructed. The spectrum resource allocation strategy and power allocation strategy are used as optimization objects. The maximization of multimodal perception utility of multiple users is used as the objective function. The user audiovisual frame size, spectrum resources and power resource restrictions are used as constraints. A resource scheduling model for the integrity transmission of multimodal streams is constructed. S22. Solve the resource scheduling model to obtain the number of resources that maximizes the multi-user multi-modal sensing utility under the constraints of spectrum and power resources.

5. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 4 is characterized in that: In step S21, a resource scheduling model for multimodal stream integrity transmission is constructed, and the specific method is as follows: S211. Constructing user multimodal perceived utility function based on multi-attribute utility theory. In time slot The calculation expression of the multimodal perceived utility obtained is: ； in, For users In time slot The transmission rate of the internal audio and video stream, For users In time slot The transmission time of the internal audio-visual stream, For users The size of the requested audio-visual chunk, is the minimum value of the audio-visual block, is the maximum value of the tactile stream transmission delay, is the minimum value of the tactile stream transmission delay, For users In time slot Transmission delay of the internal tactile stream; S212. Establish a resource scheduling model for multimodal stream integrity transmission to represent the optimization problem: ； ； ； ； ； ； ； in, is the set of base stations, is the number of base stations; is a collection of users, is the number of users; is a collection of spectrum resource blocks, is the total number of spectrum resource blocks; Indicates base station The number of occupied spectrum resource blocks; is the set of time slots, is the total number of time slots prepared for the transmission of each audiovisual frame; For time slot The collection of mini-slots contained in is the number of mini-slots contained in each time slot; Spectrum resource scheduling strategy for audio-visual streams, Indicates spectrum resource block In time slot Assigned to users ,otherwise ; represents the spectrum resource scheduling strategy of tactile flow, Indicates spectrum resource block In micro-slot Assigned to users ,otherwise ; Indicates the user's power allocation strategy. The total transmission power of each base station in each time slot cannot exceed ; Indicates the duration of each time slot; Indicates user The requested audio-visual frame size. Each audio-visual frame is divided into audio-visual blocks; Step S22 solves the resource scheduling model to obtain the number of resources that maximizes the multi-user multi-modal sensing utility under the constraints of spectrum and power resources. The specific method is: Firstly, based on Lyapunov theory, the optimization problem of step S212 is decomposed into the multimodal perception utility maximization problem for each time slot; then, spectrum resources are allocated based on a one-to-many matching algorithm, and power resources are allocated based on the alternating direction multiplier method. Finally, the spectrum resource and power resource allocation algorithms are combined to obtain stable spectrum resource allocation strategies and power resource allocation strategies.

6. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 1, characterized in that: Step S3 allocates power and spectrum resources to multiple users according to the number of resources required by the users; The transmission of the tactile stream adopts a resource puncture method, that is, the transmission of the tactile stream occupies the spectrum resources that have been allocated to the audio-visual stream.

7. The resource scheduling method for multi-user multi-modal stream integrity transmission task according to claim 6, characterized in that: In the user When the current audiovisual frame has been transmitted, the spectrum and power resources it occupies are allocated to the user whose audiovisual frame has not been transmitted yet; at this time, the user The tactile streaming transmission occupies the spectrum of other users' audio-visual frame transmission.

8. A resource scheduling system for multi-user multi-modal stream integrity transmission tasks, characterized in that: include: A transmission task acquisition module, used to obtain the transmission rate of the audiovisual stream and the transmission delay of the tactile stream for a multi-user multimodal stream transmission task; A transmission strategy optimization module is used to establish an optimization mathematical model for resource scheduling based on the audio-visual stream transmission rate and the tactile stream transmission delay, with the goal of maximizing the multi-user multi-modal perception utility, and solve the resource scheduling model to obtain the number of resources that maximize the multi-user multi-modal perception utility under the constraints of spectrum and power resources; The transmission task allocation module is used to allocate power and spectrum resources to multiple users according to the number of resources required by the users.

9. A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions enable a computer to execute the resource scheduling method for a multi-user multi-modal stream integrity transmission task as described in any one of claims 1-7.

10. An electronic device comprising a processor, a communication interface, a memory and a communication bus; in, The processor, the communication interface, and the memory communicate with each other via a communication bus; The processor is used to call the logic instructions in the memory to execute the resource scheduling method for the multi-user multi-modal stream integrity transmission task described in any one of claims 1 to 7.

Images