Satellite network-oriented resource scheduling method for user satisfaction

Abstract

The invention provides a satellite network-oriented user satisfaction resource scheduling method, which comprises the following steps of: acquiring an initial environment state of a satellite network, including satellite resource information and respective demand information of a plurality of user services, and constructing a satellite network topological structure; according to the topological structure, setting a corresponding action space for the service, which comprises a plurality of different link selections; constructing an initial Q table according to each service and the corresponding action space, and recording the initialized user long-term satisfaction of each service under different link selections; and adopting a reinforcement learning algorithm to maximize the long-term satisfaction of the user as a target, carrying out multiple rounds of iterative updating on the initial Q table to obtain links distributed for each service, and selecting links for each service based on the action space of each service in each round, and feeding back the current satisfaction degree based on the evaluated perception index of the user on the network service quality of the selected link by using a preset reward function, and updating the Q table according to the current satisfaction degree.

Description

Technical Field

[0001] The present invention relates to the field of satellite communications, in particular to the field of satellite resource scheduling, and more particularly to a resource scheduling method oriented to user satisfaction of a satellite network. Background Art

[0002] With the rapid development of communications technology, mobile multimedia services and emerging applications such as artificial intelligence (AI), augmented reality (AR), and cloud-native services are constantly emerging. Compared with traditional applications, these emerging applications can provide richer interactive experiences and support real-time processing and analysis of massive amounts of data. The growth of user base, the diversification of content, the increasing complexity of applications, the popularity of IoT devices, and the increasing demand for real-time performance have driven explosive growth in data traffic. However, the coverage and capacity limitations of traditional terrestrial cellular networks are becoming increasingly prominent. Although fifth-generation mobile communication technology (5G) has achieved significant improvements in connection speed, latency, and device connectivity, it still has shortcomings in coverage, spectrum efficiency, and energy consumption.

[0003] Sixth-generation mobile communication technology (6G) is expected to achieve faster data speeds, lower latency, greater network capacity, and wider coverage than 5G, further promoting the realization of the Internet of Everything. To meet the need for universal coverage, 6G networks will complement terrestrial networks with space networks such as satellite communications, maritime communications, and drone communications, building a three-dimensional information network that integrates air, space, and land. Non-geostationary satellite constellations, particularly low-Earth orbit (LEO) satellite constellations, leverage their wide coverage, high throughput, and high bandwidth. Through satellite networking, they can provide ubiquitous internet services globally, particularly in remote areas that are difficult to reach with traditional cellular networks. Therefore, LEO satellite constellations are considered a highly promising solution for future wireless network architectures.

[0004] Standards organizations such as the International Telecommunication Union (ITU) are gradually incorporating a user-centric approach in the development of next-generation satellite communication standards. As user needs evolve and diversify, network Quality of Service (QoS) is no longer sufficient to fully describe user perceived quality. Multi-service quality assurance requirements are no longer sufficient, and a new assurance assessment system tailored to the specific industry characteristics and needs is urgently needed. Quality of Experience (QoE) focuses more on overall user satisfaction and experience, better reflecting how QoS factors impact end-user perception of communication service quality.

[0005] The QoE-based evaluation system has been developed to a certain extent. In the field of real-time communications, several evaluation methods that are closely related to QoE have emerged. These evaluation methods can be divided into the following three types:

[0006] ① Subjective quality assessment methods: These include the Customer Satisfaction Opinion Score (MOSS) for voice service evaluation and the subjective video quality assessment method specified in ITU-T P.910. These methods provide a relatively objective understanding of user perceptions of QoE in different situations, highlighting the impact of various factors on the quality of experience through comparison. However, subjective methods have the disadvantages of requiring specialized environments and equipment and requiring a large number of test participants.

[0007] ② Objective quality assessment methods: These are categorized into parameterized and non-parametric assessments. Parameterized assessment is used when a reference signal is known and can more accurately assess audio quality. Non-parametric assessment, however, is used when a reference signal is unavailable and can utilize both traditional and machine learning methods. Traditional methods evaluate based on signals and parameters, while machine learning methods use data-based training models to assess audio quality.

[0008] ③ Pseudo-subjective evaluation methods: This method combines the advantages of subjective and objective evaluation methods. By simulating the characteristics of the human perceptual system, it uses objective or signal processing techniques to extract characteristic parameters related to human subjective perception. Based on these parameters, an evaluation model is then constructed to quantitatively assess the quality of multimedia content. This method avoids the subjectivity and instability of purely subjective evaluation while overcoming the limitation of purely objective evaluation, which cannot fully reflect the characteristics of human perception. It can, to a certain extent, more accurately predict human subjective perception of multimedia content quality.

[0009] Based on user QoE requirements for different services, rationally allocating resources such as bandwidth and power consumption can ensure the quality of communication for critical services and users. For example, in video conferencing, resource allocation must be prioritized to ensure video clarity and smoothness. To optimize resource scheduling strategies, ensuring resource efficiency while improving user satisfaction, it is crucial to establish a corresponding QoE evaluation system for satellite resource scheduling.

[0010] In traditional terrestrial wireless networks, the correlation model between QoS and QoE has been studied to some extent, but there are fewer studies on satellite networks. For example, Reference [1] uses the cancellation rate of web browsing to represent user satisfaction. Reference [2] combines the reference-free QoE evaluation model and sorts and assigns weights to QoS parameters through the pairwise comparison method, and finally predicts the QoE value of IPTV video services by normalizing the QoS parameters. Reference [3] proposes the IQX hypothesis, which describes the exponential relationship between QoE and QoS through a universal quantitative formula. Reference [4] uses the same method to establish a QoE / QoS correlation model for video services in wireless networks. Reference [5] proposes a QoE evaluation method for video services in 3G LTE networks based on BP neural network and particle swarm optimization (PSO). Reference [6] collects actual video session data for statistical analysis and calculates the mean opinion score based on the ITU-TP.1201.1 standardized model to construct a QoE / QoS correlation model based on transmission rate and packet loss rate. Reference [7] comprehensively considers user, background and technical factors, and constructs a QoE / QoS association model for 5G mobile wireless network communication services based on the logarithmic relationship of the IQX hypothesis. Reference [8] uses a subjective method to evaluate QoE. The authors analyze the impact of single QoS parameters (such as packet loss rate, delay and jitter) on the QoE of satellite network multimedia services by using actual network parameters and analyzing user feedback. Reference [9] focuses on the resource allocation problem under VLC / RF network, comprehensively considers system, environment and human factors, and constructs a QoE / QoS association model through a weighted method. This association model is relatively simple and no further research based on the three factors is done. Most of the above studies are aimed at terrestrial wireless network scenarios, and there is little research on satellite communications.

[0011] Based on the above analysis, it can be seen that the evaluation method centered on QoS has certain limitations in reflecting the quality of user experience. QoS indicators mainly focus on technical parameters at the network level, such as bandwidth, latency, jitter, and packet loss rate, and cannot fully reflect the user's actual perceived experience. Research has shown that under the same QoS conditions, different users may have significant differences in their perception of service quality. Currently, most research focuses on QoE evaluation for traditional terrestrial networks, while research on user quality evaluation in satellite communication scenarios is still in the stage of continuous exploration and improvement. The evaluation method centered on QoS will lead to particularly prominent differences in the service quality perception of different users in low-orbit satellite communication scenarios.

[0012] Therefore, the existing user experience quality evaluation in satellite communication scenarios is not perfect, and the evaluation based on QoS indicators cannot fully reflect the user's actual perceived experience.

[0013] It should be noted that this background information is intended solely to introduce relevant information related to the present invention to facilitate understanding of the present invention's technical solution. It does not necessarily constitute prior art. Relevant information submitted and disclosed together with the present invention's solution should not be considered prior art unless there is evidence that the relevant information was disclosed prior to the filing date of the present invention.

[0014] References are as follows:

[0015] [1] Relationship between quality-of-service and quality-of-experience for public internet service.

[0016] [2] Hyun, JK and GC Seong. A study on a QoS / QoE correlation model for QoE evaluation on IPTV service. in 2010 The 12th International Conference on Advanced Communication Technology (ICACT). 2010.

[0017] [3] M., F., HT and TP, A generic quantitative relationship between quality of experience and quality of service. IEEE Network, 2010. 24(2): p. 36-41.

[0018] [4] Z., W., et al. A study on QoS / QoE correlation model in wireless-network. in Signal and Information Processing Association Annual Summit andConference (APSIPA), 2014 Asia-Pacific. 2014.

[0019] [5] Quality-of-experience assessment and its application to videoservices in lte networks.

[0020] [6] Gu Hongcheng, Research on QoE-QoS Correlation Model for Video Conversation Service, 2020, Nanjing University of Posts and Telecommunications.

[0021] [7] Zhang, X. and J. Wang. Joint heterogeneous statistical-QoS / QoEprovisionings for edge-computing based WiFi offloading over 5G mobilewireless networks. 2018: IEEE.

[0022] [8] Xu, S., X. Wang and M. Huang. A study on QoE-QoS relationship for multimedia services in satellite networks. 2018: IEEE.

[0023] [9] Sun Hongliang, Chen Tongfei and Qian Lei. QoS-QoE driven VLC / RF heterogeneous network resource allocation algorithm. Optical Communication Technology, 2024. 48(06): pp. 16-22. Summary of the Invention

[0024] Therefore, the purpose of the present invention is to overcome the above-mentioned defects of the prior art and provide a resource scheduling method oriented to user satisfaction of satellite networks.

[0025] The purpose of the present invention is achieved through the following technical solutions:

[0026] According to a first aspect of the present invention, a resource scheduling method for user satisfaction in a satellite network is provided. The method comprises: S1. obtaining an initial environmental state of the satellite network, which includes satellite resource information and service demand information of multiple users, and constructing a satellite network topology; S2. setting a link set for each service based on the satellite network topology, which includes multiple different links that meet the service requirements, and constructing an action space for the service based on the link set, including multiple different link selections; S3. constructing an initial Q-table based on each service and its action space, which records the initialized long-term user satisfaction of each service under different link selections in its action space; S4. using a reinforcement learning algorithm, based on the initial environmental state, and with the goal of maximizing long-term user satisfaction, iterating the initial Q-table multiple times to obtain links allocated to each service while maximizing long-term user satisfaction. In each round, a selected link is determined based on the service's action space, and a preset reward function is used to provide feedback on the current satisfaction based on an evaluated user perception indicator of the network service quality of the selected link, and the Q-table is updated based on the current satisfaction.

[0027] In some embodiments of the present invention, the service includes a video service. In S4, an evaluation model is used to evaluate the user's perception index of the network service quality of the selected link based on the mapping relationship between the parameters of the network service quality and the user experience quality; wherein the perception index includes a first index and a second index, the first index is determined based on a first sub-mapping relationship between the packet loss rate of the selected link and the video quality, and the second index is determined based on a second sub-mapping relationship between the video playback time of the selected link jointly fitted by the rate ratio and the delay and the video playback smoothness.

[0028] In some embodiments of the present invention, the evaluation model is as follows:

[0029] ,

[0030] in, represents the perceptual index of the calculation, represents the dynamic weight coefficient, Represents the first indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the packet loss rate, express The lower limit of represents the dynamic weight coefficient, represents the second indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the video playback duration of the combined rate ratio and delay fitting, represents the rate ratio, Indicates delay, express The lower limit of .

[0031] In some embodiments of the present invention, in S4, each round of iterative process includes: obtaining a Q table and an environmental state, using an initial Q table and an initial environmental state in the first round, and using the updated Q table and the updated environmental state in each round after the first round; with the goal of maximizing long-term user satisfaction, updating the Q table multiple times based on the environmental state and the services of multiple users, and using the Q table and environmental state after the last update as the updated Q table and the updated environmental state in the current round. Each update process includes:

[0032] Based on the action space of the business obtained this time, a link is selected using the ε-greedy decay strategy. Based on the selected link, the preset reward function is used to calculate the current satisfaction of the corresponding user, and the environment state is updated to obtain the updated environment state. The latest Q-table is obtained, and the TD difference is calculated based on the Q-table and the current satisfaction of the corresponding user. Based on the TD difference, the long-term user satisfaction corresponding to the link selection of the business in the corresponding action space in the Q-table is updated to obtain the updated Q-table.

[0033] In some embodiments of the present invention, during each update process, the current satisfaction level is determined by checking whether the resources of the selected link meet the business requirements. If so, the current satisfaction level is the user's perception index of the network service quality of the selected link; otherwise, the current satisfaction level is a preset penalty value.

[0034] In some embodiments of the present invention, the video playback duration is calculated as follows:

[0035] ,

[0036] in, 、 、 、 、 、 、 、 、 、 are fitting parameters.

[0037] In some embodiments of the present invention, in S2, the method of setting a link set for each service includes: constructing a corresponding initial link set for each user service, which includes links corresponding to all satellites visible to the user and which meet the needs of the user service; and using a preset link pre-screening mechanism to remove links corresponding to satellites whose satellite communication quality is less than a preset quality threshold, to obtain a final link set.

[0038] In some embodiments of the present invention, the satellite communication quality is evaluated based on the satellite's elevation angle, signal strength, and signal-to-noise ratio.

[0039] According to a second aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the steps of the method of the first aspect of the present invention.

[0040] According to the third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method of the first aspect of the present invention by executing the executable instructions.

[0041] Compared with the prior art, the advantages of the present invention are:

[0042] The method of the present invention evaluates the user's perception of the link's network quality of service (QoS) to provide feedback on current satisfaction, achieving a comprehensive consideration of the satellite network's QoS and the user's actual perceived quality of experience (QoE), thereby more comprehensively reflecting the user's actual perceived experience of network satisfaction. On this basis, with the goal of maximizing long-term user satisfaction, the link allocation strategy is dynamically adjusted over multiple rounds, ensuring that the allocated link resources both meet business needs and improve long-term user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0044] Figure 1 2. A schematic flow chart of a resource scheduling method for user satisfaction in a satellite network according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram showing the effect of statistical packet loss rate on satisfaction according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram showing the combined effect of the statistical rate ratio and delay on satisfaction according to an embodiment of the present invention;

[0047] Figure 4Schematic diagram showing the combined impact of statistical packet loss rate and latency on satisfaction according to an embodiment of the present invention;

[0048] Figure 5 Schematic diagram showing the combined impact of the statistical rate ratio and packet loss rate on satisfaction according to an embodiment of the present invention;

[0049] Figure 6 Schematic diagram of the combined impact of statistical rate ratio, packet loss rate and delay on satisfaction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] As mentioned in the background technology section, the existing user experience quality evaluation in satellite communication scenarios is not perfect, and the evaluation based on QoS indicators cannot fully reflect the user's actual perceived experience.

[0052] To address the above issues, according to one embodiment of the present invention, a resource scheduling method for user satisfaction in satellite networks is proposed. The method includes: setting an action space for each service in the satellite network, which includes multiple different link options that meet service requirements; constructing an initial Q-table that records the initial long-term user satisfaction for each service under different link options in its action space; and performing multiple rounds of iterations on the Q-table using a reinforcement learning algorithm to determine the links allocated to each service that maximize long-term user satisfaction. In each round, the current user satisfaction is fed back based on an assessed user perception of the network service quality of the selected link, thereby updating the long-term user satisfaction in the Q-table based on the current satisfaction.

[0053] The technical solutions of the above-described embodiments can achieve at least the following beneficial technical effects: by evaluating user perception indicators of link network quality of service (QoS), current satisfaction is provided. This allows the present invention to comprehensively consider both the satellite network QoS and the user's actual perceived quality of experience (QoE) when scheduling resources based on long-term user satisfaction, thereby more comprehensively reflecting the user's actual perceived satisfaction with the network. Furthermore, a reinforcement learning algorithm is used to dynamically adjust the link allocation strategy, ensuring that allocated link resources both meet service needs and enhance long-term user satisfaction. This present invention not only further improves user quality of experience evaluation in satellite communication scenarios but also achieves link resource allocation that enhances user satisfaction.

[0054] According to one embodiment of the present invention, see Figure 1, which is a flow chart of a resource scheduling method for user satisfaction in satellite networks. The method includes steps S1, S2, S3, and S4. This method can be used for resource scheduling in low-orbit satellite networks or medium-orbit satellite networks. To better understand the present invention, the following uses a low-orbit satellite network as an example, with each step described in detail in conjunction with specific embodiments.

[0055] Step S1: Acquire the initial environment state of the satellite network, which includes satellite resource information and service demand information of multiple users, and construct a satellite network topology.

[0056] According to one embodiment of the present invention, the environmental status includes information about satellite orbits, satellite resource information, and service information for each user. Each user's service information includes their geographic location, the service's transmission rate requirements, and the service's minimum network Quality of Service (QoS) requirements. The service requirements include minimum QoS requirements, which include minimum data transmission rate ratio, maximum latency, and maximum packet loss rate requirements. The rate ratio is the ratio of the link's achievable rate (i.e., the service's actual transmission rate) to the service's transmission rate requirements.

[0057] According to one embodiment of the present invention, taking a low-orbit satellite network as an example, the service of each user in the low-orbit satellite network is a video service. Assume that there are U users performing video transmission services simultaneously, including users uploading and downloading videos. The transmission link directions are from user to satellite to base station (uplink) and from base station to satellite to user (downlink). Uploading and downloading videos are both unidirectional services, and each unidirectional service is regarded as a video service. The set of services of multiple users can be expressed as , , where any business There are minimum network quality of service (QoS) requirements, defined as The minimum network quality of service (QoS) requirement is =( , respectively representing the user's business The minimum data transfer rate is , maximum delay and maximum packet loss rate of.

[0058] According to an embodiment of the present invention, the constructed satellite network topology structure includes various users, various base stations, various satellites, and the link relationships between them.

[0059] Step S2: According to the satellite network topology, a link set is set for each service, which includes multiple different links that meet the service requirements, and an action space for the service is constructed based on the link set, including multiple different link options.

[0060] According to one embodiment of the present invention, in a low-orbit satellite network, the entire uplink (or the entire downlink) of a service during transmission is regarded as one link.

[0061] According to one embodiment of the present invention, in S2, the method of setting a link set for each service includes steps S21 and S22:

[0062] Step S21: constructing a corresponding initial link set for each user service, which includes links corresponding to all satellites visible to the user and which meet the needs of the user service.

[0063] According to one embodiment of the present invention, all satellites visible to the user indicate that satellite signals can cover the user to communicate with the user. The links corresponding to all visible satellites are used as optional links for the service, and all optional links are complete links for the service to be transmitted from the source node to the target node. For business A set of optional links from the source node to the destination node. The link in the constructed action space can be expressed as , and Satisfy user service requirements. If a link satisfies service requirements, it means that the link satisfies the following constraint formula:

[0064] , (1)

[0065] in, Indicates business In the link The actual rate ratio, Indicates business In the link The actual end-to-end delay, Indicates business In the link The actual packet loss rate. Indicates business In the link Actual transmission rate, for and business The ratio of the transmission rate requirements to the optional link set. The set of all links that meet the business requirements is used as the initial link set.

[0066] Step S22: Using a preset link pre-screening mechanism, remove links corresponding to satellites whose satellite communication quality is less than a preset quality threshold, and obtain a final link set.

[0067] According to one embodiment of the present invention, satellite communication quality is evaluated based on the satellite's elevation angle, signal strength, and signal-to-noise ratio (SNR). For example, satellite communication quality is assessed by weighting the satellite's elevation angle, signal strength, and SNR, and links corresponding to satellites with quality below a preset threshold are removed. Links corresponding to satellites with large elevation angles, strong signals, and high SNRs are prioritized. Alternatively, satellite communication quality can be comprehensively assessed by weighting transmit power, elevation angle, and signal-to-noise ratio (SNR), prioritizing links corresponding to satellites with high transmit power, large elevation angles, and high SNRs.

[0068] The technical solution of the embodiment of step S22 described above can achieve at least the following beneficial technical effects: optimizing the action space, reducing the number of selectable links, avoiding dimensionality explosion, and thus improving computational efficiency. Furthermore, satellites with large elevation angles are preferentially selected. A larger elevation angle means a closer-to-vertical line-of-sight path between the satellite and the ground base station, which can improve signal transmission quality and stability.

[0069] According to one embodiment of the present invention, step S2 further includes step S23: constructing an action space for the service based on the final link set obtained in step S24, including multiple different link options. Since each service selects one link or no link at a time, if the final link set includes N paths, the action space includes N+1 different link options. The size of the action space for this service is N+1. Due to satellite visibility and resource constraints, the size of the action space for each service varies.

[0070] Indicatively, As the final link set of the service, it is a discrete index set. Select a link or do not select any link, so the action space size corresponding to this service is .in, express The number of links. Select Link , select link , ..., not selecting any link represents multiple different link selections in the action space.

[0071] Step S3: construct an initial Q table based on each service and its action space, which records the initialized long-term user satisfaction of each service under different link selections in its action space.

[0072] According to one embodiment of the present invention, the Q table includes multiple columns of data. The number of columns in the Q table is the same as the total number of all user services, and the number of rows in each column is the same as the size of the action space of the corresponding user service. That is, each column records the initialization of the long-term user satisfaction of a user service under different link selections in its action space. Schematically, the first column and the first row of the Q table record the initialization of the first user service in its action space. The initial long-term user satisfaction under the link selection, the first column and second row of the Q table records the first user service in its action space. The initial long-term user satisfaction under the link selection. Among them, the long-term satisfaction of all users in the initial Q table is set to 0.

[0073] Step S4: Using a reinforcement learning algorithm, based on the initial environment state and with the goal of maximizing long-term user satisfaction, the initial Q-table is iterated multiple times to obtain the links allocated to each service while maximizing long-term user satisfaction. In each round, the selected link is determined based on the action space of the service, and the current satisfaction is fed back based on the user's perceived network service quality indicator of the selected link using a preset reward function, and the Q-table is updated based on the current satisfaction.

[0074] According to one embodiment of the present invention, in a low-orbit satellite network, each link includes multiple sub-links, and the multiple sub-links include inter-satellite links, feeder links, and user links according to link categories. In a low-orbit satellite network, the total resources of all low-orbit satellites are recorded as , which includes the resources of each low-orbit satellite. low-orbit satellites The resource is represented as .in, represents an intersatellite link; represents a feeder link; Indicates user link, represents the intersatellite link resources, represents the feeder link resources, Represents user link resources. Each sub-link resource includes transmission power resources, bandwidth resources and time slot resources. Therefore, for sub-links The resources on ,in, Indicates a sublink Transmit power resources; Indicates a sublink bandwidth resources; Indicates a sublink In the process of allocating links to each service, the total link resources allocated to all services must not exceed the total resources of all low-orbit satellites. This resource constraint.

[0075] The following describes the perception index evaluation and reinforcement learning algorithm in step S4 in turn:

[0076] 1. Perception Index Evaluation

[0077] According to one embodiment of the present invention, in step S4, an evaluation model is used to evaluate a user's perceived network service quality indicator for a selected link based on a mapping relationship between network service quality parameters and user experience quality. The perceived indicator includes a first indicator and a second indicator. The first indicator is determined based on a first sub-mapping relationship between the packet loss rate of the selected link and video quality, and the second indicator is determined based on a second sub-mapping relationship between video playback duration and video playback smoothness, which is a joint fit of the rate ratio and latency of the selected link. Latency represents the end-to-end latency of the link.

[0078] The technical solution of this embodiment can achieve at least the following beneficial technical effects: The inventors have analyzed and discovered that in satellite communication systems, key indicators such as throughput, latency, and packet loss rate are mutually constrained (for example, increasing throughput may result in increased latency). Traditional multi-objective optimization methods struggle to balance these conflicting objectives, resulting in an ineffective improvement in user Quality of Experience (QoE). Furthermore, satellite networks are characterized by high dynamics, limited resources, and time-varying channels. Existing resource scheduling algorithms fail to fully integrate network-layer technical indicators with actual user experience perceptions, making it difficult to accurately adapt resource allocation in complex network environments. Therefore, the present invention evaluates user perception of network service quality based on a mapping relationship between network Quality of Service (QoS) and User Quality of Experience (QoE). In establishing this mapping, the present invention selects three key parameters: packet loss rate, data rate ratio, and latency, to construct an evaluation model that comprehensively reflects user perceptions of video quality and playback smoothness. This model provides a practical and intuitive assessment of user Quality of Experience, enabling better link allocation in complex network environments.

[0079] According to one embodiment of the present invention, the evaluation model is as follows:

[0080] , (2)

[0081] in, represents the perceptual index of the calculation, represents the dynamic weight coefficient, Represents the first indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the packet loss rate, express The lower limit of represents the dynamic weight coefficient, represents the second indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the video playback duration of the combined rate ratio and delay fitting, represents the rate ratio, Indicates delay, express The lower limit of . 、 The exponential function is used to construct the first sub-mapping relationship: And the second sub-mapping relationship: .

[0082] The technical solution of this embodiment can at least achieve the following beneficial technical effects: The inventors have analyzed and found that when QoS is poor (low), users are very sensitive to its changes, while when QoS is high, the growth rate of QoE gradually slows down. That is, QoE exhibits an exponential decay or exponential growth relationship with QoS. Therefore, the present invention constructs a corresponding sub-mapping relationship through an exponential function to better reflect the user's actual experience. Based on the more realistic user satisfaction obtained from this actual experience, the link allocation strategy is subsequently optimized by optimizing user satisfaction, so that the final optimized link allocation strategy not only achieves multi-parameter optimization of rate ratio, delay, and packet loss rate at the network technology level, but also optimizes user satisfaction at the user level.

[0083] According to one embodiment of the present invention, the video playback duration is calculated as follows:

[0084] , (3)

[0085] in, 、 、 、 、 、 、 、 、 、 The technical solution of this embodiment can at least achieve the following beneficial technical effects: taking into account the rate ratio and delay The combined impact on video playback duration can result in a more accurate video playback duration.

[0086] According to one embodiment of the present invention, the rate ratio , delay and packet loss rate These are all statistical parameters of the actual network service quality of the link. The bandwidth constraints in actual business scenarios are also taken into account. That is, when the allocated bandwidth exceeds the business demand, the business satisfaction reaches saturation and will not continue to improve as the bandwidth increases. Therefore, the rate ratio is used in the calculation process of the evaluation model. Perform the following constraint processing: When This method will not cause the actual rate of the allocated link to be much higher than the rate required by the business, thus avoiding resource waste and improving resource utilization.

[0087] According to one embodiment of the present invention, the principle of the evaluation model construction process of the above embodiment is explained below through formulas (4) to (10):

[0088] 1) Construction basis:

[0089] When establishing an evaluation model, it is necessary to quantitatively evaluate the network quality of service (QoE). Currently, methods for quantifying QoE are broadly categorized into the two-category method, the paired comparison method, and the mean opinion score (MOS). MOS is a widely used standard for subjective evaluation, quantifying service quality by collecting users' subjective ratings of service samples. MOS typically uses a 5-point or 7-point scale. This is not a limitation of the present invention, and a 10-point scale may also be used. The following description uses the 5-point scale as an example. The 5-point scoring criteria are shown in Table 1 below:

[0090] Table 1: MOS scores

[0091]

[0092] MOS scores can systematically quantify users' perception of QoE, providing an important basis for establishing evaluation models.

[0093] 2) Construct an evaluation model through experimental fitting.

[0094] 2.1) This model maps the impact of packet loss rate, delay, and rate ratio on video quality and video smoothness into user QoE perception indicators , including the first and second indicators. Based on the MOS 5-point scoring mechanism, the values of the first and second indicators are mapped to a range of 1 to 5. The higher the video smoothness, the less noticeable the corresponding video distortion. Therefore, we first quantify the user's actual experience perception based on the packet loss rate, delay, and rate ratio in QoE, and obtain the following quantitative form:

[0095] , (4)

[0096] in, Indicates the quantitative form, Indicates the actual rate ratio based on the link , delay and packet loss rate Quantify the user's actual experience perception.

[0097] 2.2) Introduce the Weber-Fechner law and the IQX (Exponential IQX Hypothesis) as theoretical inositol. The Weber-Fechner law consists of two parts: the Weber law and the Fechner law.

[0098] Among them, Weber's law describes the just noticeable difference ( ) and stimulus intensity ( ) is the linear relationship between:

[0099] , (5)

[0100] in, Represents the coefficient.

[0101] Fechner's law further proposes the perception intensity based on Weber's law. ) and stimulus intensity ( ) have the following logarithmic relationship:

[0102] , (6)

[0103] The above law reveals a nonlinear relationship between QoS technical indicators and user experience perception: when technical parameters (i.e., QoS technical indicators) are small, users are very sensitive to parameter changes, and the perceived intensity changes rapidly; when technical parameters are large, users are relatively insensitive to parameter changes, and the perceived intensity changes slowly. For example, in video services, when bandwidth is low, users will notice a significant improvement in video quality when the bandwidth increases from 1Mbps to 2Mbps; however, when bandwidth is already high, users will perceive a less noticeable improvement in video quality.

[0104] The IQX hypothesis believes that the relationship between QoE and QoS can be described by an exponential function, that is, when QoS is poor (low), users are very sensitive to its changes, and when QoS is high, the growth rate of QoE gradually slows down. In other words, QoE shows an exponential decay or exponential growth relationship with QoS. The formula is as follows:

[0105] , (7)

[0106] in, 、 denote the scaling factor, attenuation coefficient, and The lower limit of .

[0107] 2.3) Based on the theoretical analysis in 2.2), the inventors constructed the first sub-mapping relationship between packet loss rate and video quality based on the IQX hypothesis through simulation experimental results. :

[0108] , (8)

[0109] in, 、 and The specific data are determined by fitting the experimental data.

[0110] 2.4) Comprehensive consideration of rate ratio ( ) and delay ( ) on the combined effect of rate ratio and delay on video playback duration. Based on the simulation results, a bivariate cubic polynomial fitting method is used to construct the effect of rate ratio and delay on video playback duration. The influence function of , the expression of the influence function is shown in the above formula (3), where the fitting parameters in formula (3) are determined by fitting the experimental data.

[0111] Similarly, based on the index form, the video playback time is compared with the second indicator The mapping relationship is:

[0112] , (9)

[0113] in, 、 and The specific data of each are determined by fitting the experimental data.

[0114] In addition, to ensure the consistency of the evaluation model results with the MOS 5-point scoring system, the first indicator calculated by formula (8) and the second indicator calculated by formula (9) are mapped to within the closed interval of .

[0115] Finally, the perception index is calculated by dynamic weighting method (i.e. MOS value):

[0116] , (10)

[0117] in, and The respective values can be adjusted according to the specific business scenario to better reflect the actual user experience. and The sum is equal to 1.

[0118] 2. Reinforcement Learning Algorithm (also known as Q-learning Algorithm)

[0119] 1) Design principles

[0120] In the low-orbit satellite network, according to the resource constraints and evaluation model of the above embodiment, the resource allocation scheme is solved with the optimization goal of maximizing the long-term user satisfaction, so as to realize the allocation of links for each service of the satellite network. Based on the constructed evaluation model, by optimizing the rate ratio in the evaluation model, , delay and packet loss rate These three QoS parameters are used to maximize long-term user satisfaction. That is, according to one embodiment of the present invention, the optimization objective function with maximizing long-term user satisfaction as the optimization goal is as follows:

[0121] , (11)

[0122] in, Indicates allocating links to each user's business while maximizing the overall long-term user satisfaction. Indicates the total number of user services. Indicates the user service number. Indicates the first The optimization objective function of the present invention is multi-objective collaborative optimization, that is, optimizing the rate ratio, delay and packet loss rate at the same time, ultimately maximizing the overall long-term user satisfaction.

[0123] 2) Execution process of reinforcement learning algorithm

[0124] According to one embodiment of the present invention, multiple rounds of iterations are performed on the initial Q table based on the resource constraints, evaluation model, and optimization objective function of the above-described embodiment to obtain links allocated to each service while maximizing long-term user satisfaction. In step S4, each round of iteration includes step S41: obtaining the Q table and environmental state. The initial Q table and environmental state are used in the first round, and each subsequent round uses the updated Q table and environmental state from the previous round. Step S42: with the goal of maximizing long-term user satisfaction, the Q table is updated multiple times based on the environmental state and the services of multiple users, and the Q table and environmental state from the last update are used as the updated Q table and environmental state for the current round.

[0125] According to one embodiment of the present invention, each update process includes steps S421, S422, S423 and S424:

[0126] Step S421: Based on the action space of the service currently acquired, a link is selected using the ε-greedy attenuation strategy.

[0127] According to one embodiment of the present invention, the ε-greedy decay strategy includes: setting the exploration rate in each round , initial settings , and with multiple rounds of iterations When selecting a link, the exploration rate Select the link corresponding to the maximum long-term user satisfaction in the action space. The probability of selecting other links in the action space.

[0128] Step S422: Based on the selected link, a preset reward function is used to calculate the current satisfaction of the corresponding user, and the environment state is updated to obtain the updated environment state.

[0129] According to one embodiment of the present invention, during each update process, the current satisfaction is determined by checking whether the resources of the selected link meet the business requirements. If so, the current satisfaction is the user's perception of the network service quality of the selected link. Otherwise, the current satisfaction is a preset penalty value. The penalty value is the negative of the highest score in the MOS score. Taking the MOS score as an example, the preset reward function is as follows:

[0130] , (12)

[0131] According to an embodiment of the present invention, updating the environmental status includes: updating satellite resource information corresponding to the selected link to obtain the updated environmental status.

[0132] Step S423: Obtain the latest Q table, and calculate the TD difference based on the Q table and the current satisfaction of the corresponding user.

[0133] According to one embodiment of the present invention, if this is the first time in a round, the latest Q-table is the Q-table updated in the previous round. For each subsequent time in the round, the latest Q-table is the Q-table updated in the previous round. The TD difference is calculated based on the user's current satisfaction, the user's long-term satisfaction with the service under the corresponding link selection in the latest Q-table, and the maximum long-term user satisfaction across all link selections in the action space of the user's service for the next time.

[0134] Step S424: updating the long-term user satisfaction corresponding to the link selection of the service in the corresponding action space in the Q-table according to the TD difference, and obtaining the updated Q-table.

[0135] According to one embodiment of the present invention, the update rule for the long-term user satisfaction in the Q-table in the Q-learning algorithm is as follows:

[0136] , (13)

[0137] in, Indicates business Link selection in the corresponding action space The corresponding long-term user satisfaction after the update is: Indicates business Link selection in the corresponding action space The corresponding long-term user satisfaction before the update is: Indicates the current business. Indicates a link selection in the action space corresponding to the service, that is, selecting a link for resource allocation or not selecting a link for the service at that time. represents the learning rate used to control the update step size of the user’s long-term satisfaction, The value range is , represents the current satisfaction calculated according to the preset reward function after selecting a link for the business, represents the discount factor used to measure the importance of future satisfaction, The value is , Indicates the next user's business The maximum long-term user satisfaction under all link selections in the action space.

[0138] The reinforcement learning algorithm of the above embodiment is described below in pseudo code:

[0139] 1: Input: Initial environment state of satellite network

[0140] 2: Output: Link selection strategy for each service while maximizing long-term user satisfaction

[0141] 3: for simulation step τ=1 to T do

[0142] 4: Get the satellite network topology

[0143] 5: for episode=1 to G do

[0144] 6: Reset the satellite resources in the environment state, set the link set and action space for each business, G represents the total number of businesses, and U represents the set of all businesses

[0145] 7: for business u∈U do

[0146] 8: Obtain business demand information

[0147] 9: Get the link set of the service in the current time slot t And resource information of each link

[0148] 10: if Empty then

[0149] 11: continue

[0150] 12: else

[0151] 13: Based on the exploration rate from Select link a,

[0152] 14: Check whether the selected link resources meet the business requirements

[0153] 15: if satisfies then

[0154] 16: Update the satellite resource information corresponding to the link and calculate the link 、 and

[0155] 17: According to 、 and calculate

[0156] 18: Set the current satisfaction equal to the calculated

[0157] 19: else

[0158] 20: Set current satisfaction equal to −5 (resource shortage penalty)

[0159] 21: end if

[0160] 22: end if

[0161] 23: Calculate TD difference and update Q table:

[0162] 24: Get the next user service

[0163] 25: end for

[0164] 26: end for

[0165] 27: end for

[0166] 28: Output the links finally allocated to each service and calculate user perception indicators

[0167] In general, according to one embodiment of the present invention, the overall process of the resource scheduling method is as follows:

[0168] ① Read the initial environmental status of the satellite network, including satellite orbit elevation angle, satellite resource information, user service information, resource information of each node in the satellite network, gateway station and gateway station resource information;

[0169] ② Based on the initial environmental status, analyze the satellites that each user can access and output the satellite network topology;

[0170] ③ According to the satellite network topology, a link set is set for each service, which includes multiple different links that meet the service requirements. Each link can be used to transmit the service from the source node to the destination node;

[0171] ④Set the simulation duration of the satellite network ;

[0172] ⑤ Design reinforcement learning algorithm, including constructing initial Q table and parameter initialization, where all users’ long-term satisfaction in Q table is set to 0, and setting learning rate α, discount factor γ, exploration rate ;

[0173] ⑥ Execute the reinforcement learning algorithm to dynamically adjust the long-term user satisfaction of each business under different link selections through multiple rounds of iterations, and realize dynamic optimization of the link selection strategy.

[0174] ⑦ Select links to allocate to user services while maximizing overall long-term user satisfaction.

[0175] In order to verify the beneficial effects of the method of the present invention, the inventors conducted the following comparative experiments:

[0176] First, we conducted simulation experiments to analyze the impact of rate ratio, packet loss rate, and latency on the MOS value (a perceptual indicator, also known as long-term user satisfaction in reinforcement learning algorithms) of video services. The results include the following:

[0177] 1) See Figure 2 This is a diagram showing the statistical impact of packet loss rate on satisfaction. The vertical axis represents satisfaction, and the horizontal axis represents packet loss rate. As the packet loss rate increases, satisfaction decreases exponentially.

[0178] 2) See Figure 3 This diagram shows the combined impact of rate ratio and latency on satisfaction. The figure is a three-dimensional graph representing satisfaction, latency (in milliseconds), and rate ratio. As the rate ratio decreases and latency increases, satisfaction decreases.

[0179] 3) See Figure 4 This diagram shows the combined impact of packet loss rate and latency on satisfaction. The figure is a three-dimensional graph representing satisfaction, latency (in milliseconds), and packet loss rate. As latency and packet loss rate increase, satisfaction decreases.

[0180] 4) See Figure 5 This diagram shows the combined impact of rate ratio and packet loss rate on satisfaction. The figure is a three-dimensional graph representing satisfaction, rate ratio, and packet loss rate. As the rate ratio decreases and the packet loss rate increases, satisfaction decreases.

[0181] 5) See Figure 6 This diagram shows the combined impact of rate ratio, packet loss rate, and latency on satisfaction. The figure is a three-dimensional graph representing packet loss rate, latency (in milliseconds), and rate ratio. Satisfaction decreases as the rate ratio decreases, latency increases, and packet loss rate increases.

[0182] The above statistical results show that overall, higher satisfaction is achieved with higher rate ratios, lower packet loss rates, and lower latency. This means that the evaluation model of this invention comprehensively considers the overall mapping relationship between rate ratio, packet loss rate, and latency on user experience quality, enabling users to achieve a better quality of experience.

[0183] 2. Comparative analysis of the method of the present invention with the existing method

[0184] Existing methods include the following: Method 1, which schedules resources with the goal of maximizing throughput, and Method 2, which schedules resources with the goal of minimizing latency. When traffic is low, user satisfaction can improve by 10%. As traffic increases to near saturation, user satisfaction improves by 5% compared to Method 1 and by 20% compared to Method 2, demonstrating its potential and value in practical applications. Experimental results show that the method of the present invention has certain advantages over existing methods in terms of user satisfaction, resource utilization, and satellite access success rate. It can significantly improve user satisfaction while maintaining high resource utilization.

[0185] The existing methods are as follows:

[0186] Method 1: Yuan Y, Lei L, Vu TX, et al. Adapting to dynamic LEO-B5Gsystems: Meta-critic learning based efficient resource scheduling[J]. IEEETransactions on Wireless Communications, 2022, 21(11): 9582-9595.

[0187] Method 2: Song Y, Li

[0188] It should be noted that although the above describes the various steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0189] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0190] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove having instructions stored thereon, and any suitable combination thereof.

[0191] While various embodiments of the present invention have been described above, the above descriptions are intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A resource scheduling method for user satisfaction in satellite networks, characterized in that: Methods include: S1. Obtain the initial environmental state of the satellite network, which includes satellite resource information and service demand information of multiple users, and construct a satellite network topology structure; S2. According to the satellite network topology, a link set is set for each service, which includes multiple different links that meet the service requirements, and an action space for the service is constructed based on the link set, including multiple different link options; S3. Based on each service and its action space, an initial Q table is constructed to record the initial long-term user satisfaction of each service under different link selections in its action space; S4. A reinforcement learning algorithm is used to iterate the initial Q-table multiple times based on the initial environment state with the goal of maximizing long-term user satisfaction. The links allocated to each service are obtained while maximizing long-term user satisfaction. In each round, the selected link is determined based on the action space of the service. A preset reward function is used to feedback the current satisfaction based on the user's perceived network service quality indicators of the selected link, and the Q-table is updated based on the current satisfaction.

2. The method according to claim 1, characterized in that The service includes a video service. In S4, an evaluation model is used to evaluate the user's perception index of the network service quality of the selected link based on the mapping relationship between the network service quality parameters and the user experience quality; Among them, the perception index includes a first index and a second index. The first index is determined based on the first sub-mapping relationship between the packet loss rate of the selected link and the video quality, and the second index is determined based on the second sub-mapping relationship between the video playback time jointly fitted by the rate ratio and delay of the selected link and the video playback smoothness.

3. The method according to claim 2, characterized in that The evaluation model is as follows: ， in, represents the perceptual index of the calculation, represents the dynamic weight coefficient, Represents the first indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the packet loss rate, express The lower limit of represents the dynamic weight coefficient, represents the second indicator, , represents the scaling factor, represents the attenuation coefficient, Indicates the video playback duration of the combined rate ratio and delay fitting, represents the rate ratio, Indicates delay, express The lower limit of .

4. The method according to claim 1, wherein In S4, each round of iteration includes: Get the Q table and environment state. The initial Q table and initial environment state are used in the first round. After the first round, each round uses the updated Q table and updated environment state of the previous round. To maximize long-term user satisfaction, the Q table is updated multiple times based on the environment status and the services of multiple users. The Q table and environment status after the last update are used as the updated Q table and environment status for the current round. Each update process includes: Based on the action space of the service obtained at that time, a link is selected using the ε-greedy decay strategy; Based on the selected link, the preset reward function is used to calculate the current satisfaction of the corresponding user, and the environment state is updated to obtain the updated environment state; Get the latest Q table and calculate the TD difference based on the Q table and the current satisfaction of the corresponding user; The long-term user satisfaction corresponding to the link selection of the service in the corresponding action space in the Q-table is updated according to the TD difference to obtain the updated Q-table.

5. The method according to claim 4, characterized in that During each update, the current satisfaction level is determined by: Check whether the resources of the selected link meet the business requirements. If so, the current satisfaction is the user's perception index of the network service quality of the selected link. Otherwise, the current satisfaction is the preset penalty value.

6. The method according to claim 2, characterized in that The video playback duration is calculated as follows: ， in, 、 、 、 、 、 、 、 、 、 are fitting parameters.

7. The method according to claim 1, characterized in that In S2, the method of setting a link set for each service includes: For each user service, a corresponding initial link set is constructed, which includes links corresponding to all satellites visible to the user and which meet the needs of the user service; By using the preset link pre-screening mechanism, the links corresponding to satellites whose satellite communication quality is less than the preset quality threshold are removed to obtain the final link set.

8. The method according to claim 7, characterized in that The satellite communication quality is evaluated based on the satellite's elevation angle, signal strength and signal-to-noise ratio.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 8 by executing the executable instructions.

Images