Industrial wireless network resource scheduling method based on multi-weight quality evaluation model

Abstract

The invention discloses an industrial wireless network resource scheduling method based on a multi-weight quality evaluation model, and the method comprises the steps: collecting the related data of a recent network state through an internal node of a network by applying a sliding window mechanism, and constructing the multi-weight quality evaluation model to calculate the transmission quality of a time slot in a candidate pool; dynamically adjusting the weight by using a Pareto optimization and self-adaptive exploration Q learning algorithm; based on the transmission quality of the time slot output by the multi-weight quality evaluation model, performing action state management on the time slot resource of the node by combining a 6P protocol, including dynamically releasing and supplementing the time slot and maintaining and optimizing the time slot pool, and selecting the most suitable channel combination through a pseudo-random channel selection mechanism, so as to realize the optimal channel selection. And the node performs data transmission by using the selected channel and updates the scheduling table. According to the method, a multi-weight quality evaluation model and Pareto optimization are utilized, dynamic adjustment of time slots and channels is achieved, and high reliability and end-to-end low delay of wireless sensor node data transmission are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things resource scheduling, and in particular to an industrial wireless network resource scheduling method based on a multi-weight quality assessment model. Background Art

[0002] In the Industrial Internet of Things (IIoT) environment, communication between devices needs to be highly reliable, low-latency, and low-energy to support complex automation control and monitoring applications such as smart manufacturing and process control. The large scale and strong dynamics of networks in industrial environments pose great challenges to resource management. To address these challenges, the IETF 6TiSCH working group proposed the 6TiSCH network, a network architecture based on IEEE 802.15.4 TSCH (Time Slotted Channel Hopping) and IPv6, which aims to provide efficient multi-hop wireless network connections for low-power Internet of Things (IoT) devices.

[0003] The 6TiSCH network combines the time synchronization and channel hopping mechanism of TSCH with 6LoWPAN and RPL (IPv6 routing protocol) to provide a standardized scheduling mechanism for low-power, multi-hop networks. In the 6TiSCH network, a reliable and efficient scheduling mechanism is indispensable, which determines when devices communicate and directly affects the energy efficiency, latency and reliability of the network. The scheduling algorithm ensures that each node exchanges data with neighboring nodes at the appropriate time by reasonably allocating time slots and channels, maximizing resource utilization, reducing transmission conflicts and extending node battery life.

[0004] The 6TiSCH network uses TSCH technology to divide the communication time into time slots of fixed length and jump between multiple channels, forming the basic unit in scheduling - "cell". Each cell consists of a slot offset and a channel offset. The slot offset defines the transmission time of the node, and the channel offset is used to determine the frequency of data transmission. Although the existing scheduling scheme provides a lightweight scheduling method, it still has the following shortcomings: (1) Randomly select time slots in the candidate pool for scheduling. This randomness ignores the transmission performance differences of each time slot, resulting in poor network performance in high interference environments, high transmission failure rate, and large network delay. (2) Lack of intelligent evaluation of the transmission quality of time slot channel combinations. Different time slot channel combinations have significant differences in transmission quality, which leads to inefficient utilization of some time slots and increases the number of retransmissions and delays. Summary of the invention

[0005] In order to overcome the shortcomings of the existing methods, the purpose of the present invention is to provide an industrial wireless network resource scheduling method based on a multi-weight quality assessment model, combined with an improved Q learning algorithm, to optimize the time slot selection in the TSCH network, and solve the problem that the time slot selection in the existing resource scheduling method lacks pertinence and cannot effectively evaluate the transmission quality. This method uses edge computing to process complex network state analysis and policy optimization, reduces the computing pressure and energy consumption of sensor nodes, and achieves efficient resource scheduling. At the same time, using a multi-weight quality assessment model and Pareto optimization, dynamic adjustment of time slots and channels is achieved to ensure high reliability and end-to-end low latency of wireless sensor node data transmission in a complex and changeable industrial network environment. The present invention uses the 6TiSCH simulator to perform multi-faceted evaluation of quantitative and qualitative indicators, and verifies the significant advantages of the scheme in transmission efficiency, network adaptability and energy consumption.

[0006] The present invention is implemented by the following technical solution: an industrial wireless network resource scheduling method based on a multi-weight quality assessment model, comprising: Step S1A: The internal nodes of the network apply a sliding window mechanism to collect relevant data of the recent network status, including the expected number of transmissions, queue length, packet retransmission rate, and receiving forwarding distance, and perform data preprocessing to obtain the average value of the expected number of transmissions, the median value of the queue length, the truncated average value of the retransmission rate, and the receiving forwarding distance, and use them as inputs of the multi-weight quality assessment model to calculate the transmission quality of the time slots in the candidate pool; the multi-weight quality assessment model is expressed as: ; in, is the transmission quality of the time slot, The average expected number of transmissions , median queue length , retransmission rate truncated mean And receiving forwarding distance The weights are adjusted dynamically using the Pareto optimization and adaptive exploration Q learning algorithm; Step S2A: Based on the transmission quality of the time slot output by the multi-weight quality assessment model, the 6P protocol is combined to dynamically manage the time slot resources of the node, including dynamically releasing and replenishing time slots, maintaining and optimizing the time slot pool, and selecting the most suitable channel combination through a pseudo-random channel selection mechanism. The node uses the selected channel for data transmission and updates the scheduling table.

[0007] It is further preferred that the most suitable channel combination is selected through a pseudo-random channel selection mechanism, specifically: by combining the channel record table and the setting of the active channel set, all channels in the network are divided into available channels and an active channel set, wherein the active channel set has a channel record table that records the channels that have been transmitted in several time slots and their front and back adjacent channels, and the available channels are the remaining channels after the division; for multiple continuous channel blocks divided by the channels, the largest channel block is selected; and then a linear congruential formula is applied to generate random numbers, and the channel with the smallest probability of conflict and interference is selected from the largest channel block.

[0008] Further preferably, the node uses the selected channel for data transmission and updates the scheduling table, including: the node uses the selected channel for data transmission, and after the transmission is completed, stores the channel information used in the channel recorder for reference during subsequent channel selection; and through the 6P protocol, the node transmits the relevant data back to the parent node, updates the scheduling table, thereby further dynamically releasing and supplementing time slots, maintaining the dynamic update of the time slot candidate pool, and continuously optimizing the time slot selection strategy according to the new transmission data.

[0009] Furthermore, the dynamic adjustment of weights using the Pareto optimization and adaptive exploration Q learning algorithm includes the following steps: Step S1B: The edge routing node collects real-time status data of the network, including packet transmission rate PDR, traffic load TL, and delay; the real-time status data is used as the input of the Q learning algorithm, and the key parameters of Q learning are initialized according to the current network state, and a Q table is constructed; the Q table is used to store the Q value of each state-action pair, and the Q value will be used as a decision-making basis to guide the routing node on how to allocate resources and adjust the network strategy; a reward function is designed to quantify the performance of the network and support multi-objective optimization; by introducing Pareto optimization, the preference weight vector of each target in the reward function is determined to represent the priority of different optimization targets, to ensure that the priority of different targets is reasonably considered, and to achieve multi-objective optimization of the reward function; Step S2B: Update the value in the Q table according to the output of the reward function in step S1B; adopt an adaptive exploration strategy for the action selection control of each iteration after updating the Q value, balance between exploration and utilization by monitoring the network delay, packet transmission rate and traffic load fluctuations, dynamically adjust the exploration rate, and perform convergence tests on the updated iterations of the Q value to monitor whether the change in the Q value between two consecutive iterations is less than a given threshold.

[0010] The beneficial effect of the present invention is that compared with the prior art, the present invention optimizes the time slot selection in the TSCH network by adopting the edge computing idea, combining the multi-weight quality assessment model and the improved Q learning algorithm, and solves the problems of lack of pertinence, lack of transmission quality assessment and high node computing pressure in the existing scheduling method. A multi-weight quality assessment model based on edge computing is proposed, which comprehensively considers multiple parameters such as the expected number of transmissions, the number of retransmissions, the queue length and the time slot distance factor, and intelligently evaluates the transmission performance of each time slot channel combination. This method ensures that the node can give priority to the time slot with high transmission success rate and low delay for data packet transmission, while reducing the computational overhead of the node. In addition, by introducing the improved Q learning algorithm through the edge router, the system can dynamically adjust the weight parameters in the multi-weight quality assessment model using Pareto optimization according to the real-time changes of the network status, and adjust the exploration rate of Q learning through the adaptive exploration strategy, thereby realizing dynamic network adaptability. This mechanism enhances the system's responsiveness to network load and interference, ensures efficient and reliable wireless transmission, and meets the strict requirements for sensor node data interaction in the industrial Internet of Things environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A structural diagram of an industrial wireless network resource scheduling method based on a multi-weight quality assessment model of the present invention; Figure 2 A time slot-channel selection structure diagram for implementing the edge computing driven multi-weight quality assessment model method of the present invention; Figure 3 The structure diagram of the Q learning algorithm for Pareto optimization and adaptive exploration of the present invention; Figure 4 This is a diagram of the pseudo-random channel selection structure of the present invention. DETAILED DESCRIPTION

[0012] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0013] like Figure 1As shown, this embodiment provides an industrial wireless network resource scheduling method based on a multi-weight quality assessment model, which aims to solve the randomness of the existing resource scheduling time slot selection and the lack of time slot quality assessment problems, and at the same time, ensure the high reliability of wireless sensor node data transmission and the low latency of end-to-end data transmission in a complex and changeable industrial network environment. The present invention proposes a unique multi-weight quality assessment model: in the process of selecting and evaluating the time slot-channel, a multi-weight index calculation scheme is invented, and the relevant data of the recent network status is used as the data input and preprocessed; after preprocessing, the sliding window mechanism is applied to select the performance index that best reflects the recent candidate time slot of the current node; the historical data is processed by arithmetic mean, median, truncated mean and other methods to obtain robust statistical characteristics, and the receiving and forwarding distance is comprehensively considered to form the calculation input of the multi-weight quality assessment model, so as to obtain the time slot with the highest quality score in the candidate pool, and according to the output, the node is dynamically released and supplemented with the 6P protocol, and the time slot pool is maintained and dynamically optimized. Next, a pseudo-random channel selection method is used to select a low-interference, low-conflict channel to form a time slot channel combination for task transmission.

[0014] In the dynamic adjustment process of the weight parameters of the multi-weight quality assessment model: the Q learning algorithm of Pareto optimization and adaptive exploration was invented. By collecting basic performance parameters in the network at the edge router, the dynamic adjustment of the reward function by Pareto optimization and the balance between exploration and utilization of the adaptive exploration strategy are combined to evaluate the overall state of the network, and the weight parameters of the multi-weight quality assessment model that best suits the current network state are calculated. These weight parameters will be fed back to the internal nodes in the network through specific data packets (EB packets) to assist them in time slot channel selection.

[0015] In one embodiment, an industrial wireless network resource scheduling method based on a multi-weight quality assessment model includes using the multi-weight quality assessment model to perform time slot-channel quality assessment and selection and using a Q-learning algorithm of Pareto optimization and adaptive exploration to dynamically adjust weights.

[0016] The multi-weight quality assessment model performs time slot-channel quality assessment and selection, including the following steps: Step S1A: The internal nodes of the network apply a sliding window mechanism to collect relevant data on the recent network status, including the expected number of transmissions, queue length, packet retransmission rate, and receiving forwarding distance, and perform data preprocessing to reduce computing overhead while retaining the key features of the data, and obtain the average value of the expected number of transmissions, the median value of the queue length, the truncated average value of the retransmission rate, and the receiving forwarding distance, and use them as inputs to the multi-weight quality assessment model to calculate the transmission quality of the time slots in the candidate pool. These data are used for the calculation of the multi-weight quality assessment model to ensure that the multi-weight quality assessment model can accurately assess the transmission quality of each time slot channel combination and provide high-quality decision support for resource scheduling.

[0017] Further, in an embodiment, the process is described as: The nodes inside the network collect relevant transmission performance data (expected number of transmissions, queue length, i.e., number of packets in the transmission queue, packet retransmission rate) within the recent sliding window time, regard it as the first preprocessing, and use the processed data for the calculation of the multi-weight quality assessment model. In addition, the receiving and forwarding distance between the node receiving time slot and the forwarding time slot is considered. The above data are combined to ensure that the multi-weight quality assessment model can evaluate the true quality of each time slot and provide high-quality decision support for scheduling, which is expressed as: (1); (2); (3); (4); in, It represents the expected number of transmissions required to transmit a data packet, that is, the expected number of transmissions, where and They are the success rates of forward and reverse transmissions, respectively. The lower the ETX value, the better the link quality and the fewer retransmissions required for data transmission. Therefore, ETX is an important indicator reflecting the stability of time slot transmission; The node periodically monitors the number of packets in its transmission queue. is the number of packets currently waiting to be transmitted, i.e., the queue length. The queue length is the result of direct counting and indicates the current load; The node counts the packet retransmission rate in the same time slot. Indicates the number of retransmissions, that is, the number of times a data packet is resent in the same time slot in order to successfully transmit it. Indicates the total number of transmission attempts, including all initial transmissions and all retransmissions. Indicates the distance between a time slot and the receiving time slot, that is, the receiving forwarding distance. The shorter the distance, the faster the time slot can be used to transmit data, reducing waiting time and delay. Therefore, SDF is an important parameter for optimizing delay. Weight in the multi-weight quality assessment model: For delay-sensitive scenarios, increase the weight and give priority to time slots with good time slot proximity for transmission, thereby reducing transmission delay. represents the slot position in the candidate slot pool, Indicates the time slot position of the node signal reception, Indicates the length of the slot frame.

[0018] The data obtained above is processed for the second time to generate the data input for the function in the multi-weight quality assessment model, and the arithmetic mean of the data collected by the sliding window is used to obtain the average value of the expected number of transmissions. , smooth the fluctuation of transmission success rate and avoid misleading due to short-term link fluctuations; use the median of the data collected by the sliding window to obtain the median value of the queue length , to avoid the impact of occasional load peaks on model evaluation; the truncated average of the data collected by the sliding window is used to obtain the truncated average of the retransmission rate , to avoid the impact of extreme retransmission times on the multi-weight quality assessment model and enhance the stability of the multi-weight quality assessment model. It can be expressed as: (5); (6); (7); in, is the size of the sliding window (e.g. the last 10 cycles), The retransmission rate is truncated data, removing extreme values ​​in the window (such as abnormally large or small values). For the The transmission Value, QL i For the The queue length at the time of transmission, () is the median function, which means taking the median of a set of values. is the number of data points remaining after truncation (after removing the highest and lowest 10%), For the The retransmission rate of the transmission; The processed data above is used as the input of the multi-weight quality assessment model to calculate the transmission quality of the time slots in the candidate pool, which is expressed as: (8); in, is the transmission quality of the time slot; They are the weights of the average expected number of transmissions, the median value of the queue length, the truncated average value of the retransmission rate, and the receiving and forwarding distance. The Q-learning algorithm of Pareto optimization and adaptive exploration is used to dynamically adjust the weights to balance the impact of different indicators. In summary, the internal nodes can obtain the transmission quality of the relevant time slots through a processing flow with less computational effort, and further combine this with the 6P protocol to dynamically manage the time slot resources of the nodes, including dynamically releasing and replenishing time slots, maintaining and optimizing the time slot pool, thereby improving network transmission efficiency. Through the pseudo-random channel selection mechanism, low-interference channels are selected and combined with the previously output time slots to ensure the stability and reliability of transmission. Furthermore, step S1A has selected time slots with higher transmission quality. For channel selection, a pseudo-random channel selection mechanism is used to select low-interference, low-conflict channels to form a time slot channel combination for task transmission.

[0019] Step S2A: Based on the transmission quality of the time slot output by the multi-weight quality assessment model, the 6P protocol is combined to dynamically manage the time slot resources of the node, including dynamically releasing and replenishing time slots, maintaining and optimizing the time slot pool, and selecting the most suitable channel combination through a pseudo-random channel selection mechanism. The node uses the selected channel for data transmission and updates the scheduling table.

[0020] Specifically, the most suitable channel combination is selected through the pseudo-random channel selection mechanism, specifically: by combining the channel record table and the setting of the active channel set, all channels in the network are divided into available channels and active channel sets, wherein the active channel set has a channel record table that records the channels that have been transmitted in several time slots and their adjacent channels before and after, and the available channels are the remaining channels after the division; for multiple continuous channel blocks divided by the channels, the largest channel block is selected; and then a linear congruential formula is applied to generate random numbers, and the channel with the smallest probability of conflict and interference is selected from the largest channel block.

[0021] More specifically, step S2A includes: Step S2A1: First, there is a set of available channels in the system to form a channel candidate pool, denoted as , They represent the 1st, 2nd, ..., nth channels respectively. For example, the 16 channels available in the network are Each node maintains a channel record table , the storage space of this structure is not large, and it is only used to record the channels used by the node in several past transmissions. At this time, according to the channel record table The system will set an active channel set for the historical channel records in , including the most recently used channels and their adjacent channels, to avoid these channels being reused in a short period of time. The formula for setting the isolation zone is: (9); in, represents the r-1th channel, represents the rth channel, represents the r+1th channel; The active channel set is set to reduce interference problems caused by frequent or adjacent use of channels. Next, the system generates a new pool of available channel candidates. , obtained by excluding the active channel set channels from the channel candidate pool: (10); Then, the available channel candidate pool Divide into several channel blocks according to the continuity of the channel , channel block number i={1,2,…,m}, m is the number of channel blocks, where each channel block Contains several continuous channels. All channel blocks can be expressed as: (11); Among these channel blocks, the system selects the largest channel block containing the most channels ,The purpose of selecting the largest channel block is to ensure that the channels are distributed as evenly as possible within the available channel range, reducing the possibility of interference: (12); Step S2A2: In order to select a specific channel in the largest channel block, a linear congruential formula is used and based on the seed value Generates a random number to determine the channel. Seed value By function Generate, where the function The current timeslot number is used as input to ensure the randomness and dynamics of channel selection: (13); (14); Wherein, time slot number: the unique number of the current time slot, which is used to provide the time sequence information of each transmission and ensure that each time slot has a unique seed value; To calculate the output random number, The random number generated last time, the random number initialized here ; is the multiplier to ensure the distribution characteristics of random numbers, c is the increment, and M is the modulus. The hash function is used to convert the input (time slot number) into a numerical value with good randomness. Using the hash function to generate the seed value can ensure the randomness of the seed value, while avoiding the periodicity and repetitiveness of pseudo-random numbers that may be caused by directly using the number. Random numbers generated using the linear congruential formula Used to index in the largest channel block, the final selected channel is: (15); (16); Where k is the channel index, For the selected channel, The largest channel block channels.

[0022] Step S2A3: After the channel is selected After that, the node will use the channel for data transmission in the current time slot t, and after the transmission is completed, the channel information used will be stored in the channel recorder for reference in subsequent channel selection. And through the 6P protocol, the node will return the relevant data to the parent node and update the scheduling table, so as to further dynamically release and supplement time slots, maintain the dynamic update of the time slot candidate pool, and continuously optimize the time slot selection strategy according to the new transmission data. The update of the channel recorder follows the following rules: (17); in, is the updated channel record table, The original channel.

[0023] The pseudo-random channel selection mechanism reduces channel usage conflicts and channel interference by avoiding the recent use of channels in the schedule as much as possible. It combines the setting of the active channel set and the linear congruential formula, and adopts the strategy of dividing the channels into blocks. First, there is a set of available channels in the system that constitute the channel candidate pool, such as the 16 channels available in the network. Furthermore, based on the historical channel records, the system will set an active channel set, including the most recently used channels and their adjacent channels, to avoid these channels being reused in a short period of time. The available channel candidate pool is then divided into several channel blocks according to the continuity of the channels, and the linear congruential formula is used to randomly select channels in the largest channel block, so as to ensure that the probability of conflict and interference caused by each channel selection is greatly reduced, thereby ensuring efficient, safe, and interference-resistant time slot channel allocation in complex industrial wireless network environments.

[0024] Furthermore, the weights are dynamically adjusted using the Pareto optimization and adaptive exploration Q learning algorithm, including the following steps: Step S1B: The edge routing node collects real-time status data of the network, including packet transmission rate PDR, traffic load TL, delay and other information; these real-time status data are used as inputs to the Q learning algorithm, and the key parameters of Q learning are initialized according to the current state of the network, and the Q table structure is constructed. The Q table is used to store the expected return (Q value) of each state-action pair. These Q values ​​will be used as a basis for decision-making to guide routing nodes on how to allocate resources and adjust network strategies. In addition, a reward function needs to be designed, which will be used to quantify the performance of the network and support multi-objective optimization. By introducing Pareto optimization, the preference weight vector of each objective in the reward function is determined to represent the priority of different optimization objectives, ensuring that the priority of different objectives is reasonably considered, and realizing multi-objective optimization of the reward function.

[0025] Step S2B: Update the values ​​in the Q table based on the output of the reward function in step S1B. The Q value reflects the expected return of selecting an action under a specific network state. The action selection control for each iteration after updating the Q value adopts an adaptive exploration strategy, which dynamically adjusts the exploration rate by balancing exploration and utilization by monitoring the network's state fluctuations such as latency, packet transmission rate, and traffic load. The greater the network fluctuation, the higher the exploration rate. The larger the value, the higher the exploration frequency; when the network is stable, the exploration rate is reduced , make more use of the existing optimal strategy, and perform convergence test on the updated iteration of Q value, monitoring whether the change of Q value between two consecutive iterations is less than a given threshold.

[0026] More specifically, step S1B includes: Step S1B1: Initialize the Q table to store the Q value of each state and action combination, set the learning rate α, discount factor γ, exploration rate and other parameters and initialize the Q table.

[0027] Step S1B2: Collect network status and pre-process data according to the edge router to provide basic input for Q learning. The packet transmission rate is the core indicator for evaluating the quality of network links. It is calculated by monitoring the number of successfully sent and received packets. The traffic load is used to reflect the buffer (queue) utilization of the node and indicate the load status of the network. By monitoring the number of packets in the node queue, the system can determine the network congestion; the delay reflects the time from sending to receiving data, which is a key indicator for evaluating network transmission efficiency. It is particularly suitable for time-sensitive application scenarios. Each node records the timestamp when sending a data packet, and calculates the total delay of the transmission when receiving the confirmation packet. The average delay of several past data transmission cycles is used to calculate the delay performance of the time slot to ensure its effectiveness in time-sensitive applications. It is expressed as: (18); (19); (20); in: is the packet transmission rate of the jth node, using a sliding window method, such as the average packet transmission rate of the past sliding window size = 10 time slots; Indicates the traffic load of the jth node. When the traffic load reaches or exceeds a certain threshold (for example, 80% of the maximum capacity of the queue), the node sends a Boolean alarm signal = +1 indicates that the current load is high; otherwise, send = 0, indicating that the load is normal. This solution reduces the communication overhead of scheduled monitoring and triggers an alarm only when the node load is close to saturation; represents the delay of the jth node and uses the average delay of several past data transmission cycles to calculate the delay performance of the time slot to ensure effectiveness in time-sensitive applications.

[0028] Based on the three selected key indicators: packet transmission rate, traffic load and delay, a comprehensive reward function can be constructed. Further, by combining the design of the Pareto optimization method, the multi-objective optimization idea is introduced to dynamically adjust the reward function to improve the adaptability to complex scenarios and the ability to balance multiple objectives. First, the input of the reward function includes the current network status , the action selected by the current strategy , the change in the target value, and the dynamic or fixed preference weight vector , used to indicate the priority of each optimization objective. The calculation process is centered on constructing a multi-dimensional reward vector. , calculate its multi-objective performance and generate a reward vector , where each dimension represents the optimization effect of an objective. For example, the change in packet transmission rate , the change in traffic load , the change in delay The reward vectors of all actions constitute the solution set . (twenty one); (twenty two); (twenty three); (twenty four); in, is the PDR value at the current moment, is the PDR value at the previous moment, , For the current moment value, is the delay of the previous moment, is the delay at the current time, is the reward vector solution set, , , are the target rewards represented by the 1st, 2nd, and nth components respectively.

[0029] Subsequently, the solution set is screened by the Pareto optimization method, the inferior solutions dominated by other solutions are eliminated, and the non-dominated solutions that achieve a balance between all objectives are retained to form the Pareto frontier. The criterion for determining a non-dominated solution is that if a solution If the solution is not completely superior to other solutions in all objectives, it is a non-dominated solution. Then, the final solution is selected from the Pareto frontier, which can be based on random selection, preference weight selection, or ideal solution distance selection. The preference weight selection strategy calculates the weighted score of each Pareto solution. Then choose the solution with the highest score as the optimal solution ; The ideal solution distance is selected by calculating the distance between each Pareto solution and the ideal solution Euclidean distance , select the solution with the smallest distance. The output result is the selected reward vector and its corresponding actions , this reward vector achieves the optimal balance among multiple objectives.

[0030] (25); (26); (27); (28); in, For a solution, For the ideal solution, is the preference weight of the indicator, is the Pareto solution of the indicator; The selected reward vector is used to update the Q-value table after being scalarized. The scalarization method uses a scoring method based on the ideal solution distance: (29); in, is the reward value of the action at time t.

[0031] Step S2B is specifically as follows: Step S2B1: Based on the setting of multi-objective optimization of reward function in step S1B, the reward value is obtained according to formulas (25)-(29): , substitute into the following formula to get the update of Q value: (30); in: is the state at time t Next, take action Q value; is the reward value of the action at time t, Indicates in status Take action The Q value (expected return) of. Indicates action The next state after execution. Represents the state at time t The best possible action Q value; is the learning rate, which is used to balance between new and old information. is a discount factor, which is used to indicate the influence of future rewards on the current action. Through the above update, the Q value can reflect the comprehensive contribution of the current action to the multi-objective optimization.

[0032] Furthermore, regarding the action control strategy of the Q-learning iteration process, an adaptive exploration strategy is adopted. According to the network status data collected by the edge router, the exploration rate is dynamically set in consideration of the abnormal fluctuation of the status. According to the fluctuation of the network status, the exploration rate is dynamically adjusted, so as to achieve an effective balance between exploration and utilization in the Q-learning algorithm: when the network fluctuates greatly, the exploration rate is increased. When the network is relatively stable, the exploration rate is reduced and the current strategy is used more.

[0033] (31); (32); in, is the exploration rate at the Tth iteration, which determines the trade-off between exploration and exploitation of the Q-learning algorithm; is the highest exploration rate, is the minimum exploration rate, is the decay rate, which controls how fast the exploration rate decreases; is the delay at time t, is the delay at time t-1, is the packet transmission rate at time t, is the data packet transmission rate at time t-1, is the traffic load at time t, is the traffic load at time t-1, , are the weight parameters for latency, packet transmission rate, and traffic load fluctuation, respectively, controlling the contribution of each network metric to the fluctuation metric. The maximum exploration rate indicates that the possibility of exploration should be increased when the network state fluctuates greatly; the minimum exploration rate ensures that a certain degree of exploration is retained at all times, even if the network state is relatively stable. Function It is used to measure the fluctuation of the network state at time t, and the result is used to dynamically adjust the exploration rate.

[0034] : represents the current time t and the previous time The change of network delay can reflect the real-time performance of the network.

[0035] : Indicates the data packet transmission rate at the current moment and the previous moment. Its change reflects the reliability of the network.

[0036] Indicates the network load at the current moment and the previous moment. The fluctuation of network load reflects the change in the amount of data transmitted by the node.

[0037] Step S2B2: For the above dynamically adjusted exploration rate, select an action in the action space, that is, select a weighted combination, according to the selected action Calculate the reward function to evaluate the effect of the action, which is used to update the Q value in the Q table. And gradually optimize the weights of the multi-weight quality assessment model; by adjusting the exploration rate , and make convergence judgments to ensure that the algorithm finds the optimal weight combination in the process of continuous learning . express The optimal value of express The optimal value of express The optimal value of express The optimal value of .

[0038] Action space A: Action space A is defined as a combination of multiple weight parameters: (33); Generate a random number when the action is selected If ζ< , then explore (randomly select an action combination). Otherwise, exploit (select the action combination with the highest current Q value). The action selection formula for exploration and exploitation is: (34); (35); in, For random actions, is the optimal action, i.e. the action corresponding to the maximum Q value, is the state at time t-1 Next, take action The Q value of is a given threshold. When the change in Q value between two consecutive iterations is less than a given threshold, When , it can be considered to have reached a certain stable state, that is, convergence. This means that the strategy has become stable.

[0039] like Figure 2As shown, in this embodiment, the key performance parameters of each candidate time slot are first collected at the internal node, including the expected number of transmissions, the transmission queue length, the packet retransmission rate, etc., to help evaluate the quality of the time slot in the future. These performance parameters are the basic data for the subsequent multi-weight quality evaluation, which directly determines the quality of the transmission of the time slot. Next, the collected performance parameter data is preprocessed to remove outliers and smooth the data, so as to ensure that the subsequent evaluation results are more accurate and robust, eliminate the impact of short-term network fluctuations, and improve the reliability of evaluating the performance of the time slot. Then, the above data is processed using relevant statistical data processing methods. For each candidate time slot, the distance between it and the receiving unit is directly calculated. The receiving forwarding distance is used as an important parameter to evaluate the spatial distance between the time slot and the receiving unit. Generally speaking, when the distance is far, there is less interference. In the next step, it is necessary to assign weights obtained by the Q learning algorithm of Pareto optimization and adaptive exploration to each performance indicator, substitute the preprocessed performance parameters and the corresponding weights into the quality evaluation function, and calculate the comprehensive quality score of each candidate time slot. This quality evaluation function is the core of the entire solution, which uses different performance indicators to comprehensively calculate the final score of each time slot. The quality of the time slots in the candidate pool is detected based on this score. If the quality is abnormal, such as a significant decrease in quality or frequent changes, the time slots in the candidate pool need to be adjusted. By dynamically adjusting the candidate pool, the overall quality of the candidate time slots can be maintained at a high level. Based on the quality detection of the candidate pool, the time slots in the candidate pool are maintained, including replacing low-quality time slots or adding new high-quality time slots to ensure the quality of the entire candidate pool, thereby ensuring the reliability and stability of the time slots in subsequent scheduling. Finally, according to the comprehensive score calculated by the quality evaluation function, the time slot with the highest score is selected as the priority transmission time slot. This process ensures that under the current network state, the selected time slot can optimally meet the transmission requirements, with a higher transmission success rate and lower latency. For example, node A needs to select the best time slot from three candidate time slots S1, S2, and S3 for data transmission. First, node A collects the transmission performance parameters of each time slot, including the expected number of transmissions, the length of the transmission queue, the packet retransmission rate, and the receiving and forwarding distance. In order to ensure the accuracy and robustness of the data, node A preprocesses the collected data to remove outliers and eliminate the impact of accidental fluctuations. After the preprocessing is completed, node A uses different methods to process the expected number of transmissions, transmission queue length, and packet retransmission rate in the performance indicators to obtain more robust data. For the expected number of transmissions (ETX), node A uses the arithmetic mean method to calculate the average value using the data of the last five times. For example, for time slot S1, the data of the last five times of ETX are [1.1, 1.2, 1.3, 1.2, 1.1], and the average value is 1.18.For the transmission queue length, node A uses the median for processing. For example, for time slot S1, the queue length data for the last five times is [3, 4, 3, 5, 3], and the median is 3. For the packet retransmission rate, node A uses the truncated average method, that is, the remaining values ​​are averaged after removing the highest and lowest values. For example, for time slot S1, the data of the retransmission rate for the last five times is [0.05, 0.04, 0.06, 0.05, 0.04]. After removing the highest 0.06 and the lowest 0.04, the remaining data is [0.05, 0.05, 0.04], and its truncated average is 0.047. At the same time, for the receive forwarding distance (SDF), node A directly uses the logical distance between the current candidate time slot and the receiving unit. For example, the SDF of time slot S1 is 2, indicating that the logical distance between it and the RX receiving unit is 2 channels. Next, node A determines the weights of each indicator. The weights of the four indicators are assumed to be: w1=0.3, w2=0.2, w3=0.2, w4=0.3, corresponding to ETX, transmission queue length, retransmission rate and receiving forwarding distance, respectively. These weights are used to reflect the importance of different indicators in comprehensive quality evaluation. Node A then substitutes the preprocessed performance parameters and weights into the quality evaluation function to calculate the comprehensive quality score of each time slot. For example, for time slot S1, the score is:. During the evaluation process, node A also needs to detect and maintain the quality of the time slots in the candidate pool. If it detects that the quality evaluation results of some time slots are continuously below a certain threshold, the system will perform candidate pool quality maintenance, such as replacing low-quality time slots and adding new high-quality time slots to maintain the overall quality of the candidate pool. Finally, node A selects the time slot with the highest score as the priority time slot for data transmission based on the comprehensive score of each time slot. In this example, the highest score is time slot S3, Therefore, node A selects time slot S3 for data transmission. The entire process ensures that node A can efficiently select the best time slot for data transmission in a complex industrial network environment through steps such as data collection, preprocessing, selection of different data processing methods (average, median, truncated average), calculation of receiving and forwarding distance, weight allocation, quality assessment, and candidate pool maintenance, achieving high efficiency, reliability, and adaptability to dynamic changes.

[0040] like Figure 3As shown, in one embodiment, the entire process of the Q learning algorithm of Pareto optimization and adaptive exploration starts when the node receives the transmission task, and the scheduling mechanism is started by collecting the TSCH network performance parameters. These network performance parameters include network delay: reflecting the time for data packets to be transmitted in the network; data packet transmission rate: measuring the proportion of successfully transmitted data packets to the total transmitted data packets, reflecting the stability of the transmission link; network load: reflecting the changes in the communication volume between nodes in the network. These data are collected by the edge router using a sliding window mechanism to accumulate historical data and monitor changes in network status. The collected data is preprocessed to ensure the accuracy and stability of the data by filtering out outliers and using methods such as truncated averaging. The processed sliding window data provides a continuous performance trend for the model to ensure that the scheduling decision is based on the actual network status. By processing these complex computing tasks on the edge side, the computing pressure of the sensor node is greatly reduced, saving energy consumption, making the network node more suitable for resource-constrained industrial Internet of Things environments.

[0041] Based on the collected data, the edge router initializes the Q table, state space, and key parameters. The Q table is used to record the value estimate of each state-action pair to provide a basis for subsequent action selection. The initial Q value is usually set to a small random value to allow more exploration in the initial stage. When selecting actions, we adopted an adaptive exploration strategy to balance between exploration and utilization, and dynamically adjust the balance between exploration and utilization under different network conditions to ensure efficient use of network resources while minimizing energy consumption and latency. By monitoring network state fluctuations such as latency, packet transmission rate, and load, the ε value is dynamically adjusted. The greater the network fluctuation, the larger the ε value and the increase in the exploration frequency; when the network is stable, the ε value is reduced to make more use of the existing optimal strategy. Action space A: Define action space A as a combination of multiple weight parameters: Generate a random number If ζ< , then explore (randomly select an action combination). Otherwise, exploit (select the action combination with the highest current Q value). The action selection for exploration and exploitation adopts formula (31)-formula (34).

[0042] In order to optimize the scheduling strategy, we use Pareto optimization to dynamically adjust the weights in the reward function. Specifically, the edge router dynamically adjusts the priorities of these goals according to the current network latency, packet transmission rate, and load. For example, in high-load situations, latency is prioritized, and when node energy consumption is high, energy consumption is prioritized to achieve the best balance in different network environments. First, the reward function is designed as a multidimensional reward vector to retain the independence of each indicator. The constructed reward vector is = , , the reward vectors of all actions constitute the solution set Then, the solution set is screened by the Pareto optimization method, the inferior solutions dominated by other solutions are eliminated, and the non-dominated solutions that achieve a balance between all objectives are retained to form the Pareto frontier. The criterion for determining a non-dominated solution is that if a solution If a solution is not completely superior to other solutions in all objectives, it is a non-dominated solution. Next, the final solution is selected from the Pareto frontier, which can be based on strategies such as random selection, preference weighted selection, or ideal solution distance selection. The weighted selection strategy calculates the weighted score of each Pareto solution: , and then select the solution with the highest score as the optimal solution: ; The ideal solution distance is selected by calculating the distance between each Pareto solution and the ideal solution The Euclidean distance selects the solution with the smallest distance. The output result is the selected reward vector and its corresponding actions , the reward vector achieves the optimal balance among multiple objectives. The selected reward is used to update the Q-value table after scalarization. The scalarization method uses a scoring method based on the ideal solution distance. Finally, the Q-value update follows the classic Q-learning formula, that is, formula (30). Through the above update, the Q-value can reflect the comprehensive contribution of the current action to the multi-objective optimization.

[0043] The key to the whole process is to adopt the idea of ​​edge computing, concentrate the complex network status analysis and optimization calculations on the edge router for processing, and the sensor nodes only need to perform simple local calculations, thereby effectively reducing the computational burden and energy consumption of the nodes. Through the multi-weight quality assessment model driven by the edge router and the improved Q-learning algorithm, changes in network performance (such as delay, packet transmission rate and load) can be monitored in real time and used to dynamically optimize the selection of time slot-channel combinations and the maintenance of the candidate pool. The edge router makes the optimal scheduling decision based on this comprehensive information and then sends the result to each node for execution. The sliding window mechanism is used to collect and pre-process network status data in real time, so that the edge router can efficiently evaluate the network status and make intelligent decisions. Therefore, the sensor node only needs to operate according to the optimal strategy provided by the edge router, ensuring that the system can maintain low latency, high reliability and low energy consumption in a dynamic environment. For channel allocation after time slot selection, this scheme adopts a random channel selection method to maintain the security and anti-interference ability of the TSCH network. In this way, the channel selection is not fixed, which effectively prevents channel monitoring and attacks, and further improves the security and robustness of the network. At the same time, through the 6P protocol to dynamically manage time slot resources, the edge router continuously optimizes the maintenance of the candidate pool. Combined with the multi-weight quality assessment model, the system can adaptively improve transmission performance and achieve efficient and flexible resource scheduling.

[0044] Figure 4 It shows how the slot offset and channel offset affect the final channel selection.

[0045] Currently, node A records its most recently used channel at time t as These channels have been used recently for data transmission. To reduce possible interference, these channels will not be used for this selection. Based on the historical records in the recorder, node A uses the recently used channels and adjacent channels as the recently used area: Recently used area includes channels . Exclude the recently used area channels from the channel candidate pool to obtain the remaining available channel candidate pool: ; The available channel candidate pool is further divided into several channel blocks according to continuity: ; Compare the sizes of each channel block and select the channel block with the largest number of channels: Maximum channel block Contains 10 channels, which is the block with the largest number of channels. Next, use the current time slot number (assuming the time slot number is Generate a seed value). Assume that the hash function converts the slot number 42 into a seed value .

[0046] Select LCG parameters: a=1664525, c=1013904223, m=2 32 =4294967296; Generate the next pseudo-random number using the LCG formula: ; Now we need to select a channel in the largest channel block. The largest channel block contains 10 channels: Use the generated random number modulo the channel block size to get the index of the channel: . According to the index , select channels in the largest channel block: Therefore, the final selected channel is Node A uses the selected channel Perform data transmission. After the transmission is completed, update the channel recorder and record the latest used channel in it: ; The present invention proposes an industrial wireless network resource scheduling method based on a multi-weight quality assessment model, which effectively solves the problems of randomness of time slot selection and lack of transmission quality assessment in existing resource scheduling, and at the same time ensures high reliability of data transmission of wireless sensor nodes and low end-to-end latency in complex and changeable industrial network environments. First, each sensor node collects the performance data of the time slots in the candidate pool through a sliding window mechanism, and performs reasonable preprocessing to ensure that the data input into the multi-weight quality assessment model is accurate and robust. In order to reduce the computing pressure of the node, the edge router undertakes the complex quality assessment calculation task. The edge router uses the multi-weight quality assessment model to comprehensively score each time slot and generates dynamic weight parameters suitable for the current network environment. This method not only improves the accuracy of the evaluation, but also significantly saves the computing resources and energy consumption of the nodes inside the network, and extends the working life of the nodes. Then, the edge router feeds back the evaluation results and weight parameters to each node, and the node performs dynamic management of the time slot pool in combination with the 6P protocol, including releasing and replenishing time slots, to ensure that resources are optimally utilized. Subsequently, the node selects the most appropriate time slot for data transmission based on the output of the multi-weight quality assessment model, and selects the optimal channel in combination with the TSCH channel frequency hopping mechanism, further improving the stability and reliability of network transmission. By using edge computing to concentrate complex computing tasks on edge routers, the present invention effectively reduces the computing burden and energy consumption loss of sensor nodes, while ensuring the efficiency and reliability of transmission tasks in complex network environments, meeting the requirements for efficiency and stability of data transmission in industrial Internet of Things environments.

[0047] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, some details in the embodiments should not be construed as limiting the present invention, and the present invention shall be protected by the scope defined by the attached claims.

Claims

1. An industrial wireless network resource scheduling method based on a multi-weight quality assessment model, characterized in that: include: Step S1A: The internal nodes of the network apply a sliding window mechanism to collect relevant data of the recent network status, including the expected number of transmissions, queue length, packet retransmission rate, and receiving forwarding distance, and perform data preprocessing to obtain the average value of the expected number of transmissions, the median value of the queue length, the truncated average value of the retransmission rate, and the receiving forwarding distance, and use them as inputs of the multi-weight quality assessment model to calculate the transmission quality of the time slots in the candidate pool; the multi-weight quality assessment model is expressed as: ； in, is the transmission quality of the time slot, The average expected number of transmissions , median queue length , retransmission rate truncated mean And receiving forwarding distance The weights are adjusted dynamically using the Pareto optimization and adaptive exploration Q learning algorithm; Step S2A: Based on the transmission quality of the time slot output by the multi-weight quality assessment model, the 6P protocol is combined to dynamically manage the time slot resources of the node, including dynamically releasing and replenishing time slots, maintaining and optimizing the time slot pool, and selecting the most suitable channel combination through a pseudo-random channel selection mechanism. The node uses the selected channel for data transmission and updates the scheduling table.

2. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model as claimed in claim 1, characterized in that: The most suitable channel combination is selected by the pseudo-random channel selection mechanism, specifically: by combining the channel record table and the setting of the active channel set, all channels in the network are divided into available channels and active channel sets, wherein the active channel set is the channel that has been transmitted in several time slots recorded in the channel record table and its adjacent channels before and after, and the available channel is the remaining channel after the division; for multiple continuous channel blocks divided by the channel, the largest channel block is selected; and then a linear congruential formula is applied to generate random numbers, and a channel with the smallest probability of conflict and interference is selected from the largest channel block.

3. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model according to claim 1, characterized in that: The node uses the selected channel for data transmission and updates the scheduling table, including: the node uses the selected channel for data transmission, and after the transmission is completed, stores the used channel information in the channel recorder for reference during subsequent channel selection; and through the 6P protocol, the node transmits the relevant data back to the parent node, updates the scheduling table, thereby further dynamically releasing and supplementing time slots, maintaining the dynamic update of the time slot candidate pool, and continuously optimizing the time slot selection strategy according to the new transmission data.

4. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model according to claim 1, characterized in that: The Q learning algorithm using Pareto optimization and adaptive exploration is used to dynamically adjust the weights, including the following steps: Step S1B: The edge routing node collects real-time status data of the network, including packet transmission rate PDR, traffic load TL, and delay; the real-time status data is used as the input of the Q learning algorithm, and the key parameters of Q learning are initialized according to the current network state, and a Q table is constructed; the Q table is used to store the Q value of each state-action pair, and the Q value will be used as a decision-making basis to guide the routing node on how to allocate resources and adjust the network strategy; a reward function is designed to quantify the performance of the network and support multi-objective optimization; by introducing Pareto optimization, the preference weight vector of each target in the reward function is determined to represent the priority of different optimization targets, to ensure that the priority of different targets is reasonably considered, and to achieve multi-objective optimization of the reward function; Step S2B: Update the value in the Q table according to the output of the reward function in step S1B; adopt an adaptive exploration strategy for the action selection control of each iteration after updating the Q value, balance between exploration and utilization by monitoring the network delay, packet transmission rate and traffic load fluctuations, dynamically adjust the exploration rate, and perform convergence tests on the updated iterations of the Q value to monitor whether the change in the Q value between two consecutive iterations is less than a given threshold.

5. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model according to claim 1, characterized in that: Step S2A includes: Step S2A1: First, there is a set of available channels in the system to form a channel candidate pool, denoted as , Represents the 1st, 2nd, ..., nth channels respectively. Each node maintains a channel record table ,exist At this time, according to the channel record table The system will set an active channel set for the historical channel records in , including the most recently used channel and its adjacent channels; the formula for setting the isolation zone is: （9）； in, represents the r-1th channel, represents the rth channel, represents the r+1th channel; Next, the system generates a new pool of available channel candidates , obtained by excluding the active channel set channels from the channel candidate pool: （10）； Then, the available channel candidate pool Divide into several channel blocks according to the continuity of the channel , channel block number i={1,2,…,m}, m is the number of channel blocks, where each channel block Contains several continuous channels; all channel blocks are represented as: （11）； The system selects the largest channel block containing the most channels : （12）； Step S2A2: In order to select a specific channel in the largest channel block, a linear congruential formula is used and based on the seed value Generate a random number to determine the channel; seed value By function Generate, where the function Taking the current time slot number as input: （13）； （14）； in, To calculate the output random number, The random number generated last time, the random number initialized here ; is the multiplier, c is the increment, and M is the modulus; the hash function is used to convert the time slot number into a value with good randomness; the random number generated by the linear congruential formula Used to index in the largest channel block, the final selected channel is: （15）； （16）； Where k is the channel index, For the selected channel, The largest channel block channels; Step S2A3: After the channel is selected After that, the node will use the channel for data transmission in the current time slot t, and after the transmission is completed, the channel information used will be stored in the channel recorder for reference in subsequent channel selection; and through the 6P protocol, the node will return the relevant data to the parent node and update the scheduling table, so as to further dynamically release and supplement time slots, maintain the dynamic update of the time slot candidate pool, and continuously optimize the time slot selection strategy according to the new transmission data; the update of the channel recorder follows the following rules: （17）； in, is the updated channel record table, The original channel.

6. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model as claimed in claim 4, characterized in that: Step S1B includes: Step S1B1: Initialize the Q table to store the Q value of each state and action combination, set the learning rate α, discount factor γ, exploration rate and other parameters and initialize the Q table; Step S1B2: Collect network status and pre-process data based on edge routers to provide basic input for Q learning; including packet transmission rate, traffic load and delay; construct a comprehensive reward function based on packet transmission rate, traffic load and delay; introduce multi-objective optimization ideas by combining the design of Pareto optimization method, and dynamically adjust the reward function.

7. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model as claimed in claim 6, characterized in that: The step S1B2 comprises: First, the input of the reward function includes the current network state , the action selected by the current strategy , the change in the target value, and the dynamic or fixed preference weight vector , used to indicate the priority of each optimization goal; the calculation process is centered on constructing a multidimensional reward vector. , calculate its multi-objective performance and generate a reward vector , where each dimension represents the optimization effect of an objective; the reward vectors of all actions constitute the solution set ; Subsequently, the solution set is screened by the Pareto optimization method, the inferior solutions dominated by other solutions are eliminated, and the non-dominated solutions that achieve a balance between all objectives are retained to form the Pareto frontier. ; Next, the final solution is selected from the Pareto frontier based on random selection, preference weighted selection, or ideal solution distance selection strategy; the preference weighted selection strategy calculates the weighted score of each Pareto solution Then choose the solution with the highest score as the optimal solution ; The ideal solution distance is selected by calculating the distance between each Pareto solution and the ideal solution Euclidean distance , select the solution with the smallest distance; the output result is the selected reward vector and its corresponding actions , the reward vector achieves the optimal balance among multiple objectives; the selected reward vector is used to update the Q value table after scalarization.

8. The industrial wireless network resource scheduling method based on a multi-weight quality assessment model as claimed in claim 7, characterized in that: The step S2B is specifically as follows: Step S2B1: According to the setting of multi-objective optimization of reward function in step S1B, the reward value is obtained, and the reward value is used to update the Q value; the action control strategy of the Q learning iteration process adopts an adaptive exploration strategy, and according to the network status data collected by the edge router, the exploration rate is dynamically set in consideration of the abnormal fluctuation of the status; Step S2B2: For the dynamically adjusted exploration rate, select actions in the action space, that is, select weight combinations according to the selected actions. Calculate the reward function to evaluate the effect of the action, which is used to update the Q value in the Q table; and gradually optimize the weights of the multi-weight quality assessment model; by adjusting the exploration rate , and make convergence judgments to ensure that the Q learning algorithm finds the optimal weight combination in the process of continuous learning ; express The optimal value of express The optimal value of express The optimal value of express The optimal value of .

Images