Resource dynamic scheduling method, equipment and medium

Abstract

The invention relates to the technical field of resource scheduling, and discloses a resource dynamic scheduling method and device and a medium, and the method comprises the steps: packaging the crowd information corresponding to a target crowd into a file package, and analyzing the information data of each user in the file package into message data; performing priority identification on the message data, and querying a target transmission channel corresponding to the message data according to the message data subjected to the priority identification; the total flow of all the message transmission channels and the real-time flow of each transmission channel are collected, and the distributable flow shared by all the message transmission channels is calculated according to the real-time flow and the total flow; distributing the distributable flow to the target distribution channel according to the channel weight of the target distribution channel; and updating the flow limiting threshold values of all the message transmission channels according to the distributed flow of the target distribution channel, and scheduling the message data in the message queue corresponding to the target transmission channel through the updated flow limiting threshold values. According to the invention, the resource scheduling efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of resource scheduling, and particularly to a method, device and medium for dynamic resource scheduling. Background Art

[0002] In the Internet finance industry, in order to improve marketing value, corresponding marketing strategies need to be constructed for different customer groups. Since the number of users involved is very large and the resources of the system are limited, some channels have no traffic during certain time periods, but still occupy resources, while other channels with traffic do not have enough resources. Therefore, dynamic invocation of resources is required.

[0003] Existing resource scheduling divides different channels according to the priorities of different messages, and resource scheduling is carried out through channels with different priorities. In actual applications, when different priorities use different channels, there will still be accumulation among messages with the same priority, which will have a greater impact on the business. Moreover, the consumption rates between different channels are fixed and are controlled by message queues, resulting in inaccurate flow control and no precise monitoring mechanism, and unable to dynamically and real-time adjust system resources according to the traffic sizes of different channels, thus resulting in low efficiency in resource scheduling. Summary of the Invention

[0004] The present invention provides a method, device and medium for dynamic resource scheduling, and its main purpose is to solve the problem of low efficiency in resource scheduling.

[0005] To achieve the above object, a method for dynamic resource scheduling provided by the present invention includes:

[0006] Encapsulate the population information corresponding to the pre-screened target population into a file package, and parse the information data of each user in the file package into message data;

[0007] Perform priority identification on the message data, and query the target transmission channel corresponding to the message data from all message transmission channels according to the message data after priority identification;

[0008] Collect the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel, and calculate the allocable traffic shared by all message transmission channels according to the real-time traffic and the total traffic;

[0009] Select the channels with real-time traffic greater than the preset allocated traffic as target allocation channels among all message transmission channels, and allocate the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels;

[0010] Update the flow limit thresholds of all message transmission channels according to the traffic of the target allocation channels after allocation, and schedule the message data in the message queue corresponding to the target transmission channels through the updated flow limit thresholds.

[0011] Optionally, parsing the information data of each user in the file package into message data includes:

[0012] Extract the data fields corresponding to the information data of each user in the file package;

[0013] Generate message format attributes according to the data fields;

[0014] Concatenate the message format attributes and the preset message attribute fields into message fields;

[0015] Convert the information data of each user into message data according to the message fields.

[0016] Optionally, performing priority identification on the message data includes:

[0017] Calculate the correlation degree between the preset marketing requirements and the message data;

[0018] Filter the message data according to the correlation degree to obtain a message data filtering set;

[0019] Perform priority division on the message data filtering set;

[0020] Identify each message data in the message data filtering set after priority division with the corresponding priority of the message data filtering set.

[0021] Optionally, querying the target transmission channel corresponding to the message data from all message transmission channels according to the message data after priority identification includes:

[0022] Extract the priorities corresponding to the pre-configured message transmission channels, and match the priority corresponding to the message data with the priority corresponding to the message transmission channel;

[0023] Filter the message transmission channels after successful matching according to the theme of the message data to obtain the target transmission channel;

[0024] Push the message data into the target transmission channel and store the message data in the message queue corresponding to the target transmission channel.

[0025] Optionally, collecting the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel includes:

[0026] Superimpose the traffic thresholds corresponding to each message transmission channel to obtain the total traffic of all message transmission channels;

[0027] Collect the actual channel traffic of each transmission channel according to a preset timing task;

[0028] Determine the actual channel traffic as the real-time traffic of each message transmission channel.

[0029] Optionally, calculating the allocable traffic shared by all message transmission channels according to the real-time traffic and the total traffic includes:

[0030] Superimpose the real-time traffic of each message transmission channel to obtain the total real-time traffic of all message transmission channels;

[0031] Take the traffic difference between the total traffic of all message transmission channels and the total real-time traffic as the allocable traffic shared by all message transmission channels.

[0032] Optionally, allocating the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels includes:

[0033] Calculate the weight ratio of the target allocation channel according to the channel weight of the target allocation channel and the weight value after adding the channel weights of all message transmission channels;

[0034] Divide the allocable traffic according to the weight ratio to obtain the channel allocation traffic corresponding to each target allocation channel;

[0035] Allocate the channel allocation traffic corresponding to each target allocation channel to the target allocation channel.

[0036] Optionally, scheduling the message data in the message queue corresponding to the target transmission channel through the updated flow limiting threshold includes:

[0037] Extract the target transmission channel corresponding to the message data according to the priority identifier corresponding to the message data in the message queue;

[0038] Obtain the updated flow limiting threshold of the target transmission channel corresponding to the message data, and collect the real-time channel traffic of the target transmission channel corresponding to the message data;

[0039] Judge whether the real-time channel traffic exceeds the updated flow limiting threshold;

[0040] When the real-time channel traffic exceeds the updated flow limiting threshold, put the message data in the message queue of the target transmission channel into sleep;

[0041] When the real-time channel traffic does not exceed the updated flow limiting threshold, schedule the message data in the message queue of the target transmission channel.

[0042] To solve the above problems, the present invention also provides an electronic device, which includes:

[0043] At least one processor; and,

[0044] A memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the above-mentioned resource dynamic scheduling method.

[0046] To solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is executed by a processor in an electronic device to implement the above-mentioned resource dynamic scheduling method.

[0047] In the embodiments of the present invention, through the reasonable use of priority identifiers and message queues, it is ensured that high-priority messages can be processed preferentially, the accumulation of low-priority messages is reduced, and the efficiency of message processing is improved; through the timed execution of the dynamic scheduling module and the reasonable allocation of channel weights, it is ensured that the traffic is reasonably allocated in different time periods, so as to maximize the utilization of system resources and reduce resource waste; through the token bucket algorithm and the dynamic adjustment of the flow limiting threshold, the traffic of each channel is effectively controlled, and system overload caused by sudden increase in traffic is avoided, protecting the stable operation of downstream systems and databases. Therefore, the resource dynamic scheduling method, device and medium proposed by the present invention can solve the problem of low efficiency in resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of the resource dynamic scheduling method provided by an embodiment of the present invention;

[0049] Figure 2 It is a flowchart of the channel dynamic scheduling mechanism provided by an embodiment of the present invention;

[0050] Figure 3 It is a structural diagram of an electronic device for implementing the resource dynamic scheduling method provided by an embodiment of the present invention.

[0051] The realization, functional features and advantages of the objectives of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] It should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0053] An embodiment of the present application provides a resource dynamic scheduling method. The execution subject of the resource dynamic scheduling method includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the resource dynamic scheduling method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0054] Refer to Figure 1 As shown, it is a flowchart of the resource dynamic scheduling method provided by an embodiment of the present invention. In this embodiment, the resource dynamic scheduling method includes:

[0055] S1. Package the population information corresponding to the pre-screened target population into a file package, and parse the information data of each user in the file package into message data.

[0056] In the embodiment of the present invention, the pre-screened target population refers to the population screened according to different data tags. For example, under the conditions that the data tags are age and order amount, users with an age greater than 25 years old and the last order amount greater than 100 yuan are screened out, and all eligible users are gathered together. And for each user ID and the relevant attributes of the user corresponding to each user in the target population, they are packaged into a file package. The file package is a structured data file for storing population information, such as in JSON or CSV format. Then, the population information of the target population is packaged into a file package through computer statements (such as Java statements, Python statements, etc.).

[0057] Exemplarily, if there is a list containing target population information, each entry is a dictionary containing information such as the user's ID, name, age, etc. The attributes in each dictionary are determined according to the data tags. For example, if the data tags for screening the target population are age and order amount, the corresponding dictionary attributes are user ID, age, and order amount. Then, use the standard library csv of Python to save the data as a CSV file, and then specify the column names as field names. Then, write the information of each person row by row through the write interface. Each dictionary corresponds to one row of data, so as to obtain the file package corresponding to the target population.

[0058] Further, the selected population is used as a file package, and the file package is parsed. The corresponding population information in the file package is parsed into individual messages, and then each message is tagged with a priority for differentiation. The messages are sent to the corresponding transmission channels based on the priority.

[0059] In an embodiment of the present invention, the message data refers to a structured data format generated based on the information data of each user and is used to transmit information.

[0060] In an embodiment of the present invention, parsing the information data of each user in the file package into message data includes:

[0061] Extracting the data fields corresponding to the information data of each user in the file package;

[0062] Generating a message format attribute according to the data fields;

[0063] Concatenating the message format attribute and the preset message attribute fields into a message field;

[0064] Converting the information data of each user into message data according to the message field.

[0065] Specifically, if the file package is a CSV file, the information data of each user is read from the CSV file. For example, the data corresponding to ID is 1, 2, 3; the data corresponding to Name is Alice, Bob, Char; the data corresponding to Age is 30, 25, 26; the data corresponding to Money is 120, 160, 500. Then, the information data of each read user is parsed, and a message format attribute is defined. According to the fields ID, Name, Age, Money corresponding to the user's information data, the message format attribute is defined as 'user_id': user_data['ID'], 'name': user_data['Name'], 'age': user_data['Age'],'money': user_data['Money']. Further, the information data of each user is converted into message data, that is, by traversing the data dictionary of each user in the list, the message sending content content in the message data is generated according to the message format attribute.

[0066] Specifically, the message attribute fields further include message_id: the unique identifier of the message; timestamp: the timestamp when the message is sent; sender_id: the user ID of the sender. Then, the information corresponding to message_id, timestamp, sender_id, and content is encapsulated into message data, so that the information data of each user corresponds to a message data.

[0067] Further, in order to build corresponding marketing strategies for different customer groups, it is necessary to divide different channels according to the priorities of different messages. Furthermore, it is necessary to identify different priorities for the message data corresponding to each user, so that different priorities go through different channels, avoiding unreasonable utilization of traffic.

[0068] S2. Identify the priority of the message data, and query the target transmission channel corresponding to the message data from all message transmission channels according to the message data after priority identification.

[0069] In the embodiment of the present invention, by identifying the priority of the message data, the information most relevant to the marketing needs can be quickly identified and distinguished, which helps to optimize the allocation of marketing resources. For example, when conducting advertising or social media promotion, high-priority messages can be preferentially selected to ensure the maximum utilization of resources, and the messages are pushed to the corresponding transmission channels based on the priority, facilitating the reasonable allocation of traffic.

[0070] In the embodiment of the present invention, the identification of the priority of the message data includes:

[0071] Calculate the correlation degree between the preset marketing needs and the message data;

[0072] Screen the message data according to the correlation degree to obtain a set of screened message data;

[0073] Divide the priority of the set of screened message data;

[0074] Identify each message data in the set of screened message data after priority division as the priority corresponding to the set of screened message data.

[0075] Specifically, the marketing needs refer to specific needs achieved in marketing activities, such as promotional activities, special offers, brand activities, etc. Then, calculate the similarity between the features in the marketing needs and the message content attributes in the message data. For example, if the marketing needs are mainly for users over 25 years old with medium consumption amounts, then screen according to the conditions in the marketing needs in the message content conditions of the message data, so as to obtain the similarity between this marketing need and the message data corresponding to each user. If all conditions are met, the correlation degree is determined to be 1. If any one condition is met, the correlation degree is determined to be 0.5. If none of the conditions are met, the correlation degree is determined to be 0. Then, screen the message data with a correlation degree of 1 into one set, screen the message data with a correlation degree of 0.5 into one set, and screen the message data with a correlation degree of 0 into one set.

[0076] Specifically, the message data set corresponding to a correlation degree of 1 is divided into high priority, the message data set corresponding to a correlation degree of 0.5 is divided into medium priority, and the message data set corresponding to a correlation degree of 0 is divided into low priority. Furthermore, each message data in the filtered message data set after priority division is marked with its corresponding priority. For example, if the message data set is divided into high priority, each message data in the message data set is marked as high priority, and the priority corresponding to each message data can be customized according to business requirements.

[0077] Furthermore, different channels are taken according to the different business lines and priorities of the messages. Therefore, the corresponding channel is queried in the configuration center through the priority of the message for message delivery.

[0078] In the embodiment of the present invention, selecting an appropriate transmission channel according to the priority of the message can ensure that the message is conveyed to the relevant audience in an appropriate manner and at an appropriate time, thereby maximizing the effect and influence of the information. Therefore, the corresponding priority transmission channel should be selected according to the message priority.

[0079] In the embodiment of the present invention, querying the target transmission channel corresponding to the message data from all message transmission channels for the message data marked according to the priority includes:

[0080] Extracting the priority corresponding to the pre-configured message transmission channel, and matching the priority corresponding to the message data with the priority corresponding to the message transmission channel;

[0081] Filtering the message transmission channels after successful matching according to the theme of the message data to obtain the target transmission channel;

[0082] Pushing the message data into the target transmission channel, and storing the message data into the message queue corresponding to the target transmission channel.

[0083] Specifically, query the priority corresponding to each pre-configured message transmission channel in the configuration center. The priority of the message transmission channel is custom-configured through the configuration center, and match the priority corresponding to the message data with the priority corresponding to the transmission channel. For example, if the priority corresponding to the message data is high priority, the matched message transmission channel is high priority, that is, the priority of the message data matching the message transmission channel is the same priority. And there may be more than one channel that matches the message data, and there may be a backlog of messages with the same priority among the same-priority messages. Then, it is necessary to further screen the message transmission channels with the same priority after successful matching based on the topic of the message data, that is, use different topics in RocketMQ to distinguish different channels, so as to ensure that different priorities do not affect each other. RocketMQ is a distributed message middleware, which can classify and distinguish messages through different topics. Each topic can be regarded as an independent message channel, which is used to distinguish message flows of different types or different business logics. For example, a topic can be defined for different business logics or modules respectively, such as order-related messages, payment notification messages, user behavior logs, etc.

[0084] Specifically, according to the message transmission channel corresponding to the screened message data, this message needs to be pushed to the corresponding message transmission channel, and each piece of message data is stored in the message queue corresponding to the selected target transmission channel. The message queue can be a queue dedicated to this channel to ensure that messages are processed in order and effectively. Furthermore, through the reasonable use of priority identification and message queue, it is ensured that high-priority messages can be processed first, reducing the backlog of low-priority messages and improving the efficiency of message processing.

[0085] Furthermore, during the message push process, since the resources of the system are limited, some channels have no traffic during certain time periods, but still occupy resources, while the resources of other channels with traffic are insufficient. Therefore, it is necessary to dynamically schedule system resources in real time according to the traffic volume of different channels.

[0086] S3. Collect the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel, and calculate the allocable traffic common to all message transmission channels according to the real-time traffic and the total traffic.

[0087] In the embodiment of the present invention, the total traffic of the transmission channel refers to the total traffic determined based on the sum of the pre-configured traffic thresholds of each transmission channel, and the real-time traffic refers to the actual data volume transmitted through this channel at the current moment.

[0088] In the embodiment of the present invention, the collection of the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel includes:

[0089] Add up the traffic thresholds corresponding to each message transmission channel to obtain the total traffic of all message transmission channels;

[0090] Collect the actual channel traffic of each message transmission channel according to a preset scheduled task;

[0091] Determine the actual channel traffic as the real-time traffic of each message transmission channel.

[0092] Specifically, add up the preset traffic thresholds corresponding to each message transmission channel to obtain the total traffic of all message transmission channels, and collect the current actual channel traffic of each message transmission channel every 10 minutes through a scheduled script, and use the collected actual channel traffic as the real-time traffic of each message transmission channel.

[0093] Exemplarily, the flow limiting threshold of channel 1 is 60, the flow limiting threshold of channel 2 is 80, and the flow limiting threshold of channel 3 is 60, then the total traffic of all message transmission channels is 200, and the real-time traffic of the current channel is collected through a scheduled task. For example, the actual channel traffic of channel 1 is 0, the actual channel traffic of channel 2 is 90, and the actual channel traffic of channel 3 is 50. Since there is no traffic in channel 1 and the traffic resources of channel 2 are not enough, therefore, it is necessary to dynamically allocate traffic resources, calculate the allocable traffic in the message transmission channels, so as to give more resources to the channels with large traffic and the channels with small traffic do not occupy too many resources.

[0094] In the embodiment of the present invention, the allocable traffic refers to taking the traffic corresponding to the message transmission channels through which no traffic passes as additional traffic, and then taking the additional traffic as the allocable traffic.

[0095] In the embodiment of the present invention, calculating the allocable traffic shared by all message transmission channels according to the real-time traffic and the total traffic includes:

[0096] Add up the real-time traffic of each message transmission channel to obtain the total real-time traffic of all message transmission channels;

[0097] Take the traffic difference between the total traffic of all message transmission channels and the total real-time traffic as the allocable traffic shared by all message transmission channels.

[0098] Specifically, continuously collect the real-time traffic corresponding to each message transmission channel according to the scheduled task, add up the real-time traffic corresponding to each collected message transmission channel to obtain the total real-time traffic corresponding to all message transmission channels, compare the total traffic with the real-time total traffic of the current channel, that is, calculate the traffic difference between the total traffic and the real-time total traffic as the allocable traffic of the message transmission channel. For example, if the total traffic is 200 and the real-time total traffic is 130, the allocable traffic is 70. Among them, when the real-time traffic of the message transmission channel is greater than the current limiting threshold, the current limiting threshold of the corresponding message transmission channel is used as the real-time traffic.

[0099] Further, regard the extra traffic as the traffic that can be redistributed, and then, according to the weight of the channel, allocate it to the channels where the current traffic is greater than the already allocated traffic for redistribution, so as to give more resources to the channels with large traffic and prevent the channels with small traffic from occupying too many resources.

[0100] S4. Select the channels with real-time traffic greater than the preset already allocated traffic among all message transmission channels as the target allocation channels, and allocate the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels.

[0101] In the embodiment of the present invention, the target allocation channel refers to the channel whose real-time traffic of the current transmission channel is greater than the current limiting threshold of this transmission channel screened out among all message transmission channels, where the preset already allocated traffic is the current limiting threshold of this message transmission channel. For example, if the real-time traffic of channel 1 is 70 and the current limiting threshold is 60, then channel 1 is used as the target allocation channel.

[0102] Further, based on the channel weights corresponding to each message transmission channel, the allocable traffic can be used to redistribute the traffic of the message transmission channels, giving more resources to the channels with large traffic and preventing the channels with small traffic from occupying too many resources.

[0103] In the embodiment of the present invention, the channel weight of the message transmission channel refers to the one preset in the configuration center, and the weight of each message transmission channel is custom-configured in the configuration center. The channel weight represents the importance of the message transmission channel. The greater the channel weight, the higher the priority. Through scheduled execution and reasonable allocation of the channel weights, it is ensured that the traffic is reasonably allocated in different time periods, so as to maximize the utilization of system resources and reduce resource waste.

[0104] In the embodiment of the present invention, the step of allocating the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels includes:

[0105] Calculate the weight ratio of the target allocation channel according to the channel weight of the target allocation channel and the weight value after adding up the channel weights of all message transmission channels;

[0106] Divide the allocable traffic according to the weight ratio to obtain the channel allocation traffic corresponding to each target allocation channel;

[0107] Allocate the channel allocation traffic corresponding to each target allocation channel to the target allocation channel.

[0108] Specifically, calculate the ratio of the channel weight of the target allocation channel to the sum of the weights of all message transmission channels. For example, if the channel weight of the target allocation channel is x and the sum of the weights of all message transmission channels is y, then the weight ratio of the target allocation channel is x / y, and divide the allocable traffic according to the weight ratio. For example, if the allocable traffic is z, then the channel allocation traffic corresponding to the target allocation channel is (x / y)×z, and then allocate it to the target allocation channel according to the channel allocation traffic corresponding to each target allocation channel.

[0109] Furthermore, reallocating the traffic resources to the target allocation channels means that the flow limiting thresholds of some channels in the message transmission channels have changed. Therefore, updating the flow limiting thresholds of each transmission channel according to the dynamically allocated traffic can control the traffic allocation of the system more precisely and reduce the overload of the downstream system or data caused by the appearance of glitches.

[0110] S5. Update the flow limiting thresholds of all message transmission channels according to the traffic of the allocated target allocation channels, and schedule the message data in the message queue corresponding to the target transmission channel through the updated flow limiting thresholds.

[0111] In the embodiment of the present invention, updating the flow limiting threshold according to the real-time traffic allocation situation can make the system more dynamically adapt to the current traffic load. If the traffic of some channels increases, the system can adjust the flow limiting threshold accordingly to avoid overload and performance degradation.

[0112] In the embodiment of the present invention, the updating the flow limiting thresholds of all message transmission channels according to the traffic of the allocated target allocation channels includes:

[0113] Extract the channel traffic corresponding to each message transmission channel after allocation;

[0114] Take the allocated channel traffic as the updated flow limiting threshold corresponding to each message transmission channel.

[0115] Specifically, the channel traffic corresponding to each message transmission channel after allocation is used as the updated flow limiting threshold corresponding to each message transmission channel. For example, the flow limiting threshold of channel 1 is 60, the flow limiting threshold of channel 2 is 80, and the flow limiting threshold of channel 3 is 60. After traffic allocation, the traffic of channel 1 is 30, the traffic of channel 2 is 100, and the traffic of channel 3 is 70. Then, the traffic of the channel at this time is used as the updated flow limiting threshold, that is, the updated flow limiting threshold of channel 1 is 30, the updated flow limiting threshold of channel 2 is 100, and the updated flow limiting threshold of channel 3 is 70.

[0116] Further, through the dynamically updated channel traffic, after receiving a marketing message, the messages in the message queue corresponding to the target transmission channel will be scheduled according to the priority identifier on the message.

[0117] In the embodiment of the present invention, after receiving a marketing message, the execution module will obtain the corresponding flow limiting threshold from REDIS according to the priority identifier on the message, and then judge whether the traffic of the current channel exceeds the threshold according to the token bucket algorithm. If so, it will sleep and not perform scheduling, so as to control the call volume to the downstream system and the call volume to the database, thereby reducing the resource occupation of the system.

[0118] In the embodiment of the present invention, scheduling the message data in the message queue corresponding to the target transmission channel through the updated flow limiting threshold includes:

[0119] Extracting the target transmission channel corresponding to the message data according to the priority identifier corresponding to the message data in the message queue;

[0120] Obtaining the updated flow limiting threshold of the target transmission channel corresponding to the message data, and collecting the real-time channel traffic of the target transmission channel corresponding to the message data;

[0121] Judging whether the real-time channel traffic exceeds the updated flow limiting threshold;

[0122] When the real-time channel traffic exceeds the updated flow limiting threshold, sleeping the message data in the message queue of the target transmission channel;

[0123] When the real-time channel traffic does not exceed the updated flow limiting threshold, scheduling the message data in the message queue of the target transmission channel.

[0124] Specifically, determine the target transmission channel corresponding to the message data according to the priority identifier corresponding to the message data stored in the message queue, obtain the flow limiting threshold corresponding to the channel from the REDIS that stores the flow limiting threshold of the channel according to the priority identifier on the message, and collect the current channel traffic of the target transmission channel corresponding to the message data. Then, judge whether the real-time traffic of the current channel exceeds the traffic threshold according to the token bucket algorithm. If the real-time traffic of the current channel exceeds the updated traffic threshold, sleep the message data in the message queue at this time and do not perform scheduling until several pieces of data before the message data in the message queue at this time are processed completely, and then this message data will be scheduled; if the real-time traffic of the current channel does not exceed the updated traffic threshold, schedule this message in the message queue, so as to control the call volume to the downstream system and the call volume to the database, thereby reducing the resource occupancy of the system. Through the dynamic adjustment of the token bucket algorithm and the flow limiting threshold, effectively control the traffic of each channel, avoid system overload caused by sudden increase in traffic, and protect the stable operation of the downstream system and the database. Among them, the token bucket algorithm is often used to limit the rate of network traffic or control the access frequency of application programs. The token bucket can be regarded as a bucket with a fixed capacity, which stores tokens; the tokens are put into the bucket at a fixed rate at a constant speed (i.e., the token generation rate). When the token bucket is full, the extra tokens will be discarded. That is, when a request arrives, the system will try to obtain a token from the token bucket; if there is an available token in the token bucket, the request is allowed to pass and a token is taken from the token bucket; if there are not enough tokens in the token bucket, the request is rejected or put into the queue to wait; the capacity and generation rate of the token bucket determine the maximum traffic and rate limit that can be processed.

[0125] Specifically, as Figure 2As shown in the figure, it is a schematic flowchart of the channel dynamic scheduling mechanism. Through the file parsing module, user population packages are generated and parsed into individual messages. Tags are added to the messages to distinguish priorities, and the messages are sent to the message queues corresponding to different priorities according to different type-based routing. That is, the module that uses the message queue to implement message forwarding stores the user information pushed by the file parsing module in the message queue, and uses different topics of RocketMQ to distinguish different channels, so as to ensure that different priorities do not affect each other. To achieve dynamic scheduling, the dynamic scheduling module is executed once every 10 minutes. It first collects the current traffic of all channels, then compares it with the traffic currently allocated to the channels, and regards the extra traffic as the traffic that can be redistributed. Then, through the weights of the channels, it is allocated to the channels with a larger current traffic than the allocated traffic for redistribution, so as to give more resources to the channels with large traffic and prevent the channels with small traffic from occupying too many resources. The new flow limiting threshold is written into REDIS. Furthermore, after receiving the marketing message, the execution module will obtain the corresponding flow limiting threshold from REDIS according to the priority identifier on the message, and then judge whether the traffic of the current channel exceeds the threshold according to the token bucket algorithm. If so, it will go to sleep and not perform scheduling, so as to control the number of calls to the downstream system and the database, thereby reducing the resource occupancy of the system. In addition, the configuration center will configure the channels corresponding to the priority identifiers of the messages, and then configure the weights of each channel.

[0126] In the embodiment of the present invention, through the reasonable use of priority identifiers and message queues, it is ensured that high-priority messages can be processed preferentially, reducing the backlog of low-priority messages and improving the efficiency of message processing; through the periodic execution of the dynamic scheduling module and the reasonable allocation of channel weights, it is ensured that the traffic is reasonably allocated during different time periods, thereby maximizing the utilization of system resources and reducing resource waste; through the token bucket algorithm and the dynamic adjustment of the flow limiting threshold, the traffic of each channel is effectively controlled, avoiding system overload caused by sudden traffic increase and protecting the stable operation of the downstream system and the database. Therefore, the resource dynamic scheduling method, device and medium proposed by the present invention can solve the problem of low efficiency in resource scheduling.

[0127] As Figure 3 shown, it is a schematic structural diagram of an electronic device for implementing the resource dynamic scheduling method provided by an embodiment of the present invention.

[0128] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a resource dynamic scheduling program.

[0129] Among them, in some embodiments, the processor 10 may be composed of an integrated circuit. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting various components of the entire electronic device through various interfaces and circuits. By running or executing programs or modules stored in the memory 11 (such as executing a resource dynamic scheduling program, etc.), and calling data stored in the memory 11, it performs various functions of the electronic device and processes data.

[0130] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disks, multimedia cards, card-type memories (such as SD or DX memories, etc.), magnetic memories, magnetic disks, optical discs, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device, such as the mobile hard disk of the electronic device. In some other embodiments, the memory 11 may also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can not only be used to store application software installed on the electronic device and various types of data, such as the code of the resource dynamic scheduling program, etc., but also be used to temporarily store data that has been output or will be output.

[0131] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is set to achieve connection communication between the memory 11 and at least one processor 10, etc.

[0132] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is generally used to establish a communication connection between this electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, and is used to display the information processed in the electronic device and to display a visual user interface.

[0133] Only the electronic device with components is shown in the figure. Those skilled in the art can understand that the structure shown in the figure does not constitute a limitation on the electronic device, and it may include fewer or more components than shown in the figure, or combine certain components, or have different component arrangements.

[0134] For example, although not shown, the electronic device may further include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so as to implement functions such as charge management, discharge management, and power consumption management through the power management device. The power source may also include any components such as one or more DC or AC power sources, a recharge device, a power failure detection circuit, a power converter or an inverter, and a power status indicator. The electronic device may also include a variety of sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated here.

[0135] It should be understood that the above embodiments are only for illustration purposes and are not limited by this structure in the scope of the patent application.

[0136] The resource dynamic scheduling program stored in the memory 11 in the electronic device is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0137] Encapsulate the population information corresponding to the pre-screened target population into a file package, and parse the information data of each user in the file package into message data;

[0138] Perform a priority identification on the message data, and query the target transmission channel corresponding to the message data from all message transmission channels according to the message data after the priority identification;

[0139] Collect the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel, and calculate the allocable traffic shared by all message transmission channels according to the real-time traffic and the total traffic;

[0140] Select, from all message transmission channels, channels with real-time traffic greater than the preset allocated traffic as target allocation channels, and allocate the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels;

[0141] Update the traffic limiting thresholds of all message transmission channels according to the traffic of the target allocation channels after allocation, and schedule the message data in the message queue corresponding to the target transmission channels through the updated traffic limiting thresholds.

[0142] Specifically, the specific implementation method of the above instructions by the processor 10 can refer to the description of the relevant steps in the corresponding embodiments of the attached drawings, which will not be elaborated here.

[0143] Furthermore, if the modules / units integrated in the electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory).

[0144] The present invention also provides a computer-readable storage medium, and the readable storage medium stores a computer program, which, when executed by a processor of an electronic device, can implement:

[0145] Package the population information corresponding to the pre-screened target population into a file package, and parse the information data of each user in the file package into message data;

[0146] Perform priority identification on the message data, and query the target transmission channels corresponding to the message data from all message transmission channels according to the message data after priority identification;

[0147] Collect the total traffic after superimposing the traffic thresholds of each message transmission channel and the real-time traffic of each message transmission channel, and calculate the allocable traffic shared by all message transmission channels according to the real-time traffic and the total traffic;

[0148] Select, from all message transmission channels, channels with real-time traffic greater than the preset allocated traffic as target allocation channels, and allocate the allocable traffic to the target allocation channels according to the channel weights of the target allocation channels;

[0149] Update the current limiting thresholds of all message transmission channels according to the traffic of the target assigned channels after assignment, and schedule the message data in the message queue corresponding to the target transmission channels by using the updated current limiting thresholds.

[0150] In several embodiments provided by the present invention, it should be understood that the disclosed devices, media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation.

[0151] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, in each embodiment of the present invention, the functional modules can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0153] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms.

[0154] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is not limited only by the above description. Therefore, it is intended to embrace all changes within the meaning and scope of the equivalent elements falling within the protection scope of the present invention.

[0155] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0156] In addition, obviously, the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described in the system can also be implemented by one unit or device through software or hardware. Words such as first and second are used to represent names and do not represent any specific order.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A resource dynamic scheduling method, characterized in that: The method comprises: Encapsulating the crowd information corresponding to the pre-screened target crowd into a file package, and parsing the information data of each user in the file package into message data; Priority marking is performed on the message data, and according to the message data after the priority marking, a target transmission channel corresponding to the message data is searched from all message transmission channels; Collect the total flow after the flow threshold of each message transmission channel is superimposed and the real-time flow of each message transmission channel, and calculate the allocatable flow shared by all message transmission channels according to the real-time flow and the total flow; Selecting a channel whose real-time traffic is greater than a preset allocated traffic from all message transmission channels as a target allocation channel, and allocating the allocable traffic to the target allocation channel according to the channel weight of the target allocation channel; The flow limiting thresholds of all message transmission channels are updated according to the flow of the allocated target allocation channel, and the message data in the message queue corresponding to the target transmission channel is scheduled using the updated flow limiting thresholds.

2. The resource dynamic scheduling method according to claim 1, characterized in that: The step of parsing the information data of each user in the file package into message data includes: Extracting the data field corresponding to the information data of each user in the file package; generating a message format attribute according to the data field; The message format attribute and the preset message attribute field are spliced ​​into a message field; The information data of each user is converted into message data according to the message fields.

3. The resource dynamic scheduling method according to claim 1, characterized in that: The step of marking the priority of the message data comprises: Calculating the correlation between the preset marketing demand and the message data; Filtering the message data according to the correlation to obtain a message data filter set; Prioritizing the message data screening set; Each message data in the message data screening set after priority division is identified as the priority corresponding to the message data screening set.

4. The resource dynamic scheduling method according to claim 1, characterized in that: The step of searching the target transmission channel corresponding to the message data from all message transmission channels according to the message data marked with the priority level includes: Extracting the priority corresponding to the pre-configured message transmission channel, and matching the priority corresponding to the message data with the priority corresponding to the message transmission channel; Filter the message transmission channels that have been successfully matched according to the subject of the message data to obtain a target transmission channel; The message data is pushed to the target transmission channel, and the message data is stored in a message queue corresponding to the target transmission channel.

5. The resource dynamic scheduling method according to claim 1, characterized in that: The collecting of the total flow after the flow threshold of each message transmission channel is superimposed and the real-time flow of each message transmission channel includes: The traffic threshold corresponding to each message transmission channel is added together to obtain the total traffic of all message transmission channels. Collect the actual channel traffic of each message transmission channel according to the preset timed tasks; The actual channel traffic is determined as the real-time traffic of each message transmission channel.

6. The resource dynamic scheduling method according to claim 5, characterized in that: The calculating the allocatable flow shared by all message transmission channels according to the real-time flow and the total flow includes: The real-time traffic of each message transmission channel is superimposed to obtain the real-time total traffic of all message transmission channels; The flow difference between the total flow of all message transmission channels and the real-time total flow is used as the allocatable flow shared by all message transmission channels.

7. The resource dynamic scheduling method according to claim 1, characterized in that: The allocating the allocable flow to the target allocation channel according to the channel weight of the target allocation channel comprises: Calculating the weight ratio of the target allocation channel according to the weight value obtained by adding the channel weight of the target allocation channel and the channel weights of all message transmission channels; Dividing the allocable flow according to the weight ratio to obtain a channel allocation flow corresponding to each target allocation channel; The channel allocation flow corresponding to each target allocation channel is allocated to the target allocation channel.

8. The resource dynamic scheduling method according to claim 1, characterized in that: The scheduling of the message data in the message queue corresponding to the target transmission channel by using the updated current limiting threshold includes: Extracting the target transmission channel corresponding to the message data according to the priority identifier corresponding to the message data in the message queue; Obtaining an updated current limiting threshold of the target transmission channel corresponding to the message data, and collecting real-time channel traffic of the target transmission channel corresponding to the message data; Determine whether the real-time channel flow exceeds the updated flow limiting threshold; When the real-time channel traffic exceeds the updated current limiting threshold, the message data in the message queue of the target transmission channel is put into hibernation; When the real-time channel traffic does not exceed the updated current limiting threshold, the message data in the message queue of the target transmission channel is scheduled.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the resource dynamic scheduling method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the resource dynamic scheduling method according to any one of claims 1 to 8 is implemented.

Images