Resource recommendation method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a resource recommendation method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: constructing a resource popularity prediction model based on user historical behavior data and resource state information; obtaining a resource matching degree according to the resource state information and the user service demand information; based on the resource popularity prediction model and the resource matching degree, performing reinforcement learning training on a to-be-trained computing power resource recommendation model to obtain a trained computing power resource recommendation model; and through the trained computing power resource recommendation model, obtaining a computing power resource recommendation result for the target user. By adopting the method, the computing power resources meeting user preferences and matching user business requirements can be recommended.

Description

Technical Field

[0001] The present application relates to the field of computing power network technology, and in particular to a resource recommendation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] The computing power network is a new type of information infrastructure that allocates and flexibly schedules computing power resources between the cloud, edge, and end on demand based on the user's business needs. It can realize flexible scheduling of computing tasks and orchestration and management of computing power resources.

[0003] However, computing power resources are difficult to measure, have a wide variety, are widely distributed, and have complex ownership. Users often find it difficult to make clear computing power resource demands based on their business needs, making it difficult for computing power resource trading platforms to recommend computing power resources that match their business needs to users. Summary of the invention

[0004] Based on this, it is necessary to provide a resource recommendation method, device, computer equipment, computer-readable storage medium and computer program product that can recommend computing resources that match the business needs of users in response to the above technical problems.

[0005] In a first aspect, in one embodiment, the present application provides a resource recommendation method, the method comprising:

[0006] Build a resource popularity prediction model based on user historical behavior data and resource status information; resource status information is used to characterize the performance status and price status of computing resources;

[0007] Obtain resource matching degree based on resource status information and user business demand information; resource matching degree is used to characterize the matching degree between computing resources and user business;

[0008] Based on the resource popularity prediction model and resource matching degree, the computing resource recommendation model to be trained is reinforced with learning training to obtain a trained computing resource recommendation model.

[0009] Obtain the computing power resource recommendation results for target users through the trained computing power resource recommendation model.

[0010] In one embodiment, a resource popularity prediction model is constructed based on user historical behavior data and resource status information, including:

[0011] Based on the user's historical behavior data, mark the popularity of the browsed computing resources;

[0012] Using the recurrent neural network model, iterative training is performed to obtain a resource popularity prediction model;

[0013] Among them, the input of the recurrent neural network model includes the resource status information and popularity of all computing resources in the computing resource pool in the previous time slot, and the resource status information of all computing resources in the computing resource pool in the current time slot; the output of the recurrent neural network model includes the popularity of all computing resources in the computing resource pool in the current time slot.

[0014] In one embodiment, obtaining resource matching degree according to resource status information and user service requirement information includes:

[0015] According to the user's business demand information and the resource status information of the computing resources under the load state, the resource matching degree between any computing resource and any user's business demand information is obtained based on vector similarity.

[0016] In one embodiment, based on the resource popularity prediction model and the resource matching degree, the computing resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing resource recommendation model, including:

[0017] Determine the state space of the computing resource recommendation model based on the popularity of computing resources, resource status information, and resource matching degree;

[0018] Based on all computing resources in the current computing resource pool, determine the action space of the computing resource recommendation model;

[0019] Through the preset reward function, the computing power resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing power resource recommendation model.

[0020] In one embodiment, determining the state space of the computing resource recommendation model according to the popularity of the computing resource, the resource state information, and the resource matching degree includes:

[0021] Construct a computing network resource state vector based on the resource state information of each computing resource in the computing resource pool;

[0022] Construct a resource matching vector based on the resource matching degree of computing resources for each user's business;

[0023] Construct a popularity vector based on the popularity of each computing resource in the computing resource pool;

[0024] Based on the computing network resource state vector, resource matching vector and popularity vector, the state space of the computing power resource recommendation model is determined.

[0025] In one embodiment, a preset reward function is used to perform reinforcement learning training on the computing resource recommendation model to be trained to obtain a trained computing resource recommendation model, including:

[0026] Initialize the training environment and the agent to be trained according to the state space, action space and reward function;

[0027] Based on the environment and the agent to be trained, strategy training is performed through a reinforcement learning algorithm until the agent to be trained converges to obtain a trained computing resource recommendation model.

[0028] In a second aspect, in one embodiment, the present application provides a resource recommendation device, the device comprising:

[0029] The prediction model building module is used to build a resource popularity prediction model based on user historical behavior data and resource status information; the resource status information is used to characterize the performance status and price status of computing resources;

[0030] The resource matching degree acquisition module is used to obtain the resource matching degree based on the resource status information and the user business demand information; the resource matching degree is used to characterize the matching degree between the computing power resources and the user business;

[0031] The recommendation model building module is used to perform reinforcement learning training on the computing resource recommendation model to be trained based on the resource popularity prediction model and resource matching degree to obtain a trained computing resource recommendation model;

[0032] The recommendation result output module is used to obtain the computing power resource recommendation results for the target user through the trained computing power resource recommendation model.

[0033] In a third aspect, in one embodiment, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any one of the methods described in the method embodiments of the first aspect are implemented.

[0034] In a fourth aspect, in one embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the method embodiment of the first aspect above.

[0035] In a fourth aspect, in one embodiment, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in the method embodiment of the first aspect above.

[0036] The above-mentioned resource recommendation method, device, computer equipment, computer-readable storage medium and computer program product construct a resource popularity prediction model based on user historical behavior data and resource status information; the resource status information is used to characterize the performance status and price status of computing power resources; the resource matching degree is obtained according to the resource status information and user business demand information; the resource matching degree is used to characterize the degree of matching between the computing power resources and the user business; based on the resource popularity prediction model and the resource matching degree, the computing power resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing power resource recommendation model; through the trained computing power resource recommendation model, the computing power resource recommendation result for the target user is obtained. By constructing a computing power resource recommendation model, the present application can recommend computing power resources that match the business needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 An application environment diagram of a resource recommendation method in an embodiment;

[0039] Figure 2 A schematic diagram of a process flow of a resource recommendation method in one embodiment;

[0040] Figure 3 A schematic diagram of a process for training a popularity prediction model in one embodiment;

[0041] Figure 4 A schematic diagram of an iterative training process of a popularity prediction model in one embodiment;

[0042] Figure 5 A schematic diagram of a process of reinforcement learning training in one embodiment;

[0043] Figure 6 A schematic diagram of a process for determining a state space in one embodiment;

[0044] Figure 7 A schematic diagram of a process of training an intelligent agent in one embodiment;

[0045] Figure 8 A schematic diagram of data transmission for agent training in one embodiment;

[0046] Fig. 9 is a structural block diagram of a resource recommendation device in an embodiment;

[0047] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0050] The computing network is a technology that distributes computing power, algorithms, and storage information based on the network control plane. It can combine network information to realize the distribution, association, transaction, and deployment of computing, network, storage, and other resources with business needs as the core, so as to achieve the goal of optimizing the configuration of the entire network resources. In addition, the computing network is a new type of information infrastructure that can allocate and flexibly schedule computing resources, storage resources, and network resources on demand between the cloud, edge, and end according to the business needs of users. Its essence is a computing resource service that can realize the flexible scheduling of computing tasks and the orchestration and management of computing resources.

[0051] However, computing resources are difficult to measure, have a wide variety, are widely distributed, and have complex attribution. In addition, due to the continuous changes in user business types and the continuous evolution of computing resources, computing power orchestration and scheduling are highly complex.

[0052] Traditional computing network technology is mainly guided by computing network operation indicators such as energy consumption, revenue conversion rate, and network latency, but does not consider user experience indicators such as user browsing and feedback. This leads to poor user experience and the final computing resource recommendation results often deviate from user expectations.

[0053] In addition, it is difficult for ordinary users to put forward clear computing power requirements for specific business needs, which also affects the user experience and hinders the promotion of computing power networks.

[0054] The resource recommendation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0055] In an exemplary embodiment, Figure 2 As shown, a resource recommendation method is provided, which is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, which includes the following steps 202 to 206. Among them:

[0056] Step 202: construct a resource popularity prediction model based on user historical behavior data and resource status information.

[0057] Among them, resource status information is used to characterize the performance status and price status of computing resources.

[0058] Exemplarily, the user historical behavior data may represent the user's behavior on computing resources. Optionally, the user historical behavior data may include browsing history (HB), click history (HC), purchase history (HP), feedback history (HF), etc. In some examples, the user historical behavior data may be represented by the following mathematical form:

[0059] (Formula 1)

[0060] in For users User historical behavior data, For users Browsing history, For users Click records, For users Purchase records, For users Feedback record.

[0061] It is understandable that the above-mentioned user historical behavior data is not limited to the implementation methods mentioned in the above embodiments. As long as it is data that can characterize the user's historical behavior with respect to computing resources, the embodiments of the present application do not specifically limit the specific type of user historical behavior data.

[0062] Exemplarily, the resource status information may represent status information such as the performance status and price status of the computing power resources. Optionally, the resource status information may include the computing power (CP), communication capability (CC), memory capability (MC), storage capacity (SC), load condition (LC), and price per unit (PPU) of the computing power resources. In some examples, the resource status information may be represented by the following mathematical form:

[0063] (Formula 2)

[0064] in, Indicates computing resources status, For computing resources The computing power of For computing resources communication capabilities, For computing resources Memory capacity, For computing resources storage capacity, For computing resources The load condition, Current computing resources Unit price.

[0065] It can be understood that the above-mentioned resource status information is not limited to the implementation methods mentioned in the above embodiments. As long as it can characterize the performance status and price status of computing resources, the embodiments of the present application do not specifically limit the specific type of resource status information.

[0066] For example, based on the resource status information, the status of each computing resource in the current computing resource pool can be obtained; based on the user's historical behavior data, the user's historical behavior towards each computing resource in the computing resource pool can be learned. It can be understood that by analyzing and simulating the user's historical behavior towards computing resources, the user's behavior prediction can be further achieved.

[0067] In some possible implementations, a resource popularity prediction model can be constructed through a recurrent neural network based on user historical behavior data and resource status information; wherein the resource popularity prediction model can be used to obtain the popularity of each computing resource in the computing resource pool.

[0068] Specifically, the resource popularity prediction model can be constructed by combining user historical behavior data and resource status information.

[0069] Step 204: Obtain resource matching degree according to resource status information and user business requirement information.

[0070] Among them, resource matching degree is used to characterize the matching degree between computing resources and user services.

[0071] Exemplarily, user business demand information can be used to characterize the computing resource requirements required by the user's current business. Optionally, user business demand information may include computing power requirements (CPR), communication capability requirements (CCR), memory capability requirements (MCR), storage capacity requirements (SCR), and project planning budget (PPB). In some examples, user business demand information can be expressed in the following mathematical form:

[0072] (Formula 3)

[0073] in, Indicates user service demand, For user business Computing power requirements, For user business Communication capability requirements, For user business Memory capacity requirements, For user business Storage capacity requirements, For user business project plan budget.

[0074] It can be understood that the above-mentioned user business demand information is not limited to the implementation methods mentioned in the above embodiments. As long as it can characterize the computing power resource requirements required for the user's current business, the embodiments of the present application do not specifically limit the specific type of user business demand information.

[0075] Specifically, based on the resource status information of the computing resources and the user's business demand information, the resource matching degree can be obtained to determine the matching degree between the current computing resources and the user's business.

[0076] Step 206: Based on the resource popularity prediction model and the resource matching degree, the computing power resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing power resource recommendation model.

[0077] Among them, reinforcement learning, also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. It can be used to describe and solve the problem of how an agent can maximize rewards or achieve specific goals through learning strategies during its interaction with the environment.

[0078] For example, based on the resource popularity prediction model, the popularity of each computing resource in the current computing resource pool among users can be obtained; based on the resource matching degree, the matching degree between the current user's business needs and each computing resource can be obtained, so that the connection between user business needs, user behavior and computing resources can be established through the popularity and resource matching degree of computing resources, so as to construct a computing resource recommendation model. Furthermore, through the computing resource recommendation model, it is possible to output computing resource recommendation results that meet user preferences and user business needs.

[0079] Specifically, the computing power resource recommendation model to be trained can be subjected to reinforcement learning training according to the resource popularity prediction model and the resource matching degree, so as to obtain a trained computing power resource recommendation model.

[0080] Step 208: Obtain the computing power resource recommendation result for the target user through the trained computing power resource recommendation model.

[0081] The target users may include platform users who expect to obtain computing resources on the computing network. The trained computing resource recommendation model can output computing resource recommendation results that meet the user's preferences and meet the user's business needs according to the target user's business needs.

[0082] Specifically, the trained computing resource recommendation model can be used to obtain computing resource recommendation results for target users.

[0083] This application can build a resource popularity prediction model based on user historical behavior data and resource status information, and obtain resource matching based on resource status information and user business demand information; then, based on the resource popularity prediction model and resource matching, the trained computing resource recommendation model is subjected to reinforcement learning training to obtain a trained computing resource recommendation model. This application can use the trained computing resource recommendation model to obtain computing resource recommendation results that meet user preferences and meet user business needs. On the one hand, this application optimizes the operation of the computing network, and on the other hand, it maximizes the user experience of using the computing network.

[0084] In one embodiment, if Figure 3 As shown, based on the user historical behavior data and resource status information, a resource popularity prediction model is constructed, including the following steps S302 to S304. Among them:

[0085] Step S302: Mark the popularity of the browsed computing resources based on the user's historical behavior data.

[0086] Among them, the popularity of computing resources can be used to indicate the popularity of the computing resources among users on the computing network. Exemplarily, the popularity of computing resources can be represented by parameters such as the click-through rate (CTR), purchase rate (PR), and favorable rate (FR) of computing resources, and the above parameters can be obtained based on the statistics of historical user behavior data. In some examples, the popularity of computing resources can be represented by the following mathematical form:

[0087] (Formula 4)

[0088] in For computing resources Popularity among users, For computing resources Click-through rate, For computing resources The purchase rate, For computing resources The praise rate.

[0089] It can be understood that the above parameters used to represent the popularity of computing power resources are not limited to the implementation methods mentioned in the above embodiments. As long as they can characterize the popularity of the computing power resources among users on the computing power network, the embodiments of the present application do not specifically limit the specific types of the above parameters.

[0090] Specifically, based on the user's historical behavior data, the popularity of the browsed computing resources can be marked in the computing resource pool.

[0091] Step S304, using a recurrent neural network model, iteratively trains to obtain a resource popularity prediction model.

[0092] Among them, the input of the recurrent neural network model includes the resource status information and popularity of all computing resources in the computing resource pool in the previous time slot, and the resource status information of all computing resources in the computing resource pool in the current time slot; the output of the recurrent neural network model includes the popularity of all computing resources in the computing resource pool in the current time slot.

[0093] In order to make the technical means and advantages of the embodiments of the present application more clearly understood, the following Figure 4 It should be understood that the Figure 4 The specific embodiments described are only used to explain the method steps of the embodiments of the present application and are not used to limit the present application.

[0094] For example, Figure 4 As shown, this embodiment builds a recurrent neural network model (RNN) to learn the user's historical behavior to predict the popularity of any given computing resources (including computing resources that the user has not browsed). Figure 4 The input of the recurrent neural network shown can be the resource status information of all computing resources in the computing resource pool at time t-1 and its corresponding popularity , and the resource status information of all computing resources in the computing resource pool at time t ; The output of the recurrent neural network is the popularity of all computing resources at time t .

[0095] Further, Figure 4 The "hidden state" in the RNN can refer to the state information saved by the recurrent neural network in the current time step. It can be used to store information inherited from the previous time step and combined with the current input to calculate the new output. It can be understood that the hidden state is the core of the RNN, which can capture the temporal dependency in the sequence data, that is, the memory of the previous input. Figure 4 The "RNN cell" in the figure is the basic unit for processing single time step data in a recurrent neural network.

[0096] Specifically, this embodiment can utilize Figure 4 The recurrent neural network model shown is iteratively trained to obtain a resource popularity prediction model.

[0097] In one embodiment, obtaining resource matching degree according to resource status information and user service requirement information includes the following steps:

[0098] According to the user's business demand information and the resource status information of the computing resources under the load state, the resource matching degree between any computing resource and any user's business demand information is obtained based on vector similarity.

[0099] The resource status information of computing resources under load may refer to the resource status information of computing resources that is actually effective under working conditions. It is understood that in the matching process, using the resource status information of computing resources under load has higher accuracy than using the resource status information calculated by the theoretical maximum value, which can make the allocation of resources more reasonable.

[0100] In some possible implementations, the user traffic can be calculated User business demand information Resource status information of each computing resource in the computing resource pool at time t , we can get the computing power resource matching vector as shown below:

[0101] (Formula 5)

[0102] in, It can represent the maximum value of the computing resource index tag in the current computing resource pool, and the resource matching degree The computing resources can be determined by load status. Resource status information Business with users User business demand information The vector similarity of get.

[0103] Specifically, based on the user business demand information and the resource status information of the computing resources under the load state in the computing resource pool, the resource matching degree between any computing resource and any user business demand information can be obtained by using vector similarity.

[0104] In one embodiment, if Figure 5 As shown, based on the resource popularity prediction model and the resource matching degree, the computing resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing resource recommendation model, including the following steps S502 to S506. Among them:

[0105] Step S502, determining the state space of the computing resource recommendation model according to the popularity of the computing resource, resource status information, and resource matching degree.

[0106] The computing resource recommendation model may be a reinforcement learning model (RL), which may include a state space, an action space, and a corresponding agent. It is understood that the state space may be a set that includes all possible states in the environment of reinforcement learning training, and each state is a description of the environment of the reinforcement learning model.

[0107] In some examples, the popularity of computing resources can be obtained by outputting a constructed resource popularity prediction model. Resource matching can be obtained by performance information and price information of each computing resource in the computing resource pool. Resource matching can be obtained by calculating vector similarity based on user business demand information and resource status information of computing resources under load in the computing resource pool.

[0108] Specifically, based on the popularity of computing resources, resource status information, and resource matching degree, the state space of the computing resource recommendation model can be constructed.

[0109] Step S504: Determine the action space of the computing resource recommendation model based on all computing resources in the current computing resource pool.

[0110] The action space of the computing resource recommendation model can be a set, which can include all actions that the agent of the computing resource recommendation model can perform in a specific state. For example, suppose there is a task in each time slot T. , the actions in the action space can be represented as giving tasks to the user The above actions can be expressed in the following mathematical form:

[0111] (Formula 6)

[0112] in, For task Actions, is the computing resource k in the computing resource pool, For the action space.

[0113] Specifically, based on all the computing resources in the current computing resource pool, the action space of the computing resource recommendation model can be constructed.

[0114] Step S506: Perform reinforcement learning training on the computing resource recommendation model to be trained through a preset reward function to obtain a trained computing resource recommendation model.

[0115] Exemplarily, the reward function may be the reward data (reward value) fed back by the environment of the computing power resource recommendation model when the intelligent agent of the computing power resource recommendation model takes action according to a specific state; based on the reward data, the intelligent agent of the computing power resource recommendation model may be guided to converge toward the optimal strategy.

[0116] Optionally, in the embodiment of the present application, the reward function defines when the agent is based on the state Take Action When the environment gives the reward value In some examples, the reward function can be set to be expressed in the following mathematical form:

[0117] (Formula 7)

[0118] in, Represents reward data, For Action A single computing resource in the recommended computing resource group popularity; is a coefficient weight vector, which is used to represent the weights assigned to click rate, purchase rate, and praise rate; Indicates the expected transaction amount; represents the expected transaction amount coefficient; For Action A single computing resource in the recommended computing resource group The degree of resource matching with user business; is the weight coefficient for resource matching; Indicates the selection range or allocation quantity of computing resources, usually referring to the current selection. A computing resource or the number of computing resources involved in the calculation.

[0119] Specifically, based on a preset reward function, the computing power resource recommendation model to be trained can be subjected to reinforcement learning training to obtain a trained computing power resource recommendation model.

[0120] In one embodiment, if Figure 6 As shown, according to the popularity of computing resources, resource status information and resource matching degree, the state space of the computing resource recommendation model is determined, including the following steps S602 to S608. Among them:

[0121] Step S602: construct a computing network resource status vector based on the resource status information of each computing resource in the computing resource pool.

[0122] The state space of the computing resource recommendation model can be composed of a computing network resource state vector, a resource matching vector, and a popularity vector. In some examples, the state s in the computing resource recommendation model can be expressed as ,in Indicates the resource status information of computing resources. Indicates the resource matching degree of computing resources. For the popularity of computing resources, is the state space.

[0123] Exemplarily, the computing network resource state vector may include resource state information of each computing resource obtained from the computing resource pool. Optionally, the computing network resource state vector may be expressed in the following mathematical form:

[0124] (Formula 8)

[0125] (Formula 9)

[0126] in, Indicates time slot The computing network resource state vector of the medium computing resource pool; Indicates computing resources Resource status information; For computing resources computing power; Can be used for computing resources communication capabilities; For computing resources Memory capacity; Can be used for computing resources Storage capacity; Can be used for computing resources The load condition, Can be used for computing resources The unit price, It can refer to the set of computing network resource status vectors of the computing power resource recommendation model.

[0127] Specifically, a computing network resource status vector can be constructed based on the resource status information of each computing resource in the computing resource pool.

[0128] Step S604: construct a resource matching vector according to the resource matching degree of the computing resources for each user service.

[0129] Exemplarily, the resource matching vector is used to reflect the matching degree of each computing resource to the user's business requirements. Optionally, the resource matching vector is expressed in the following mathematical form:

[0130] (Formula 10)

[0131] in, For time slot The resource matching vector in can be used to represent the time slot Computing resources and user services in the medium computing resource pool The matching situation; Can represent computing resources For user business MR can represent the set of resource matching vectors in the computing resource recommendation model.

[0132] Specifically, a resource matching vector may be constructed according to the resource matching degree of computing resources for each user's business.

[0133] Step S606: construct a popularity vector according to the popularity of each computing resource in the computing resource pool.

[0134] For example, the popularity vector can be expressed in the following mathematical form:

[0135] (Formula 11)

[0136] (Formula 12)

[0137] in, Can refer to time slot The popularity vector in ; Indicates computing resources popularity; is the click-through rate; is the purchase rate; For the favorable rate; It can represent the collection of popularity vectors in the computing resource recommendation model.

[0138] Specifically, a popularity vector may be constructed based on the popularity of each computing resource in the computing resource pool.

[0139] Step S608, determining the state space of the computing resource recommendation model based on the computing network resource state vector, the resource matching vector and the popularity vector.

[0140] It can be understood that through the computing network resource status vector, the overall resource status of the computing resources in the computing resource pool can be obtained; through the resource matching vector, the overall matching situation between each computing resource in the computing resource pool and the user business can be obtained; through the popularity vector, the popularity of each computing resource in the computing resource pool can be obtained.

[0141] Specifically, based on the computing network resource state vector, resource matching vector and popularity vector, the state space of the computing resource recommendation model can be constructed.

[0142] In one embodiment, if Figure 7 As shown, using a preset reward function, the computing resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing resource recommendation model, including the following steps S702 to S704. Among them:

[0143] Step S702, initialize the training environment and the agent to be trained according to the state space, action space and reward function.

[0144] Exemplarily, a reinforcement learning training framework can be constructed based on a preset reward function, state space, and action space, that is, the environment of the computing resource recommendation model and the intelligent agent to be trained are initialized.

[0145] Optionally, when initializing the environment of the computing power resource recommendation model, the initial environment parameters (environment, ENV) of the computing power resource recommendation model are set; wherein, the environment parameters (ENV parameters) can be used to characterize the initial state or dynamically changing factors of the environment in the simulator of the computing power resource recommendation model, thereby dynamically adjusting the performance of the environment; exemplarily, the environment parameters may include the initial configuration of the computing power resource state, input variables for simulating user behavior, and the changing patterns of resource load or requests, etc.

[0146] In some examples, initial ENV parameters can be input into the model's environment when the computing resource recommendation model is initialized, and the ENV parameters can be updated in each round of agent training (or interaction) as the environment changes dynamically (such as resource failure, resource increase, network link increase, network link failure, etc.).

[0147] It is understandable that the above-mentioned initialization setting and update method for environmental parameters are not limited to the implementation method mentioned in the above-mentioned embodiment, and the embodiment of the present application does not specifically limit the initialization setting and update method for environmental parameters.

[0148] In some possible implementations, initial RL parameters may be input to the to-be-trained agent of the model when the computing resource recommendation model is initialized. Optionally, the RL parameters may include the learning rate, discount factor, exploration rate, etc. of the model. It is to be understood that the setting of the above RL parameters is not limited to the implementation methods mentioned in the above embodiments, and the embodiments of the present application do not specifically limit the setting of the RL parameters.

[0149] Specifically, the environment for training the computing resource recommendation model and the intelligent agent to be trained can be initialized according to the state space, action space and reward function.

[0150] Step S704: Based on the environment and the agent to be trained, strategy training is performed through a reinforcement learning algorithm until the agent to be trained converges to obtain a trained computing resource recommendation model.

[0151] Optionally, the reinforcement learning algorithm may include a Q-learning algorithm or a policy gradient method.

[0152] In an exemplary embodiment, in order to make the technical means and technical effects of step S704 more clear, the following is combined with the attached Figure 8 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0153] For example, please combine Figure 8 , the strategy training is performed through the reinforcement learning algorithm, including the following steps 1 to 4. Among them:

[0154] Step 1: Make the computing resource recommendation model (RL model) environment change its current state Input agent to instruct the agent to and optimization strategies Generate corresponding actions (i.e. Figure 8 in the "Recommended Resources" section) and set the action Feedback to the environment.

[0155] Step 2: Make the computing resource recommendation model environment based on the received action And the preset reward function , calculate the action The corresponding reward data (reward value) is fed back to the agent.

[0156] Step 3: Use the received reward data and the corresponding state-action pairs to update the optimization strategy of the computing resource recommendation model through reinforcement learning algorithms (such as Q-learning or policy gradient method, etc.) , to gradually optimize the recommendation ability of the model.

[0157] Step 4: Repeat steps 1 to 3 until the agent converges, and finally obtain a trained computing resource recommendation model (i.e., RL result).

[0158] For example, the above optimization strategy (also called policy model) can be a policy function that the agent gradually learns through interaction with the environment. In some possible implementations, the optimization policy The initial setting can be based on random or simple rule strategies, and the ultimate goal of its training is to optimize the strategy through reinforcement learning. Converge to the optimal strategy. Optionally, Figure 8 The ENV parameters and the RL parameters shown can be set during the model initialization phase. Figure 8 The ENV parameters shown can be set so that parameter updates are required during each round of agent training.

[0159] Specifically, according to the environment and intelligent agent of the constructed computing power resource recommendation model, a reinforcement learning algorithm can be used to perform strategy training until the intelligent agent to be trained converges, and finally a trained computing power resource recommendation model is obtained.

[0160] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0161] Based on the same inventive concept, the embodiment of the present application also provides a resource recommendation device for implementing the resource recommendation method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more resource recommendation device embodiments provided below can refer to the limitations on the resource recommendation method above, and will not be repeated here.

[0162] In an exemplary embodiment, Fig. 9 As shown, the present application provides a resource recommendation device 900, comprising:

[0163] Prediction model building module 902, used to build a resource popularity prediction model based on user historical behavior data and resource status information; the resource status information is used to characterize the performance status and price status of computing resources;

[0164] The resource matching degree acquisition module 904 is used to acquire the resource matching degree according to the resource status information and the user service demand information; the resource matching degree is used to characterize the matching degree between the computing power resources and the user service;

[0165] The recommendation model building module 906 is used to perform reinforcement learning training on the computing resource recommendation model to be trained based on the resource popularity prediction model and the resource matching degree to obtain a trained computing resource recommendation model;

[0166] The recommendation result output module 908 is used to obtain the computing power resource recommendation results for the target user through the trained computing power resource recommendation model.

[0167] In one embodiment, the prediction model building module 902 is further configured to:

[0168] Based on the user's historical behavior data, mark the popularity of the browsed computing resources;

[0169] Using the recurrent neural network model, iterative training is performed to obtain a resource popularity prediction model;

[0170] Among them, the input of the recurrent neural network model includes the resource status information and popularity of all computing resources in the computing resource pool in the previous time slot, and the resource status information of all computing resources in the computing resource pool in the current time slot; the output of the recurrent neural network model includes the popularity of all computing resources in the computing resource pool in the current time slot.

[0171] In one embodiment, the resource matching degree obtaining module 904 is further used to:

[0172] According to the user's business demand information and the resource status information of the computing resources under the load state, the resource matching degree between any computing resource and any user's business demand information is obtained based on vector similarity.

[0173] In one embodiment, the recommendation model building module 906 is further configured to:

[0174] Determine the state space of the computing resource recommendation model based on the popularity of computing resources, resource status information, and resource matching degree;

[0175] Based on all computing resources in the current computing resource pool, determine the action space of the computing resource recommendation model;

[0176] Through the preset reward function, the computing power resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing power resource recommendation model.

[0177] In one embodiment, the recommendation model building module 906 is further configured to:

[0178] Construct a computing network resource state vector based on the resource state information of each computing resource in the computing resource pool;

[0179] Construct a resource matching vector based on the resource matching degree of computing resources for each user's business;

[0180] Construct a popularity vector based on the popularity of each computing resource in the computing resource pool;

[0181] Based on the computing network resource state vector, resource matching vector and popularity vector, the state space of the computing power resource recommendation model is determined.

[0182] In one embodiment, the recommendation model building module 906 is further configured to:

[0183] Initialize the training environment and the agent to be trained according to the state space, action space and reward function;

[0184] Based on the environment and the agent to be trained, strategy training is performed through a reinforcement learning algorithm until the agent to be trained converges to obtain a trained computing resource recommendation model.

[0185] Each module in the resource recommendation device 900 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0186] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store resource status information, user business demand information, user historical behavior data, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a resource recommendation method is implemented.

[0187] Those skilled in the art will understand that Fig.10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0188] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0189] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0190] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0191] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0192] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0193] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0194] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A resource recommendation method, characterized in that: The method comprises: Building a resource popularity prediction model based on user historical behavior data and resource status information; the resource status information is used to characterize the performance status and price status of computing resources; Obtaining a resource matching degree according to the resource status information and the user service demand information; the resource matching degree is used to characterize the matching degree between the computing power resources and the user service; Based on the resource popularity prediction model and the resource matching degree, the computing power resource recommendation model to be trained is subjected to reinforcement learning training to obtain a trained computing power resource recommendation model; The computing power resource recommendation results for the target user are obtained through the trained computing power resource recommendation model.

2. The method according to claim 1, characterized in that: The resource popularity prediction model is constructed based on user historical behavior data and resource status information, including: According to the user's historical behavior data, mark the popularity of the browsed computing resources; Using a recurrent neural network model, iterative training is performed to obtain the resource popularity prediction model; Among them, the input of the recurrent neural network model includes the resource status information and popularity of all the computing power resources in the computing power resource pool in the previous time slot, and the resource status information of all the computing power resources in the computing power resource pool in the current time slot; the output of the recurrent neural network model includes the popularity of all the computing power resources in the computing power resource pool in the current time slot.

3. The method according to claim 1, characterized in that The obtaining of resource matching degree according to the resource status information and the user service requirement information includes: According to the user business demand information and the resource status information of the computing power resources under the load state, the resource matching degree between any of the computing power resources and any of the user business demand information is obtained based on vector similarity.

4. The method according to claim 2, characterized in that: The step of performing reinforcement learning training on the computing resource recommendation model to be trained based on the resource popularity prediction model and the resource matching degree to obtain a trained computing resource recommendation model includes: Determining a state space of the computing resource recommendation model according to the popularity of the computing resource, the resource state information, and the resource matching degree; Determine the action space of the computing resource recommendation model based on all the computing resources in the current computing resource pool; Through a preset reward function, reinforcement learning training is performed on the computing power resource recommendation model to be trained to obtain the trained computing power resource recommendation model.

5. The method according to claim 4, characterized in that Determining the state space of the computing resource recommendation model according to the popularity of the computing resource, the resource state information, and the resource matching degree includes: Constructing a computing network resource state vector according to the resource state information of each computing resource in the computing resource pool; Constructing a resource matching vector according to the resource matching degree of the computing power resources for each user service; Constructing a popularity vector according to the popularity of each of the computing resources in the computing resource pool; Based on the computing network resource state vector, the resource matching vector and the popularity vector, the state space of the computing resource recommendation model is determined.

6. The method according to claim 4, characterized in that The method of using a preset reward function to perform reinforcement learning training on the computing resource recommendation model to be trained to obtain the trained computing resource recommendation model includes: Initialize the training environment and the agent to be trained according to the state space, the action space and the reward function; Based on the environment and the agent to be trained, strategy training is performed through a reinforcement learning algorithm until the agent to be trained converges, so as to obtain the computing power resource recommendation model after the training is completed.

7. A resource recommendation device, characterized in that: The device comprises: A prediction model building module, used to build a resource popularity prediction model based on user historical behavior data and resource status information; the resource status information is used to characterize the performance status and price status of computing resources; A resource matching degree acquisition module, used to acquire the resource matching degree according to the resource status information and the user service demand information; the resource matching degree is used to characterize the matching degree between the computing power resources and the user service; A recommendation model building module, used to perform reinforcement learning training on the computing resource recommendation model to be trained based on the resource popularity prediction model and the resource matching degree, to obtain a trained computing resource recommendation model; The recommendation result output module is used to obtain the computing power resource recommendation result for the target user through the computing power resource recommendation model that has been trained.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images