Information decision-making method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides an information decision method and device, electronic equipment and a medium, and relates to the technical field of computers. The method comprises: according to related information of a plurality of historical travel orders, obtaining sample feature data, the related information of the historical travel orders including subsidy rates corresponding to the historical travel orders, and the sample feature data including order features, budget control features and environment features; and inputting the sample feature data as input data into a pre-constructed reinforcement learning model, training the pre-constructed reinforcement learning model to obtain a trained reinforcement learning model, predicting the to-be-processed travel order by using the trained reinforcement learning model, and determining a subsidy rate corresponding to the to-be-processed travel order. According to the method, the order features, the budget control features and the environment features related to the historical travel order are learned through the reinforcement learning model, the subsidy rate is dynamically predicted and adjusted in real time based on external environment perception, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to an information decision method, device, electronic equipment and medium. Background Art

[0002] With the development of mobile Internet, more and more people choose online ride-hailing for travel, and the competition among manufacturers on online ride-hailing aggregation platforms has become increasingly fierce. In order to obtain more orders on the platform, online ride-hailing manufacturers usually provide certain subsidies or discounts by manually monitoring the bidding ranking in real time. In order to provide users with efficient and high-quality services, the bidding time is very short, generally only a few seconds or even tens of milliseconds. However, manual processing is not only costly, but also has time delays and delayed responses, which cannot meet real-time needs, and thus cannot respond to market changes in a timely manner, and may miss the best time for adjustment. At the same time, manual processing may not be able to obtain all the necessary information in time to make decisions, and subsidy adjustment efficiency is low. Summary of the invention

[0003] In order to solve the above technical problems or at least partially solve the above technical problems, an embodiment of the present invention provides an information decision method, device, electronic device and medium.

[0004] In a first aspect, an embodiment of the present invention provides an information decision method, comprising:

[0005] Acquire a historical travel order sequence, where the historical travel order sequence includes multiple historical travel orders;

[0006] According to the relevant information of the plurality of historical travel orders, sample feature data of each historical travel order is obtained; the relevant information of the historical travel order includes the subsidy rate corresponding to the historical travel order; the sample feature data includes order features, budget control features and environmental features;

[0007] Inputting the sample feature data as input data into a pre-constructed reinforcement learning model, training the pre-constructed reinforcement learning model, and obtaining a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the pending travel order based on the relevant information of the travel orders processed in the current cycle and the current environmental status;

[0008] Obtain pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

[0009] In an optional embodiment, the order characteristics include one or more of the following: the pre-coupon price of the historical travel order, the subsidy rate corresponding to the historical travel order, and the position of the historical travel order in the historical travel order sequence; the budget control characteristics include a preset target subsidy rate and / or a subsidy rate deviation threshold; the environmental characteristics include one or more of the following: the popularity coefficient, competition degree coefficient and weather characteristics of the area where the historical travel order is located.

[0010] In an optional embodiment, the pre-built reinforcement learning model includes an actor network and a critic network, the actor network is used to select a subsidy rate under a current environmental state, and the critic network is used to evaluate whether the subsidy rate selected by the actor network meets the business goal;

[0011] The step of inputting the sample feature data as input data into a pre-built reinforcement learning model to train the pre-built reinforcement learning model includes:

[0012] For each historical travel order, the sample feature data of the historical travel order is input into the actor network as input data, and the actor network determines the probability values ​​of multiple candidate subsidy rates being selected according to the sample feature data, and takes the candidate subsidy rate with the largest probability value as the subsidy rate to be evaluated;

[0013] Inputting the sample feature data and the subsidy rate to be evaluated as input data into the critic network to obtain estimated value information output by the critic network;

[0014] According to the estimated value information output by the critic network, the parameters of the critic network and the actor network are updated respectively;

[0015] Repeat the above steps until the training stop condition is met.

[0016] In an optional embodiment, the critic network determines the penalty item corresponding to the subsidy rate to be evaluated based on the current cumulative subsidy rate and the target subsidy rate; the current cumulative subsidy rate is the sum of the subsidy rate corresponding to the historical travel orders and the subsidy rate corresponding to the travel orders that are located before the historical travel order in the historical travel order sequence; the critic network calculates the reward value based on the penalty item corresponding to the subsidy rate to be evaluated and the post-voucher income of the historical travel orders at the subsidy rate to be evaluated; the critic network calculates the time difference error based on the reward value, and uses the time difference error as the estimated value information output by the critic network.

[0017] In an optional embodiment, the critic network determines the current cumulative subsidy rate based on the subsidy rate to be evaluated; when the current cumulative subsidy rate is greater than the target subsidy rate, the penalty item corresponding to the subsidy rate to be evaluated is determined, and the reward value is determined based on the post-coupon income of the historical travel orders at the subsidy rate to be evaluated and the penalty item corresponding to the subsidy rate to be evaluated; when the current cumulative subsidy rate is less than or equal to the target subsidy rate, the post-coupon income of the historical travel orders at the subsidy rate to be evaluated is determined as the reward value.

[0018] In an optional embodiment, when the difference between the current cumulative subsidy rate and the target subsidy rate is greater than or equal to the subsidy rate deviation threshold, the critic network determines the penalty item corresponding to the subsidy rate to be evaluated, and determines the reward value based on the post-coupon income of the historical travel orders at the subsidy rate to be evaluated and the penalty item corresponding to the subsidy rate to be evaluated; when the difference between the current cumulative subsidy rate and the target subsidy rate is less than the subsidy rate deviation threshold, the post-coupon income of the historical travel orders at the subsidy rate to be evaluated is determined as the reward value.

[0019] In an optional embodiment, the critic network calculates the reward value according to the following formula (1):

[0020] return=reward*(1-(p action -p target ))*max((1-i / len(arr)), β) (1)

[0021] Wherein, return represents the reward value, reward represents the post-coupon income of the historical travel order under the subsidy rate to be evaluated, (1-(p action -p target ))*max((1-i / len(arr)), β) represents the penalty term corresponding to the subsidy rate to be evaluated, p action represents the subsidy rate to be evaluated, p target represents the target subsidy rate, i represents the position of the historical travel order in the historical travel order sequence, len(arr) represents the preset total travel order volume of the current period, and β represents the preset hyperparameter.

[0022] In a second aspect, an embodiment of the present invention provides an information decision device, including:

[0023] An acquisition module, used to acquire a historical travel order sequence, wherein the historical travel order sequence includes a plurality of historical travel orders;

[0024] A feature extraction module, configured to obtain sample feature data of each historical travel order according to the relevant information of the plurality of historical travel orders; the relevant information of the historical travel orders includes a subsidy rate corresponding to the historical travel orders; the sample feature data includes order features, budget control features and environmental features;

[0025] A training module, used to input the sample feature data as input data into a pre-built reinforcement learning model, train the pre-built reinforcement learning model, and obtain a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the travel order to be processed according to the relevant information of the travel orders processed in the current cycle and the current environmental status;

[0026] The prediction module is used to obtain the pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

[0027] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the information decision method provided by any embodiment of the present invention when executing the program stored in the memory.

[0028] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information decision method provided by any embodiment of the present invention.

[0029] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0030] The information decision method provided by the embodiment of the present invention obtains a historical travel order sequence, obtains sample feature data based on the relevant information of each historical travel order in the historical travel order sequence, the relevant information of the historical travel order includes the subsidy rate, and the sample feature data includes order features, budget control features and environmental features, and then the order features, budget control features and environmental features are input as input data into a pre-constructed reinforcement learning model, and the pre-constructed reinforcement learning model is trained to obtain a trained reinforcement learning model, which is used to determine the subsidy rate corresponding to the pending travel order based on the relevant information of the processed travel orders in the current cycle and the current environmental state. The method learns the order features, budget control features and environmental features related to the historical travel orders through the reinforcement learning model, realizes the real-time and dynamic prediction and adjustment of the subsidy rate based on the perception of the external environment, and reduces the labor cost. At the same time, because the reinforcement learning model learns the budget control features, the subsidy rate determined by the model can achieve a balance between cost and benefit and achieve the bidding target. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0032] Figure 1 A schematic diagram showing a flow chart of an information decision-making method provided by an embodiment of the present invention;

[0033] Figure 2 A schematic diagram showing a flow chart of an information decision method provided by another embodiment of the present invention;

[0034] Figure 3 A schematic diagram showing the structure of an information decision-making device provided by an embodiment of the present invention is shown;

[0035] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0037] Figure 1 FIG. 2 is a flow chart showing the information decision-making method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0038] Step S101: Obtain a historical travel order sequence, wherein the historical travel order sequence includes multiple historical travel orders. Optionally, historical travel orders within a period of time (e.g., 10 days, 30 days, etc.) can be obtained from the online car-hailing platform, and the obtained historical travel orders are sorted according to the order generation time to generate a historical travel order sequence.

[0039] Step S102: acquiring sample feature data of each historical travel order based on the relevant information of the plurality of historical travel orders.

[0040] Optionally, the relevant information of historical travel orders includes the subsidy rate corresponding to the historical travel orders. The subsidy rate refers to the ratio of the subsidy amount corresponding to the historical travel orders to the total pre-coupon price of all historical travel orders. The pre-coupon price refers to the price of the travel order before the subsidy is used, and the post-coupon price refers to the price of the travel order after the subsidy is used. Still optionally, the relevant information of historical travel orders may also include, but is not limited to: identification information of the travel order such as ID, generation time of the travel order, whether an order is given (i.e., whether the travel order is assigned to a certain online car-hailing manufacturer), the pre-coupon price of the travel order, the post-coupon price of the travel order, etc.

[0041] Optionally, the sample feature data includes order features, budget control features, and environmental features. Order features are used to describe travel orders. Budget control features are used to describe the control rules for subsidies for travel orders. Environmental features are used to describe environmental conditions, such as the popularity, competition intensity, and weather conditions of the area where the travel order is located.

[0042] In an optional embodiment, the order characteristics include one or more of the following: the pre-coupon price of the historical travel order, the subsidy rate of the historical travel order, and the position of the historical travel order in the historical travel order sequence (for example, the historical travel order sequence includes 10,000 orders, and the historical travel order is the 2000th order, then the position of the historical travel order is 2000 / 10000, which can also be expressed as 20%).

[0043] In an optional embodiment, the budget control feature includes a preset target subsidy rate and / or a subsidy rate deviation threshold. The target subsidy rate refers to the preset expected subsidy rate. The subsidy rate deviation refers to the difference between two subsidy rates, for example, the difference between the target subsidy rate and the cumulative subsidy rate at the current moment. The cumulative subsidy rate at the current moment (referred to as the current cumulative subsidy rate) refers to the ratio of the total amount of subsidies provided at the current moment to the sum of the post-coupon prices of all travel orders before the current moment. The subsidy rate deviation threshold refers to the maximum subsidy rate deviation, for example 1%.

[0044] In an optional embodiment, the environmental characteristics include one or more of the following: the heat coefficient of the area where the historical travel orders are located, the competition coefficient of the area, and weather characteristics. Among them, the heat coefficient is used to describe the degree of attention the area has received over a period of time. The higher the heat coefficient, the higher the degree of attention the area has received, and the area is a popular area. The competition coefficient is used to describe the degree of competition among multiple online ride-hailing platforms in the area where the historical travel orders are located. The higher the competition coefficient, the higher the degree of competition for online ride-hailing in this area. Weather characteristics are used to describe weather conditions, such as sunny, light rain, heavy rain, light snow, etc.

[0045] Step S103: Input the sample feature data as input data into a pre-built reinforcement learning model, train the pre-built reinforcement learning model, and obtain a trained reinforcement learning model. The trained reinforcement learning model is used to determine the subsidy rate corresponding to the pending travel order based on the relevant information of the travel orders processed in the current cycle and the current environmental status.

[0046] The reinforcement learning model constructed by the embodiment of the present invention uses a reward function to judge the quality of performing a certain action under a certain state. Its learning process is to change the strategy of executing the action through the reward signal, and finally form a strategy that maximizes the reward. The state of the reinforcement learning model constructed by the embodiment of the present invention includes sample feature data obtained from the relevant information of historical travel orders, the reward is the post-coupon income (that is, the income after the travel order is subsidized), and the strategy is to determine the subsidy rate corresponding to the travel order from the preset discrete subsidy rate, so that the post-coupon income of the travel order is maximized. For example, the discrete subsidy rate set is {0.0, 0.05, 0.1, 0.15, 0.2}, and the trained reinforcement learning model determines the subsidy rate corresponding to the travel order to be processed from the set.

[0047] The reinforcement learning model constructed in the embodiment of the present invention aims to maximize the post-coupon income, learns the order characteristics, budget control characteristics and environmental characteristics related to historical travel orders, and determines the subsidy rate corresponding to each travel order in real time and dynamically. In an optional embodiment, with an hourly cycle, the sample feature data obtained based on the relevant information of the historical travel orders for each hour is used as a set of training data to input into the reinforcement learning model for learning.

[0048] Step S104: Obtain the travel orders to be processed, use the trained reinforcement learning model to predict the travel orders to be processed, and determine the subsidy rate corresponding to the travel orders to be processed.

[0049] Optionally, relevant information of travel orders processed in the current cycle (such as the subsidy rate of processed travel orders), current environmental status (such as the heat coefficient and competition degree coefficient of the area where the travel orders to be processed are located), information of travel orders to be processed, and budget control characteristics are input as input data into the trained reinforcement learning model to predict the travel orders to be processed, determine the subsidy rate corresponding to the travel orders to be processed, and then determine the price information of the travel orders to be processed after the subsidy.

[0050] The information decision method provided by the embodiment of the present invention obtains sample feature data by obtaining a historical travel order sequence and obtaining sample feature data according to the relevant information of each historical travel order in the historical travel order sequence. The relevant information of the historical travel order includes the subsidy rate. The sample feature data includes order features, budget control features and environmental features. The order features, budget control features and environmental features are then input into a pre-constructed reinforcement learning model as input data. The pre-constructed reinforcement learning model is trained to obtain a trained reinforcement learning model. The reinforcement learning model is used to determine the subsidy rate corresponding to the pending travel order according to the relevant information of the processed travel orders in the current cycle and the current environmental state. The method learns the order features, budget control features and environmental features related to the historical travel orders through the reinforcement learning model, realizes the real-time and dynamic prediction and adjustment of the subsidy rate based on the external environment perception, reduces the labor cost, and at the same time, because the reinforcement learning model learns the budget control features, the subsidy rate determined by the model can achieve a balance between cost and benefit and achieve the bidding target.

[0051] In an optional embodiment, the pre-built reinforcement learning model includes an actor network (Actor) and a critic network (Critic). Among them, the actor network selects the action a to be performed in state s based on the current strategy π(a|s). In this embodiment, the actor network is used to select the subsidy rate under the current environmental state. Optionally, the strategy of the actor network can be expressed as a strategy distribution, which can be discrete or continuous. The goal of the actor network is to maximize the cumulative expected return. By adjusting the strategy parameters, the probability of selecting a high-quality action in a given state is higher. The actor network updates its strategy parameters based on the feedback information provided by the critic network (usually the advantage function or Q value). The method for updating the strategy is usually the policy gradient method (Policy Gradient), such as the REINFORCE algorithm. The critic network is used to evaluate the performance of the current strategy of the actor network. In this embodiment, the critic network is used to evaluate whether the subsidy rate selected by the actor network meets the business objectives to provide feedback information to the actor network. The critic network evaluates whether the subsidy rate selected by the actor network meets the business objectives by estimating the state value function V(s) or the action-state value function Q(s, a), that is, providing feedback information to the actor network. Optionally, the critic network updates the state value function V(s) or the action-state value function Q(s, a) by minimizing the temporal difference error (TD Error). The definition of the temporal difference error is as follows:

[0052] δ=r+γV(s')-V(s)

[0053] r represents the reward value estimated by the critic network, γ represents the discount factor, s represents the current state, and s' represents the next state.

[0054] In an optional embodiment, both the actor network and the critic may adopt a three-layer neural network, for example, including an input layer, one or more hidden layers (the hidden layer usually uses a fully connected layer, a convolutional layer or a recurrent layer) and an output layer.

[0055] Figure 2 FIG. 2 is a flow chart showing a process of training a reinforcement learning model in the information decision-making method provided by an embodiment of the present invention. Figure 2 As shown in Figure 1, the process of training a reinforcement learning model includes:

[0056] Step S201: For each historical travel order, the sample feature data of the historical travel order is input as input data into the actor network, and the actor network determines the probability values ​​of multiple candidate subsidy rates being selected according to the sample feature data, and takes the candidate subsidy rate with the largest probability value as the subsidy rate to be evaluated. Optionally, the preset candidate subsidy rates include: {0.0, 0.05, 0.1, 0.15, 0.2}.

[0057] Step S202: Input the sample feature data and the subsidy rate to be evaluated as input data into the critic network to obtain the estimated value information output by the critic network.

[0058] Optionally, the critic network determines the penalty item corresponding to the subsidy rate to be evaluated based on the current cumulative subsidy rate and the target subsidy rate, calculates the reward value based on the penalty item corresponding to the subsidy rate to be evaluated and the post-voucher income of the historical travel orders under the subsidy rate to be evaluated, and then calculates the time series difference error based on the reward value, and uses the time series difference error as the estimated value information provided to the actor network. Among them, the current cumulative subsidy rate is the sum of the subsidy rate corresponding to the historical travel order and the subsidy rate corresponding to the travel order before the historical travel order in the historical travel order sequence. For example, if the historical travel order is the 200th order in the travel order sequence, the current cumulative subsidy rate is the sum of the subsidy rates from the 1st historical travel order to the 200th historical travel order (including the 200th historical travel order).

[0059] In order to prevent the total subsidy for travel orders from being controlled and to prevent excessive subsidies, the embodiment of the present invention determines a reward value based on the gap between the current cumulative subsidy rate and the target subsidy rate. For example, when the current cumulative subsidy rate is less than or equal to the target subsidy rate, the post-coupon income of historical travel orders at the subsidy rate to be evaluated is determined as the reward value. When the current cumulative subsidy rate is greater than the target subsidy rate, the penalty item of the subsidy rate to be evaluated is determined, and the reward value is determined based on the post-coupon income of historical travel orders at the subsidy rate to be evaluated and the penalty item of the subsidy rate to be evaluated. For example, the sum of the post-coupon income of historical travel orders at the subsidy rate to be evaluated and the penalty item of the subsidy rate to be evaluated is used as the reward value.

[0060] Furthermore, when the difference between the current cumulative subsidy rate and the target subsidy rate is less than or equal to the subsidy rate deviation threshold, the post-coupon income of historical travel orders at the subsidy rate to be evaluated is used as the reward value. When the difference between the current cumulative subsidy rate and the target subsidy rate is greater than the subsidy rate deviation threshold, the penalty item of the subsidy rate to be evaluated is determined; the sum of the penalty item of the subsidy rate to be evaluated and the post-coupon price of historical travel orders at the subsidy rate to be evaluated is used as the reward value. Optionally, the penalty item corresponding to different subsidy rates can be determined according to a preset functional relationship, and the functional relationship can be flexibly set according to application requirements.

[0061] As an optional example, the penalty term of the subsidy rate to be evaluated can be determined according to the following formula:

[0062] (1-(p action -P target ))*max((1-i / len(arr)),β)

[0063] p action represents the subsidy rate to be evaluated, p target Represents the target subsidy rate, i represents the position of the travel order in the historical travel order sequence, len(arr) represents the preset total travel order volume of the current cycle, and β represents the preset hyperparameter. Among them, the preset total travel order volume represented by len(arr) is the estimated total travel order volume of the current cycle. Optionally, the preset total travel order volume represented by len(arr) can be determined based on the total volume of historical travel orders, for example, by calculating the sum of the total volume of travel orders in the previous N cycles, and then calculating the average value of the N cycles, and using the average value as the estimated total volume of travel orders. β is a real number greater than 0 and less than 1, for example 0.1.

[0064] The critic network determines the reward value according to the following formula:

[0065] return=reward*(1-(p action -p target ))*max((1-i / len(arr)),β)

[0066] Where return represents the reward value, and reward represents the post-coupon income of historical travel orders under the subsidy rate to be evaluated, (1-(p action -p target ))*max((1-i / len(arr)), β) represents the penalty term corresponding to the subsidy rate to be evaluated, p action represents the subsidy rate to be evaluated, p target represents the target subsidy rate, i represents the position of the travel order in the historical travel order sequence, len(arr) represents the total number of travel orders in the current period, and β represents the preset hyperparameter.

[0067] After the reward value is determined, the temporal difference error is calculated according to the reward value, and the temporal difference error is used as feedback information output by the critic network.

[0068] Step S203: According to the estimated value information output by the critic network, the parameters of the critic network and the actor network are updated respectively.

[0069] Step S204: Repeat the above steps until the training stop condition is met, for example, when the reinforcement learning model converges or the number of training times of the reinforcement learning model reaches a specified value, the training is stopped.

[0070] The information decision-making method provided by the embodiment of the present invention can timely and accurately determine the subsidy rate corresponding to the pending travel orders through the trained actor network-critic network to meet the business objectives.

[0071] Figure 3 FIG. 1 is a schematic diagram showing the structure of an information decision device provided by an embodiment of the present invention. Figure 3 As shown, the information decision device 300 includes:

[0072] An acquisition module 301 is used to acquire a historical travel order sequence, where the historical travel order sequence includes a plurality of historical travel orders;

[0073] The feature extraction module 302 is used to obtain sample feature data of each historical travel order according to the relevant information of the plurality of historical travel orders; the relevant information of the historical travel orders includes the subsidy rate corresponding to the historical travel orders; the sample feature data includes order features, budget control features and environmental features;

[0074] The training module 303 is used to input the sample feature data as input data into a pre-built reinforcement learning model, train the pre-built reinforcement learning model, and obtain a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the travel order to be processed according to the relevant information of the travel orders processed in the current cycle and the current environmental status;

[0075] The prediction module 304 is used to obtain the pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

[0076] Optionally, the pre-built reinforcement learning model includes an actor network and a critic network, the actor network is used to select the subsidy rate under the current environmental state, and the critic network is used to evaluate whether the subsidy rate selected by the actor network meets the business objectives; the training module 303 is also used to: for each historical travel order, input the sample feature data of the historical travel order as input data into the actor network, the actor network determines the probability values ​​of multiple candidate subsidy rates being selected based on the sample feature data, and takes the candidate subsidy rate with the largest probability value as the subsidy rate to be evaluated; input the sample feature data and the subsidy rate to be evaluated as input data into the critic network to obtain the estimated value information output by the critic network; according to the estimated value information output by the critic network, update the parameters of the critic network and the actor network respectively; repeat the above steps until the training stop condition is met.

[0077] The above device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the information decision method provided by the embodiment of the present invention.

[0078] Figure 4 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, the electronic device includes:

[0079] Processor 401, communication interface 402, memory 403 and communication bus 404, wherein processor 401, communication interface 402, memory 403 communicate with each other via communication bus 404.

[0080] Memory 403, used for storing computer programs;

[0081] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0082] Acquire a historical travel order sequence, where the historical travel order sequence includes multiple historical travel orders;

[0083] According to the relevant information of the plurality of historical travel orders, sample feature data of each historical travel order is obtained; the relevant information of the historical travel order includes the subsidy rate corresponding to the historical travel order; the sample feature data includes order features, budget control features and environmental features;

[0084] Inputting the sample feature data as input data into a pre-constructed reinforcement learning model, training the pre-constructed reinforcement learning model, and obtaining a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the pending travel order based on the relevant information of the travel orders processed in the current cycle and the current environmental status;

[0085] Obtain pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

[0086] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0087] The communication interface is used for communication between the above terminal and other devices.

[0088] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0089] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0090] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the information decision-making method described in any of the above embodiments.

[0091] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the information decision method described in any one of the above embodiments.

[0092] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0093] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0094] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An information decision method, characterized in that: include: Acquire a historical travel order sequence, where the historical travel order sequence includes multiple historical travel orders; According to the relevant information of the plurality of historical travel orders, sample feature data of each historical travel order is obtained; the relevant information of the historical travel order includes the subsidy rate corresponding to the historical travel order; the sample feature data includes order features, budget control features and environmental features; Inputting the sample feature data as input data into a pre-constructed reinforcement learning model, training the pre-constructed reinforcement learning model, and obtaining a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the pending travel order based on the relevant information of the travel orders processed in the current cycle and the current environmental status; Obtain pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

2. The method according to claim 1, characterized in that The order characteristics include one or more of the following: the pre-coupon price of the historical travel order, the subsidy rate corresponding to the historical travel order, and the position of the historical travel order in the historical travel order sequence; The budget control features include a preset target subsidy rate and / or subsidy rate deviation threshold; The environmental characteristics include one or more of the following: a heat coefficient, a competition coefficient, and weather characteristics of the area where the historical travel orders are located.

3. The method according to claim 1 or 2, characterized in that: The pre-built reinforcement learning model includes an actor network and a critic network, wherein the actor network is used to select a subsidy rate under a current environment state, and the critic network is used to evaluate whether the subsidy rate selected by the actor network meets the business goal; The step of inputting the sample feature data as input data into a pre-built reinforcement learning model to train the pre-built reinforcement learning model includes: For each historical travel order, the sample feature data of the historical travel order is input into the actor network as input data, and the actor network determines the probability values ​​of multiple candidate subsidy rates being selected according to the sample feature data, and takes the candidate subsidy rate with the largest probability value as the subsidy rate to be evaluated; Inputting the sample feature data and the subsidy rate to be evaluated as input data into the critic network to obtain estimated value information output by the critic network; According to the estimated value information output by the critic network, the parameters of the critic network and the actor network are updated respectively; Repeat the above steps until the training stop condition is met.

4. The method according to claim 3, characterized in that The critic network determines the penalty item corresponding to the subsidy rate to be evaluated according to the current cumulative subsidy rate and the target subsidy rate; the current cumulative subsidy rate is the sum of the subsidy rate corresponding to the historical travel order and the subsidy rate corresponding to the travel order before the historical travel order in the historical travel order sequence; The critic network calculates the reward value according to the penalty item corresponding to the subsidy rate to be evaluated and the post-coupon income of the historical travel order under the subsidy rate to be evaluated; The critic network calculates a temporal difference error according to the reward value, and uses the temporal difference error as estimated value information output by the critic network.

5. The method according to claim 4, characterized in that The critic network determines the penalty item corresponding to the subsidy rate to be evaluated when the current cumulative subsidy rate is greater than the target subsidy rate, and determines the reward value according to the post-coupon income of the historical travel order at the subsidy rate to be evaluated and the penalty item corresponding to the subsidy rate to be evaluated; When the current cumulative subsidy rate is less than or equal to the target subsidy rate, the post-coupon income of the historical travel order at the subsidy rate to be evaluated is determined as the reward value.

6. The method according to claim 5, characterized in that The critic network determines the penalty item corresponding to the subsidy rate to be evaluated when the difference between the current cumulative subsidy rate and the target subsidy rate is greater than or equal to the subsidy rate deviation threshold, and determines the reward value according to the post-coupon income of the historical travel order at the subsidy rate to be evaluated and the penalty item corresponding to the subsidy rate to be evaluated; When the difference between the current cumulative subsidy rate and the target subsidy rate is less than the subsidy rate deviation threshold, the post-coupon income of the historical travel order at the subsidy rate to be evaluated is determined as the reward value.

7. An information decision device, characterized in that: include: An acquisition module, used to acquire a historical travel order sequence, wherein the historical travel order sequence includes a plurality of historical travel orders; A feature extraction module, configured to obtain sample feature data of each historical travel order according to the relevant information of the plurality of historical travel orders; the relevant information of the historical travel orders includes a subsidy rate corresponding to the historical travel orders; the sample feature data includes order features, budget control features and environmental features; A training module, used to input the sample feature data as input data into a pre-built reinforcement learning model, train the pre-built reinforcement learning model, and obtain a trained reinforcement learning model; the trained reinforcement learning model is used to determine the subsidy rate corresponding to the travel order to be processed according to the relevant information of the travel orders processed in the current cycle and the current environmental status; The prediction module is used to obtain the pending travel orders, use the trained reinforcement learning model to predict the pending travel orders, and determine the subsidy rate corresponding to the pending travel orders.

8. The device according to claim 7, characterized in that The pre-built reinforcement learning model includes an actor network and a critic network, wherein the actor network is used to select a subsidy rate under a current environment state, and the critic network is used to evaluate whether the subsidy rate selected by the actor network meets the business goal; The training module is also used to: For each historical travel order, the sample feature data of the historical travel order is input into the actor network as input data, and the actor network determines the probability values ​​of multiple candidate subsidy rates being selected according to the sample feature data, and takes the candidate subsidy rate with the largest probability value as the subsidy rate to be evaluated; Inputting the sample feature data and the subsidy rate to be evaluated as input data into the critic network to obtain estimated value information output by the critic network; According to the estimated value information output by the critic network, the parameters of the critic network and the actor network are updated respectively; Repeat the above steps until the training stop condition is met.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, configured to implement the method according to any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images