Malicious bidding behavior detection method and device in auction bidding, and computer equipment

Abstract

The invention relates to a malicious bidding behavior detection method and device in auction bidding, computer equipment, a storage medium and a computer program product, and the method comprises the steps: obtaining historical auction bidding data of a user; calculating a malicious coefficient corresponding to the historical auction bidding based on the historical auction bidding data; training the multiple agents according to the malicious coefficient to obtain the corresponding trained multiple agents when the action value is optimal; and obtaining real-time bidding data of the user, and performing malicious bidding behavior detection according to the real-time bidding data of the user and the trained multiple agents. In the whole process, the malicious coefficient is determined based on the historical auction bidding data of the user, then the bidding action under the malicious coefficient is trained and learned by the multiple agents, detection is performed based on the real-time bidding data of the user and the trained multiple agents, and malicious bidding behaviors in auction bidding can be efficiently and accurately detected.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for detecting malicious bidding behavior in auctions. Background Art

[0002] With the rapid development of internet technology, market bidding, as an efficient method for resource allocation, has gained widespread application in various fields, including advertising, finance, and e-commerce. Market bidding allows multiple participants to compete for the same resource or product, with the highest bidder securing the right to purchase it. However, with the widespread adoption of market bidding, malicious bidding behavior has become increasingly prominent, becoming a significant factor impacting market fairness and efficiency.

[0003] Malicious bidding refers to the use of illegal means, such as automated tools, fake accounts, and bid collusion, to intentionally inflate prices, manipulate the market, or deplete competitors' budgets. These actions not only harm the interests of other participants but also disrupt the normal competitive order of the market and reduce the efficiency of resource allocation. Traditional methods for detecting malicious bidding rely primarily on manual review and rule-matching techniques. However, these methods have the following limitations: 1) high cost and low efficiency of manual review; 2) limited flexibility in rule-matching; and 3) poor real-time performance.

[0004] Based on the above-mentioned shortcomings of traditional technologies, there is an urgent need for an efficient and accurate malicious bidding behavior detection solution. Summary of the Invention

[0005] Based on this, it is necessary to provide an efficient and accurate method, device, computer equipment, computer-readable storage medium and computer program product for detecting malicious bidding behavior in auction bidding in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for detecting malicious bidding behavior in an auction. The method comprises:

[0007] Obtain the user's historical auction bidding data;

[0008] Calculating the maliciousness coefficient corresponding to the historical auction bidding data;

[0009] Training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0010] Acquire real-time bidding data of users, and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agent.

[0011] In one embodiment, the calculating, based on the historical auction bidding data, the malicious coefficient corresponding to the historical auction bidding includes:

[0012] Extracting a bid vector corresponding to each piece of auction bidding data based on the historical auction bidding data;

[0013] Calculating a median bid based on the bid vector;

[0014] Obtaining a preset deviation threshold, and calculating a malicious value corresponding to each piece of auction bidding data based on the preset deviation threshold and the median bid; the preset deviation threshold is used to represent the market's tolerance for malicious bids;

[0015] The average value of the maliciousness value corresponding to each piece of auction bidding data is calculated to obtain a maliciousness coefficient.

[0016] In one embodiment, the step of training the multi-agent according to the maliciousness coefficient to obtain the trained multi-agent corresponding to the optimal action value includes:

[0017] Passing the malicious coefficient to the initial multi-agent;

[0018] Record the bids of the initial multi-agents and calculate the returns and rewards corresponding to the bids of the initial multi-agents in each round;

[0019] Based on the bid, return, reward and the policy network of the initial agent, an action-value function and an optimal action-value function of the agent are obtained;

[0020] The initial multi-agent is controlled to perform agent strategy learning, and the action-value function of the agent is updated using the Bellman equation so that the updated action-value function approaches the optimal action-value function, thereby obtaining a trained multi-agent.

[0021] In one embodiment, controlling the initial multi-agent to perform agent strategy learning, updating the action-value function of the agent using the Bellman equation so that the updated action-value function approaches the optimal action-value function, and obtaining the trained multi-agent includes:

[0022] Control the initial multi-agent to perform agent strategy learning, and obtain the corresponding four-tuple in the agent strategy learning process, wherein the four-tuple includes the bids of all agents at the previous moment, the bids of the agents at the current moment, the agent rewards, and the bids of all agents at the current moment;

[0023] Calculate the action value generated by the current action value function based on the four-tuple;

[0024] According to the TD algorithm and Monte Carlo, the action value under the optimal action value function is obtained;

[0025] The action-value function is updated using the gradient ascent method so that the updated action-value function approaches the optimal action-value function, thereby obtaining a trained multi-agent.

[0026] In one embodiment, the agent reward is obtained in the following manner:

[0027] Initialize the agent's reward vector;

[0028] Select the agent whose bid is larger than the median bid to form the first agent index vector;

[0029] Calculating the deviation of the agent bid corresponding to the first agent index vector;

[0030] Select the agent whose deviation is greater than the preset deviation threshold and obtain the second agent index vector;

[0031] Giving negative rewards to the agents in the second agent index vector;

[0032] If the auction ends, the winning agent is rewarded;

[0033] Returns the reward vector of the initialized agent until the auction ends.

[0034] In one embodiment, the method for detecting malicious bidding behavior in an auction further includes:

[0035] Determining whether the maliciousness coefficient meets a preset dynamic adjustment condition;

[0036] If not, the malicious coefficient is passed to the trained user agent;

[0037] If so, the malicious coefficient is adjusted to a preset fixed malicious coefficient, and the preset fixed malicious coefficient is transferred to the trained user agent.

[0038] In a second aspect, the present application also provides a device for detecting malicious bidding behavior in auctions. The device comprises:

[0039] Historical data acquisition module, used to obtain the user's historical auction bidding data;

[0040] A malicious coefficient calculation module, configured to calculate a malicious coefficient corresponding to a historical auction bid based on the historical auction bid data;

[0041] A training module, configured to train the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0042] The detection module is used to obtain real-time bidding data of users and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agent.

[0043] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0044] Obtain the user's historical auction bidding data;

[0045] Based on the historical auction bidding data, calculating the maliciousness coefficient corresponding to the historical auction bidding;

[0046] Training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0047] Acquire real-time bidding data of users, and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agent.

[0048] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0049] Obtain the user's historical auction bidding data;

[0050] Based on the historical auction bidding data, calculating the maliciousness coefficient corresponding to the historical auction bidding;

[0051] Training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0052] Acquire real-time bidding data of users, and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agent.

[0053] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0054] Obtain the user's historical auction bidding data;

[0055] Calculating the maliciousness coefficient corresponding to the historical auction bidding data;

[0056] Training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0057] Acquire real-time bidding data of users, and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agent.

[0058] The above-mentioned method, apparatus, computer equipment, storage medium, and computer program product for detecting malicious bidding behavior in auctions obtain a user's historical auction bidding data; based on the historical auction bidding data, calculate the malicious coefficient corresponding to the historical auction bidding; train a multi-agent based on the malicious coefficient to obtain a trained multi-agent corresponding to the optimal action value; obtain the user's real-time bidding data, and detect malicious bidding behavior based on the user's real-time bidding data and the trained multi-agent. Throughout the entire process, the malicious coefficient is determined based on the user's historical auction bidding data, and the multi-agent is then trained to learn bidding actions under the malicious coefficient. Detection is performed based on the user's real-time bidding data and the trained multi-agent, which can efficiently and accurately detect malicious bidding behavior in auctions. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A diagram illustrating an application environment of a method for detecting malicious bidding behavior in an auction in one embodiment;

[0060] Figure 2 1 is a flow chart of a method for detecting malicious bidding behavior in an auction in one embodiment;

[0061] Figure 3 1 is a flow chart of a method for detecting malicious bidding behavior in an auction in another embodiment;

[0062] Figure 4 1. A structural block diagram of a device for detecting malicious bidding behavior in an auction according to an embodiment;

[0063] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0065] The method for detecting malicious bidding behavior in auctions provided in the embodiments of the present application can be applied to Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store data that server 104 needs to process. The data storage system can be integrated with server 104, or placed on a cloud or other network server. Terminal 102 sends a malicious bidding behavior detection request to server 104. Server 104 responds to the request by obtaining the user's historical auction bidding data; based on the historical auction bidding data, calculates the malicious coefficient corresponding to the historical auction bidding; trains a multi-agent based on the malicious coefficient to obtain a trained multi-agent corresponding to the optimal action value; obtains the user's real-time bidding data, and performs malicious bidding behavior detection based on the user's real-time bidding data and the trained multi-agent. Furthermore, server 104 can feed back the malicious bidding behavior detection results to terminal 102. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can include smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0066] In one embodiment, Figure 2 As shown, a method for detecting malicious bidding behavior in auction bidding is provided. Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0067] S200: Obtain the user's historical auction bidding data.

[0068] A user's historical auction bidding data refers to the user's bids in each round of historical auctions, the auction item's value, and the user's index. Specifically, a piece of historical data consists of two vectors and a positive integer: the bid vector, the value vector, and the user index to be detected. The bid vector represents the final bids placed by all bidders in a particular auction. The value vector represents the perceived value of the auction item by each bidder. The index is used to locate the user's data within these two vectors. Here, we are obtaining a historical dataset containing several vectors of historical data.

[0069] S400: Calculate the maliciousness coefficient corresponding to the historical auction bidding data.

[0070] Analyze each piece of historical auction bidding data to analyze the malicious coefficient of the user in the historical auction bidding process. Specifically, we can first determine the median bid based on the historical auction bidding data. iIn addition, a deviation threshold θ is obtained. The deviation threshold θ is preset and determined by the market, which reflects the market's tolerance for malicious high prices. In addition, a function f is introduced i , indicating whether the user has the probability of being a malicious user under data i, based on the median of the bid i And the deviation threshold θ is used to calculate the f corresponding to each historical auction bidding data, and then the malicious coefficient corresponding to the historical auction bidding is calculated by accumulating and averaging.

[0071] S600: Train the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value.

[0072] The malicious coefficient calculated in step S400 is passed to the multi-agent, which then begins training for the auction environment. During the training process, the action value corresponding to each (bid) action is continuously calculated, and training continues until the trained multi-agent corresponding to the optimal action value is obtained. Specifically, the multi-agent here refers to all agents, including the user agent and other agents. During the training of all agents, the index starts with the user agent.

[0073] Specifically, during the training process, the malicious coefficient is first passed to the multi-agent, and the multi-agent initializes the auction bidding environment and state, starts bidding, initializes the bid vector, adds the bid of each agent to the bid vector, and continuously calculates the action value corresponding to each (bid) action. Each agent continuously trains and adjusts the bidding strategy until the trained multi-agent corresponding to the optimal action value is obtained.

[0074] S800: Obtain user real-time bidding data, and detect malicious bidding behavior based on the user real-time bidding data and trained multi-agents.

[0075] In actual applications, the detection of malicious bidding behavior in auction bidding is to first obtain the user's real-time bidding data, and then input the user's real-time bidding data into the trained multi-agent for detection to determine the malicious probability of the user being a malicious bidding user.

[0076] The above-mentioned method for detecting malicious bidding behavior in auctions obtains the user's historical auction bidding data; based on the historical auction bidding data, calculates the malicious coefficient corresponding to the historical auction bidding; trains a multi-agent based on the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value; obtains the user's real-time bidding data, and detects malicious bidding behavior based on the user's real-time bidding data and the trained multi-agent. Throughout this process, the malicious coefficient is determined based on the user's historical auction bidding data, and the multi-agent is then trained to learn bidding actions under this malicious coefficient. Detection is performed based on the user's real-time bidding data and the trained multi-agent, which can efficiently and accurately detect malicious bidding behavior in auctions.

[0077] like Figure 3 As shown, in one embodiment, S400 includes:

[0078] S420: Extracting the bid vector corresponding to each piece of auction bidding data based on the historical auction bidding data.

[0079] From the historical auction bidding data data, for each auction, the user bids are extracted to form a bid vector. As mentioned above, a historical data consists of two vectors and a positive integer, namely the bid vector bids and the value vector values, and the index of the user to be detected index. The bid vector bids is the final bid of everyone in a certain auction. The value vector values ​​is the value of a certain auction item in everyone's mind. The role of the index is to find the user's data in the two vectors. The historical auction bidding data set D contains several vectors of historical auction bidding data. That is, D = (data1, data2,…, data m ).

[0080] S440: Calculate the bid median based on the bid vector.

[0081] The bid vectors for each auction data item (i.e., each auction) are sorted and the median is found. As a robust statistic, the median can better reflect the central trend of bids in the auction while reducing the impact of extreme values.

[0082] For each piece of historical data i , find the median bid i :The specific calculation formula is as follows:

[0083]

[0084] The return value of sort is a vector in ascending order.

[0085] S460: Obtain a preset deviation threshold, and calculate the malicious value corresponding to each auction bidding data according to the preset deviation threshold and the median bid; the preset deviation threshold is used to represent the market's tolerance for malicious bids.

[0086] The preset deviation threshold is a value set based on factors such as market conditions, auction rules, and historical data. It represents the market's tolerance for malicious bids. This value can be a percentage (relative to the median bid) or a fixed amount. Its purpose is to distinguish between legitimate bids and potentially malicious bids. For each bid in each auction data entry, the deviation from the median bid is calculated. The deviation can be the absolute difference between the bid and the median, or a relative value adjusted based on the preset deviation threshold. If a bid's deviation exceeds the preset deviation threshold, the bid is considered malicious and assigned a maliciousness value. The maliciousness value can be a fixed value (such as 1) or a function of the deviation exceeding the threshold (such as linear or exponential growth).

[0087] Specifically, a function f can be introduced here i , indicating whether the user has a probability of being a malicious user under data i. Its specific function expression is as follows:

[0088]

[0089] Where θ is the deviation threshold, which is determined by the market and reflects the market's tolerance for malicious high prices.

[0090] S480: Calculate the average value of the maliciousness value corresponding to each piece of auction bidding data to obtain a maliciousness coefficient.

[0091] For each auction (i.e., each piece of auction bid data), the average or weighted average of the maliciousness coefficients corresponding to all bids is calculated. This average reflects the overall level of malicious bidding in that auction. It is important to note that if no bids in a particular auction exceed the preset deviation threshold, the maliciousness coefficient for that auction can be set to 0 or a very small value, indicating that no malicious bidding was detected in that auction.

[0092] Calculate f for each data and add the result to cnt:

[0093]

[0094] Finally, calculate the malicious coefficient ξ:

[0095]

[0096] In the above formula, m is the number of auction bidding data.

[0097] In general, the meanings of the symbols involved in the above embodiments are shown in Table 1 below.

[0098] Table 1 is a table showing the meanings of symbols involved in auction bidding.

[0099] symbol meaning D Represents a historical dataset data Represents a piece of historical data ξ Indicates the malicious coefficient p Indicates the probability of maliciousness θ Market deviation coefficient

[0100] In one embodiment, the multi-agent is trained based on the maliciousness coefficient, and the trained multi-agent corresponding to the optimal action value includes:

[0101] Step 1: Pass the malicious coefficient to the initial multi-agent.

[0102] Basic parameters such as the initial number of multi-agents, the policy network structure (e.g., a deep neural network), and the learning rate are set. Each agent is given the ability to process the auction environment (including the malicious coefficient). Before each auction round begins, the calculated malicious coefficient is passed to all agents as part of the environment state. This allows agents to take potential malicious behavior into account when making bidding decisions.

[0103] Step 2: Record the initial multi-agent bids and calculate the returns and rewards corresponding to the initial multi-agent bids in each round.

[0104] In each auction round, each agent's bid is recorded. Based on the auction results (such as the final transaction price and whether the agent won the bid) and a pre-set reward function, each agent's return and reward for that round are calculated. The reward function can be designed to encourage agents to bid at reasonable prices while discouraging overly high or low bids.

[0105] Step 3: Based on the bid, return, reward, and the initial agent’s policy network, obtain the agent’s action-value function and the optimal action-value function.

[0106] The action-value function (Q-function) estimates the expected reward from taking an action in a given state. The optimal action-value function (Q*) is the action-value function with the maximum value among all possible strategies. In a multi-agent system, each agent needs to learn its own action-value function while also taking into account the behavior of other agents. Reinforcement learning algorithms (such as Q-learning, DQN, MADDPG, etc.) are used to update the agent's policy network. After each auction round, the Bellman Equation is used to update the agent's action-value function based on the recorded bids, returns, rewards, and the current agent's policy network. The Bellman Equation allows the agent to learn from experience and approximate the optimal action-value function by iteratively updating its action-value function.

[0107] Step 4: Control the initial multi-agent to perform agent strategy learning, and use the Bellman equation to update the action-value function of the agent so that the updated action-value function approaches the optimal action-value function to obtain the trained multi-agent.

[0108] Auction simulations are repeated, and the agent's policy network is updated after each simulation. As training progresses, the agent learns how to make better bidding decisions based on the maliciousness coefficient and other environmental conditions. Training can be stopped by setting a certain number of training rounds or performance criteria. When the training process meets the stopping criteria, a trained multi-agent system is obtained. These agents are able to make bidding decisions with high action values ​​in auction environments containing malicious behavior.

[0109] To further explain the multi-agent training process in detail, the following will further expand on the relevant content.

[0110] The multi-agent system consists of a user to be detected and multiple adversarial agents. The user is a special agent.

[0111] system=(user,agent1,…,agent n )

[0112] When searching for the corresponding agent through the index, 1 is specified as the user to be detected, that is, the index starts from 1.

[0113] The meanings of the symbols involved in the entire multi-agent training process are shown in Table 2 below.

[0114] Table 2 is the corresponding table of the symbol meanings involved in the multi-agent training process

[0115]

[0116]

[0117] The action-value function is as follows:

[0118]

[0119] U t is the reward of the agent at time t, r t is the agent’s reward at time t, and γ represents the discount factor.

[0120] Q(s t ,a t ,π)=E(U t |S=s t ,A=a t ,π)(6)

[0121] Where π represents the agent's policy network, and its output value is the probability density of the action to be taken by the agent, that is, a~π.

[0122]

[0123] Q * is the optimal action-value function. The decision-making process of the agent during training is as follows:

[0124]

[0125] When rand>=ε, the action may be empty. In this case:

[0126]

[0127] In one embodiment, the initial multi-agent is controlled to perform agent strategy learning, and the action-value function of the agent is updated using the Bellman equation so that the updated action-value function approaches the optimal action-value function. The trained multi-agent includes:

[0128] Step 1: Control the initial multi-agents to conduct agent strategy learning and obtain the corresponding four-tuples in the agent strategy learning process. The four-tuples include the bids of all agents at the previous moment, the bids of the agents at the current moment, the rewards of the agents, and the bids of all agents at the current moment.

[0129] During the interaction between the agent and the environment, the system records a series of tuples. Each tuple contains: the bids of all agents at the previous moment, the bids of the agents at the current moment, the rewards of the agents, and the bids of all agents at the current moment. Specifically, the tuple is represented as (s t ,a t ,r t ,st+1 ).

[0130] Step 2: Calculate the action value generated by the current action value function based on the four-tuple.

[0131] Based on these four-tuples, the value of the current action-value function Q(s,a) can be calculated, where s is the state (which can be simplified to the bid here) and a is the action. The action-value function usually represents the expected future reward that can be obtained by taking a specific action in a given state. It is as follows:

[0132] q c =Q(s t ,a t )

[0133] q c The action value generated by the current action-value function.

[0134] q r =r t +maxQ(s t+1 ,a t+1 ).

[0135] Step 3: Get the action value under the optimal action value function according to the TD algorithm and Monte Carlo. Next, use the TD (Temporal Difference) algorithm or the Monte Carlo method to estimate or calculate the optimal action value function Q * The TD algorithm updates the value function by combining the current estimate and the future estimate, while the Monte Carlo method estimates the value by averaging the rewards of the complete round. That is, here, according to the TD algorithm and Monte Carlo approximation, q r is considered as the action value under the optimal action-value function.

[0136] Q(s t ,a t )+=α(q r -q c ).

[0137] Step 4: Use the gradient ascent method to update the action-value function so that the updated action-value function approaches the optimal action-value function to obtain the trained multi-agent.

[0138] Use the gradient ascent method to update the current action value function Q(s,a), with the goal of making it as close as possible to the optimal action value function Q * (s,a). The gradient ascent method calculates Q(s,a) and Q *(s, a), and adjust the parameters of Q(s, a) along the gradient direction of this difference to maximize Q(s, a).

[0139] In one embodiment, the agent reward is obtained in the following manner:

[0140] Initialize the agent's reward vector; select agents whose bids are larger than the median bid to form the first agent index vector; calculate the deviation of the corresponding agent bids in the first agent index vector; select agents whose deviations are larger than the preset deviation threshold to obtain the second agent index vector; give negative rewards to the agents in the second agent index vector; if the auction ends, reward the winning agent; return the reward vector of the initialized agent until the auction ends.

[0141] Specifically, the environment must be initialized before the agent interacts with it. The auction environment is set to a first-price sealed-bid auction. The symbols and their corresponding meanings are shown in Table 3.

[0142] Table 3 is the correspondence between the auction environment and its meaning

[0143] symbol meaning θ Market deviation coefficient n Number of agents value The value vector of the agent <![CDATA[bid r ]]> Reserve price, the minimum selling price of the auction item <![CDATA[bid c ]]> Bid vector, the current bids of all agents done Logical variable indicating whether the current auction is over

[0144] Environment initialization

[0145]

[0146] Set the current bid vector to an all-zero row vector and whether the auction is over to no.

[0147] A step is the process of transitioning from one state to another in the current round. The return value is the next state, the agent's reward, and whether the round is over.

[0148] First, state transition and determine whether the auction is over:

[0149]

[0150] That is, the next state is the bid of all agents at the current moment, and the initial state is the initial bid vector.

[0151] In a first-price sealed-bid auction, the auction ends if the highest bid is greater than 0.

[0152]

[0153] Then the reward setting during the auction process begins.

[0154] The first step is to initialize the reward vector rewards.

[0155] The second step is to select the agents whose bids are larger than the median bids to form the agent index vector

[0156] The third step is to calculate the vector The corresponding agent bid deviation θ i :

[0157]

[0158] Among them, bid median is the median bid price. The calculation formula can refer to formula (1), θ i is the bias of the agent with index i.

[0159] Step 4: Select The corresponding agent constitutes the agent index vector

[0160]

[0161] Step 5, yes Negative rewards are given to the agents in , also called biased over-rewards:

[0162]

[0163] Step 6: Determine whether the auction is over. If the auction is over, issue the winner a reward:

[0164]

[0165] That is to say, only when the winner bids no less than the reserve price can the winner win the auction and obtain the auction item.

[0166] Finally, step returns (s t+1 ,rewards,done).

[0167] In one embodiment, the method for detecting malicious bidding behavior in an auction further includes:

[0168] Determine whether the malicious coefficient meets the preset dynamic adjustment conditions; if not, pass the malicious coefficient to the trained user agent; if so, adjust the malicious coefficient to a preset fixed malicious coefficient, and pass the preset fixed malicious coefficient to the trained user agent.

[0169] Preset dynamic adjustment conditions dynamically adjust (constrain) the maliciousness coefficient during the testing phase. These conditions can be pre-set and are generally correlated with the user's bid and the deviation threshold. Specifically, during the testing phase, the maliciousness coefficient of the user agent being tested is dynamically adjusted to estimate a more accurate malicious probability.

[0170] The check function detects malicious behavior, which is as follows:

[0171]

[0172] The above function can detect whether the user to be detected triggers malicious bidding behavior.

[0173] Dynamically adjust the malicious coefficient as follows:

[0174]

[0175] The purpose of dynamic adjustment is to allow the agent to simulate "repentance".

[0176] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0177] Based on the same inventive concept, the embodiments of the present application also provide a device for detecting malicious bidding behavior in auction bidding, which is used to implement the aforementioned method for detecting malicious bidding behavior in auction bidding. The implementation solution provided by the device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more embodiments of the device for detecting malicious bidding behavior in auction bidding provided below can be found in the above-mentioned limitations on the method for detecting malicious bidding behavior in auction bidding, and will not be repeated here.

[0178] In one embodiment, Figure 4 As shown, a device for detecting malicious bidding behavior in auction bidding is provided, comprising:

[0179] A historical data acquisition module 200 is used to obtain historical auction bidding data of users;

[0180] The malicious coefficient calculation module 400 is used to calculate the malicious coefficient corresponding to the historical auction bidding data;

[0181] A training module 600 is used to train the multi-agent according to the maliciousness coefficient to obtain the trained multi-agent corresponding to the optimal action value;

[0182] The detection module 800 is used to obtain real-time bidding data of users and detect malicious bidding behavior based on the real-time bidding data of users and trained multi-agents.

[0183] In one embodiment, the malicious coefficient calculation module 400 is also used to extract the bid vector corresponding to each auction bidding data based on historical auction bidding data; calculate the bid median based on the bid vector; obtain a preset deviation threshold, and calculate the malicious value corresponding to each auction bidding data based on the preset deviation threshold and the bid median; the preset deviation threshold is used to represent the market's tolerance for malicious bids; calculate the average value of the malicious value corresponding to each auction bidding data to obtain the malicious coefficient.

[0184] In one embodiment, the training module 600 is also used to transfer the malicious coefficient to the initial multi-agent; record the bids of the initial multi-agent, and calculate the returns and rewards corresponding to the bidding of the initial multi-agent in each round; based on the bids, returns, rewards and the strategy network of the initial agent, obtain the action value function and the optimal action value function of the agent; control the initial multi-agent to perform agent strategy learning, and use the Bellman equation to update the action value function of the agent so that the updated action value function approaches the optimal action value function, thereby obtaining a trained multi-agent.

[0185] In one embodiment, the training module 600 is also used to control the initial multi-agent to perform agent strategy learning, obtain the corresponding quadruple in the agent strategy learning process, the quadruple includes the bids of all agents at the previous moment, the bids of the agents at the current moment, the agent rewards and the bids of all agents at the current moment; calculate the action value generated by the current action-value function based on the quadruple; obtain the action value under the optimal action-value function according to the TD algorithm and Monte Carlo; use the gradient ascent method to update the action-value function so that the updated action-value function approaches the optimal action-value function, and obtain the trained multi-agent.

[0186] In one embodiment, the training module 600 is also used to initialize the reward vector of the agent; select the agents whose bids are larger than the median bid to form a first agent index vector; calculate the deviation of the corresponding agent bids in the first agent index vector; select the agents whose deviations are larger than a preset deviation threshold to obtain a second agent index vector; give negative rewards to the agents in the second agent index vector; if the auction ends, reward the winning agent; and return the reward vector of the initialized agent until the auction ends.

[0187] In one embodiment, the training module 600 is also used to determine whether the malicious coefficient meets the preset dynamic adjustment conditions; if not, the malicious coefficient is passed to the trained user agent; if so, the malicious coefficient is adjusted to a preset fixed malicious coefficient, and the preset fixed malicious coefficient is passed to the trained user agent.

[0188] Each module in the aforementioned apparatus for detecting malicious bidding behavior in auctions can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0189] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for detecting malicious bidding behavior in an auction is implemented.

[0190] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0191] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method for detecting malicious bidding behavior in auction bidding when executing the computer program.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting malicious bidding behavior in auction bidding is implemented.

[0193] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the above-mentioned method for detecting malicious bidding behavior in auction bidding.

[0194] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0195] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to these.

[0196] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting malicious bidding behavior in auction bidding, characterized in that: The method comprises: Obtain the user's historical auction bidding data; Based on the historical auction bidding data, calculating the malicious coefficient corresponding to the historical auction bidding; Training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value; Acquire user real-time bidding data, and detect malicious bidding behavior based on the user real-time bidding data and the trained multi-agent.

2. The method according to claim 1, characterized in that The calculating, based on the historical auction bidding data, a malicious coefficient corresponding to the historical auction bidding includes: Extracting a bid vector corresponding to each piece of auction bidding data according to the historical auction bidding data; Calculate a median bid based on the bid vector; Obtaining a preset deviation threshold, and calculating a malicious value corresponding to each piece of auction bidding data according to the preset deviation threshold and the median bid; the preset deviation threshold is used to characterize the market's tolerance for malicious bids; The average value of the maliciousness value corresponding to each piece of auction bidding data is calculated to obtain the maliciousness coefficient.

3. The method according to claim 2, characterized in that The training of the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value comprises: Passing the malicious coefficient to the initial multi-agent; Record the bids of the initial multi-agents, and calculate the returns and rewards corresponding to the bids of the initial multi-agents in each round; Based on the bid, return, reward and the strategy network of the initial agent, an action value function and an optimal action value function of the agent are obtained; The initial multi-agent is controlled to perform agent strategy learning, and the action-value function of the agent is updated using the Bellman equation so that the updated action-value function approaches the optimal action-value function, thereby obtaining a trained multi-agent.

4. The method according to claim 3, characterized in that The controlling the initial multi-agent to perform agent strategy learning, using the Bellman equation to update the action value function of the agent so that the updated action value function approaches the optimal action value function, and obtaining the trained multi-agent includes: Control the initial multi-agent to perform agent strategy learning, and obtain the corresponding quaternary group in the agent strategy learning process, wherein the quaternary group includes the bids of all agents at the last moment, the bids of the agents at the current moment, the rewards of the agents, and the bids of all agents at the current moment; Calculate the action value generated by the current action value function based on the four-tuple; According to the TD algorithm and Monte Carlo, the action value under the optimal action value function is obtained; The action-value function is updated by using the gradient ascent method so that the updated action-value function approaches the optimal action-value function to obtain a trained multi-agent.

5. The method according to claim 4, characterized in that Agent rewards are obtained in the following way; Initialize the agent’s reward vector; Select the agent whose bid is larger than the median bid to form the first agent index vector; Calculate the deviation of the corresponding agent bid in the first agent index vector; Select the agent whose deviation is greater than the preset deviation threshold, and obtain the second agent index vector; Giving negative rewards to the agents in the second agent index vector; If the auction ends, the winning agent is rewarded; Returns the reward vector of the initialized agent until the auction ends.

6. The method according to claim 3, characterized in that: Also includes: Determining whether the malicious coefficient meets a preset dynamic adjustment condition; If not, the malicious coefficient is passed to the trained user agent; If satisfied, the maliciousness coefficient is adjusted to a preset fixed maliciousness coefficient, and the preset fixed maliciousness coefficient is transferred to the trained user agent.

7. A device for detecting malicious bidding behavior in auction bidding, characterized in that: The device comprises: A historical data acquisition module is used to obtain the user's historical auction bidding data; A malicious coefficient calculation module, used to calculate the malicious coefficient corresponding to the historical auction bidding based on the historical auction bidding data; A training module, used for training the multi-agent according to the malicious coefficient to obtain the trained multi-agent corresponding to the optimal action value; The detection module is used to obtain real-time bidding data of users and detect malicious bidding behavior based on the real-time bidding data of users and the trained multi-agents.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images