Bidding model training, bidding ranking method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a bidding model training, bidding ranking method, device, equipment and storage medium. The bidding model training method includes: obtaining a first training sample, the first training sample at least including the characteristics of enterprises with bidding relationships; based on the first training sample, training the bidding model, the bidding model is used to evaluate bids. In the scheme of the embodiment of the present invention, since the training sample includes the characteristics of enterprises with bidding relationships, the trained bidding model can evaluate bids more accurately.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a bidding model training, bidding ranking method, device, equipment and storage medium. Background Art

[0002] In the process of exceeding the standard, the existing technology can use the principal component analysis (PCA) and random forest (RF) model to evaluate the credit status of the bidding enterprises. The specific method is to collect factors such as the financial status and operating capacity of the bidding enterprises, and then use some known or manually labeled credit ratings of enterprises for supervised training, which can be a regression model or an ordered multi-classification model. The result is that each enterprise will get a score or a grade after being evaluated by the model, and finally the experts will proceed to the next step based on this score.

[0003] However, when training enterprise bidding evaluation models in this way, the manual labeling process of data often produces deviations, resulting in poor accuracy of the final evaluation model. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a bidding model training, bidding ranking method, device, equipment and storage medium to solve or alleviate the above problems.

[0005] According to a first aspect of an embodiment of the present invention, a bidding model training method is provided, comprising: obtaining a first training sample, the first training sample at least including characteristics of enterprises having a bidding relationship; based on the first training sample, training the bidding model, the bidding model being used to evaluate bids.

[0006] According to a second aspect of an embodiment of the present invention, a bidding ranking method is provided, comprising: obtaining enterprise bidding information of bidding enterprises; inputting the enterprise bidding information into a bidding model to obtain enterprise bidding ranking results, wherein the bidding model is obtained by training according to the method described in the first aspect.

[0007] According to a third aspect of an embodiment of the present invention, a bidding model training device is provided, comprising: an acquisition module, which acquires a first training sample, wherein the first training sample at least includes characteristics of enterprises that have a bidding relationship; a training module, which trains the bidding model based on the first training sample, and the bidding model is used to evaluate bids.

[0008] According to a fourth aspect of an embodiment of the present invention, a bidding ranking device is provided, comprising: an acquisition module for acquiring enterprise bidding information of bidding enterprises; a prediction module for inputting the enterprise bidding information into a bidding model to predict enterprise bidding ranking results, wherein the bidding model is obtained by training according to the method described in the first aspect.

[0009] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect or the second aspect.

[0010] According to a sixth aspect of an embodiment of the present invention, there is provided a storage medium on which a computer program is stored, and when the program is executed by a processor, the method as described in the first aspect or the second aspect is implemented.

[0011] In the solution of the embodiment of the present invention, since the training samples include the characteristics of enterprises that have bidding relationships, the bidding model obtained through training can evaluate bids more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] Figure 1 A schematic diagram of a model training method according to an embodiment of the present invention;

[0014] Figure 2 A schematic diagram of a model training method according to another embodiment of the present invention;

[0015] Figure 3A A schematic diagram of a model training method according to another embodiment of the present invention;

[0016] Figure 3B A schematic diagram of a model training method according to another embodiment of the present invention;

[0017] Figure 4 A schematic flow chart of a bidding ranking method according to another embodiment of the present invention;

[0018] Figure 5 A schematic diagram of a bidding ranking method according to another embodiment of the present invention;

[0019] Figure 6 is a schematic block diagram of a device according to another embodiment of the present invention;

[0020] Figure 7 is a schematic block diagram of a device according to another embodiment of the present invention;

[0021] Figure 8 The present invention is another embodiment of the hardware structure of an electronic device. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0023] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0024] In the bidding and tendering field, reasonable credit rating annotations in the bidding and tendering field are difficult to obtain. The first reason is that there is a small amount of annotation data for reasonable single-enterprise credit ratings, and even through manual annotation, there are inevitably deviations. In addition, the credit rating of an enterprise is often a manifestation of the overall strength of the enterprise, but it is not necessarily directly linked to the bidding and tendering rating. For example, an enterprise with a high comprehensive credit rating may rarely participate in bidding projects, so the enterprise's score in the bidding and tendering field should be lower. In addition, there are very few corporate credit ratings specifically for the bidding and tendering field. In short, the score in the bidding and tendering field is not necessarily related to the credit rating of the enterprise, which greatly increases the difficulty of evaluating the enterprise's rating in the bidding and tendering field.

[0025] In addition, the bidding and tendering levels of different companies in different project types should be different. For example, a company with strong comprehensive capabilities in the education field may not necessarily have high value in medical bidding and tendering projects, which increases the difficulty of evaluating the company's level in the bidding and tendering field.

[0026] Figure 1 A schematic diagram of a model training method according to an embodiment of the present invention. Figure 1 The model training method can be applied to any appropriate electronic device with data processing capabilities, including but not limited to servers and PCs, and the model training method includes:

[0027] 110: Obtain a first training sample, where the first training sample at least includes features of enterprises that have a bidding relationship.

[0028] It should be understood that the bid winning announcement data can be obtained by collecting bidding winning announcement data. Public bidding winning announcement data can be collected by means such as web crawlers or data purchase. Extract the bidding announcement by any technical means. For example, search the key fields of the data, including but not limited to the name of the tenderer, the bidding agency, the project name, the bidding release date, the bid winning date or the bid determination date or the transaction date, the name of the subject matter, the name of the winning enterprise, the winning amount, the ranking of other enterprises participating in the bidding, etc. Among them, the name of the subject matter includes but is not limited to the bid item, equipment name, sub-item name, service content, etc.

[0029] 120: Based on the first training sample, a bidding model is trained, and the bidding model is used to evaluate the bids.

[0030] It should be understood that the neural network can be trained to generate a bidding model. The neural network in this article includes but is not limited to a feedforward neural network such as a multi-layer perceptron (MLP), a BP (back propagation) neural network, a convolutional neural network (CNN) or other classification neural networks.

[0031] It should also be understood that the bidding model can be deployed on the server side or on the front end. When the bidding model is deployed on the front end, when the model is used for prediction, the prediction request can be received on the front end, and reasoning can be performed on the front end, and the model can be updated through interaction with the server. When the bidding model is deployed on the server side, the prediction request can be received through the front end, and reasoning processing can be implemented through interaction with the back end.

[0032] In the solution of the embodiment of the present invention, since the training samples include the characteristics of enterprises that have bidding relationships, the bidding model obtained through training can evaluate bids more accurately.

[0033] In a specific example, the first training sample may include the enterprise winning bid announcement and its ranking label determined by the enterprise bidding relationship in the winning bid announcement data. Since the training sample includes the enterprise winning bid announcement and its ranking label determined by the enterprise bidding relationship in the winning bid announcement data, the ranking label can better reflect the real enterprise status and improve the accuracy of the label, so that the trained model can perform more accurate bidding ranking. In other words, the embodiment of the present invention avoids the inevitable labeling deviation caused by the different historical experience and supervisory feelings of different bid evaluation experts.

[0034] In addition, the embodiment of the present invention also avoids the phenomenon of data islands, realizes information sharing, and improves utilization.

[0035] The bidding model can be trained in any of the following ways. In one example, it can be trained based on enterprise characteristics and information about the subject matter. The matching degree between the enterprise and the subject matter can be marked, for example, the first rank can be 1, the second rank can be 0.5, the third rank can be 0.25, and random generation can be marked as 0.

[0036] In another implementation of the present invention, a first training sample is obtained in the following manner: obtaining a bid winning announcement with multiple bidding companies; sorting the multiple bidding companies in the bid winning announcement to obtain a ranking feature of each bidding company; obtaining corporate features of each bidding company, the corporate features including a ranking feature, and one or more of the following features: registered capital, financial capacity, industry, qualification certificate, and innovation capability.

[0037] The winning announcements of bidding enterprise pairs can achieve accurate, fast and efficient training. In addition, the sorting feature can accurately reflect the sorting relationship between bidding enterprise pairs, further improving the efficiency of bidding model training.

[0038] In one example, the enterprise winning bid announcement includes winning bid announcements of multiple bidding enterprises determined from enterprise bidding relationships, and the ranking label represents the winning probability score of the first enterprise in each pair of enterprises compared with the second enterprise based on the enterprise bidding relationship.

[0039] In another implementation of the present invention, a bidding model is trained based on the first training sample, including: obtaining an enterprise feature vector according to the enterprise characteristics of the bidding enterprise; obtaining the score probability of the bidding enterprise after passing the enterprise feature vector through a fully connected layer; and training the bidding model according to the score probabilities and ranking characteristics of multiple bidding enterprises.

[0040] Since the scoring probability of the bidding companies obtained after passing the fully connected layer is an important indicator for bidding ranking, the bidding model is trained according to the scoring probabilities and ranking characteristics of multiple bidding companies, thereby improving the model training efficiency.

[0041] In one example, the first enterprise's bid winning announcement and the second enterprise's bid winning announcement can be used as inputs of the first neural network and the second neural network, respectively, and the first enterprise's bid winning probability score and the second enterprise's bid winning probability score can be used as outputs of the first neural network and the second neural network, respectively, to obtain the bidding model through training. For example, the first neural network and the second neural network can both include a feature vector calculation layer and a fully connected layer for the enterprise.

[0042] Specifically, Figure 2 FIG. 1 is a schematic diagram of a model training method according to another embodiment of the present invention. Figure 2 As shown, for any enterprise pair (C i ,Cj ), Enterprise C i After the feature vector of passes through several layers of fully connected layers, the score probability of the last 1-dimensional fully connected layer is S i , Enterprise C j After the feature vector of passes through several layers of fully connected layers, the score probability of the last 1-dimensional fully connected layer is S j Then according to the model's prediction, C i Than C j The probability of winning the bid is more likely to be:

[0043]

[0044] Since the network structure of the neural network to be trained is bilaterally symmetrical, the network structure of the first neural network can be defined, and the second neural network shares network parameters. The loss function calculation layer uses the predicted probability P ij And real business labels Cross entropy is used as the metric loss function and can be optimized using gradient descent methods.

[0045]

[0046] In addition, in another implementation of the present invention, the first training sample also includes the subject matter information, wherein the bidding model is trained based on the first training sample, including: generating a subject matter feature vector according to the subject matter information; concatenating the enterprise feature vector with the subject matter feature vector to obtain a concatenated vector; correspondingly, after passing the concatenated vector through a fully connected layer, the scoring probability of the bidding enterprise is obtained; and the bidding model is trained based on the scoring probabilities and ranking characteristics of multiple bidding enterprises.

[0047] Since the training samples include the feature vector of the target object, prediction can be made based on the information of the target object, thereby improving the prediction accuracy of the trained model.

[0048] In one example, word embedding processing can be performed on enterprise information and target object information respectively to generate enterprise feature vectors and target object feature vectors; the enterprise feature vectors and target object feature vectors can be concatenated to obtain a concatenated feature vector of the enterprise's winning bid announcement; and a neural network can be trained based on the concatenated feature vector of the enterprise's winning bid announcement.

[0049] Specifically, Figure 3A This is a schematic diagram of a model training method of another embodiment of the present invention. As shown in the figure, the subject information includes but is not limited to the subject item, equipment name, sub-item name, service content, etc. These contents can use the average of the word vectors after word segmentation as the vector of the subject matter, and then splice it to the current enterprise feature vector as the input vector for calculation.

[0050] Figure 3B It is a schematic diagram of a model training method of another embodiment of the present invention. As shown in the figure, after the bidding model training is completed, the structure of the inference network is saved. The bidding model can be deployed on the server side or on the front end. When reasoning is performed online, when multiple companies bid together, for example, the information of the bidding companies includes C1, C2, C3, ..., Cn, and each company is calculated through this inference network in combination with the subject matter, and each company is given a score, and finally sorted according to the size of the score Cr1, Cr2, Cr3, ..., Crn.

[0051] In another implementation of the present invention, the first training sample also includes bid co-occurrence frequency information, wherein the bidding model is trained based on the first training sample, including: generating a co-occurrence feature vector based on the bid co-occurrence frequency information; concatenating the enterprise feature vectors of at least two co-appearing bidding companies and the co-occurrence feature vector to obtain a co-occurrence vector; correspondingly, after passing the co-occurrence vector through a fully connected layer, the scoring probability of the bidding company is obtained; and the bidding model is trained based on the scoring probabilities and ranking characteristics of multiple bidding companies.

[0052] Since the company's winning bid announcement includes information on the co-occurrence frequency of bids, the trained model can effectively predict risks such as bid rigging and collusion.

[0053] In one example, the at least two bidding companies that appear together may be the first company and the second company. The company's bid winning announcement may include the company information of the first company and the second company and the bid co-occurrence frequency information. The company information of the first company and the second company and the bid co-occurrence frequency information may be word-embedded to generate feature vectors of the first company and the second company, and a bid co-occurrence frequency vector; the feature vectors of the first company and the second company and the bid co-occurrence frequency vector are concatenated to obtain a feature vector of the company's bid winning announcement; and a neural network is trained based on the feature vector of the company's bid winning announcement.

[0054] In actual bidding scenarios, if two companies engage in bid rigging and collusion, then a very likely manifestation is that the probability of these two companies appearing in the same bidding project is higher than that of two normal companies. The higher the co-occurrence probability, the greater the risk of bid rigging and collusion in theory. Specifically, when Company A and Company B agree that they hope Company A will win the bid (bid rigging and collusion), if the combination of these two companies is used as a training sample, there are unreasonable factors, which reduces the accuracy of the bidding prediction model obtained by learning the bidding relationship from the samples.

[0055] In another implementation of the present invention, the enterprise feature vectors of at least two co-appearing bidding enterprises and the co-occurrence feature vector are spliced, including: splicing the co-occurrence feature vector between the enterprise feature vectors of at least two bidding enterprises.

[0056] In one example, if two companies in one enterprise pair A are very small, and the other enterprise pair B has two larger companies. If the co-occurrence times of the two enterprise pairs are the same, the probability that there are many normal competition relationships between the enterprise pairs with larger scale is higher, while the probability that the small enterprise pairs have bid rigging and collusion is higher. Therefore, the cross-features of co-occurrence times / probability (example of enterprise bidding co-occurrence frequency information) and enterprise size (example of enterprise information) will jointly affect the risk of bid rigging and collusion.

[0057] In another example, if the two companies belong to different industries but have a high number of co-occurrences, the risk of bid rigging is also greater than that of companies in the same industry. In other words, the number of co-occurrences and the cross-industry characteristics of the two companies will also jointly affect the risk of bid rigging.

[0058] Thus, the co-occurrence relationship features of both parties (including co-occurrence times, co-occurrence probabilities, etc.) and various features of both enterprises are integrated together to jointly train the bid-rigging model, and the resulting model can greatly reduce the impact of bid-rigging on prediction accuracy. In this example, since the bid co-occurrence frequency vector is spliced ​​between the feature vector of the first enterprise and the feature vector of the second enterprise, the input vector of the model is simplified, thereby improving the training efficiency of the model.

[0059] In another implementation of the present invention, at least two bidding companies include a winning company and an accompanying bidding company, wherein the co-occurrence feature vector is spliced ​​between the company feature vectors of the at least two bidding companies, including: placing the company feature vector of the winning company before the co-occurrence feature vector, and placing the company feature vector of the accompanying bidding company after the co-occurrence feature vector.

[0060] Placing the enterprise feature vector of the successful bidder before the co-occurrence feature vector and placing the enterprise feature vectors of the accompanying bidders after the co-occurrence feature vector facilitates data processing such as vector calculation and improves data processing efficiency.

[0061] In other words, the first enterprise may be the winning enterprise, and the second enterprise may be the accompanying bidding enterprise. The feature vector of the first enterprise may be placed before the bid co-occurrence frequency vector, and the feature vector of the second enterprise may be placed after the bid co-occurrence frequency vector.

[0062] In another implementation of the present invention, the corporate features and co-occurrence feature vectors of at least two bidding companies that appear together are spliced, including: determining weight information of the co-occurrence feature vector through the suspicion information of bid rigging or collusion between the at least two bidding companies; and splicing the corporate feature vectors and co-occurrence feature vectors of the at least two bidding companies based on the weight information.

[0063] Since the bid-rigging suspicion information can effectively reflect the correlation between the bid co-occurrence frequency vector and the bid-rigging phenomenon, the prediction accuracy of the trained model is improved.

[0064] In one example, the weight information of the bid co-occurrence frequency vector can be determined through the bid-rigging suspicion information of each pair of enterprises; based on the weight information, the enterprise feature vector and the bid co-occurrence frequency vector are concatenated.

[0065] In one example, all co-occurrence relationship features and enterprise features can be integrated together to directly train the bidding model. The weight of the suspected bid rigging sample is automatically adjusted according to the suspicion. For samples with the risk of bid rigging, it is sometimes difficult to determine whether bid rigging has occurred based on the existing data. The suspicion in this example can indicate the probability of bid rigging. Based on the above probability, samples with high suspicion are assigned lower weights in the training of the bidding prediction model, so that the bidding model eliminates the influence of bid rigging as much as possible.

[0066] In another implementation of the present invention, the suspicion information of bid rigging or collusion is determined through the historical bidding information of the winning bidder and the accompanying bidders.

[0067] In this example, since the information on the suspicion of bid rigging and collusion is determined by the historical bidding information of the first enterprise and the second enterprise, the data reference value of the information on the suspicion of bid rigging and collusion is improved.

[0068] The above-mentioned suspicion calculation can be based on the historical bidding relationship diagram (an example of historical bidding information). The historical bidding relationship diagram includes but is not limited to the following dimensions of information: entity information, relationship type, etc. Entity information includes but is not limited to the issuing enterprise, agency, bidding enterprise and subject matter, etc. Relationship types include but are not limited to bidding relationships, bidding relationships, winning relationships and subject matter association relationships, etc. The suspicion calculation can be based on the number of times two companies bid together (that is, the greater the number of co-occurrences, the higher the suspicion, and the ratio of the number of winning bids of the two companies after the two companies have bid together for many times (that is, the greater the difference in the ratio of the number of winning bids, the higher the suspicion). A feasible method for calculating suspicion is shown below:

[0069] The winning enterprise A in the current bidding document can be represented as S_win, and the unsuccessful enterprise can be represented as S_loss. S_win and any unsuccessful enterprise in the current bidding document can be combined as S_loss into an enterprise pair. The unique identification number of the enterprise pair is P_AB, and its co-occurrence number is P_AB_cnt CASE WHEN S_win>S_loss THENconcat(S_win,"_",S_loss)ELSE concat(S_loss,"_",S_win).

[0070] In addition, in another winning bid announcement document, there is a situation where company B won the bid but company A did not. The unique identification number of the company pair generated is still the same as above, but each P_AB corresponds to two values: A_B_cnt and B_A_cnt.

[0071] If A_B_cnt / P_AB is closer to 1, it indicates that every time A and B bid together, A wins the bid more often and B bids with him more often. In addition, if A_B_cnt / P_AB is closer to 0, it indicates that every time A and B bid together, B wins the bid more often and A bids with him very often. If A_B_cnt / P_AB is closer to 0, it indicates that there is no obvious bidding relationship between the two companies. That is, the greater the number of P_AB times, the greater the suspicion. As an example, the formula for suspicion can be expressed as follows:

[0072] Anm_deg=log(P_AB_cnt)*|A_B_cnt / P_AB_cnt-0.5|

[0073] By cross-integrating the suspicion information with other features, the bidding model can automatically adjust the weight of the features of interest. That is, if the suspicion is high, the bidding model is more inclined to directly use the co-occurrence related features for bidding ranking prediction, while if the suspicion is low, the bidding model is more inclined to use various factors such as scale, strength and matching degree in normal bidding for ranking prediction.

[0074] By splicing the bid co-occurrence frequency vector between the feature vector of the first enterprise and the feature vector of the second enterprise, the model can greatly alleviate the problem of inaccurate ranking when predicting normal bidding data caused by models trained with abnormal samples such as bid rigging and collusion, and avoid directly judging whether the data is bid rigging or collusion. In addition, the accuracy of the current model can be effectively improved by continuously adjusting the suspicion calculation rules, so the suspicion calculation rules themselves can also become an evaluation indicator for bid rigging and collusion.

[0075] In another implementation of the present invention, the historical bidding information of the first enterprise and the second enterprise includes at least one of the following: co-occurrence frequency information of bids of the first enterprise and the second enterprise, probability information of winning the bid of the first enterprise or the second enterprise, information of the jointly-agented enterprises of the first enterprise and the second enterprise, and interactive information of the corporate community relationship of the first enterprise and the second enterprise.

[0076] For example, for the number of times two companies in a pair of companies bid together, the greater the number of co-occurrences, the higher the suspicion. For example, after two companies in a pair of companies bid together for many times, the greater the difference in the number of times the two companies win bids, the higher the suspicion. For example, for two companies in a pair of companies, if the same agency is used in multiple biddings, the fewer the number of agencies, the lower the suspicion. For example, if the bidding unit conducts multiple biddings, and the same company wins each time, and is accompanied by other companies, the suspicion is high. For example, if the two companies in a pair of companies always appear in projects represented by the same agency, the suspicion is high.

[0077] For another example, the density of a community can also indicate the degree of suspicion. If the number of edges in a community divided by the number of nodes in the community is too large, it means that many nodes in the community have a history of bidding against each other. If multiple companies form a small group of k companies and there are bidding records between each other, the suspicion is high. In addition, the average edge weight of a small community (the number of joint biddings between two enterprise nodes) can be considered: for example, first divide it into several small communities through a community partitioning algorithm. If a community has a total of k edges, calculate the median of the edge weights of the community. If the median edge weight is greater than the threshold (that is, more than half of the edge weights in the community exceed the threshold), the risk of collusion in bidding by the community is greater and the suspicion is higher.

[0078] In another implementation of the present invention, obtaining a first training sample includes: based on a bid-rigging and bid-collusion risk prediction model generated by using a second training sample, screening the enterprise information in the bid-winning announcement data to obtain the first training sample, wherein the second training sample is generated by the enterprise information of the bid-rigging and bid-collusion enterprises and the bid co-occurrence frequency information.

[0079] One is that there are many blacklists of bid rigging and collusion on the Internet. The enterprise pairs related to these blacklists are used as black samples. After randomly generating white samples, a bid rigging and collusion risk determination model is trained using the various co-occurrence relationship features and enterprise characteristics mentioned above. Then, the risk model is used to filter all enterprise pairs in the training samples, and the data of enterprise pairs with lower bid rigging and collusion risks are used to train the bidding model to improve the reliability of the bidding model. The advantage of this solution is that it is more interpretable and can directly predict the risk of bid rigging and collusion. The disadvantage is that the amount of effective bid rigging and collusion blacklist data may not be large and is relatively difficult to find.

[0080] In this example, since the bid-rigging and bid-collusion risk prediction model can effectively predict the risk of bid-rigging and bid-collusion, the bid-rigging and bid-collusion risk prediction model is used for screening, thereby improving the confidence of the obtained first training sample.

[0081] Figure 4 A schematic flowchart of a bidding ranking method according to another embodiment of the present invention. Figure 4 The bidding ranking method can be applied to any appropriate electronic device with data processing capability, including but not limited to servers, mobile terminals (such as mobile phones, PADs, etc.) and PCs, etc. The bidding ranking method includes:

[0082] 410: Obtain the bidding information of the bidding enterprise.

[0083] 420: Input the enterprise bidding information into the bidding model to obtain the enterprise bidding ranking results, wherein the bidding model is trained by the bidding model training method.

[0084] In the solution of the embodiment of the present invention, since the training samples include the characteristics of enterprises that have bidding relationships, the bidding model obtained through training can evaluate bids more accurately.

[0085] Figure 5 FIG. 1 is a schematic diagram of a bidding ranking method according to another embodiment of the present invention. Figure 5 As shown,

[0086] In step S511, the winning bid announcement data is collected, and then the process proceeds to step S512.

[0087] In step S512, the winning bid announcement data and the subject matter information are parsed to obtain bidding relationship pair data, and then proceed to step S513.

[0088] In step S513, the training features are combined with the subject matter and the characteristics of the bidding companies, and the process proceeds to step S514.

[0089] In step S514, the suspicion level of bid rigging or collusion among enterprises is calculated, and the process proceeds to step S515.

[0090] In step S515, the bidding model is trained, and the process proceeds to steps S516 and S517.

[0091] In step S516, the scores and ratings given by the bidders for different projects in the prediction stage are determined.

[0092] In step S517, it is determined whether the suspiciousness feature improves the model effect to the expected level (for example, based on the accuracy threshold)? If yes, step S518 is executed; if no, step S514 is executed.

[0093] Figure 6 A schematic block diagram of a device according to another embodiment of the present invention. Figure 6 The model training device can be applied to any appropriate electronic device with data processing capability, including but not limited to a server and a PC, etc. The model training device includes:

[0094] An acquisition module 610 acquires a first training sample, where the first training sample at least includes characteristics of enterprises that have a bidding relationship;

[0095] The training module 620 trains the bidding model based on the first training sample, and the bidding model is used to evaluate the bids.

[0096] In the solution of the embodiment of the present invention, since the training samples include the characteristics of enterprises that have bidding relationships, the bidding model obtained through training can evaluate bids more accurately.

[0097] As an example, the first training sample may include enterprise winning bid announcements and their ranking labels determined through enterprise bidding relationships in the winning bid announcement data.

[0098] In another implementation of the present invention, the acquisition module is specifically used to: obtain a winning bid announcement with multiple bidding companies; sort the multiple bidding companies in the winning bid announcement to obtain the sorting characteristics of each bidding company; obtain the enterprise characteristics of each bidding company, the enterprise characteristics include sorting characteristics, and one or more of the following characteristics: registered capital, financial capacity, industry, qualification certificate, and innovation ability.

[0099] In another implementation of the present invention, the training module is specifically used to: obtain an enterprise feature vector based on the enterprise characteristics of the bidding enterprise; obtain the scoring probability of the bidding enterprise after passing the enterprise feature vector through the fully connected layer; and train the bidding model based on the scoring probabilities and ranking characteristics of multiple bidding enterprises.

[0100] In another implementation of the present invention, the first training sample also includes the subject matter information, and the training module is specifically used to: generate a subject matter feature vector based on the subject matter information; concatenate the enterprise feature vector with the subject matter feature vector to obtain a concatenated vector; accordingly, after passing the concatenated vector through a fully connected layer, obtain the scoring probability of the bidding enterprise; and train the bidding model based on the scoring probabilities and ranking characteristics of multiple bidding enterprises.

[0101] In another implementation of the present invention, the first training sample also includes bid co-occurrence frequency information, and the training module is specifically used to: generate a co-occurrence feature vector based on the bid co-occurrence frequency information; concatenate the enterprise feature vectors of at least two co-appearing bidding companies and the co-occurrence feature vector to obtain a co-occurrence vector; accordingly, after passing the co-occurrence vector through a fully connected layer, the scoring probability of the bidding company is obtained; and the bidding model is trained based on the scoring probabilities and ranking characteristics of multiple bidding companies.

[0102] In another implementation of the present invention, the training module is specifically used to: splice the co-occurrence feature vector between the enterprise feature vectors of at least two bidding enterprises.

[0103] In another implementation of the present invention, the at least two bidding enterprises include a winning enterprise and an accompanying bidding enterprise, wherein the training module is specifically used to: place the enterprise feature vector of the winning enterprise before the co-occurrence feature vector, and place the enterprise feature vector of the accompanying bidding enterprise after the co-occurrence feature vector.

[0104] In another implementation of the present invention, the training module is specifically used to: determine the weight information of the co-occurrence feature vector through the suspicion information of bid rigging and collusion of at least two bidding enterprises; and concatenate the enterprise feature vectors and co-occurrence feature vectors of at least two bidding enterprises based on the weight information.

[0105] In another implementation of the present invention, the suspicion information of bid rigging or collusion is determined through the historical bidding information of the winning bidder and the accompanying bidders.

[0106] In another implementation of the present invention, the at least two bidding enterprises include a first enterprise and a second enterprise. The historical bidding information of the first enterprise and the second enterprise includes at least one of the following: information on the co-occurrence frequency of bids of the first enterprise and the second enterprise, information on the probability of winning the bid of the first enterprise or the second enterprise, information on the common agent enterprises of the first enterprise and the second enterprise, and information on the corporate community relationship interaction of the first enterprise and the second enterprise.

[0107] In another implementation of the present invention, the acquisition module is specifically used to: screen the enterprise information in the winning bid announcement data based on the bid rigging and collusion risk prediction model generated by the second training sample to obtain the first training sample, wherein the second training sample is generated by the enterprise information of the bid rigging and collusion enterprises and the bid co-occurrence frequency information.

[0108] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0109] Figure 7 A schematic block diagram of a device according to another embodiment of the present invention. Figure 7 The bidding ranking device can be applied to any appropriate electronic device with data processing capability, including but not limited to a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC, etc. The bidding ranking device includes:

[0110] An acquisition module 710 is used to acquire the enterprise bidding information of the bidding enterprise;

[0111] The prediction module 720 inputs the enterprise bidding information into the bidding model to predict the enterprise bidding ranking results, wherein the bidding model is trained according to the bidding model training method.

[0112] In the solution of the embodiment of the present invention, in the solution of the embodiment of the present invention, since the training samples include the characteristics of enterprises that have bidding relationships, the trained bidding model can evaluate the bids more accurately.

[0113] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0114] Figure 8 The hardware structure of the electronic device of another embodiment of the present invention is as follows; Figure 8 As shown, the hardware structure of the electronic device may include: a processor 801, a communication interface 802, a storage medium 803 and a communication bus 804; wherein the processor 801, the communication interface 802, and the storage medium 803 communicate with each other via the communication bus 804;

[0115] Optionally, the communication interface 802 may be an interface of a communication module;

[0116] The processor may be specifically configured to: obtain a first training sample, wherein the first training sample at least includes characteristics of enterprises that have a bidding relationship; based on the first training sample, train the bidding model, wherein the bidding model is used to evaluate bids;

[0117] Alternatively, the processor may be specifically configured to: obtain enterprise bidding information of bidding enterprises; input the enterprise bidding information into a bidding model to obtain enterprise bidding ranking results, wherein the bidding model is trained by a bidding model training method.

[0118] The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0119] The storage medium may be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable programmable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0120] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present invention are executed. It should be noted that the storage medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, an apparatus or a device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in conjunction with an instruction execution system, an apparatus or a device. The program code contained on the storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0121] Computer program code configured to perform the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more or the execution order may be adjusted. That is, in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The names of these modules do not limit the modules themselves in some cases.

[0124] As another aspect, the present invention further provides a storage medium having a computer program stored thereon, which implements the method described in the above embodiment when executed by a processor.

[0125] As another aspect, the present invention further provides a storage medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above storage medium carries one or more programs, and when the above one or more programs are executed by the device, the device: obtains a first training sample, the first training sample at least includes the characteristics of enterprises in a bidding relationship; based on the first training sample, trains the bidding model, and the bidding model is used to evaluate bids;

[0126] Alternatively, the device is configured to: obtain enterprise bidding information of bidding enterprises; input the enterprise bidding information into a bidding model to obtain enterprise bidding ranking results, wherein the bidding model is trained by a bidding model training method.

[0127] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0128] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0129] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A bidding model training method, comprising: Acquire a first training sample, wherein the first training sample at least includes characteristics of enterprises that have a bidding relationship; Based on the first training sample, training the bidding model, wherein the bidding model is used to evaluate the bid; Among them, the first training sample also includes bid co-occurrence frequency information, and the bidding model is trained based on the first training sample, including: generating a co-occurrence feature vector according to the bid co-occurrence frequency information; concatenating the enterprise feature vectors of at least two co-occurring bidding enterprises and the co-occurrence feature vector to obtain a co-occurrence vector; after passing the co-occurrence vector through a fully connected layer, obtaining the score probability of the bidding enterprise; and training the bidding model according to the score probabilities and ranking characteristics of multiple bidding enterprises.

2. The method according to claim 1, wherein: The first training sample is obtained by: Obtain a winning bid announcement with multiple bidding companies; Sorting multiple bidding companies in the bid winning announcement to obtain a ranking feature of each bidding company; The enterprise characteristics of each bidding enterprise are obtained, wherein the enterprise characteristics include ranking characteristics and one or more of the following characteristics: registered capital, financial capacity, industry, qualification certificate, and innovation capacity.

3. The method according to claim 2, wherein: The step of training the bidding model based on the first training sample includes: Obtaining enterprise feature vectors according to enterprise features of bidding enterprises; After passing the enterprise feature vector through the fully connected layer, the scoring probability of the bidding enterprise is obtained; The bidding model is trained according to the scoring probabilities and ranking characteristics of multiple bidding companies.

4. The method according to claim 3, wherein: The first training sample also includes subject matter information, wherein the step of training the bidding model based on the first training sample includes: Generating a target object feature vector according to the target object information; Concatenating the enterprise feature vector with the subject matter feature vector to obtain a concatenated vector; Correspondingly, after the concatenated vector passes through the fully connected layer, the scoring probability of the bidding enterprise is obtained; and the bidding model is trained according to the scoring probabilities and ranking features of multiple bidding enterprises.

5. The method according to claim 1, wherein: The step of concatenating the enterprise feature vectors of at least two co-occurring bidding enterprises and the co-occurrence feature vector comprises: The co-occurrence feature vector is spliced ​​between the enterprise feature vectors of the at least two bidding enterprises.

6. The method according to claim 5, wherein: The at least two bidding enterprises include a successful bidder and an accompanying bidding enterprise, wherein the step of splicing the co-occurrence feature vector between the enterprise feature vectors of the at least two bidding enterprises comprises: The enterprise feature vector of the successful bidder is placed before the co-occurrence feature vector, and the enterprise feature vectors of the accompanying bidders are placed after the co-occurrence feature vector.

7. The method according to claim 1, wherein: The step of concatenating the enterprise features of at least two co-occurring bidding enterprises and the co-occurrence feature vector comprises: Determining weight information of the co-occurrence feature vector based on the suspicion information of bid rigging and collusion between the at least two bidding enterprises; Based on the weight information, the enterprise feature vectors of the at least two bidding enterprises and the co-occurrence feature vector are concatenated.

8. The method according to claim 7, wherein: The information on the suspicion of bid rigging or collusion is determined through the historical bidding information of the successful bidder and the accompanying bidders.

9. A bidding ranking method, comprising: Obtain enterprise bidding information of bidding enterprises; The enterprise bidding information is input into a bidding model to obtain enterprise bidding ranking results, wherein the bidding model is trained by the method according to any one of claims 1-8.

10. A bidding model training device, comprising: An acquisition module is used to acquire a first training sample, wherein the first training sample at least includes characteristics of enterprises that have a bidding relationship; A training module, which trains the bidding model based on the first training sample, wherein the bidding model is used to evaluate the bid; Among them, the first training sample also includes bidding co-occurrence frequency information, and the training module is also used to generate a co-occurrence feature vector based on the bidding co-occurrence frequency information; concatenate the enterprise feature vectors of at least two bidding companies that appear together and the co-occurrence feature vector to obtain a co-occurrence vector; after passing the co-occurrence vector through a fully connected layer, the scoring probability of the bidding company is obtained; and the bidding model is trained based on the scoring probabilities and ranking characteristics of multiple bidding companies.

11. A bidding ranking device, comprising: An acquisition module is used to acquire the bidding information of bidding enterprises; A prediction module inputs the enterprise bidding information into a bidding model to predict the enterprise bidding ranking results, wherein the bidding model is trained by the method according to any one of claims 1-8.

12. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1 to 9.

13. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images