Abnormal bidding identification model training method, abnormal bidding identification method and device

Abstract

The present disclosure provides an abnormal bid identification model training method, an abnormal bid identification method and a device, which can be used in the field of artificial intelligence or finance. The method for training the abnormal bid identification model includes: extracting a bid subject related information set from historical bidding data; expanding the bid subject related information set to obtain a candidate abnormal bid feature set; determining abnormal bid features from the candidate abnormal bid features based on the correlation between the candidate abnormal bid features and the abnormal bids in the candidate abnormal bid feature set; and training the abnormal bid identification model based on the abnormal bid features.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and finance, and more specifically, to an abnormal bid identification model training method, an abnormal bid identification method and a device. Background Art

[0002] In related technologies, bidding is a way to select the best for a deal. The tenderer needs to conduct a qualification review of the bidders to prevent the bidders from using unfair means to collude on bidding matters, restrict competition, exclude other bidders, and make a certain stakeholder win the bid, thereby seeking profits and disrupting the market order.

[0003] In the process of realizing the concept of the present disclosure, the applicant discovered that there are at least the following problems in the related technology: there are various scenarios requiring bidding, and the characteristics of the bidding entities are diverse, making it difficult to accurately identify abnormal bids based on preset rules. Summary of the invention

[0004] In view of this, the present disclosure provides an abnormal bid identification model training method, an abnormal bid identification method and an apparatus, so as to at least partially solve the problem that it is difficult to identify abnormal bids based on preset rules, so as to improve the accuracy of abnormal bid identification.

[0005] One aspect of the present disclosure provides a method for training an abnormal bid identification model, including: extracting a set of bidder-related information from historical bidding data; expanding the set of bidder-related information to obtain a set of candidate abnormal bid features; determining abnormal bid features from the candidate abnormal bid features based on the correlation between the candidate abnormal bid features and the abnormal bids in the candidate abnormal bid feature set; and training an abnormal bid identification model based on the abnormal bid features.

[0006] One aspect of the present disclosure provides an abnormal bid identification method, comprising: obtaining input bid data; and processing the input bid data using an abnormal bid identification model trained by the above method to obtain an identification result for the input bid data.

[0007] One aspect of the present disclosure provides an abnormal bid identification device, comprising: a bid data acquisition module and an abnormal bid identification module. The bid data acquisition module is used to acquire input bid data; and the abnormal bid identification module is used to process the input bid data using the abnormal bid identification model trained by the above method to obtain an identification result for the input bid data.

[0008] Another aspect of the present disclosure provides an electronic device, including one or more processors and a storage device, wherein the storage device is used to store executable instructions, and when the executable instructions are executed by the processor, the above method is implemented.

[0009] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the above-mentioned method for training an abnormal bid identification model and / or abnormal bid identification method.

[0010] Another aspect of the present disclosure provides a computer program, which includes computer executable instructions, and when the instructions are executed, are used to implement the above-mentioned method for training an abnormal bid identification model and / or abnormal bid identification method.

[0011] The abnormal bid identification model training method, abnormal bid identification method and device provided by the embodiments of the present disclosure are based on the bidding subject association information, expand the bidding subject association information in the historical bidding data, obtain candidate abnormal bid features, and then automatically screen out abnormal bid features that can characterize whether there are abnormal bids in the current bidding data based on the correlation between each candidate abnormal bid feature and the abnormal bid. Compared with the abnormal bid features constructed based on expert experience, they have more dimensions and are more comprehensive, which helps to improve the prediction accuracy of the abnormal bid identification model.

[0012] The abnormal bidding identification model training method, abnormal bidding identification method and device provided by the embodiments of the present disclosure realize intelligent detection of abnormal bidding and improve the detection efficiency of abnormal bidding. It is helpful to quickly and effectively identify abnormal bidding behaviors in the bidding process, so as to provide risk warnings in a timely manner and ensure the smooth progress of the bidding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0014] Figure 1 The application scenarios of the abnormal bidding identification model training method, abnormal bidding identification method and device according to the embodiments of the present disclosure are schematically shown;

[0015] Figure 2 Schematically illustrates an exemplary system architecture to which an abnormal bidding identification method and an abnormal bidding identification device according to an embodiment of the present disclosure can be applied;

[0016] Figure 3 A flowchart of a method for training an abnormal bidding identification model according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 4 A schematic diagram schematically shows a schematic diagram of expanding a bidding subject association information set according to an embodiment of the present disclosure;

[0018] Figure 5A schematic diagram schematically shows an abnormal bidding feature vector according to an embodiment of the present disclosure;

[0019] Figure 6 A data flow diagram according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 7 A logic diagram of a method for training an abnormal bidding identification model according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 8 A flowchart of an abnormal bidding identification method according to an embodiment of the present disclosure is schematically shown;

[0022] Fig. 9 A block diagram of an abnormal bidding identification device according to an embodiment of the present disclosure is schematically shown;

[0023] Fig.10 A block diagram schematically shows a system for identifying abnormal bidding according to an embodiment of the present disclosure; and

[0024] Fig.11 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] When using expressions such as "at least one of A, B or C", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0029] In order to facilitate understanding of the technical solution of the embodiments of the present disclosure, the bidding and tendering process is firstly exemplified.

[0030] Tendering is an international practice. It is an organized way of selecting the best bidders by applying technology, economic methods and the competitive mechanism of the market economy. When organizing the qualification review of bidders, the tenderer can adopt the process of issuing a prequalification announcement or tender documents to clarify the qualification requirements for bidders. Bidders prepare corresponding qualification materials such as business licenses and qualification certificates according to the requirements. The tenderer then conducts a conformity review of the materials submitted by the bidders. At present, there are still some problems when the tenderer organizes the qualification review of bidders. For example, bidders use unfair means to collude on bidding matters, restrict competition, exclude other bidders, and make a certain stakeholder win the bid, thereby seeking benefits and disrupting the market order. This behavior is called bid rigging and collusion.

[0031] In the current bidding and tendering field, there are a large number of illegal and irregular behaviors such as bid rigging and collusion, which undermine the open, fair and just bidding documents.

[0032] However, there are many companies bidding during the bidding process. The current manual reading and comparison is inefficient and inaccurate, making it difficult to accurately identify bid rigging and collusion.

[0033] In the related art, the bidding system cannot efficiently and accurately identify bid rigging and collusion in the bidding process. Therefore, there is an urgent need for a method that can efficiently and accurately identify bid rigging and collusion. At present, there is a complete lack of technology in the field of using machine learning to build an abnormal bidding identification model to analyze supplier bid rigging and collusion.

[0034] The abnormal bidding identification model training method, abnormal bidding identification method and device provided by the embodiments of the present disclosure include an abnormal bidding feature determination process and a model training process. In the abnormal bidding feature determination process, first, a bidding subject association information set is extracted from historical bidding data, then the bidding subject association information set is expanded to obtain a candidate abnormal bidding feature set, and then, based on the correlation between the candidate abnormal bidding features and the abnormal bids in the candidate abnormal bidding feature set, the abnormal bidding features are determined from the candidate abnormal bidding features. After the abnormal bidding feature determination process is completed, the model training process is entered to train the abnormal bidding identification model based on the abnormal bidding features.

[0035] The results output by the artificial intelligence machine learning algorithm of the disclosed embodiment based on historical bidding data, historical tender data, industrial and commercial information data, etc., are significantly helpful in discovering and warning of supplier bid rigging and collusion.

[0036] Figure 1 The application scenarios of the abnormal bidding identification model training method, abnormal bidding identification method and device according to the embodiments of the present disclosure are schematically illustrated.

[0037] like Figure 1 As shown, the tenderer can issue a tender document, which may include but is not limited to at least one of the following: a tender invitation letter, instructions to bidders, technical requirements and attachments for the tender project, tender document format, tender guarantee documents, contract conditions, technical standards and specifications, tenderer qualification documents and contract format, etc. The tender document can effectively regulate the format and content of the tender document, so that the tenderer can review the bidders, etc. For a tender document, one or more bidders can bid, such as providing their own tender documents, Figure 1 In the bidding document, there are bidding documents 1, ..., bidding documents n, etc., where n is a positive integer greater than 1.

[0038] The bidding document is a document prepared by the bidder for the bid to show the tenderer the products, services, quotations, qualifications and other information provided. For example, the bidding document includes but is not limited to at least one of the following: a total bidding quotation table, a detailed bidding quotation table, a brief introduction of the bidding company, a company qualification certificate, a bidder qualification table, a basic situation table of project staffing, a project staffing qualification table and personnel qualification certificate, etc. This application extracts the bidding subject related information from these bidding data and expands it based on industrial and commercial information, etc., so as to determine abnormal bidding features from the extended information and improve the correlation between abnormal bidding features and abnormal bidding results.

[0039] Figure 2 The exemplary system architecture to which the abnormal bidding identification method and abnormal bidding identification device according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 2What is shown is only an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. It should be noted that the abnormal bid identification model training method, abnormal bid identification method and device provided in the embodiments of the present disclosure can be used in the field of artificial intelligence in the field of abnormal bid identification, and can also be used in various fields other than the field of artificial intelligence, such as the field of abnormal bid identification. The application field of the abnormal bid identification model training method, abnormal bid identification method and device provided in the embodiments of the present disclosure is not limited.

[0040] like Figure 2 As shown, the system architecture 200 according to this embodiment may include terminal devices 201, 202, 203, a network 204 and a server 205. The network 204 may include multiple gateways, routers, hubs, network cables, etc., which are used to provide a medium for communication links between the terminal devices 201, 202, 203 and the server 205. The network 204 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0041] The user can use the terminal devices 201, 202, 203 to interact with other terminal devices and the server 205 through the network 204 to receive or send information, such as sending model training instructions, historical bidding data identifiers, bidding data, etc. The terminal devices 201, 202, 203 can be installed with various communication client applications, such as web browser applications, banking applications, e-commerce applications, search applications, office applications, instant messaging tools, email clients, social platform software and other applications (only for example).

[0042] The terminal devices 201, 202, 203 include but are not limited to smart phones, desktop computers, augmented reality devices, tablet computers, laptop computers, and other electronic devices that can support functions such as surfing the Internet and interacting with web pages. The abnormal bidding identification model can be downloaded on the terminal device for offline identification.

[0043] The server 205 may receive model training requests, abnormal bid identification requests, model download requests, etc., and process the requests. For example, the server 205 may be a background management server, a server cluster, etc. The background management server may analyze and process the received service requests, information requests, etc., and feed back the processing results (such as model training results, abnormal bid identification results, business information, model parameters obtained by training the model, etc.) to the terminal device.

[0044] It should be noted that the method for training the abnormal bidding identification model provided in the embodiment of the present disclosure can be executed by the server 205, and the abnormal bidding identification method can be executed by the terminal device 201, 202, 203 or the server 205. Accordingly, the abnormal bidding identification device provided in the embodiment of the present disclosure can be set in the terminal device 201, 202, 203 or the server 205. It should be understood that the number of terminal devices, networks and servers is only illustrative. According to the implementation requirements, there can be any number of terminal devices, networks and servers.

[0045] Figure 3 The flowchart of the method for training an abnormal bidding identification model according to an embodiment of the present disclosure is schematically shown.

[0046] like Figure 3 As shown, the method for training an abnormal bidding identification model may include operations S302 to S308.

[0047] In operation S302, a bidding entity associated information set is extracted from the historical bidding data.

[0048] In this embodiment, the historical bidding data can be data obtained from a business system or from a dedicated training data set. The bidding subject related information includes but is not limited to: the name of the bidding enterprise, the address of the bidding enterprise, the name of the person in charge of the bidding enterprise, the name of the shareholder of the bidding enterprise, the contact information of the bidding enterprise, etc.

[0049] In one embodiment, extracting a bidding entity associated information set from historical bidding data may include the following operations: obtaining historical bidding data from a data set with historical bidding data by calling a script or query statement through a batch scheduling program, and assigning index information to the historical bidding data so as to determine the historical bidding data based on the index information.

[0050] For example, obtain the full amount of historical bidding and tendering, industrial and commercial information and other data. Enterprise-related information includes: enterprise name, former name of the enterprise, enterprise address, legal representative, registered address administrative number, enterprise contact number, etc., enterprise actual controller information, enterprise overseas investment information, enterprise executive information, enterprise shareholder information, etc.

[0051] In operation S304, the bidding subject association information set is expanded to obtain a candidate abnormal bidding feature set.

[0052] In this embodiment, since the bidding subject association information included in the historical bidding data is not comprehensive, the abnormal bidding features determined based only on the historical bidding data are not perfect and may not be able to well characterize whether the current bidding data is abnormal bidding data. By expanding the bidding subject association information, such as splitting, merging or expanding the bidding subject association information, the dimension of the optional abnormal bidding features is effectively improved, which helps to improve the applicability of the constructed abnormal bidding features.

[0053] In one embodiment, expanding the bid subject association information set to obtain a candidate abnormal bidding feature set may include the following operations. First, obtaining extended information matching the bid subject from a data set including the bid subject registration information. Then, performing at least one of splitting, combining, and associating on the bid subject association information and the extended information matching the bid subject association information to obtain a candidate abnormal bidding feature set.

[0054] For example, regarding feature combination: such as combining the two features of date and time, constructing the feature of whether it is working time (working time on weekdays is 1, otherwise it is 0). The purpose of feature combination is usually to obtain new features that are more expressive and informative.

[0055] Regarding feature splitting: For example, complex business features can be split up, such as splitting the login feature into multiple dimensions of login count statistics. The benefits of feature splitting include: on the one hand, expressing information from multiple dimensions; on the other hand, multiple split features can be combined in more ways.

[0056] Regarding external association features: for example, associating time information with weather information is very meaningful. First of all, weather data is not part of the original data set, so this is equivalent to enriching the original data, and generally speaking, a better result will be obtained than using only the original data. Not only weather, but a lot of information can be associated in this way (for example, in a housing prediction problem on Kaggle, the year can be associated with some local policies, international events, etc. at the time, which are all influential, such as the financial crisis). In this embodiment, the bidding entity association information is mainly expanded through industrial and commercial registration information. Specifically, the extended information that matches the bidding entity association information can be obtained by calling the interface provided by the industrial and commercial information platform.

[0057] Figure 4 The schematic diagram schematically shows the expansion of the bidding subject association information set according to an embodiment of the present disclosure.

[0058] like Figure 4As shown, the bidding subject associated information set can be expanded based on the information of the industrial and commercial registration information set. The expanded bidding subject associated information set includes not only the information of the bidding subject associated information set and the information of the industrial and commercial registration information set, but also the information obtained after splitting and combining. For example, the enterprise address and the enterprise registration address both include the address. After splitting, the address (associated address) can be used as a candidate abnormal bidding feature. For another example, the former name of the enterprise can be combined with the executive information obtained by splitting to obtain the executive information of the former name of the enterprise (to facilitate the collection of executive information of the former name of the enterprise). This effectively improves the scope of application of abnormal bidding features.

[0059] In operation S306, an abnormal bid feature is determined from the candidate abnormal bid features based on the correlation between the candidate abnormal bid features and the abnormal bids in the candidate abnormal bid feature set.

[0060] Data mining and machine learning are the processes of extracting implicit and potentially useful information and knowledge from large amounts of incomplete, noisy, fuzzy, and random practical application data. The data source must be real, large, and noisy; the knowledge discovered is of interest to the user; the discovered knowledge must be acceptable, understandable, and applicable; and the discovered knowledge is not required to be universal, but only supports specific discovery problems.

[0061] Due to the variety of bidding documents and the variety of enterprises participating in the bidding, it is difficult to determine abnormal bidding features only through machine learning without guidance. The abnormal bidding features constructed based on expert experience can only cover limited scenarios, resulting in the breadth of application failing to meet the needs. In the disclosed embodiment, by calculating the correlation between the features and the target, the abnormal bidding features are determined from the expanded set of associated information of the bidding entities, which effectively improves the scope of application and accuracy of the abnormal bidding features.

[0062] In one embodiment, based on the correlation between the candidate abnormal bid features and the abnormal bids in the candidate abnormal bid feature set, determining the abnormal bid features from the candidate abnormal bid features may include the following operations.

[0063] First, the correlation between the candidate abnormal bidding feature and the abnormal bid is determined based on the quotient of the covariance and the standard deviation between the candidate abnormal bidding feature and the abnormal bid, and / or the correlation between the candidate abnormal bidding feature and the abnormal bid is determined based on the information entropy gain of the candidate abnormal bidding feature. Then, the candidate abnormal bidding feature that meets the correlation requirement is used as the abnormal bidding feature. The correlation requirement includes but is not limited to the correlation being greater than or equal to a preset correlation threshold.

[0064] For example, the correlation between features and targets can be judged by the Pearson product-moment correlation coefficient (PPMCC) and information entropy gain. The idea is that if the changes in a feature are highly consistent with those in the target, then it is of great guiding significance for predicting the target.

[0065] The Pearson product-moment correlation coefficient is used to measure the correlation between two variables X and Y. The value is between -1 and 1. The Pearson correlation coefficient between two variables is defined as the quotient of the covariance and standard deviation between the two variables.

[0066] It should be noted that in the process of determining abnormal bidding features, the distribution of the feature values ​​itself needs to be considered. For example, the variance filtering method can be used. For example, for gender features, 1,000 data, 999 of which are for men and 1 for women. This feature is too skewed and therefore cannot provide sufficient help to the results.

[0067] In one embodiment, the abnormal bidding features include at least one of the following: the bidding subject has a repeated phone number, the number of telephone numbers of the bidding subject, the bidding subject has a repeated email address, the bidding subject has a repeated fax number, the similarity of the registered address of the bidding subject meets the registered address similarity condition, the similarity of the address of the bidding subject meets the address similarity condition, the number of addresses of the bidding subject, the legal person of the bidding subject is repeated, the former name of the bidding subject, the repeated shareholders of the bidding subject meet the repeated shareholder condition, the repeated senior executives of the bidding subject meet the repeated senior executive condition, and the repeated controllers of the bidding subject meet the repeated controller condition. Among them, the registered address similarity condition can be greater than a preset similarity threshold, etc.

[0068] In operation S308, an abnormal bid identification model is trained based on the abnormal bid features.

[0069] In this embodiment, the process of training the abnormal bidding identification model can adopt a supervised training method. Supervised learning can be used in data mining and machine learning. A model (function / learning model) can be learned or established from the training data, and the model can be used to infer new instances. The training data consists of input objects (such as vectors) and expected outputs. The output of the function can be a continuous value (called regression analysis) or a predicted classification label (called classification). The task of the supervised learner is to predict the output value of the function for any possible input after observing some training examples (input and expected output).

[0070] In one embodiment, the historical bidding data includes tag information, and the tag information identifies abnormal bidding identification result information of the current bidding data.

[0071] Accordingly, the above method also includes: using the label information as supervision information for training the abnormal bid identification model.

[0072] As a supervised mining model, data classification finds and expands the distribution law of each data mark through the classification observation mark of the samples in the training set of data mining, and then generalizes this law to the subsequent large-scale data set. The abnormal bidding identification model provided by the embodiment of the present disclosure can be applied to risk user identification, etc.

[0073] In one embodiment, training an abnormal bid recognition model based on abnormal bid features may include the following operations: inputting a feature vector corresponding to the abnormal bid feature into the abnormal bid recognition model, and adjusting the model parameters of the abnormal bid recognition model so that the recognition result output by the abnormal bid recognition model approaches the label information.

[0074] In addition, in order to facilitate machine identification of abnormal bidding features, the abnormal bidding features can be vectorized.

[0075] Figure 5 A schematic diagram of an abnormal bidding feature vector according to an embodiment of the present disclosure is schematically shown.

[0076] like Figure 5 As shown in the figure, the abnormal bidding features correspond to the independent variables, which serve as the input variables of the abnormal bidding identification model. The category labels correspond to the dependent variables, which correspond to the output variables of the abnormal bidding identification model. Vectorization is the reprocessing of the feature extraction results, with the aim of enhancing the representation ability of the features and preventing the model from being too complex and difficult to learn. For example, continuous feature values ​​are discretized, and label values ​​are mapped into enumeration values, which are identified with numbers or characters. This stage will generate a very important file: the correspondence between labels and enumeration values, which will also be used in the prediction stage. Labels can be added manually or automatically by the system. For example, labels representing abnormal categories are automatically added to abnormal bidding data in historical bidding data.

[0077] For example, for supplier bid collusion, the result data of feature combinations include: companies with duplicate phone numbers (1 can be used to represent them, and 0 can be used if they do not exist); number of company phone numbers; companies with duplicate email addresses (1 can be used to represent them, and 0 can be used if they do exist); companies with duplicate fax numbers (1 can be used to represent them, and 0 can be used if they do not exist); companies with 80% similarity in registered addresses (1 can be used to represent them, and 0 can be used if they do not exist); companies with 80% similarity in company addresses (1 can be used to represent them, and 0 can be used if they do not exist); number of company addresses; companies with duplicate legal persons; former company names; companies with duplicate shareholders; companies with duplicate executives; companies with duplicate controlling persons; and highly consistent quotations from two bidding companies (if the difference is 2% or less, 1 will be used, otherwise 0 will be used).

[0078] After completing the model training, you can test the trained model to determine the accuracy of the model processing results.

[0079] In one embodiment, a suitable abnormal bid identification model may be selected from multiple models based on the accuracy of the model processing results.

[0080] For example, training an abnormal bid identification model based on abnormal bid features may include the following operations: first, calling at least two models with different model structures to perform model training separately by calling an application interface, and then selecting a model with the highest test accuracy from the at least two models with different model structures as the abnormal bid identification model.

[0081] Figure 6 A data flow diagram according to an embodiment of the present disclosure is schematically shown.

[0082] like Figure 6 As shown, first, determine the abnormal bidding features in the manner shown in operation S306. Since the supplier bid-rigging model has a class label, it belongs to supervised learning. The target variable is discrete (whether there is bid-rigging or not), which belongs to a classification model. Select the classification model algorithm. It includes four major algorithms: K-nearest neighbor algorithm; Naive Bayes algorithm; Support vector machine; Decision tree. The training set constructs the training model, and the test set selects the optimal model. If the accuracy of the results is compared, the XGboost algorithm is finally selected to predict whether the supplier is bid-rigging or not.

[0083] The data set obtained by the supplier bid-rigging model is split into a training set and a test set in a ratio of 8:2. For example, the data is divided into two parts: (1) training set and (2) test set. The first part is a larger subset of the data, which is used as a training set (such as 80% of the original data), and the second part is usually a smaller subset, which is used as a test set (the remaining 20% ​​of the data). The training set is used to train the abnormal bidding identification model, and then the test set is input into the trained model (i.e., as new, unseen data) for prediction. The best model is selected based on the performance of the model on the test set. In order to obtain the best model, hyperparameter optimization (such as the number of layers of the neural network) can also be performed.

[0084] In the testing phase, evaluation indicators can be selected according to the issues that the regression model is concerned about. The evaluation indicators are shown in Table 1.

[0085] Table 1 Evaluation indicators

[0086]

[0087] During the test, the "overfitting" and "underfitting" of the model can be judged. If the data is overfitted, it means that the noise is also regarded as a general feature of the data during the model training process. The overfitting problem can be solved by increasing the proportion of the training set or regularization. If the data is not well fitted, it means that the data training is not in place and the general features of the data cannot be extracted. The underfitting problem should be solved by increasing the polynomial dimension, reducing the regularization parameter, etc. In addition, time and space complexity, stability, migration, etc. should also be considered when testing.

[0088] After you have completed testing, if you want to further improve your training, you can repeat the training and testing process.

[0089] In one embodiment, before extracting the bidder-related information set from the historical bidding data, the above method may also include the following operations: performing missing processing and / or exception processing on the historical bidding data, wherein the abnormal bidding data subjected to exception processing is abnormal data determined based on statistical results, or is determined based on an analysis of the context of the abnormal bidding data.

[0090] For example, various checks can be performed on the data, removing missing values, splitting the data, and normalizing / standardizing the values.

[0091] Regarding missing data: missing data may be caused by program defects (bugs). Such missing data are usually rare and can be filled.

[0092] Regarding missing data caused by normal business situations: For example, the gender field itself can be left blank, but there will be missing data for gender, and this missing data may be large. Here, we must first evaluate the importance and missing rate of the field, and then consider whether to fill it in or discard it.

[0093] Regarding exception handling. Among them, absolute exceptions: for example, a person’s age is 200 years old. This data is abnormal in any scenario. Statistical exceptions: for example, a user logs in 100 times in one minute. Although each login looks normal, it is abnormal when counted (it may be a script operating automatically). Contextual exceptions: For example, in Beijing in winter, the temperature at night is 30 degrees Celsius. Although the data looks normal, it is abnormal when associated with the current date and time.

[0094] Figure 7 A logic diagram of a method for training an abnormal bidding identification model according to an embodiment of the present disclosure is schematically shown.

[0095] like Figure 7 As shown in the figure, after completing the collection of historical bidding data, determine the abnormal bidding features. Among them, the collection of historical bidding data is very important. The quality and quantity of the collected data directly determine whether the prediction model can be built. The collected data can be deduplicated, standardized, error corrected, etc., and saved as a database file or csv format file to prepare for the next step of data loading. The data set is essentially an M×N matrix, where M represents columns (features) and N represents rows (samples). Figure 5 As shown. The columns can be decomposed into X and Y. First, X represents the features, independent variables, and input variables. Y represents the category labels, dependent variables, and output variables. Among them, M and N are positive integers greater than 1.

[0096] During data preprocessing, various checks are performed on the data, including removing missing values, splitting data, and normalizing / standardizing values.

[0097] In the process of splitting the data set, you can split it according to a preset ratio, such as 8:2 or 7:3, to obtain a training set and a test set.

[0098] After the training is completed, the model file (Model file) is output. After the model is trained, four types of files can be sorted out to ensure that the model can run correctly, such as Model file, label encoding file, metadata file (algorithm, parameters and results), variable file (independent variable name list, dependent variable name list).

[0099] The method for training an abnormal bidding identification model in the embodiment of the present disclosure can effectively improve the model training effect and improve the prediction accuracy of the trained model.

[0100] Another aspect of the present disclosure provides a method for identifying abnormal bids.

[0101] Figure 8 The flowchart of the abnormal bidding identification method according to the embodiment of the present disclosure is schematically shown.

[0102] like Figure 8 As shown, the abnormal bidding identification method includes operations S802 to S804.

[0103] In operation S802, input bidding data is obtained. The input bidding data is the current bidding data to be predicted, and its attributes can refer to historical bidding data, which will not be described in detail here.

[0104] In operation S804, the abnormal bid identification model trained by the above method is used to process the input bid data to obtain an identification result for the input bid data.

[0105] The abnormal bidding features used by the abnormal bidding identification model in the identification process are the same as the abnormal bidding features used in the model training process, and will not be described in detail here.

[0106] Another aspect of the present disclosure provides an abnormal bidding identification device.

[0107] Fig. 9 A block diagram of an abnormal bidding identification device according to an embodiment of the present disclosure is schematically shown.

[0108] like Fig. 9 As shown, the device 900 includes: a bidding data acquisition module 910 and an abnormal bidding identification module 920.

[0109] The bidding data acquisition module 910 is used to acquire input bidding data.

[0110] The abnormal bid identification module 920 is used to process the input bid data using the abnormal bid identification model trained by the above method to obtain the identification result for the input bid data.

[0111] Another aspect of the present disclosure provides an abnormal bid identification system.

[0112] Fig.10 A block diagram of an abnormal bidding identification system according to an embodiment of the present disclosure is schematically shown.

[0113] like Fig.10 As shown, the abnormal bidding identification system may include a model training system and a business system. The model training system includes a data lake, a data processing node, and a machine learning platform.

[0114] The data lake is a collection of various types of data, and users can process the data in the data lake. The data lake can be a distributed database (such as Hadoop). Using this distributed database is conducive to improving data throughput and data reliability.

[0115] Data processing nodes provide software tools and hardware equipment for processing data. Data processing nodes process data using Hive SQL scripts and Spark programs. They use batch scheduling programs to process data in the data lake, call shell scripts or Hive SQL or Spark to process data in the data lake, store the processed data in a self-built directory in the data lake, and transmit it to the machine learning platform for batch model training.

[0116] The machine learning platform is responsible for data preprocessing, feature extraction, and model training, and provides the entire process software and hardware equipment from data to model to call.

[0117] The business system can generate data to be predicted and call the prediction model in real time through the machine learning platform application interface (API) to predict the data to be predicted. In addition, batch prediction can also be performed through batch scheduling. The business database can store the prediction data transmitted by the machine learning platform, and the business system can call the prediction data in the business database. The prediction data can be displayed in the user terminal or application (APP) through network interfaces, web services, and network transmission equipment. In addition, the business database can also transfer the newly added data with tags to the data lake of the model training system for updating and iterating the prediction model.

[0118] It should be noted that the implementation methods, technical problems solved, functions realized, and technical effects achieved of each module / unit in the device part embodiment are the same or similar to the implementation methods, technical problems solved, functions realized, and technical effects achieved of each corresponding step in the method part embodiment, and they will not be repeated here one by one.

[0119] According to the embodiments of the present invention, any multiple of the modules and units, or at least part of the functions of any multiple of them, can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules and units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules and units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules and units can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0120] For example, any multiple of the bid data acquisition module 910 and the abnormal bid identification module 920 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the bid data acquisition module 910 and the abnormal bid identification module 920 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, at least one of the bid data acquisition module 910 and the abnormal bid identification module 920 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0121] Fig.11 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0122] like Fig.11As shown, the electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include an onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0123] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other through bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0124] According to an embodiment of the present disclosure, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 may further include one or more of the following components connected to the I / O interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.

[0125] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0126] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0127] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1102 and / or RAM 1103 described above and / or one or more memories other than ROM 1102 and RAM 1103.

[0128] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the image model training method or abnormal bid identification method provided by the embodiment of the present disclosure.

[0129] When the computer program is executed by the processor 1101, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0130] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0131] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0132] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. These embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the various embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, all of which should fall within the scope of the present disclosure.

Claims

1. A method for training an abnormal bidding identification model, comprising: Extracting a collection of bidding subject related information from historical bidding data; The bidding subject association information set is expanded to obtain a candidate abnormal bidding feature set, The step of expanding the bidding subject association information set to obtain a candidate abnormal bidding feature set includes: Acquire extended information matching the bidding subject from a data set including registration information of the bidding subject; and Perform at least one of splitting, combining and associating on the bidding subject association information and the extended information matching the bidding subject association information to obtain a candidate abnormal bidding feature set; determining an abnormal bidding feature from the candidate abnormal bidding features based on the correlation between the candidate abnormal bidding features and the abnormal bids in the candidate abnormal bidding feature set; and The abnormal bidding identification model is trained based on the abnormal bidding features.

2. The method according to claim 1, wherein: The determining of the abnormal bidding feature from the candidate abnormal bidding features based on the correlation between the candidate abnormal bidding features and the abnormal bid in the candidate abnormal bidding feature set comprises: determining the correlation between the candidate abnormal bid feature and the abnormal bid based on a quotient of a covariance and a standard deviation between the candidate abnormal bid feature and the abnormal bid, and / or determining the correlation between the candidate abnormal bid feature and the abnormal bid based on an information entropy gain of the candidate abnormal bid feature; and The candidate abnormal bidding features that meet the relevance requirement are used as the abnormal bidding features.

3. The method according to claim 1, wherein: The historical bidding data includes label information, and the label information identifies abnormal bidding identification result information of the current bidding data; as well as The method also includes: using the label information as supervisory information for training the abnormal bid identification model.

4. The method according to claim 3, wherein: The training of the abnormal bidding identification model based on the abnormal bidding feature includes: The feature vector corresponding to the abnormal bidding feature is input into the abnormal bidding recognition model, and the model parameters of the abnormal bidding recognition model are adjusted so that the recognition result output by the abnormal bidding recognition model approaches the label information.

5. The method according to claim 1, wherein: The extracting of the bidding subject related information set from the historical bidding data comprises: The batch scheduling program calls a script or a query statement to obtain the historical bidding data from a data set having the historical bidding data, and assigns index information to the historical bidding data so as to determine the historical bidding data based on the index information.

6. The method according to claim 1, wherein: The training of the abnormal bidding identification model based on the abnormal bidding feature includes: Calling at least two models with different model structures to perform model training respectively by calling an application interface; and A model with the highest test accuracy is selected from at least two models with different model structures as the abnormal bidding identification model.

7. The method according to any one of claims 1 to 6, before extracting the bidder association information set from the historical bidding data, the method further comprises: The historical bidding data is subjected to missing data processing and / or exception processing, wherein the abnormal bidding data subjected to the exception processing is abnormal data determined based on statistical results, or is determined based on analysis of the context of the abnormal bidding data.

8. The method according to any one of claims 1 to 6, wherein: The abnormal bidding features include at least one of the following: the bidding entity has a duplicate telephone number, the number of telephone numbers of the bidding entity, the bidding entity has a duplicate email address, the bidding entity has a duplicate fax number, the similarity of the registered addresses of the bidding entity meets the registered address similarity condition, the address similarity of the bidding entity meets the address similarity condition, the number of addresses of the bidding entity, the legal person of the bidding entity is repeated, the former name of the bidding entity, the duplicate shareholders of the bidding entity meet the duplicate shareholder condition, the duplicate senior executives of the bidding entity meet the duplicate senior executive condition, and the duplicate controllers of the bidding entity meet the duplicate controller condition.

9. A method for identifying abnormal bidding, comprising: Get input bid data; as well as The input bid data is processed using an abnormal bid identification model trained by the method according to any one of claims 1 to 8 to obtain an identification result for the input bid data.

10. An abnormal bidding identification device, comprising: A bidding data acquisition module, used to acquire input bidding data; as well as An abnormal bid identification module is used to process the input bid data using an abnormal bid identification model trained by the method according to any one of claims 1 to 8 to obtain an identification result for the input bid data.

11. An electronic device, comprising: one or more processors; A storage device for storing executable instructions, which, when executed by the processor, implements the method for training an abnormal bid identification model according to any one of claims 1 to 8, or implements the abnormal bid identification method according to claim 9.

12. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, implement the method for training an abnormal bid identification model according to any one of claims 1 to 8, or implement the abnormal bid identification method according to claim 9.

Images