Method, device and equipment for constructing training sample of classification model and storage medium

Abstract

Embodiments of the present application provide a kind of classification model training sample construction method, device and equipment and storage medium, related to computer technical field.The method comprises: obtaining the initial feature set corresponding to N objects respectively, the initial feature set of the i th object in N objects includes the M associated features of the i th object;Obtain the associated media content of the i th object, the associated media content of the i th object is the media content related to the i th object and the classification target of classification model in media content library;Determine the media feature of the i th object based on the associated media content of the i th object;In the initial feature set of the i th object, add the media feature of the i th object, obtain the training sample of the i th object;Based on the training sample corresponding to N objects respectively, generate training sample set.Can be based on media feature analysis object recent possible change, so that classification result is more accurate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for constructing training samples for a classification model. Background Technology

[0002] Classification models are used to predict the probability that an object belongs to multiple categories based on input data, in order to determine the category to which the object belongs.

[0003] In related technologies, training samples are typically mined from the object itself, and then the classification model is trained based on these training samples. For example, if the object is a company, training a classification model for companies usually involves determining the training samples based on relevant information about the company itself, such as the industry the company belongs to or its output value.

[0004] However, the training samples used to construct the classification model using the above methods are relatively fixed and conventional, which leads to the classification model trained using these training samples having inaccurate classification results. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for constructing training samples for a classification model. The technical solution is as follows:

[0006] According to one aspect of the embodiments of this application, a method for constructing training samples for a classification model is provided, the method comprising:

[0007] Obtain the initial feature set corresponding to N objects respectively. The initial feature set of the i-th object among the N objects includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0008] Obtain the associated media content of the i-th object, wherein the associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model;

[0009] Based on the associated media content of the i-th object, the media characteristics of the i-th object are determined, and the media characteristics of the i-th object are used to characterize the quantity of associated media content of the i-th object;

[0010] Add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object;

[0011] Based on the training samples corresponding to the N objects, a training sample set for training the classification model is generated.

[0012] According to one aspect of the embodiments of this application, a training sample construction apparatus for a classification model is provided, the apparatus comprising:

[0013] The initial feature acquisition module is used to acquire the initial feature sets corresponding to N objects respectively. The initial feature set of the i-th object among the N objects includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0014] The media content acquisition module is used to acquire the associated media content of the i-th object, wherein the associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model;

[0015] The media feature determination module is used to determine the media features of the i-th object based on the associated media content of the i-th object, wherein the media features of the i-th object are used to characterize the quantity of associated media content of the i-th object;

[0016] The media feature addition module is used to add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object;

[0017] The training sample set generation module is used to generate a training sample set for training the classification model based on the training samples corresponding to the N objects respectively.

[0018] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the training sample construction method of the above-described classification model.

[0019] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the training sample construction method of the above-described classification model.

[0020] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program, the computer program being loaded and executed by a processor to implement the training sample construction method of the above-described classification model.

[0021] The technical solutions provided in this application embodiment may have the following beneficial effects:

[0022] Adding media features to the initial feature set of an object takes into account the impact of media content on the classification results of the object. It allows for analysis of possible recent changes in the object based on media features, making the classification results more accurate. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of a computer system provided in one embodiment of this application;

[0024] Figure 2 This is a flowchart of a method for constructing training samples for a classification model provided in one embodiment of this application;

[0025] Figure 3 This is a flowchart illustrating the process of obtaining associated media content according to an embodiment of this application;

[0026] Figure 4 This is a schematic diagram illustrating the acquisition of associated media content according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of a feature value supplementation method provided in one embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the framework of a classification model provided in one embodiment of this application;

[0029] Figure 7 This is a flowchart illustrating a method of using a classification model provided in one embodiment of this application;

[0030] Figure 8 This is a flowchart of a method for constructing training samples for a classification model provided in another embodiment of this application;

[0031] Figure 9 This is a flowchart of a method for constructing training samples for a classification model provided in another embodiment of this application;

[0032] Figure 10 This is a block diagram of a training sample construction apparatus for a classification model provided in one embodiment of this application;

[0033] Figure 11 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] Please refer to Figure 1 This illustration shows a schematic diagram of a computer system provided in one embodiment of this application. The computer system may include: a terminal device 100 and a server 200.

[0036] Terminal device 100 may be an electronic device such as a PC (Personal Computer), mobile phone, wearable device, in-vehicle terminal, VR (Virtual Reality) device, AR (Augmented Reality) device, or MR (Mixed Reality) device. In some embodiments, a classification model is deployed in terminal device 100, or terminal device 100 establishes a communication connection with server 200 that deploys the classification model. In some embodiments, terminal device 100 may be used for training the classification model, for example, terminal device 100 may perform the steps of training the classification model. In some embodiments, terminal device 100 may assist server 200 in performing the steps of training the classification model. For example, terminal device 100 obtains initial feature sets corresponding to N objects, where the initial feature set of the i-th object includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N; it obtains associated media content of the i-th object, which is media content in the media content library related to the i-th object and the classification target of the classification model; based on the associated media content of the i-th object, it determines the media features of the i-th object, which represent the quantity of associated media content of the i-th object; it adds the media features of the i-th object to the initial feature set of the i-th object to obtain training samples of the i-th object; and based on the training samples corresponding to the N objects, it generates a training sample set for training the classification model. The terminal device sends the training sample set to server 200, and server 200 trains the classification model based on the training sample set.

[0037] Server 200 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0038] Terminal device 100 and server 200 can communicate via a network, such as a wired or wireless network.

[0039] The training sample construction method for the classification model provided in this application can be executed by a computer device at each step. The computer device refers to an electronic device capable of data computation, processing, and storage. Figure 1Taking the computer system shown as an example, the method for constructing training samples for the classification model can be executed by the terminal device 100, or by the server 200, or by the terminal device 100 and the server 200 interacting and cooperating to execute the method for constructing training samples for the classification model. This application does not limit this. For ease of explanation, in the following method embodiments, only the computer device is described as the main body executing each step of the method for constructing training samples for the classification model.

[0040] Industrial and commercial enterprise financial risk rating refers to the assessment of the risk level of industrial and commercial enterprises facing potential financial risks. It is divided into ten levels, with level one representing the lowest risk and level ten representing the highest financial risk. Risk rating of industrial and commercial enterprises helps in preparing appropriate financial risk contingency plans when selecting business entities, enabling timely reduction of holdings of bonds issued by these entities, and minimizing investment losses.

[0041] Corporate financial risk rating is essentially a multi-class classification task. Traditional classification methods mostly employ logistic regression or XGBoost to classify the data, using K-fold cross-validation to select the optimal parameters, and finally using ensemble learning for classification. However, this traditional approach has significant limitations. First, in real-world corporate financial risk rating, the distribution of risk levels exhibits a significant normal distribution, meaning that most data is concentrated in the middle levels (5, 6, 7, and 8), while levels 1, 2, 9, and 10 have less data. If the classification loss is calculated using conventional methods, these classes with less data will not receive sufficient training, resulting in poor model performance on these classes. Furthermore, unlike general classification tasks, corporate financial risk rating has different tolerance levels. In ordinary classification tasks, the penalty for predicting a sample with a true class of "10" as "9" is the same as predicting it as "1". However, in the scenario of corporate financial risk rating, the penalty for predicting a sample with a true class of "10" as "1" is much greater than that for predicting it as "9". This is because corporate financial risk rating is used as a reference for deciding whether to purchase bonds or products of that entity. If the entity is predicted as "1", the conclusion that the entity has very low risk and can be purchased with confidence would be incorrect. Therefore, this classification task involves a sequence of events, and this must be taken into account when building the model.

[0042] Meanwhile, in terms of handling missing values, conventional methods include padding with zeros or taking the mean / mode of the feature for that dimension. However, these methods often overlook the unique characteristics of a company. Companies in different industries and regions often have different values ​​for that dimension. Therefore, a more detailed feature value completion scheme is needed.

[0043] In addition, media features are rarely incorporated into traditional feature construction, but these short-term and quick features often reflect the company's recent capital market value. Therefore, media features related to the frequency of bond price fluctuations are also quite important.

[0044] The above technical solution has the following drawbacks in the actual classification process:

[0045] The model did not take into account the differences in data volume between different categories and treated all samples the same, resulting in the model only being fully trained on the class with the largest amount of data.

[0046] It does not consider the order between different categories and treats all categories the same;

[0047] The differences in feature engineering between different types of features were not fully considered;

[0048] The handling of missing values ​​did not take into account the characteristics of enterprises in different industries and regions;

[0049] The characteristics of the media were not taken into account.

[0050] This application proposes a method for constructing training samples for a base classification model. It optimizes feature engineering by using a knowledge graph to fill in missing feature values ​​for enterprises, extracts media features related to bond price fluctuations using a large model, and employs appropriate feature processing methods for different feature types to fully utilize the information from different features. It uses class-balanced cross-entropy loss as the model's loss to alleviate the problem of extremely imbalanced data distribution, enabling the model to be adequately trained even on classes with fewer samples. Finally, it adds list-wise loss as the model's loss to construct ranking relationships between categories, making the model more inclined to make predictions closer to the true categories.

[0051] Please refer to Figure 2 The diagram illustrates a flowchart of a method for constructing training samples for a classification model according to an embodiment of this application. The method is performed by a computer device. The method includes at least one of the following steps 210 to 250.

[0052] Step 210: Obtain the initial feature set corresponding to each of the N objects. The initial feature set of the i-th object among the N objects includes the M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0053] In some embodiments, a computer device refers to an electronic device capable of data computing, processing, and storage. Exemplarily, a computer device can be a terminal device or a server. In other words, steps 210-250 can be implemented solely by the terminal device, solely by the server, or through interaction between the terminal device and the server. The interaction between the terminal device and the server depends on their respective computing capabilities.

[0054] In some embodiments, the initial feature set of the i-th object includes M associated features of the i-th object, where associated features refer to features related to the classification objective of the classification model. The classification objective of the classification model refers to the application scenario of the classification result, or the expected classification effect of the classification model. For example, if the object is a company, the classification model is used to determine the risk level of the company, and the classification objective of the classification model is to classify the risk level of the company. For example, if the object is a product, the classification model is used to determine the recommendation order of the product in the application interface, and the classification objective of the classification model is to provide the application interface with products arranged in the recommendation order.

[0055] Taking a company as an example, where the classification model is used to determine its risk level, the associated features can include financial indicators such as total assets, debt-to-equity ratio, and proportion of bond financing, as well as tag data such as whether there is a substantial default and whether there are abnormal valuations on the ChinaBond platform. Taking a product as an example, where the classification model is used to determine the recommended order of products in the application interface, the associated features can include product type, number of views of similar products, price, product reviews, and whether the product or the store providing the product has been blacklisted.

[0056] Step 220: Obtain the associated media content of the i-th object. The associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model.

[0057] In some embodiments, media content refers to information content disclosed through media. Media content may include content disclosed through print media, such as news published through newspapers; it may also include content disclosed through electronic media, such as promotional videos and news videos disclosed through video software.

[0058] In some embodiments, the associated media content of the i-th object is retrieved from the media content library. The media content library is used to store media content related to the object over a period of time. Here, "related to the object" means related to the type of N objects or the common characteristics of the N objects. For example, if the object is a company, a classification model is used to determine the risk level of the company, and the media content library is used to store media content related to the company over a period of time. If all N objects are food companies, the media content library can also be used to store media content related to food companies over a period of time.

[0059] In some embodiments, the specific duration of the aforementioned period can be determined based on the nature of the object. For example, if the object is a business, the business's development is usually relatively stable, so the period can be a longer time period such as six months, a year, or several years. For example, if the object is online shopping goods, since the sales volume of online shopping goods is easily affected by media content, the period can be a shorter time period such as a month or a quarter.

[0060] In some embodiments, related media content for the i-th object is retrieved from the media content library based on the relevant information and classification target of the i-th object. For example, related media content for the i-th object is retrieved from the media content library based on the name and classification target of the i-th object. For instance, if the object is a company, and the i-th object is "xx company," related media content for the i-th object is retrieved from the media content library using "xx company" and the classification target as keywords.

[0061] In some embodiments, the publicly available media content of the i-th object is obtained from the media content library, where the publicly available media content of the i-th object is media content related to the i-th object. From the publicly available media content of the i-th object, publicly available media content related to the classification target of the classification model is filtered out to obtain the associated media content of the i-th object. For example, based on the relevant information of the i-th object, the publicly available media content of the i-th object is retrieved from the media content library, and then the publicly available media content of the i-th object is filtered out based on the classification target.

[0062] Step 230: Based on the associated media content of the i-th object, determine the media features of the i-th object. The media features of the i-th object are used to characterize the quantity of associated media content of the i-th object.

[0063] In some embodiments, the media characteristics of the i-th object are determined based on the number of associated media contents of the i-th object. For example, the number of associated media contents of the i-th object is determined as the media characteristic of the i-th object.

[0064] In some embodiments, the quantity of associated media content of the i-th object is determined as a media characteristic of the i-th object. For example, the frequency of occurrence of associated media content of the i-th object is determined as a media characteristic of the i-th object.

[0065] Step 240: Add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object.

[0066] In some embodiments, the M associated features of the i-th object and the media features of the i-th object are determined as training samples of the i-th object.

[0067] Step 250: Generate a training sample set for training the classification model based on the training samples corresponding to the N objects.

[0068] In some embodiments, the training samples corresponding to the N objects are used as a training sample set. In some embodiments, the training samples of the N objects are used as input data to train the classification model, thereby obtaining the trained classification model.

[0069] The technical solution provided in this application adds media features to the initial feature set of the object, taking into account the impact of media content on the classification results of the object. It can analyze the possible recent changes of the object based on the media features, making the classification results more accurate.

[0070] Acquisition of Media Characteristics

[0071] The following will explain in detail the contents of steps 220 and 230. The following embodiments can be combined with other embodiments in this application to obtain new embodiments.

[0072] Step 220: Obtain the associated media content of the i-th object. The associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model.

[0073] In some embodiments, the media content library is obtained from publicly available media content. For example, all publicly available media content published within a certain period is identified as the media content library. For instance, publicly available media content published within the last six months is identified as the media content library. For example, publicly available media content published within a certain period that relates to the categories of N objects is identified as the media content library. For instance, publicly available media content related to enterprises published within the last six months is identified as the media content library. For example, publicly available media content related to food enterprises in region x published within the last six months is identified as the media content library.

[0074] In some embodiments, such as Figure 3 As shown, step 220 can be implemented as at least one of steps 221 to 222 below.

[0075] Step 221: Obtain the public media content of the i-th object from the media content library. The public media content of the i-th object is the media content related to the i-th object.

[0076] In some embodiments, the publicly available media content of the i-th object is media content related to the i-th object. For example, the publicly available media content of the i-th object includes the name or abbreviation of the i-th object. For instance, if the i-th object is xx company, abbreviated as x1, the publicly available media content of the i-th object includes xx company or x1. For example, the publicly available media content of the i-th object includes the relevant responsible persons of the i-th object. For example, if the object is a company, the publicly available media content of the i-th object includes the directors, managers, etc., of the i-th object.

[0077] In some embodiments, based on the relevant information of the i-th object, the publicly available media content of the i-th object is retrieved from the media content library. In some embodiments, the relevant information of the i-th object is determined based on the nature of the i-th object. For example, if the object is a company, companies are easily influenced by the media content of relevant responsible persons, so the relevant information may include the relevant responsible persons of the company, the company name, abbreviation, the industry in which the company operates, the region in which the company is located, etc. For example, if the object is an online shopping product, online shopping products are easily influenced by marketing content, so the relevant information may include the marketing content, name, type, etc. of the online shopping product.

[0078] In some embodiments, the process of determining publicly available media content from a media content library can be performed using a first neural network model. The structure of the first neural network model is not limited in this application. Exemplarily, it can be a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), etc. For example, the publicly available media content of the i-th object can be determined from the media content library using an LLM (Large Language Model) + prompt1 approach.

[0079] Step 222: From the public media content of the i-th object, filter out the public media content that is related to the classification target of the classification model to obtain the associated media content of the i-th object.

[0080] In some embodiments, related media content for the i-th object is filtered from its publicly available media content based on a classification objective. In other embodiments, related media content for the i-th object is determined based on the semantic information of the classification objective and the semantic information of the publicly available media content of the i-th object. While this method has high accuracy, it requires parsing both the classification objective and the semantic information of the publicly available media content of the i-th object, which is computationally intensive and places high demands on the computing power of the computer equipment. Therefore, it is possible to determine the search content based on the classification objective, and then filter the related media content for the i-th object based on the search content. For example, the search content may include keywords, media content themes, etc.

[0081] In one example, keywords are determined based on the classification objective of the classification model. Based on the keywords, publicly available media content containing the keywords is filtered from the publicly available media content of the i-th object to obtain the associated media content of the i-th object. In some embodiments, publicly available media content containing the keywords is retrieved from the publicly available media content of the i-th object, and the publicly available media content containing the keywords is determined as the associated media content of the i-th object.

[0082] For example, if the classification objective is to determine a company's financial risk, then the keywords could be financial, bonds, etc. In some embodiments, there can be one or more keywords. In some embodiments, when there are multiple keywords, the keywords can be terms with the same or similar meanings, or terms with different meanings. For example, if the classification objective is to determine a company's financial risk, then the keywords could include financial, property, bonds, debt, etc.

[0083] In some embodiments, when there are multiple keyword entries, the media content corresponding to each of the multiple keyword entries is determined as associated media content. For example, if keyword entries include A and B, and media content corresponding to A is 1, 2, and 3, while media content corresponding to B is 3, 4, and 5, then media content 1, 2, 3, 4, and 5 are all determined as associated media content. In some embodiments, when there are multiple keyword entries, the intersection of the media content corresponding to each of the multiple keyword entries is determined as associated media content. For example, if keyword entries include A and B, and media content corresponding to A is 1, 2, and 3, while media content corresponding to B is 3, 4, and 5, then media content 3 is determined as associated media content.

[0084] In another example, the media content topic is determined based on the classification objective of the classification model; based on the media content topic, publicly available media content related to the media content topic is filtered from the publicly available media content of the i-th object to obtain the associated media content of the i-th object. In some embodiments, the topics corresponding to the publicly available media content of the i-th object are determined, and the publicly available media content related to the media content topic is determined as the associated media content of the i-th object. In some embodiments, in order to reduce the amount of computation required by the computer device, the steps of determining the topics corresponding to the publicly available media content of the i-th object can be simplified, for example, by determining the corresponding topics only based on the titles of the publicly available media content of the i-th object.

[0085] For example, if the classification objective is to determine a company's financial risk, then the media content topic could be defined as finance, bonds, etc. In some embodiments, the media content topic can be one or more. In some embodiments, when there are multiple media content topics, these topics can be terms with the same or similar meanings, or terms with different meanings. For example, if the classification objective is to determine a company's financial risk, then the media content topic could include finance, property, bonds, debt, etc.

[0086] In some embodiments, when there are multiple media content topics, the media content corresponding to each of the multiple media content topics is determined as associated media content. For example, if media content topics include A and B, and media content corresponding to A is 1, 2, and 3, and media content corresponding to B is 3, 4, and 5, then media content 1, 2, 3, 4, and 5 are all determined as associated media content. In some embodiments, when there are multiple keywords, the intersection of the media content corresponding to each of the multiple media content topics is determined as associated media content. For example, if media content topics include A and B, and media content corresponding to A is 1, 2, and 3, and media content corresponding to B is 3, 4, and 5, then media content 3 is determined as associated media content.

[0087] In some embodiments, the process of determining associated media content from publicly available media content can be performed using a second neural network model. The structure of the second neural network model is not limited in this application. Exemplarily, it can be a CNN, RNN, etc. For example, the publicly available media content of the i-th object can be determined from a media content library using an LLM+prompt2 approach.

[0088] For example, the object is a company, and the classification model is used to determine the company's risk level. The associated media content for the i-th object includes media content related to the profit and loss situation of the i-th object and the company. For example, media content refers to news, such as... Figure 4As shown, this method mines the feature of "abnormal bond price fluctuations" from news articles, specifically the frequency of abnormal bond price fluctuations within a certain date range. First, it pulls news articles from the past six months / year from the news database. Using LLM+prompt1, it determines whether the news is relevant to company A, filtering out irrelevant news. Then, it uses LLM+prompt2 to determine whether the remaining news is related to bond price fluctuations; if relevant, it outputs "yes," otherwise "no." Finally, it counts the frequency of "yes" in all news articles and adds this as a media feature to the feature vector.

[0089] Step 230: Determine the media characteristics of the i-th object based on the associated media content of the i-th object.

[0090] In some embodiments, the media feature of the i-th object is used to characterize the quantity of associated media content of the i-th object. For example, the quantity of associated media content of the i-th object can be determined as the media feature of the i-th object. For example, the media feature of the i-th object can be determined based on the frequency of occurrence of the associated media content of the i-th object.

[0091] In some embodiments, such as Figure 3 As shown, step 230 can be implemented as at least one of steps 231 to 232 below.

[0092] Step 231: Determine the frequency of occurrence of the associated media content of the i-th object. The frequency of occurrence is the frequency of occurrence of the associated media content of the i-th object in the public media content of the i-th object.

[0093] Step 232: Determine the media characteristics of the i-th object based on its frequency of occurrence.

[0094] In some embodiments, the frequency of occurrence of the associated media content of the i-th object can be either the number of times the associated media content of the i-th object appears or the frequency of occurrence of the associated media content of the i-th object.

[0095] In some embodiments, the frequency of occurrence is determined as the media feature of the i-th object. In some embodiments, the media feature of the i-th object is determined after processing the frequency of occurrence. For example, the frequency of occurrence is normalized to obtain the media feature of the i-th object.

[0096] Using the method described above, the associated media features of the i-th object are selected from the media content library, and then the media features of the i-th object are determined based on the associated media features. The implementation method for obtaining the media features of the i-th object is given.

[0097] Regarding eigenvalue supplements

[0098] In some embodiments, certain associated features of objects may have missing feature values, which can be supplemented by adding feature values. Exemplary embodiments of feature value supplementation methods are also provided in this application.

[0099] In some embodiments, the feature value supplementation method may include at least one of steps 1 to 3 below. This feature value supplementation method can be combined with other embodiments of this application to obtain new embodiments.

[0100] Step 1: If one or more of the associated features of the i-th object are missing, determine at least one classification node corresponding to the i-th object. Different classification nodes are used to represent different classification criteria for the i-th object.

[0101] In some embodiments, at least one classification node is determined based on the classification objective, aiming to identify classification nodes that have a strong correlation with the associated features of the object. In some embodiments, at least one influencing factor is determined based on the classification objective of the classification model. An influencing factor is a factor that affects the feature values ​​of the M associated features of the i-th object; at least one classification node is determined based on the at least one influencing factor. In some embodiments, at least one influencing factor is determined as at least one classification node. In some embodiments, the influencing factor with the highest Z-ranking influence among the at least one influencing factors is determined as a classification node, where Z is a positive integer. In some embodiments, the value of Z can be pre-configured.

[0102] For example, if the object is a business, at least one category node can be identified as industry, region, etc.; if the object is an online shopping product, at least one category node can be identified as type, place of origin, price range, etc.

[0103] Step 2: Based on the sample sets corresponding to at least one classification node, determine at least one associated sample. The associated sample belongs to the intersection of the sample sets corresponding to at least one classification node.

[0104] In some embodiments, each classification node corresponds to a sample set. For example, industry X corresponds to a sample set containing all companies belonging to industry X; region Y corresponds to a sample set containing all companies located in region Y. Since the i-th object belongs to both industry X and region Y, the companies identified by the intersection of the sample set for industry X and the sample set for region Y have a stronger correlation with the i-th object. Therefore, the intersection of the sample sets corresponding to at least one classification node is used to determine the associated samples.

[0105] Step 3: Use the feature value corresponding to at least one associated sample to fill the missing feature value in the M associated features of the i-th object.

[0106] In some embodiments, a first feature value is determined using the feature value corresponding to at least one associated sample, and the first feature value is used to fill the missing feature value among the M associated features of the i-th object. The first feature value may be the mean of the feature values ​​corresponding to at least one associated sample, or it may be the mode, arithmetic mean, or other similar values ​​among the feature values ​​corresponding to at least one associated sample.

[0107] For example, the object is a company, the classification model is used to determine the company's risk level, at least one classification node includes industry and region, and associated samples refer to companies in the same industry located in the same region as the company. Figure 5 As shown, suppose company A is missing a certain feature, "company turnover." A common approach is to use the mean or mode of all companies' "company turnover" data to fill in the missing value. However, due to the significant differences in turnover between different industries and regions, such supplementation would severely deviate from the true value. Therefore, this application proposes a knowledge graph-based missing value supplementation method. Specifically, first, find the node where company A is located in the knowledge graph. Then, find the corresponding industry X through the "company-industry" relationship, and find the related companies B / C / E of industry X. Similarly, find the region Y where company A is located through the "company-region" relationship, and find the related companies C / D / E of region Y. The corresponding feature values ​​of the intersection companies C and E can be used to fill in the missing feature of company A. If some of the above relationships do not exist, such as if the region cannot be found, only the industry's company features are used for supplementation. The basic principle of the above operation is that the relevant features of a company have a strong correlation with its industry and region, thus being more representative and achieving better supplementation results.

[0108] Training process of classification models

[0109] This application also provides exemplary embodiments regarding the training process of the classification model. In some embodiments, such as... Figure 3 As shown, the method further includes at least one of the following steps 261 to 264. The following embodiments can be combined with other embodiments of this application to obtain new embodiments.

[0110] Step 261: Using the classification model, predict the first prediction result corresponding to the training sample of the i-th object. The first prediction result includes the probability that the i-th object belongs to C categories respectively. The C categories have an order relationship, and C is an integer greater than 1.

[0111] This application does not limit the structure of the classification model. For ease of understanding, this application provides the following: Figure 6 The diagram shown illustrates the framework of the classification model. The following will use... Figure 6The technical solutions of this application embodiment are illustrated using the classification model framework shown as an example. It should be understood that the technical solutions of this application embodiment can also be applied to classification models with other structures.

[0112] Before proceeding with the detailed plan, let's explain some symbols:

[0113] C1, ..., C 10 : indicates 10 category labels;

[0114] X i :[t1,t2,…,t p ], where; X i Let t represent the i-th object. p This represents the j-th associated feature of the i-th object; assuming M1 = [t1, t2, ..., tj]. k-1 ] represents the numerical characteristic, M2 = [t k ,t2,…,t p ] represents the label feature, i.e., X i =[M1,M2]. FFN i : represents the i-th feedforward neural network.

[0115] In some embodiments, the training samples of the i-th object include at least one numerical feature and at least one label feature, wherein the value of the numerical feature is any value within a continuous range, and the value of the label feature is any value within a discrete range.

[0116] In some embodiments, step 1 above can be implemented as at least one of steps a1 to a4 below.

[0117] Step a1: Using a classification model, based on numerical features, generate the first vector feature corresponding to the training sample of the i-th object. The first vector feature is a numerical feature mapped in the feature space.

[0118] In some embodiments, numerical features can be directly used as input data for the classification model, which then processes them. For example, the processing method of the classification model is shown in formula (1) below:

[0119] Nemb i =FFN1(M i (1)

[0120] Nemb i It is a K1-dimensional vector.

[0121] For example, if the training sample of the i-th object is [2.5, 3.5, 0, 1], where the first two are numerical features, then the preceding numerical features are combined into a vector [2.5, 3.5], and this vector is then input into the neural network FFN1 to obtain the first vector feature (the first vector feature).

[0122] For example, if the object is a company, the numerical features can be fundamental features such as financial indicators.

[0123] In some embodiments, since the label features are discrete and the labels 0 / 1 do not represent size relationships, directly inputting them into the feedforward neural network would cause the classification model to over-simulate numerical values, resulting in a decrease in the classification model's performance. Therefore, the label features need to be processed.

[0124] Step a2: Using a classification model, based on the encoded label features, generate the second vector feature corresponding to the training sample of the i-th object. The second vector feature is the label feature mapped in the feature space.

[0125] In some embodiments, the method further includes step A.

[0126] Step A: Preprocess the label features to obtain encoded label features, which are vectorized representations of the label features.

[0127] For example, the label features are one-hot vectorized to obtain the encoded label features.

[0128] For example, the label features are first processed by one-hot vectorization, and then the vector obtained after one-hot vectorization (encoding the label features) is input into the feedforward neural network. Specifically, assuming that the feature t takes three values, 0, 1, and 2, this feature is first replaced with a three-dimensional vector before being input into the feedforward neural network. If t = 0, it is replaced with a vector of [1,0,0]. Similarly, if t = 1, it is [0,1,0]; and if t = 2, it is [0,0,1]. If the label features only take two values, 0 and 1, M2 will be replaced with a (n-k+1)*2 dimensional matrix. Finally, this matrix is ​​input into the feedforward neural network to obtain the corresponding embedded features, as shown in the following formula (2):

[0129] Lemb i =FFN2(L i (2)

[0130] Lemb i It is a K2-dimensional vector.

[0131] For example, if the training samples for the i-th object are [2.5, 3.5, 0, 1], where the last two are label-specific, then the label features are converted into one-hot encoding, where 0 corresponds to [1, 0] and 1 corresponds to [0, 1], thus forming a matrix. Input the matrix into FFN2 to obtain the second vector feature (the second vector feature).

[0132] Step a3: Concatenate the first vector feature corresponding to the training sample of the i-th object and the second vector feature corresponding to the training sample of the i-th object to obtain the third vector feature corresponding to the training sample of the i-th object.

[0133] To enable the classification model to better capture all information about the i-th object, the first and second vector features are concatenated to obtain the third vector feature. For example, as shown in formula (3) below:

[0134] Xemb i =[Nemb i Lemb i (3)

[0135] Xemb i It is a (K1+K2) dimensional vector.

[0136] Step a4: Using the classification model, based on the third vector feature corresponding to the training sample of the i-th object, obtain the first prediction result corresponding to the training sample of the i-th object.

[0137] The classification model determines the probability that the i-th object belongs to each of the C categories based on the third vector feature, and obtains the first prediction result.

[0138] Step 262: Based on the first prediction result, determine the first loss. The first loss is used to characterize the accuracy of the probability that the i-th object in the first prediction result belongs to each of the C categories.

[0139] In some embodiments, the first loss is primarily used to correct the accuracy of the classification model for the classification task. This application does not limit the type of the first loss. Exemplarily, the first loss can be focal loss or class-balanced cross-entropy loss.

[0140] In one example, based on the probability that the i-th object belongs to each of the C categories, the first sub-loss corresponding to each of the C categories is calculated, where the first sub-loss corresponding to the j-th category is used to characterize the probability loss of the first training sample belonging to the j-th category, and j is an integer less than or equal to C; based on the first sub-loss corresponding to each of the C categories, the first loss is determined.

[0141] For example, the process of determining the first sub-loss is shown in the following formula (4):

[0142] Loss fl =-α(1-P c ) γ log(P c (4)

[0143] Among them, Loss fl For the first loss, P c Let α be the probability that the i-th object belongs to category c, and let γ be hyperparameters.

[0144] Specifically, the aforementioned third vector feature Xemb i The input is fed into the feedforward neural network, resulting in a C-dimensional vector (the dimension must match the number of classes). Then, a softmax operation is performed on this C-dimensional vector, Sf... i The k-th dimension of the vector represents the probability that the model predicts this sample belongs to the k-th class, i.e., P. c For example, as shown in the following formula:

[0145] V i =FFN3(Xemb i )

[0146] Sf i =Softmax(FFN4(V i ))

[0147] Among them, V i Let Xemb be the C-dimensional vector mentioned above. i The third vector feature is Softmax, which is the Softmax operation. The Softmax operation transforms the input vector into a probability distribution, where each output value is between 0 and 1, and the sum of all output values ​​is 1.

[0148] The Softmax operation is shown in the following formula:

[0149]

[0150] Where, x i It is the i-th element in the input vector (third vector feature), and e is the base of the natural logarithm. This is used to sum the exponents of all elements in the input vector.

[0151] For example, we now have the output vector V = [0.234, 0.567, 0.789, 0.345, 0.678, 0.123, 0.456, 0.890, 0.234, 0.876] of the feedforward network FFN3. Then, we perform a softmax operation on this vector to get the vector sf = [0.07254646, 0.101213, 0.12637165, 0.08106305, 0.11309489, 0.06492464, 0.09057943, 0.13980201, 0.07254646, 0.13785842]. Each term represents the probability of the data belonging to that class. The probability of the i-th object belonging to the tenth class is 0.13785842.

[0152] In some embodiments, the sum of the first sub-losses corresponding to the C categories is determined as the first loss.

[0153] In another example, the first loss is determined based on the probability that the i-th object belongs to each of the C categories, and the number of objects belonging to the same category as the i-th object.

[0154] For example, the first loss is shown in the following formula (5):

[0155]

[0156] Among them, Loss bcl Indicates the first loss. P represents the Softmax operation. y Let n represent the probability that the i-th object belongs to category y. y This represents the number of training samples belonging to category y, and β is a hyperparameter.

[0157] Step 263: Based on the vector features corresponding to the training samples of N objects, determine the second loss. The second loss is used to characterize the relative order relationship between the category to which the i-th object belongs and the categories to which the N-1 objects other than the i-th object belong.

[0158] In some embodiments, the vector features corresponding to the training samples of the i-th object are used to characterize the comprehensive features of the i-th object. In some embodiments, the second loss is used to correct the ranking relationship between C categories. The category of the second loss is not limited in this application. For example, the second loss can be a rank loss or a list-wise loss.

[0159] In one example, the vector distance between the third vector feature corresponding to the training sample of the i-th object and the third vector feature corresponding to the p-th training sample among the N-1 training samples excluding the i-th object is determined; the second loss is determined based on the vector distance and the first label parameter, which is used to characterize the relative order relationship between the category to which the i-th object belongs and the category to which the p-th object belongs; where p is an integer less than N.

[0160] For example, the process of determining the second loss is shown in the following formula (6):

[0161] Loss rl (y,x i ,x j )=y||V i -V j ||+(1-y)max(0,margin-||V i -V j ||) (6)

[0162] Among them, ||V i -V j || represents the vector distance, y is the first label parameter, margin is the boundary, and is a preset parameter. i V represents the third vector feature of the i-th object. j This represents the third vector feature corresponding to the p-th training sample out of N-1 training samples.

[0163] in,

[0164]

[0165] In another example, based on the third vector feature corresponding to the training sample of the i-th object and the third vector features corresponding to the N-1 training samples excluding the i-th object, the ranking probabilities corresponding to the N training samples are determined. Here, the ranking probability corresponding to the training sample of the i-th object is the probability that the i-th training sample is in the first position, and the ranking probability corresponding to the p-th training sample among the N-1 training samples is the probability that the p-th training sample is in the second position, excluding the p-1 training samples before the p-th training sample. Based on the ranking probabilities corresponding to the N training samples, the second loss is determined. Here, p is an integer less than N.

[0166] For example, the second loss is shown in the following formula (7):

[0167]

[0168] Among them, Loss lwl (x1,…,xn ) indicates the second loss, Sf j This represents the Softmax operation applied to the j-th object. This represents the ranking probability corresponding to the j-th training sample. It represents the product of the ranking probabilities of N training samples.

[0169] Step 264: Based on the first loss and the second loss, adjust the parameters of the classification model to obtain the trained classification model.

[0170] In some embodiments, a third loss is determined based on the first loss and the second loss, and the parameters of the classification model are adjusted based on the third loss to obtain the trained classification model.

[0171] For example, the third loss is obtained based on focal loss and rank loss. For example, the third loss is shown in the following formula (8):

[0172]

[0173] Among them, Loss(x) i ) represents the third loss, and λ∈(0,1) is a preset constant.

[0174] For example, the third loss is obtained based on the class-balanced loss function and the list-wise loss. For example, the third loss is shown in the following formula (9):

[0175]

[0176] Among them, Loss(x) i ) represents the third loss, and λ∈(0,1) is a preset constant.

[0177] In some embodiments, during training, the gradient descent method is used to continuously optimize the third loss, thereby obtaining the optimal parameters.

[0178] In some embodiments, the classification model includes a first network and a second network. The first network is used to predict a first prediction result corresponding to a training sample of the i-th object, and the second network is used to generate third feature vectors corresponding to N-1 training samples respectively. The third feature vectors corresponding to the N-1 training samples are used to determine a second loss. The first network and the second network are different. For example, the first network predicts the first prediction result corresponding to a training sample of the i-th object; the second network generates third feature vectors corresponding to N-1 training samples respectively, and the third feature vectors corresponding to the N-1 training samples are used to determine the second loss.

[0179] For example, the first network uses a feedforward neural network to process the vector features of the training samples, and the second network uses XGBoost to process the vector features of the training samples.

[0180] The classification model trained as described above can be used to perform classification tasks. Because the training process considers both ranking loss and classification loss, it alleviates the problem of extremely imbalanced data distribution, allowing the model to be adequately trained even on classes with fewer samples. Constructing ranking relationships between classes penalizes cases where class predictions differ excessively, making the model more inclined to make predictions closer to the true class.

[0181] Please refer to Figure 7 The diagram illustrates a flowchart of a method for using a classification model provided in one embodiment of this application. For example, the classification model is denoted as M, and then y = M(x) is defined as follows: y = M(x) represents inputting the features of sample x into the classification model to obtain the corresponding rating label, where y = 0, 1, ..., 9.

[0182] Taking a company as an example, and using a classification model to predict the company's financial risk rating, the overall process is described below:

[0183] 1. Collect entity data: For each enterprise entity, collect its current financial indicators and data on any anomalies in the ChinaBond valuation. This data is mainly obtained from the entity's financial reports and includes financial indicators such as total assets, debt-to-asset ratio, and the proportion of bond financing, as well as label data such as whether there is a substantive default and whether there are any anomalies in the ChinaBond valuation. This type of data will be labeled 0 / 1, representing whether there is a default / abnormality. At the same time, collect regional characteristics of the region where the entity is located, mainly including the region's current GDP, special debt balance, and government fund revenue and expenditure data. This data needs to be obtained from the government's official website.

[0184] 2. Utilize large models for media content feature mining: Use large models to capture relevant news from recent news, such as abnormal fluctuations in bond prices and abnormal valuations, and aggregate them into numerical features;

[0185] 3. Determine if the data is complete: Among the data collected above, some data may be unavailable. In this case, use the knowledge graph to find the location of the enterprise node in the graph, find the related industries and regions, obtain the corresponding data for that dimension of the region and industry, and fill in the mean or mode.

[0186] 4. Statistical Data Integration: Among the data collected above, some are statistical data, which need to be integrated with all similar data within the current period to obtain the final integrated data. For example, when statistically analyzing regional characteristics, it is necessary to obtain the number of all defaulting entities in the region in previous years; this type of characteristic needs to be statistically analyzed.

[0187] 5. Feature Engineering: This part mainly distinguishes between numerical data and categorical data, converting the categorical data into one-hot encoding, and finally obtaining a list of feature values ​​x. Assume that the above processing has yielded subject-related data [d1, d2, ..., d...]. n Each data point represents a value for a feature. Some of these are label-type features, such as the feature "whether there is a substantial breach of contract," which only takes values ​​of 0 and 1. Therefore, this feature is converted to 0 corresponding to [1,0] and 1 corresponding to [0,1]. All features are converted accordingly, and finally concatenated together to obtain: in These data are the results after one-hot encoding;

[0188] 6. Model Prediction: Input the feature value list obtained above into the trained model M to obtain y = M(x). The y = 1, 2, ..., 9 is the final financial risk rating result of the entity.

[0189] The classification model provided in this application can be applied to various scenarios. The following describes, by way of example, the scenarios to which this application can be applied.

[0190] 1. The object is an enterprise. The classification model is used to determine the risk level of the enterprise. The associated media content of the i-th object includes: media content related to the profit and loss situation of the i-th object and the enterprise.

[0191] For example, such as Figure 8 As shown, the method includes at least one of the following steps 810 to 850.

[0192] Step 810: Obtain the initial feature sets corresponding to N enterprises respectively. The initial feature set of the i-th enterprise among the N enterprises includes M related features of the i-th enterprise, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0193] Step 820: Obtain the associated media content of the i-th enterprise. The associated media content of the i-th enterprise is the media content in the media content library that is related to the i-th enterprise and the classification target of the classification model.

[0194] In some embodiments, the publicly available media content of the i-th enterprise is obtained from the media content library, wherein the publicly available media content of the i-th enterprise is media content related to the i-th enterprise; from the publicly available media content of the i-th enterprise, publicly available media content related to the classification target of the classification model is selected to obtain the associated media content of the i-th enterprise.

[0195] Step 830: Based on the associated media content of the i-th enterprise, determine the media characteristics of the i-th enterprise. The media characteristics of the i-th enterprise are used to characterize the quantity of associated media content of the i-th enterprise.

[0196] In some embodiments, keyword entries are determined based on the classification objective of the classification model; based on the keyword entries, public media content containing the keyword entries is filtered from the public media content of the i-th enterprise to obtain the associated media content of the i-th enterprise.

[0197] In some embodiments, the media content theme is determined based on the classification objective of the classification model; based on the media content theme, publicly available media content related to the media content theme is selected from the publicly available media content of the i-th enterprise to obtain the associated media content of the i-th enterprise.

[0198] In some embodiments, the frequency of occurrence of the associated media content of the i-th enterprise is determined, where the frequency of occurrence is the frequency of occurrence of the associated media content of the i-th enterprise in the publicly available media content of the i-th enterprise; based on the frequency of occurrence, the media characteristics of the i-th enterprise are determined.

[0199] In some embodiments, the frequency of occurrence is normalized to obtain the media characteristics of the i-th enterprise.

[0200] Step 840: Add the media features of the i-th enterprise to the initial feature set of the i-th enterprise to obtain the training sample of the i-th enterprise.

[0201] In some embodiments, if one or more associated features are missing in the M associated features of the i-th enterprise, at least one classification node corresponding to the i-th enterprise is determined, wherein different classification nodes are used to characterize different classification criteria for the i-th enterprise; at least one associated sample is determined based on the sample set corresponding to the at least one classification node, and the associated sample belongs to the intersection of the sample sets corresponding to the at least one classification node; the feature value corresponding to the at least one associated sample is used to fill the missing feature value in the M associated features of the i-th enterprise.

[0202] In some embodiments, at least one influencing factor is determined according to the classification objective of the classification model. The influencing factor refers to the factor that affects the feature values ​​of the M related features of the i-th enterprise; at least one classification node is determined based on the at least one influencing factor.

[0203] In some embodiments, at least one classification node includes industry and region, and associated samples refer to enterprises in the same industry located in the same region as the enterprise.

[0204] Step 850: Based on the training samples corresponding to N enterprises, generate a training sample set for training the classification model.

[0205] In some embodiments, a classification model is used to predict a first prediction result corresponding to the training sample of the i-th enterprise. The first prediction result includes the probability that the i-th enterprise belongs to C categories, where the C categories have an order relationship and C is an integer greater than 1. Based on the first prediction result, a first loss is determined, which is used to characterize the accuracy of the probability that the i-th enterprise belongs to C categories in the first prediction result. Based on the training samples of N enterprises, a second loss is determined, which is used to characterize the relative order relationship between the category to which the i-th enterprise belongs and the categories to which the N-1 enterprises (excluding the i-th enterprise) belong. Based on the first loss and the second loss, the parameters of the classification model are adjusted to obtain the trained classification model.

[0206] In some embodiments, the M associated features of the i-th enterprise include at least one numerical feature and at least one tag feature, wherein the value of the numerical feature is any value within a continuous range, and the value of the tag feature is any value within a discrete range.

[0207] In some embodiments, the label features are preprocessed to obtain encoded label features, which are vectorized representations of the label features.

[0208] In some embodiments, a first vector feature corresponding to the training sample of the i-th enterprise is generated based on numerical features using a classification model. The first vector feature is a numerical feature mapped in the feature space. A second vector feature corresponding to the training sample of the i-th enterprise is generated based on encoded label features using a classification model. The second vector feature is a label feature mapped in the feature space. The first vector feature and the second vector feature corresponding to the training sample of the i-th enterprise are concatenated to obtain a third vector feature corresponding to the training sample of the i-th enterprise. Based on the third vector feature corresponding to the training sample of the i-th enterprise, a first prediction result corresponding to the training sample of the i-th enterprise is obtained using a classification model.

[0209] In some embodiments, based on the probability that the i-th enterprise belongs to each of the C categories, the first sub-loss corresponding to each of the C categories is calculated, where the first sub-loss corresponding to the j-th category is used to characterize the probability loss of the first training sample belonging to the j-th category, and j is an integer less than or equal to C; based on the first sub-loss corresponding to each of the C categories, the first loss is determined.

[0210] In some embodiments, the first loss is determined based on the probability that the i-th enterprise belongs to each of the C categories and the number of enterprises belonging to the same category as the i-th enterprise.

[0211] In some embodiments, the vector distance between the third vector feature corresponding to the training sample of the i-th enterprise and the third vector feature corresponding to the p-th training sample among the N-1 training samples other than the training sample of the i-th enterprise is determined; a second loss is determined based on the vector distance and a first label parameter, the first label parameter being used to characterize the relative order relationship between the category to which the i-th enterprise belongs and the category to which the p-th enterprise belongs.

[0212] In some embodiments, based on the third vector feature corresponding to the training sample of the i-th enterprise and the third vector features corresponding to the N-1 training samples excluding the i-th enterprise, the ranking probabilities corresponding to the N training samples are determined respectively. Here, the ranking probability corresponding to the training sample of the i-th enterprise is the probability that the i-th training sample is in the first position, and the ranking probability corresponding to the p-th training sample among the N-1 training samples is the probability that the p-th training sample is in the second position, excluding the p-1 training samples before the p-th training sample. Based on the ranking probabilities corresponding to the N training samples, the second loss is determined. Here, p is an integer less than N.

[0213] In some embodiments, the classification model includes a first network and a second network. The first network is used to predict a first prediction result corresponding to the training sample of the i-th enterprise, and the second network is used to generate a third feature vector corresponding to each of the N-1 training samples. The third feature vectors corresponding to each of the N-1 training samples are used to determine a second loss. The first network and the second network are different.

[0214] 2. The object is the information feed recommended by the application interface. The classification model is used to determine the recommendation order of the information feed. The associated media content of the i-th object includes: media content related to the number of views of the i-th object.

[0215] For example, such as Figure 9 As shown, the method includes at least one of the following steps 910 to 950.

[0216] Step 910: Obtain the initial feature sets corresponding to the N information streams respectively. The initial feature set of the i-th information stream among the N information streams includes the M associated features of the i-th information stream, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0217] Step 920: Obtain the associated media content of the i-th information stream. The associated media content of the i-th information stream is the media content in the media content library that is related to the classification target of the i-th information stream and the classification model.

[0218] In some embodiments, the public media content of the i-th information stream is obtained from the media content library, wherein the public media content of the i-th information stream is media content related to the i-th information stream; from the public media content of the i-th information stream, public media content related to the classification target of the classification model is filtered out to obtain the associated media content of the i-th information stream.

[0219] Step 930: Based on the associated media content of the i-th information stream, determine the media features of the i-th information stream. The media features of the i-th information stream are used to characterize the number of associated media contents of the i-th information stream.

[0220] In some embodiments, keyword entries are determined based on the classification objective of the classification model; based on the keyword entries, public media content containing the keyword entries is filtered from the public media content of the i-th information stream to obtain the associated media content of the i-th information stream.

[0221] In some embodiments, the media content theme is determined based on the classification objective of the classification model; based on the media content theme, publicly available media content related to the media content theme is filtered from the publicly available media content of the i-th information stream to obtain the associated media content of the i-th information stream.

[0222] In some embodiments, the occurrence frequency of the associated media content of the i-th information stream is determined, where the occurrence frequency is the frequency of the associated media content of the i-th information stream appearing in the public media content of the i-th information stream; based on the occurrence frequency, the media characteristics of the i-th information stream are determined.

[0223] In some embodiments, the frequency of occurrence is normalized to obtain the media features of the i-th information stream.

[0224] Step 940: Add the media features of the i-th information stream to the initial feature set of the i-th information stream to obtain the training sample of the i-th information stream.

[0225] In some embodiments, if one or more associated features are missing in the M associated features of the i-th information stream, at least one classification node corresponding to the i-th information stream is determined, wherein different classification nodes are used to characterize different classification criteria for the i-th information stream; at least one associated sample is determined based on the sample set corresponding to the at least one classification node, and the associated sample belongs to the intersection of the sample sets corresponding to the at least one classification node; the feature value corresponding to the at least one associated sample is used to fill the missing feature value in the M associated features of the i-th information stream.

[0226] In some embodiments, at least one influencing factor is determined according to the classification objective of the classification model. An influencing factor is a factor that affects the feature values ​​of the M associated features of the i-th information stream. At least one classification node is determined based on the at least one influencing factor.

[0227] In some embodiments, at least one classification node includes industry and region, and associated samples refer to information flows from the same industry as the information flow in the same region.

[0228] Step 950: Based on the training samples corresponding to the N information streams, generate a training sample set for training the classification model.

[0229] In some embodiments, a classification model is used to predict a first prediction result corresponding to the training sample of the i-th information stream. The first prediction result includes the probability that the i-th information stream belongs to C categories, where the C categories have an order relationship and C is an integer greater than 1. Based on the first prediction result, a first loss is determined, which is used to characterize the accuracy of the probability that the i-th information stream belongs to C categories in the first prediction result. Based on the training samples of N information streams, a second loss is determined, which is used to characterize the relative order relationship between the category to which the i-th information stream belongs and the categories to which the N-1 information streams (excluding the i-th information stream) belong. Based on the first loss and the second loss, the parameters of the classification model are adjusted to obtain the trained classification model.

[0230] In some embodiments, the M associated features of the i-th information stream include at least one numerical feature and at least one tag feature, wherein the value of the numerical feature is any value within a continuous range, and the value of the tag feature is any value within a discrete range.

[0231] In some embodiments, the label features are preprocessed to obtain encoded label features, which are vectorized representations of the label features.

[0232] In some embodiments, a first vector feature corresponding to the training sample of the i-th information stream is generated based on numerical features using a classification model. The first vector feature is a numerical feature mapped in the feature space. A second vector feature corresponding to the training sample of the i-th information stream is generated based on encoded label features using a classification model. The second vector feature is a label feature mapped in the feature space. The first vector feature and the second vector feature corresponding to the training sample of the i-th information stream are concatenated to obtain a third vector feature corresponding to the training sample of the i-th information stream. Based on the third vector feature corresponding to the training sample of the i-th information stream, a first prediction result corresponding to the training sample of the i-th information stream is obtained using a classification model.

[0233] In some embodiments, based on the probabilities of the i-th information stream belonging to C categories, the first sub-loss corresponding to each of the C categories is calculated, where the first sub-loss corresponding to the j-th category is used to characterize the probability loss of the first training sample belonging to the j-th category, and j is an integer less than or equal to C; based on the first sub-loss corresponding to each of the C categories, the first loss is determined.

[0234] In some embodiments, a first loss is determined based on the probability that the i-th information stream belongs to each of the C categories, and the number of information streams belonging to the same category as the i-th information stream.

[0235] In some embodiments, the vector distance between the third vector feature corresponding to the training sample of the i-th information stream and the third vector feature corresponding to the p-th training sample among the N-1 training samples other than the training sample of the i-th information stream is determined; a second loss is determined based on the vector distance and a first label parameter, the first label parameter being used to characterize the relative order relationship between the category to which the i-th information stream belongs and the category to which the p-th information stream belongs.

[0236] In some embodiments, based on the third vector feature corresponding to the training sample of the i-th information stream and the third vector features corresponding to the N-1 training samples other than the training sample of the i-th information stream, the ranking probabilities corresponding to the N training samples are determined respectively. Here, the ranking probability corresponding to the training sample of the i-th information stream refers to the probability that the i-th training sample is in the first position, and the ranking probability corresponding to the p-th training sample among the N-1 training samples refers to the probability that the p-th training sample is in the second position, excluding the p-1 training samples before the p-th training sample. Based on the ranking probabilities corresponding to the N training samples, the second loss is determined. Here, p is an integer less than N.

[0237] In some embodiments, the classification model includes a first network and a second network. The first network is used to predict a first prediction result corresponding to the training sample of the i-th information stream, and the second network is used to generate a third feature vector corresponding to each of the N-1 training samples. The third feature vectors corresponding to each of the N-1 training samples are used to determine a second loss. The first network and the second network are different.

[0238] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0239] Please refer to Figure 10This diagram illustrates a block diagram of a training sample construction apparatus for a classification model according to an embodiment of this application. The apparatus has the functionality to implement the method example described above; this functionality can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the computer device described above, or it can be located within a computer device. Figure 10 As shown, the device 1000 may include: an initial feature acquisition module 1010, a media content acquisition module 1020, a media feature determination module 1030, a media feature addition module 1040, and a training sample set generation module 1050.

[0240] The initial feature acquisition module 1010 is used to acquire the initial feature sets corresponding to N objects respectively. The initial feature set of the i-th object among the N objects includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N.

[0241] The media content acquisition module 1020 is used to acquire the associated media content of the i-th object. The associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model.

[0242] The media feature determination module 1030 is used to determine the media features of the i-th object based on the associated media content of the i-th object. The media features of the i-th object are used to characterize the number of associated media contents of the i-th object.

[0243] The media feature addition module 1040 is used to add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object.

[0244] The training sample set generation module 1050 is used to generate a training sample set for training the classification model based on the training samples corresponding to the N objects respectively.

[0245] In some embodiments, the media content acquisition module 1020 is configured to acquire the public media content of the i-th object from the media content library, wherein the public media content of the i-th object is media content related to the i-th object; and to filter out the public media content of the i-th object that is related to the classification target of the classification model from the public media content of the i-th object to obtain the associated media content of the i-th object.

[0246] In some embodiments, the media content acquisition module 1020 is configured to determine keyword entries based on the classification objective of the classification model; based on the keyword entries, filter out public media content containing the keyword entries from the public media content of the i-th object to obtain the associated media content of the i-th object; or, based on the classification objective of the classification model, determine the media content theme; based on the media content theme, filter out public media content related to the media content theme from the public media content of the i-th object to obtain the associated media content of the i-th object.

[0247] In some embodiments, the media feature determination module 1030 is used to determine the frequency of occurrence of the associated media content of the i-th object, wherein the frequency of occurrence is the frequency of occurrence of the associated media content of the i-th object in the public media content of the i-th object; and to determine the media features of the i-th object based on the frequency of occurrence.

[0248] In some embodiments, the media feature determination module 1030 is used to normalize the occurrence frequency to obtain the media features of the i-th object.

[0249] In some embodiments, the object is an enterprise, the classification model is used to determine the risk level of the enterprise, and the associated media content of the i-th object includes: media content related to the profit and loss situation of the i-th object and the enterprise.

[0250] In some embodiments, the device 1000 further includes a feature value supplementation module (not shown in the figure).

[0251] The feature value supplementation module is used to determine at least one classification node corresponding to the i-th object when one or more of the M associated features of the i-th object are missing feature values, wherein different classification nodes are used to characterize different classification criteria for the i-th object; determine at least one associated sample according to the sample set corresponding to the at least one classification node, wherein the associated sample belongs to the intersection of the sample sets corresponding to the at least one classification node; and fill the missing feature values ​​in the M associated features of the i-th object with the feature values ​​corresponding to the at least one associated sample.

[0252] In some embodiments, the feature value supplementation module is used to determine at least one influencing factor according to the classification objective of the classification model, wherein the influencing factor refers to the factor that affects the feature values ​​of the M associated features of the i-th object; and to determine the at least one classification node based on the at least one influencing factor.

[0253] In some embodiments, the object is an enterprise, the classification model is used to determine the risk level of the enterprise, the at least one classification node includes industry and region, and the associated sample refers to an enterprise in the same industry located in the same region as the enterprise.

[0254] In some embodiments, the device 1000 further includes a training module (not shown in the figure).

[0255] The training module is used to predict a first prediction result corresponding to the training sample of the i-th object through the classification model. The first prediction result includes the probability that the i-th object belongs to C categories, where the C categories have an order relationship and C is an integer greater than 1. Based on the first prediction result, a first loss is determined, which is used to characterize the accuracy of the probability that the i-th object belongs to C categories in the first prediction result. Based on the training samples of N objects, a second loss is determined, which is used to characterize the relative order relationship between the category to which the i-th object belongs and the categories to which the N-1 objects (excluding the i-th object) belong. Based on the first loss and the second loss, the parameters of the classification model are adjusted to obtain the trained classification model.

[0256] In some embodiments, the M associated features of the i-th object include at least one numerical feature and at least one label feature, wherein the value of the numerical feature is any value within a continuous range, and the value of the label feature is any value within a discrete range; the training module is further configured to preprocess the label feature to obtain an encoded label feature, wherein the encoded label feature is a vectorized representation of the label feature.

[0257] In some embodiments, the training module is configured to: generate a first vector feature corresponding to the training sample of the i-th object based on the numerical features using the classification model, wherein the first vector feature is the numerical feature mapped in the feature space; generate a second vector feature corresponding to the training sample of the i-th object based on the encoded label features using the classification model, wherein the second vector feature is the label feature mapped in the feature space; concatenate the first vector feature corresponding to the training sample of the i-th object and the second vector feature corresponding to the training sample of the i-th object to obtain a third vector feature corresponding to the training sample of the i-th object; and obtain the first prediction result corresponding to the training sample of the i-th object based on the third vector feature corresponding to the training sample of the i-th object using the classification model.

[0258] In some embodiments, the training module is configured to calculate a first sub-loss corresponding to each of the C categories based on the probability that the i-th object belongs to each of the C categories, wherein the first sub-loss corresponding to the j-th category is used to characterize the probability loss of the first training sample belonging to the j-th category, and j is an integer less than or equal to C; determine the first loss based on the first sub-loss corresponding to each of the C categories; or, determine the first loss based on the probability that the i-th object belongs to each of the C categories and the number of objects belonging to the same category as the i-th object.

[0259] In some embodiments, the training module is configured to determine the vector distance between the third vector feature corresponding to the training sample of the i-th object and the third vector feature corresponding to the p-th training sample among the N-1 training samples excluding the i-th object; determine the second loss based on the vector distance and a first label parameter, wherein the first label parameter is used to characterize the relative order relationship between the category to which the i-th object belongs and the category to which the p-th object belongs; or, determine the ranking probabilities corresponding to the N training samples based on the third vector feature corresponding to the training sample of the i-th object and the third vector features corresponding to the N-1 training samples excluding the i-th object, wherein the ranking probability corresponding to the training sample of the i-th object refers to the probability that the i-th training sample is in the first position, and the ranking probability corresponding to the p-th training sample among the N-1 training samples refers to the probability that the p-th training sample is in the second position, excluding the p-1 training samples before the p-th training sample; determine the second loss based on the ranking probabilities corresponding to the N training samples; wherein p is an integer less than N.

[0260] In some embodiments, the classification model includes a first network and a second network. The first network is used to predict the first prediction result corresponding to the training sample of the i-th object, and the second network is used to generate a third feature vector corresponding to each of the N-1 training samples. The third feature vectors corresponding to each of the N-1 training samples are used to determine the second loss. The first network and the second network are different.

[0261] The technical solution provided in this application adds media features to the initial feature set of the object, taking into account the impact of media content on the classification results of the object. It can analyze the possible recent changes of the object based on the media features, making the classification results more accurate.

[0262] Please refer to Figure 11 This illustration shows a schematic diagram of a computer device provided in one embodiment of this application. The computer device may be... Figure 1The terminal device 100 in the implementation environment shown can also be Figure 1 The server 200 in the implementation environment shown is used to implement the training sample construction method for the classification model provided in the above embodiments. Specifically:

[0263] Typically, terminal device 1100 includes a processor 1101 and a memory 1102.

[0264] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0265] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 are used to store a computer program configured to be executed by one or more processors to implement the training sample construction method of the classification model described above.

[0266] In some embodiments, the terminal device 1100 may also optionally include a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1104, a display screen 1105, an audio circuit 1107, and a power supply 1108.

[0267] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the terminal device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0268] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program, when executed by a processor of a computer device, implements the training sample construction method of the above-described classification model.

[0269] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0270] In an exemplary embodiment, a computer program product is also provided, the computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training sample construction method for the classification model described above.

[0271] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform users that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up. Otherwise (i.e., without receiving confirmation from the user), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected by this application (including initial feature sets, associated media features, etc.) is processed strictly in accordance with the requirements of relevant national laws and regulations. The acquisition of informed consent or separate consent from the personal information subject is only conducted with the user's consent and authorization. Subsequent data use and processing are carried out within the scope of laws and regulations and the authorization of the personal information subject. Furthermore, the collection, use, and processing of relevant user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0272] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0273] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for constructing training samples for a classification model, characterized in that, The method includes: Obtain the initial feature set corresponding to N objects respectively. The initial feature set of the i-th object among the N objects includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N. Obtain the associated media content of the i-th object, wherein the associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model; Based on the associated media content of the i-th object, the media characteristics of the i-th object are determined, and the media characteristics of the i-th object are used to characterize the quantity of associated media content of the i-th object; Add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object; Based on the training samples corresponding to the N objects, a training sample set for training the classification model is generated.

2. The method according to claim 1, characterized in that, The step of obtaining the associated media content of the i-th object includes: From the media content library, obtain the public media content of the i-th object, wherein the public media content of the i-th object is media content related to the i-th object; From the publicly available media content of the i-th object, publicly available media content related to the classification target of the classification model is selected to obtain the associated media content of the i-th object.

3. The method according to claim 2, characterized in that, The step of filtering out publicly available media content related to the classification target of the classification model from the publicly available media content of the i-th object to obtain the associated media content of the i-th object includes: Based on the classification objective of the classification model, keyword entries are determined; based on the keyword entries, public media content containing the keyword entries is filtered from the public media content of the i-th object to obtain the associated media content of the i-th object; or, Based on the classification objective of the classification model, the media content theme is determined; based on the media content theme, publicly available media content related to the media content theme is selected from the publicly available media content of the i-th object to obtain the associated media content of the i-th object.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the media characteristics of the i-th object based on the associated media content of the i-th object includes: Determine the frequency of occurrence of the associated media content of the i-th object, wherein the frequency of occurrence is the frequency of occurrence of the associated media content of the i-th object in the public media content of the i-th object; Based on the frequency of occurrence, the media characteristics of the i-th object are determined.

5. The method according to claim 4, characterized in that, Determining the media characteristics of the i-th object based on the frequency of occurrence includes: The frequency of occurrence is normalized to obtain the media characteristics of the i-th object.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: If one or more of the associated features of the i-th object are missing, at least one classification node corresponding to the i-th object is determined, wherein different classification nodes are used to characterize different classification criteria for the i-th object; Based on the sample sets corresponding to the at least one classification node, at least one associated sample is determined, wherein the associated sample belongs to the intersection of the sample sets corresponding to the at least one classification node; The feature values ​​corresponding to at least one associated sample are used to fill in the missing feature values ​​among the M associated features of the i-th object.

7. The method according to claim 6, characterized in that, Determining at least one classification node corresponding to the i-th object includes: Based on the classification objective of the classification model, at least one influencing factor is determined, wherein the influencing factor refers to the factor that affects the feature values ​​of the M associated features of the i-th object; Based on the at least one influencing factor, determine the at least one classification node.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The classification model predicts the first prediction result corresponding to the training sample of the i-th object. The first prediction result includes the probability that the i-th object belongs to C categories, where the C categories have an order relationship and C is an integer greater than 1. Based on the first prediction result, a first loss is determined to characterize the accuracy of the probability that the i-th object in the first prediction result belongs to C categories respectively; Based on the vector features corresponding to the training samples of the N objects, a second loss is determined to characterize the relative order relationship between the category to which the i-th object belongs and the categories to which the N-1 objects other than the i-th object belong. The vector features corresponding to the training samples of the i-th object are used to characterize the comprehensive features of the i-th object. Based on the first loss and the second loss, the parameters of the classification model are adjusted to obtain the trained classification model.

9. The method according to claim 8, characterized in that, The training sample of the i-th object includes at least one numerical feature and at least one label feature, wherein the value of the numerical feature is any value within a continuous range, and the value of the label feature is any value within a discrete range. The method further includes: The label features are preprocessed to obtain encoded label features, which are vectorized representations of the label features.

10. The method according to claim 9, characterized in that, The step of predicting the first prediction result corresponding to the training sample of the i-th object through the classification model includes: Based on the numerical features, the classification model generates a first vector feature corresponding to the training sample of the i-th object, where the first vector feature is the numerical feature mapped in the feature space. Based on the encoded label features, the classification model generates a second vector feature corresponding to the training sample of the i-th object, where the second vector feature is the label feature mapped in the feature space. By concatenating the first vector feature corresponding to the training sample of the i-th object and the second vector feature corresponding to the training sample of the i-th object, a third vector feature corresponding to the training sample of the i-th object is obtained; Using the classification model, based on the third vector feature corresponding to the training sample of the i-th object, the first prediction result corresponding to the training sample of the i-th object is obtained.

11. The method according to claim 10, characterized in that, The first loss, based on the first prediction result, determines the accuracy of the method for characterizing the probability that the i-th object in the first prediction result belongs to each of the C categories, including: Based on the probabilities of the i-th object belonging to the C categories, calculate the first sub-loss corresponding to each of the C categories, where the first sub-loss corresponding to the j-th category is used to characterize the probability loss of the first training sample belonging to the j-th category, and j is an integer less than or equal to C; based on the first sub-loss corresponding to each of the C categories, determine the first loss; or, The first loss is determined based on the probability that the i-th object belongs to each of the C categories, and the number of objects belonging to the same category as the i-th object.

12. The method according to claim 10 or 11, characterized in that, The second loss, determined based on the vector features corresponding to the training samples of the N objects, to characterize the relative order relationship between the category to which the i-th object belongs and the categories to which the N-1 objects (excluding the i-th object) belong, includes: Determine the vector distance between the third vector feature corresponding to the training sample of the i-th object and the third vector feature corresponding to the p-th training sample among the N-1 training samples excluding the i-th object; determine the second loss based on the vector distance and the first label parameter, where the first label parameter is used to characterize the relative order relationship between the category to which the i-th object belongs and the category to which the p-th object belongs; or, Based on the third vector features corresponding to the training sample of the i-th object and the third vector features corresponding to the N-1 training samples excluding the i-th object, the ranking probabilities corresponding to the N training samples are determined. The ranking probability corresponding to the training sample of the i-th object is the probability that the i-th training sample is in the first position, and the ranking probability corresponding to the p-th training sample among the N-1 training samples is the probability that the p-th training sample is in the second position, excluding the p-1 training samples preceding it. Based on the ranking probabilities corresponding to the N training samples, the second loss is determined. Where p is an integer less than N.

13. The method according to any one of claims 10 to 12, characterized in that, The classification model includes a first network and a second network; The step of predicting the first prediction result corresponding to the training sample of the i-th object through the classification model includes: The first prediction result corresponding to the training sample of the i-th object is obtained through the first network; The method further includes: The second network generates third feature vectors corresponding to the N-1 training samples respectively, and the third feature vectors corresponding to the N-1 training samples are used to determine the second loss; The first network and the second network are different.

14. A training sample construction device for a classification model, characterized in that, The device includes: The initial feature acquisition module is used to acquire the initial feature sets corresponding to N objects respectively. The initial feature set of the i-th object among the N objects includes M associated features of the i-th object, where M and N are integers greater than 1, and i is a positive integer less than or equal to N. The media content acquisition module is used to acquire the associated media content of the i-th object, wherein the associated media content of the i-th object is the media content in the media content library that is related to the i-th object and the classification target of the classification model; The media feature determination module is used to determine the media features of the i-th object based on the associated media content of the i-th object, wherein the media features of the i-th object are used to characterize the quantity of associated media content of the i-th object; The media feature addition module is used to add the media features of the i-th object to the initial feature set of the i-th object to obtain the training sample of the i-th object; The training sample set generation module is used to generate a training sample set for training the classification model based on the training samples corresponding to the N objects respectively.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the training sample construction method for the classification model as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the training sample construction method for the classification model as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes a computer program that is loaded and executed by a processor to implement the training sample construction method for the classification model as described in any one of claims 1 to 13.

Images