Internet of things potential user identification method and device, computer equipment and storage medium
By constructing a multi-model classification system based on the LightGBM model, the problems of low efficiency and low accuracy in the identification of potential customers in the Internet of Things were solved, and more accurate identification of potential customers was achieved.
Patent Information
- Application Number
- CN202111511879.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-07
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2041-12-07
AI Technical Summary
In existing technologies, the identification of potential customers in the Internet of Things relies on manual labor and experience, resulting in low efficiency and accuracy in potential customer discovery.
By acquiring the first and second datasets, the LightGBM model is used to transform and label the user data as feature vectors, a classification model set is constructed, and the average value of the multi-model classification is calculated to determine the label value for the user data to be predicted.
It improves the accuracy and efficiency of identifying potential customers for the Internet of Things (IoT) and reduces reliance on human experience.
Smart Images

Figure CN114155042B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of new generation information technology, and in particular to a method and device for identifying potential users of the Internet of Things, a computer device and a storage medium. BACKGROUND
[0002] As a key direction of independent innovation breakthrough of new generation information technology, the Internet of Things contains huge innovation space. In the fields of chips, sensors, short-range transmission, massive data processing, and comprehensive integration and application, innovation activities are increasingly active, and innovation elements are continuously accumulating. The application of the Internet of Things in various industries is deepening, which will give birth to a large number of new technologies, new products, new applications, and new modes.
[0003] At present, operators mainly collect target enterprise information on the network through customer managers, or understand enterprise needs through door-to-door visits, and then match the needs with operator product services, and then determine whether the enterprise is a potential customer and take corresponding marketing activities. However, this method lacks direction, is extremely dependent on manual and experience, and does not fully utilize big data (for example, enterprise business data, operator inventory customer data, etc.) and related technologies (for example, machine learning, etc.), so the potential customer mining efficiency is low and is easily affected by subjective factors to cause low accuracy. SUMMARY
[0004] Embodiments of the present application provide a method and device for identifying potential users of the Internet of Things, a computer device and a storage medium, aiming to solve the problem that the identification of potential customers of the Internet of Things in the prior art is identified by manual and experience, resulting in low potential customer mining efficiency and being easily affected by subjective factors to cause low accuracy.
[0005] In a first aspect, embodiments of the present application provide a method for identifying potential users of the Internet of Things, comprising:
[0006] obtaining a first data set and a second data set; wherein the first type user data in the first data set and the second type user data in the second data set both include a user unique identification code;
[0007] obtaining target first type user data in the first data set having the user unique identification code of the second type user data to form a labeled sample set, and obtaining an unlabeled sample set from the complement of the first data set relative to the labeled sample;
[0008] converting each piece of first type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and converting each piece of first type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy;
[0009] According to the preset labeling strategy and the labeling sample set, each first type user data in the unlabeled sample set is labeled for several times to obtain several groups of labeling sets corresponding to the unlabeled sample set;
[0010] Each LightGBM model in the several groups of labeling sets is used to train a model to obtain a LightGBM model corresponding to each group of labeling sets, and the LightGBM models corresponding to the several groups of labeling sets form a classification model set;
[0011] The user data to be predicted is obtained, and the user feature vector to be predicted corresponding to the user data to be predicted is obtained according to the field value conversion strategy, the user feature vector to be predicted is input into each LightGBM model in the classification model set for operation to obtain a classification result corresponding to each LightGBM model to form a classification result set;
[0012] The average value of the classification results in the classification result set is obtained as a classification result average value; and
[0013] If it is determined that the classification result average value is greater than a preset classification threshold, the labeling value of the user data to be predicted is set as a first labeling value.
[0014] In a second aspect, an embodiment of the present application provides an Internet of Things potential user identification device, which comprises:
[0015] A data set initial acquisition unit is configured to acquire a first data set and a second data set, wherein the first type user data in the first data set and the second type user data in the second data set both comprise a user unique identification code.
[0016] A data set classification unit is configured to acquire target first type user data in the first data set having the user unique identification code of the second type user data to form a labeling sample set, and acquire an unlabeled sample set from the complement of the labeling sample set with respect to the first data set.
[0017] A user feature vector acquisition unit is configured to convert each first type user data in the labeling sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and convert each first type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy.
[0018] A data labeling unit is configured to label each first type user data in the unlabeled sample set according to a preset labeling strategy and the labeling sample set for several times to obtain several groups of labeling sets corresponding to the unlabeled sample set.
[0019] The model set training unit is configured to obtain a plurality of sets of labeled sets corresponding to the unlabeled sample set, respectively train LightGBM models to be trained based on the plurality of sets of labeled sets, and obtain LightGBM models corresponding to the plurality of sets of labeled sets, respectively, wherein the classification model set is composed of the LightGBM models corresponding to the plurality of sets of labeled sets, respectively.
[0020] The classification result set obtaining unit is configured to obtain to-be-predicted user data, obtain a to-be-predicted user feature vector corresponding to the to-be-predicted user data based on the field value conversion strategy, input the to-be-predicted user feature vector into each LightGBM model in the classification model set for operation, and obtain classification results corresponding to each LightGBM model, respectively, to form a classification result set.
[0021] The classification result average value obtaining unit is configured to obtain an average value of the classification results in the classification result set as a classification result average value.
[0022] The first labeling unit is configured to set a labeled value of the to-be-predicted user data as a first labeled value if it is determined that the classification result average value is greater than a preset classification threshold.
[0023] In a third aspect, an embodiment of the present application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the Internet of Things potential user identification method in the first aspect when executing the computer program.
[0024] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program causes a processor to execute the Internet of Things potential user identification method in the first aspect when the processor executes the computer program.
[0025] The embodiment of the present application provides a kind of Internet of Things potential user identification method, device, computer equipment and storage medium, first based on first data set and second data set sample set and non-annotated sample set are labeled, then according to the labeling strategy and the labeled sample set, each type of user data in the unlabeled sample set is labeled several times to obtain several groups of labeled sets, then based on several groups of labeled sets, model training is carried out to the LightGBM model to be trained to obtain a classification model set, finally, based on the classification model set, the classification result of the user data to be predicted is obtained and the average value is obtained, and based on the comparison result of classification result average value and classification threshold, the labeled value of the user data to be predicted is determined. Realize that the labeled sample set and the non-annotated sample set obtained based on the first data set and the second data set are used to train multiple LightGBM models to form a classification model set, and based on the classification model set, the user data to be predicted is classified simultaneously by multiple models to obtain the average value, and more accurate classification result is obtained. BRIEF DESCRIPTION OF DRAWINGS
[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0027] Figure 1 The application scenario diagram of the Internet of Things potential user identification method provided by the embodiment of the present application is shown.
[0028] Figure 2 The flowchart of the Internet of Things potential user identification method provided by the embodiment of the present application is shown.
[0029] Figure 3 The schematic block diagram of the Internet of Things potential user identification device provided by the embodiment of the present application is shown.
[0030] Figure 4 The schematic block diagram of the computer equipment provided by the embodiment of the present application is shown. DETAILED DESCRIPTION
[0031] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0032] It should be understood that the terms "comprises" and "comprising" when used in this specification and the appended claims specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.
[0033] It should also be understood that the terms used in the present specification and the appended claims are merely for the purpose of describing particular embodiments and do not intend to limit the present application. As used in the present specification and the appended claims, singular forms "a," "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.
[0034] It should be further understood that the term "and / or" used in the present specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0035] Please refer to Figure 1 and Figure 2 , Figure 1 The application scenario diagram of the Internet of Things potential user identification method provided by the embodiment of the present application; Figure 2 The flowchart of the Internet of Things potential user identification method provided by the embodiment of the present application, which is applied to a server, and the method is executed by an application software installed in the server.
[0036] As shown in Figure 2 , the method comprises steps S101-S108.
[0037] S101, acquiring a first data set and a second data set; wherein the first type user data of the first data set and the second type user data in the second data set both comprise a user unique identification code.
[0038] In the embodiment, the technical solution is described by taking a server as an execution subject. The server can acquire a first data set from a first database. For example, the first database is an enterprise business database, and a plurality of first-type user data can be acquired from the enterprise business database. Each piece of first-type user data is multidimensional data information. Each piece of first-type user data can specifically include the feature values of dimensions such as enterprise name, unified social credit code (which can be regarded as a user unique identifier), registered capital, registered province and city, enterprise type, industry to which the enterprise belongs, scope of business, data annotation identifier, and the like. Moreover, the server can acquire a second data set from a second database. For example, the second database is an Internet of Things operator database, and a plurality of second-type user data can be acquired from the Internet of Things operator database. Each piece of second-type user data can specifically include the enterprise name and the unified social credit code. After the first data set and the second data set are acquired respectively, the first data set and the second data set are stored in the server, and the acquired first data set and second data set can be used as basic data for subsequent identification of potential Internet of Things users.
[0039] In S102, target first-type user data having a user unique identifier of second-type user data in the first data set is acquired to constitute an annotated sample set, and an unannotated sample set is acquired from the complement of the annotated sample set relative to the first data set.
[0040] In the embodiment, after the first data set and the second data set are acquired in the server, each piece of first-type user data in the first data set can be annotated by the second data set. Specifically, it is determined which first-type user data in the first data set has the same user unique identifier as the user unique identifier of the second-type user data. As long as a piece of first-type user data (for example, denoted as the first A1-type user data) in the first data set can find corresponding second-type user data in the second data set and the two have the same user unique identifier, the first A1-type user data in the first data set can be selected as annotated data, and the value of the data annotation identifier in the multidimensional data information of the first A1-type user data is assigned as 1 (at this time, the value of the data annotation identifier is assigned as 1, indicating that the first A1-type user data is annotated data). By analogy, the first A1-type user data is identified as annotated data, and whether other pieces of first-type user data in the first data set are annotated data is identified.
[0041] When the identification of whether all the first-type user data in the first data set is labeled data is completed, the labeled sample set can be composed of all the first-type user data with the identification of labeled data, and the unlabeled sample set can be composed of the first-type user data remaining in the first data after the labeled sample set is removed. In the labeled sample set, the value of the data annotation identifier in the multi-dimensional data information of each piece of first-type user data is assigned as 1, and in the unlabeled sample set, the value of the data annotation identifier in the multi-dimensional data information of each piece of first-type user data is assigned as 0.
[0042] S103, converting each piece of first-type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and converting each piece of first-type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy.
[0043] In this embodiment, in order to convert each piece of first-type user data in the labeled sample set and the unlabeled sample set into a user feature vector, a preset field value conversion strategy can be obtained first to determine how to convert the user data into a feature vector. Specifically, the field value conversion strategy includes at least a text-type field value conversion sub-strategy, a category-type field value conversion sub-strategy, and a numerical value-type field value conversion sub-strategy. The text-type field value conversion sub-strategy is used to convert the text-type field value of the first-type user data into a feature vector, the category-type field value conversion sub-strategy is used to convert the category-type field value of the first-type user data into a feature vector, and the numerical value-type field value conversion sub-strategy is used to convert the numerical value-type field value of the first-type user data into a feature vector. It can be seen that after each piece of first-type user data in the labeled sample set and the unlabeled sample set is converted into a user feature vector, the relationship between the user feature vectors can be quantitatively analyzed.
[0044] In an embodiment, the field value conversion strategy includes a text-type field value conversion sub-strategy, a category-type field value conversion sub-strategy, and a numerical value-type field value conversion sub-strategy. In step S103, converting each piece of first-type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy includes:
[0045] The i-th first-type user data in the labeled sample set is obtained, and a text-type field value set, a category-type field value set, and a numerical value-type field value set of the i-th first-type user data are obtained. The initial value of i is 1, and the value range of i is [1, N1], where N1 is the total number of first-type user data included in the labeled sample set.
[0046] convert each text type field value in the text type field value set in the i-th first type user data into a first field feature vector according to the text type field value conversion sub-strategy;
[0047] convert each category type field value in the category type field value set in the i-th first type user data into a second field feature vector according to the category type field value conversion sub-strategy;
[0048] convert each numerical value type field value in the numerical value type field value set in the i-th first type user data into a third field feature vector according to the numerical value type field value conversion sub-strategy;
[0049] connect the first field feature vector, the second field feature vector and the third field feature vector corresponding to each field value in the i-th first type user data respectively to obtain an i-th first user feature vector of the i-th first type user data;
[0050] increment i by 1 to update the value of i;
[0051] if it is determined that i does not exceed N1, return to execute the steps of obtaining the i-th first type user data in the labeled sample set, and obtaining the text type field value set, the category type field value set and the numerical value type field value set of the i-th first type user data;
[0052] if it is determined that i exceeds N1, save the first first user feature vector to the N1-th first user feature vector.
[0053] In the embodiment, the acquisition process of the first user feature vector No. 1 is illustrated by taking the first type user data No. 1 as an example. For example, it is assumed that the first type user data No. 1 includes the user unique identification code field value (such as 91440300************), the enterprise name field value (such as Shenzhen ** Co., Ltd.), the registered capital field value (such as **** million), the registered province and city field value (such as Shenzhen, Guangdong Province), the enterprise type field value (such as limited liability company), the industry field value (such as manufacturing industry), the business scope field value (such as clothing, apparel), the data annotation identifier field value (such as the field value is 1), wherein the enterprise name field value and the business scope field value are text type field values, the registered province and city field value, the enterprise type field value and the industry field value are category type field values, and the user unique identification code field value, the registered capital field value and the data annotation identifier field value are numerical value type field values. The text type field values of the enterprise name field value and the business scope field value are converted into the first field feature vector by the text type field value conversion sub-strategy, the category type field values of the registered province and city field value, the enterprise type field value and the industry field value are converted into the second field feature vector by the category type field value conversion sub-strategy, and the numerical value type field values of the user unique identification code field value, the registered capital field value and the data annotation identifier field value are converted into the third field feature vector by the numerical value type field value conversion sub-strategy, and then the field feature vectors corresponding to the respective field values in the first type user data No. 1 are obtained. Finally, the field feature vectors corresponding to the respective field values are connected according to the original field value arrangement order of the first type user data No. 1, and the first user feature vector No. 1 is obtained.
[0054] For example, taking the enterprise name field value of Shenzhen ** Co., Ltd. as an example, the keyword set corresponding to the enterprise name field value can be obtained by first segmenting the enterprise name field value and then extracting the keywords, and then the word2vec model can be used to finally convert each keyword in the keyword set into a word vector, and the first field feature vector corresponding to the enterprise name field value can be obtained by connecting the word vectors. The conversion method of converting other text type field values into the first field feature vector according to the text type field value conversion sub-strategy can refer to the conversion method of converting the enterprise name field value into the corresponding first field feature vector.
[0055] Continuing to take the registered province and city field value of Guangdong Shenzhen as an example, the province and city code can be obtained first, and then the preset category field value conversion sub-strategy is used to convert the value range of the dummy variable between 0 and 1 as the second field feature vector. Taking the registered capital field value as an example, the registered capital field value can be directly standardized to make it a numerical value with a mean of 0 and a standard deviation of 1, thereby obtaining the corresponding third field feature vector.
[0056] After converting each piece of first type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy, the difference is that the data annotation identification field value of each piece of first type user data in the unlabeled sample set is all 0, while the data annotation identification field value of each piece of first type user data in the labeled sample set is all 1. When converting the field values of each piece of first type user data in the unlabeled sample set except the data annotation identification field value according to the field value conversion strategy, the process of converting each piece of first type user data in the labeled sample set into a corresponding first user feature vector according to the preset field value conversion strategy can be referred to.
[0057] In step S104, a plurality of labeled sets corresponding to the unlabeled sample set are obtained by labeling each piece of first type user data in the unlabeled sample set according to a preset labeling strategy and the labeled sample set.
[0058] In this embodiment, in order to re-label the data annotation identification field value of each piece of first type user data in the unlabeled sample set according to the data annotation identification field value of each piece of first type user data in the labeled sample set, the labeling strategy stored in the server can be obtained first, and then the labeled sample set (since the data annotation identification field value of each piece of first type user data in the labeled sample set is known, and the first user feature vector of each piece of first type user data in the labeled sample set is also known) is obtained. After that, each piece of first type user data in the unlabeled sample set is labeled a plurality of times based on the preset labeling strategy and the labeled sample set, and a plurality of labeled sets corresponding to the unlabeled sample set are obtained. In this labeling strategy, the vector similarity between each second user feature vector in the unlabeled sample set and each first user feature vector in the labeled sample set and other second user feature vectors in the unlabeled sample set is mainly analyzed, so as to find the approximate user feature vector set of each second user feature vector in the unlabeled sample set, and thus the data annotation identification field value of each second user feature vector in the unlabeled sample set is obtained. In this way, the labeled sample set is fully utilized to realize the labeling of the unlabeled sample set.
[0059] In an embodiment, step S104 includes:
[0060] obtaining the kth group of random numbers with values in the range of [0, 1]; wherein the initial value of k is 1, and the value range of k is [1, N3], and N3 is a preset total number of annotations;
[0061] obtaining the jth second user feature vector corresponding to the jth first type user data in the unlabeled sample set; wherein the initial value of j is 1, and the value range of j is [1, N2], and N2 is the total number of first type user data included in the unlabeled sample set;
[0062] obtaining the vector similarity values between the jth second user feature vector and other second user feature vectors corresponding to other first type user data in the unlabeled sample set, and obtaining the vector similarity values between the jth second user feature vector and first user feature vectors corresponding to each first type user data in the labeled sample set, to obtain the jth vector similarity set;
[0063] obtaining the target vector similarity values in the jth vector similarity set whose descending order ranking values do not exceed the stored ranking threshold, and the target user feature vectors corresponding to each target vector similarity value, to form the jth target user feature vector set;
[0064] obtaining the jth positive example proportion in the jth target user feature vector set according to a preset positive example determination strategy, and performing numerical size comparison between the jth positive example proportion and the kth group of random numbers with values in the range of [0, 1] to obtain the jth comparison result;
[0065] if the jth comparison result is that the jth positive example proportion is greater than or equal to the random number, assigning the label value of the jth first type user data as 1;
[0066] if the jth comparison result is that the jth positive example proportion is less than the random number, assigning the label value of the jth first type user data as -1;
[0067] increasing j by 1 to update the value of j;
[0068] if it is determined that j does not exceed N2, returning to perform the step of obtaining the jth second user feature vector corresponding to the jth first type user data in the unlabeled sample set;
[0069] if it is determined that j exceeds N2, combining the label values of the 1st first type user data to the label values of the N2th first type user data with the corresponding second user feature vectors, and saving them as the kth group of label sets of the unlabeled sample set;
[0070] increasing k by 1 to update the value of k;
[0071] If it is determined that k does not exceed N3, the step of obtaining the value of k is returned to perform the step of randomly generating the kth set of random numbers with a value range of [0, 1];
[0072] If it is determined that k exceeds N3, the first set of annotations of the unlabeled sample set and the N3th set of annotations of the unlabeled sample set are obtained and saved.
[0073] In this embodiment, taking k = 1 as an example to illustrate the process of the first round of annotation of the unlabeled sample set based on the annotated sample set, if it is assumed that the first set of random numbers randomly generated when k = 1 is 0.5, then the annotation process of the first number of first type user data in the first round of annotation is as follows:
[0074] 1) Obtain the first number of second user feature vectors corresponding to the first number of first type user data in the unlabeled sample set;
[0075] 2) Obtain the vector similarity values between the first number of second user feature vectors and other second user feature vectors corresponding to other first type user data in the unlabeled sample set, and obtain the vector similarity values between the first number of second user feature vectors and first user feature vectors corresponding to each first type user data in the annotated sample set, to obtain the first number of vector similarity sets;
[0076] 3) Obtain the target vector similarity values in the first number of vector similarity sets whose descending rank values of vector similarity values do not exceed the stored rank threshold value (for example, the stored rank threshold value is 20, and the process of obtaining the rank threshold value is described in detail later), and the target user feature vectors corresponding to each target vector similarity value, to form the first number of target user feature vector sets;
[0077] 4) Obtain the first number of feature vector totals corresponding to the first user feature vectors of the annotated sample set in the first number of target user feature vector sets, then divide the first number of feature vector totals by the rank threshold value to obtain the first number of positive example proportions, and then compare the first number of positive example proportions with the first set of random numbers (i.e., 0.5 in the above example) to obtain the first number of comparison results;
[0078] 5) If the jth comparison result is that the jth positive example proportion is greater than or equal to the random number, the annotation value of the jth first type user data is assigned as 1;
[0079] 6) If the jth comparison result is that the jth positive example proportion is less than the random number, the annotation value of the jth first type user data is assigned as -1.
[0080] The labeling of other first-type user data in the first round of labeling can refer to the labeling manner of the first No. 1 first-type user data in the first round of labeling, and the difference between the labeling of each first-type user data in the other rounds of the unlabeled sample set and the first round of labeling is that the random numbers generated are not the same, which may lead to different labeling results for the same first-type user data in different rounds. It can be seen that in this way, the unlabeled sample set is fully utilized to realize the rapid labeling of the unlabeled sample set.
[0081] wherein the stored ranking threshold is a special hyperparameter that needs to be adjusted, for example, the ranking threshold is denoted as L, then E L is the expected value of the unlabeled sample set labeled as positive examples by the labeling method of steps 1)-6), within a certain range, as L increases, E L will also increase; but when L reaches a certain size, E L no longer increases, and even decreases. Since a larger L can fully utilize the information provided by the positive examples, a suitable L value should make E L as large as possible; but at the same time, a too large L value will lead to too large a calculation amount; therefore, multiple L values can be tried here, and the final L value to be used is determined by comparing the results, and L=20 is used as the value of L.
[0082] S105, obtaining a plurality of sets of labeled sets corresponding to the unlabeled sample set, respectively training the LightGBM model to be trained based on the labeled sets to obtain a plurality of LightGBM models corresponding to the plurality of sets of labeled sets, and the plurality of LightGBM models corresponding to the plurality of sets of labeled sets form a classification model set.
[0083] In this embodiment, since a plurality of sets of labeled sets are obtained based on multiple rounds of labeling in step S104, each set of labeled sets can be used to train the LightGBM model to be trained to obtain a plurality of LightGBM models. The plurality of LightGBM models can be used to operate on the to-be-predicted data to obtain a prediction result (which can also be understood as a classification result).
[0084] In an embodiment, after step S105, the method further includes:
[0085] Each LightGBM model in the classification model set is respectively subjected to five-fold cross-validation for model optimization to update the classification model set.
[0086] In this embodiment, one of the LightGBM models in the classification model set is taken as an example for illustration, for example, the selected LightGBM model is denoted as a first LightGBM model, and the first LightGBM model can be subjected to the following processing to realize five-fold cross-validation:
[0087] 11) Divide all data in the set of labeled data corresponding to the first LightGBM model into five equal parts to obtain five piles of data;
[0088] 12) Take one of the five piles of data as the test set and the other four piles as the training set to perform the first optimization of the first LightGBM model to update the first LightGBM model;
[0089] 13) Then repeat step 12) four more times, each time selecting a different pile of data as the test set, and after completing five times of model optimization, the optimization of the first LightGBM model is completed.
[0090] In the process of each optimization of the first LightGBM model, the area under the ROC curve (i.e., the area under the receiver operating characteristic curve) is used as an evaluation index, and the larger the area under the ROC curve of the LightGBM model, the more excellent the model. It can be seen that the five-fold cross-validation is performed on each LightGBM model in the set of classification models to optimize the model, which can effectively improve the prediction accuracy of the model.
[0091] S106, obtain the to-be-predicted user data, and obtain the to-be-predicted user feature vector corresponding to the to-be-predicted user data according to the field value conversion strategy, input the to-be-predicted user feature vector into each LightGBM model in the set of classification models for operation, and obtain the classification results corresponding to each LightGBM model respectively to constitute a classification result set.
[0092] In the embodiment, steps S101-S105 describe the training process of the classification model set, and step S106 starts the classification operation of the to-be-predicted user data based on the classification model set. The to-be-predicted user data is also the same as the first type of user data in the first data set, is multi-dimensional data information, and can be uploaded from the client to the server. At this time, the to-be-predicted user data cannot be directly input into each LightGBM model in the classification model set for operation, and the corresponding to-be-predicted user feature vector of the to-be-predicted user data needs to be obtained according to the field value conversion strategy. In the process of converting the to-be-predicted user data according to the field value conversion strategy, the field value of the text type in the to-be-predicted user data is converted into a feature vector according to the text type field value conversion sub-strategy, the field value of the category type in the to-be-predicted user data is converted into a feature vector according to the category type field value conversion sub-strategy, and the field value of the numerical type in the to-be-predicted user data is converted into a feature vector according to the numerical type field value conversion sub-strategy. The corresponding to-be-predicted user feature vector of the to-be-predicted user data is obtained based on the above conversion. Finally, the to-be-predicted user feature vector is input into each LightGBM model in the classification model set for operation, and the classification results corresponding to each LightGBM model are obtained to form a classification result set. Since the operation is based on each LightGBM model in the classification model set, rather than only based on one classification model, more classification results can be obtained for comprehensive analysis of the final classification result.
[0093] S107, obtaining an average value of the classification results in the classification result set as a classification result average value.
[0094] In the embodiment, when the plurality of classification results in the classification result set are obtained, the average value of the plurality of classification results is obtained as the classification result average value. Then, the classification result average value can be used as a reference value for labeling the to-be-predicted user data.
[0095] S108, if it is determined that the classification result average value is greater than a preset classification threshold, setting the label value of the to-be-predicted user data as a first label value.
[0096] In the embodiment, a classification threshold (such as 0.5) can be set in advance. If the classification result average value of the to-be-predicted user data is greater than the classification threshold, it indicates that the user corresponding to the to-be-predicted user data can be labeled with the first label value (such as 1, indicating that the user is a potential user).
[0097] In an embodiment, step S107 further includes:
[0098] If it is determined that the classification result average value is less than or equal to the classification threshold value, the label value of the to-be-predicted user data is set to a second label value.
[0099] If the classification result average value of the to-be-predicted user data is less than or equal to the classification threshold value, it indicates that the user corresponding to the to-be-predicted user data can be labeled with a second label value (for example, the second label value is 0, indicating that the user is not a potential user).
[0100] The method realizes training of multiple LightGBM models to form a classification model set based on the labeled sample set and the unlabeled sample set obtained based on the first data set and the second data set, and simultaneously performs multi-model classification on to-be-predicted user data based on the classification model set to obtain a more accurate classification result.
[0101] The embodiment of the present application also provides an Internet of Things potential user identification device, which is used to execute any one of the aforementioned embodiments of the Internet of Things potential user identification method. Specifically, please refer to Figure 3 , Figure 3 is a schematic block diagram of the Internet of Things potential user identification device 100 provided by the embodiment of the present application.
[0102] As shown in Figure 3 , the Internet of Things potential user identification device 100 comprises a data set initial acquisition unit 101, a data set classification unit 102, a user feature vector acquisition unit 103, a data labeling unit 104, a model set training unit 105, a classification result set acquisition unit 106, a classification result average value acquisition unit 107, and a first labeling unit 108.
[0103] The data set initial acquisition unit 101 is used to acquire a first data set and a second data set; wherein the first type user data in the first data set and the second type user data in the second data set both comprise a user unique identification code.
[0104] In the embodiment, the technical solution is described by taking a server as an execution subject. The server can acquire a first data set from a first database. For example, the first database is an enterprise business database, and a plurality of first-type user data can be acquired from the enterprise business database. Each piece of first-type user data is multidimensional data information. Each piece of first-type user data can specifically include the feature values of dimensions such as enterprise name, unified social credit code (which can be regarded as a user unique identifier), registered capital, registered province and city, enterprise type, industry to which the enterprise belongs, scope of business, and data annotation identifier. Moreover, the server can acquire a second data set from a second database. For example, the second database is an Internet of Things operator database, and a plurality of second-type user data can be acquired from the Internet of Things operator database. Each piece of second-type user data can specifically include the enterprise name and the unified social credit code. After the first data set and the second data set are acquired respectively, the first data set and the second data set are stored in the server, and the acquired first data set and second data set can be used as basic data for subsequent identification of potential Internet of Things users.
[0105] The data set classification unit 102 is configured to acquire target first-type user data with a user unique identifier of second-type user data in the first data set to form an annotated sample set, and acquire an unannotated sample set from a complement of the annotated sample set relative to the first data set.
[0106] In the embodiment, after the first data set and the second data set are acquired in the server, each piece of first-type user data in the first data set can be annotated by the second data set. Specifically, it is determined that which first-type user data in the first data set has the same user unique identifier as the user unique identifier of the second-type user data. As long as a piece of first-type user data (for example, denoted as the first A1-type user data) in the first data set can find corresponding second-type user data in the second data set and the two have the same user unique identifier, the first A1-type user data in the first data set can be selected as annotated data, and the value of the data annotation identifier in the multidimensional data information of the first A1-type user data is assigned as 1 (at this time, the value of the data annotation identifier is assigned as 1, indicating that the first A1-type user data is annotated data). By analogy, the first-type user data in the first data set is identified as annotated data in the same way as the first A1-type user data.
[0107] When the identification of whether all the first-type user data in the first data set is labeled data is completed, the labeled sample set can be composed of all the first-type user data with the identification of labeled data, and the unlabeled sample set can be composed of the first-type user data remaining in the first data after the labeled sample set is removed. In the labeled sample set, the value of the data annotation identifier in the multi-dimensional data information of each piece of first-type user data is assigned as 1, and in the unlabeled sample set, the value of the data annotation identifier in the multi-dimensional data information of each piece of first-type user data is assigned as 0.
[0108] The user feature vector acquisition unit 103 is configured to convert each piece of first-type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and convert each piece of first-type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy.
[0109] In this embodiment, in order to convert each piece of first-type user data in the labeled sample set and the unlabeled sample set into a user feature vector, a preset field value conversion strategy can be obtained first to determine how to convert the user data into a feature vector. Specifically, the field value conversion strategy includes at least a text-type field value conversion sub-strategy, a category-type field value conversion sub-strategy, and a numerical value-type field value conversion sub-strategy. The text-type field value conversion sub-strategy is used to convert the text-type field value of the first-type user data into a feature vector, the category-type field value conversion sub-strategy is used to convert the category-type field value of the first-type user data into a feature vector, and the numerical value-type field value conversion sub-strategy is used to convert the numerical value-type field value of the first-type user data into a feature vector. It can be seen that after each piece of first-type user data in the labeled sample set and the unlabeled sample set is converted into a user feature vector, the relationship between the user feature vectors can be quantitatively analyzed.
[0110] In an embodiment, the field value conversion strategy includes a text-type field value conversion sub-strategy, a category-type field value conversion sub-strategy, and a numerical value-type field value conversion sub-strategy. The conversion of each piece of first-type user data in the labeled sample set into a corresponding first user feature vector according to the preset field value conversion strategy in the user feature vector acquisition unit 103 includes:
[0111] The i-th first-type user data in the labeled sample set is obtained, and the text-type field value set, the category-type field value set, and the numerical value-type field value set of the i-th first-type user data are obtained. The initial value of i is 1, and the value range of i is [1, N1], where N1 is the total number of first-type user data included in the labeled sample set.
[0112] convert each text type field value in the text type field value set in the i-th first type user data into a first field feature vector according to the text type field value conversion sub-strategy;
[0113] convert each category type field value in the category type field value set in the i-th first type user data into a second field feature vector according to the category type field value conversion sub-strategy;
[0114] convert each numerical value type field value in the numerical value type field value set in the i-th first type user data into a third field feature vector according to the numerical value type field value conversion sub-strategy;
[0115] connect the first field feature vector, the second field feature vector and the third field feature vector corresponding to each field value in the i-th first type user data respectively to obtain an i-th first user feature vector of the i-th first type user data;
[0116] increment i by 1 to update the value of i;
[0117] if it is determined that i does not exceed N1, return to execute the steps of obtaining the i-th first type user data in the labeled sample set, and obtaining the text type field value set, the category type field value set and the numerical value type field value set of the i-th first type user data;
[0118] if it is determined that i exceeds N1, save the first first user feature vector to the N1-th first user feature vector.
[0119] In the embodiment, the acquisition process of the first user feature vector No. 1 is illustrated by taking the first type user data No. 1 as an example. For example, it is assumed that the first type user data No. 1 includes the user unique identification code field value (such as 91440300************), the enterprise name field value (such as Shenzhen ** Co., Ltd.), the registered capital field value (such as **** million), the registered province and city field value (such as Shenzhen, Guangdong Province), the enterprise type field value (such as limited liability company), the industry field value (such as manufacturing industry), the business scope field value (such as clothing, apparel), the data annotation identifier field value (such as the field value is 1), wherein the enterprise name field value and the business scope field value are text type field values, the registered province and city field value, the enterprise type field value and the industry field value are category type field values, and the user unique identification code field value, the registered capital field value and the data annotation identifier field value are numerical value type field values. The text type field values of the enterprise name field value and the business scope field value are converted into the first field feature vector by the text type field value conversion sub-strategy, the category type field values of the registered province and city field value, the enterprise type field value and the industry field value are converted into the second field feature vector by the category type field value conversion sub-strategy, and the numerical value type field values of the user unique identification code field value, the registered capital field value and the data annotation identifier field value are converted into the third field feature vector by the numerical value type field value conversion sub-strategy, and then the field feature vectors corresponding to the respective field values in the first type user data No. 1 are obtained. Finally, the field feature vectors corresponding to the respective field values are connected according to the original field value arrangement order of the first type user data No. 1, and the first user feature vector No. 1 is obtained.
[0120] For example, taking the enterprise name field value of Shenzhen ** Co., Ltd. as an example, the keyword set corresponding to the enterprise name field value can be obtained by first segmenting the enterprise name field value and then extracting the keywords, and then the word2vec model can be used to finally convert each keyword in the keyword set into a word vector, and the first field feature vector corresponding to the enterprise name field value can be obtained by connecting the word vectors. The conversion method of converting other text type field values into the first field feature vector according to the text type field value conversion sub-strategy can refer to the conversion method of converting the enterprise name field value into the corresponding first field feature vector.
[0121] Continuing to take the registered province and city field value of Shenzhen, Guangdong Province as an example, the province and city code of Shenzhen, Guangdong Province can be obtained first, and then the dummy variable with the value range between 0 and 1 is converted as the second field feature vector based on the preset category type field value conversion sub-strategy. Taking the registered capital field value as an example, the registered capital field value can be directly standardized to make it a numerical value with a mean value of 0 and a standard deviation of 1, so as to obtain the corresponding third field feature vector.
[0122] After converting each piece of first type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy, the difference is that the data annotation identification field value of each piece of first type user data in the unlabeled sample set is all 0, while the data annotation identification field value of each piece of first type user data in the labeled sample set is all 1. When converting the data annotation identification field value of each piece of first type user data in the unlabeled sample set according to the field value conversion strategy, the process of converting each piece of first type user data in the labeled sample set into a corresponding first user feature vector according to the preset field value conversion strategy can be referred to.
[0123] The data annotation unit 104 is configured to perform a plurality of times of annotation on each first type user data in the unlabeled sample set according to a preset annotation strategy and the labeled sample set, to obtain a plurality of groups of annotation sets corresponding to the unlabeled sample set.
[0124] In this embodiment, in order to re-annotate the data annotation identification field value of each first type user data in the unlabeled sample set according to the data annotation identification field value of each first type user data in the labeled sample set, the annotation strategy stored in the server can be obtained first, and then the labeled sample set (since the data annotation identification field value of each first type user data in the labeled sample set is known, and the first user feature vector of each first type user data in the labeled sample set is also known) can be obtained. After that, a plurality of times of annotation are performed on each first type user data in the unlabeled sample set based on the preset annotation strategy and the labeled sample set, to obtain a plurality of groups of annotation sets corresponding to the unlabeled sample set. In the annotation strategy, the vector similarity between each second user feature vector in the unlabeled sample set and each first user feature vector in the labeled sample set and other second user feature vectors in the unlabeled sample set is mainly analyzed, so as to find the approximate user feature vector set of each second user feature vector in the unlabeled sample set, and thus the data annotation identification field value of each second user feature vector in the unlabeled sample set is obtained. In this way, the labeled sample set is fully utilized to realize the annotation of the unlabeled sample set.
[0125] In an embodiment, the data annotation unit 104 is specifically configured to:
[0126] obtaining the kth group of random numbers with values in the range of [0, 1]; wherein the initial value of k is 1, and the value range of k is [1, N3], N3 is a preset total number of annotations;
[0127] obtaining the jth second user feature vector corresponding to the jth first type user data in the unlabeled sample set; wherein the initial value of j is 1 and the value range of j is [1, N2], N2 is the total number of first type user data included in the unlabeled sample set;
[0128] obtaining the vector similarity value between the jth second user feature vector and other second user feature vectors corresponding to other first type user data in the unlabeled sample set, and obtaining the vector similarity value between the jth second user feature vector and the first user feature vectors corresponding to each first type user data in the labeled sample set, to obtain the jth vector similarity set;
[0129] obtaining the target vector similarity values in the jth vector similarity set whose descending order ranking values do not exceed the stored ranking threshold, and the target user feature vectors corresponding to each target vector similarity value, to form the jth target user feature vector set;
[0130] obtaining the jth positive example proportion in the jth target user feature vector set according to a preset positive example determination strategy, and performing numerical size comparison between the jth positive example proportion and the kth group of random numbers with values in the range of [0, 1] to obtain the jth comparison result;
[0131] if the jth comparison result is that the jth positive example proportion is greater than or equal to the random number, assigning the label value of the jth first type user data as 1;
[0132] if the jth comparison result is that the jth positive example proportion is less than the random number, assigning the label value of the jth first type user data as -1;
[0133] incrementing j by 1 to update the value of j;
[0134] if it is determined that j does not exceed N2, returning to execute the step of obtaining the jth second user feature vector corresponding to the jth first type user data in the unlabeled sample set;
[0135] if it is determined that j exceeds N2, combining the label values of the 1st first type user data to the label values of the N2th first type user data with the corresponding second user feature vectors and saving them as the kth group of label sets of the unlabeled sample set;
[0136] incrementing k by 1 to update the value of k;
[0137] If it is determined that k does not exceed N3, the step of obtaining the value of k is returned to execute the step of randomly generating the kth set of random numbers with a value range of [0, 1];
[0138] If it is determined that k exceeds N3, the first set of labels of the unlabeled sample set and the N3th set of labels of the unlabeled sample set are obtained and saved.
[0139] In this embodiment, taking k = 1 as an example to illustrate the process of the first round of labeling of the unlabeled sample set based on the labeled sample set, if it is assumed that the first set of random numbers randomly generated when k = 1 is 0.5, then the labeling process of the first number of first type user data in the first round of labeling is as follows:
[0140] 21) Obtain the first number of second user feature vectors corresponding to the first number of first type user data in the unlabeled sample set;
[0141] 22) Obtain the vector similarity values between the first number of second user feature vectors and other second user feature vectors corresponding to other first type user data in the unlabeled sample set, and obtain the vector similarity values between the first number of second user feature vectors and first user feature vectors corresponding to each first type user data in the labeled sample set, to obtain the first number of vector similarity sets;
[0142] 23) Obtain the target vector similarity values in the first number of vector similarity sets whose descending order ranking values do not exceed the stored ranking threshold (for example, the stored ranking threshold is 20, and the process of obtaining the ranking threshold is described in detail later), and the target user feature vectors corresponding to each target vector similarity value, to form the first number of target user feature vector sets;
[0143] 24) Obtain the first number of feature vector totals corresponding to the first user feature vectors of the labeled sample set in the first number of target user feature vector sets, then divide the first number of feature vector totals by the ranking threshold to obtain the first number of positive example proportions, and then compare the first number of positive example proportions with the first set of random numbers (i.e., 0.5 in the above example) to obtain the first number of comparison results;
[0144] 25) If the jth comparison result is that the jth positive example proportion is greater than or equal to the random number, the labeling value of the jth first type user data is assigned as 1;
[0145] 26) If the jth comparison result is that the jth positive example proportion is less than the random number, the labeling value of the jth first type user data is assigned as -1.
[0146] The labeling of other first-type user data in the first round of labeling can refer to the labeling manner of the first No. 1 first-type user data in the first round of labeling, and the difference between the labeling of each first-type user data in the unlabeled sample set in other rounds and the first round of labeling is that the random numbers generated at random are not the same, which may lead to different labeling results for the same first-type user data in different rounds. It can be seen that through this way, the unlabeled sample set is fully utilized to realize the rapid labeling of the unlabeled sample set.
[0147] wherein the stored ranking threshold is a special hyperparameter that needs to be adjusted, for example, the ranking threshold is recorded as L, then E L is the expected value of the unlabeled sample set labeled as positive examples by the labeling method of steps 21)-26), within a certain range, as L increases, E L will also increase; but when L reaches a certain size, E L no longer increases, and even decreases. Since a large L can fully utilize the information provided by the positive examples, a suitable L value should make E L as large as possible; but at the same time, a too large L value will lead to too large calculation; therefore, multiple L values can be tried here, and the final L=20 to be used is determined by comparing the results.
[0148] The model set training unit 105 is configured to perform model training on the to-be-trained LightGBM models respectively corresponding to the multiple sets of labeled samples in the unlabeled sample set, to obtain multiple LightGBM models respectively corresponding to the multiple sets of labeled samples, and to obtain the classification model set composed of the multiple LightGBM models respectively corresponding to the multiple sets of labeled samples.
[0149] In this embodiment, since multiple sets of labeled samples are obtained based on multiple rounds of labeling in step S104, each set of labeled samples can be used to train a to-be-trained LightGBM model, so that multiple LightGBM models can be obtained. The multiple LightGBM models can be used to operate on the to-be-predicted data to obtain a prediction result (which can also be understood as a classification result).
[0150] In an embodiment, the Internet of Things potential user identification apparatus 100 further comprises:
[0151] The classification model set tuning unit is configured to perform model tuning on each LightGBM model in the classification model set through five-fold cross-validation, to update the classification model set.
[0152] In the embodiment, taking one of the LightGBM models in the classification model set as an example, and denoting the selected LightGBM model as a first LightGBM model, the first LightGBM model can be processed as follows to implement five-fold cross-validation:
[0153] 31) The first LightGBM model is used to divide all data in a set of labeled data corresponding to the first LightGBM model into five equal parts to obtain five piles of data;
[0154] 32) One of the five piles of data is used as a test set, and the other four piles are used as training sets to perform the first tuning of the first LightGBM model to update the first LightGBM model;
[0155] 33) Step 32) is repeated four more times, each time selecting a different one of the selected LightGBM models as the test set, and after five times of model tuning, the tuning of the first LightGBM model is completed.
[0156] In the process of tuning the first LightGBM model each time, the area under the ROC curve (i.e., the area under the receiver operating characteristic curve) is used as an evaluation index, and the larger the area under the ROC curve of the LightGBM model, the more superior the model. It can be seen that the model tuning of each LightGBM model in the classification model set through five-fold cross-validation can effectively improve the prediction accuracy of the model.
[0157] The classification result set acquisition unit 106 is configured to acquire the to-be-predicted user data, acquire the to-be-predicted user feature vector corresponding to the to-be-predicted user data according to the field value conversion strategy, input the to-be-predicted user feature vector into each LightGBM model in the classification model set for operation, and obtain the classification results corresponding to each LightGBM model to form a classification result set.
[0158] In the embodiment, not only the training process of the classification model set is described, but also the classification operation based on the classification model set on the to-be-predicted user data is started in the classification result set acquisition unit 106. The to-be-predicted user data is also the same as the first type of user data in the first data set, is multi-dimensional data information, and can be uploaded from the client to the server. At this time, the to-be-predicted user data cannot be directly input into each LightGBM model in the classification model set for operation, and the to-be-predicted user feature vector corresponding to the to-be-predicted user data needs to be acquired according to the field value conversion strategy. In the process of converting the to-be-predicted user data according to the field value conversion strategy, the field value of the to-be-predicted user data in the text type is converted into a feature vector according to the text type field value conversion sub-strategy, the field value of the to-be-predicted user data in the category type is converted into a feature vector according to the category type field value conversion sub-strategy, and the field value of the to-be-predicted user data in the numerical type is converted into a feature vector according to the numerical type field value conversion sub-strategy. The to-be-predicted user feature vector corresponding to the to-be-predicted user data is obtained based on the above conversion. Finally, the to-be-predicted user feature vector is input into each LightGBM model in the classification model set for operation, and the classification results corresponding to each LightGBM model are obtained to form a classification result set. Since the operation is based on each LightGBM model in the classification model set, not only one classification model, more classification results can be obtained for comprehensive analysis of the final classification result.
[0159] The classification result average value acquisition unit 107 is configured to acquire an average value of the classification results in the classification result set as a classification result average value.
[0160] In the embodiment, when the plurality of classification results in the classification result set are obtained, the average value of the plurality of classification results is calculated to obtain the classification result average value. Then, the classification result average value can be used as a reference value for labeling the to-be-predicted user data.
[0161] The first labeling unit 108 is configured to set a labeling value of the to-be-predicted user data as a first labeling value if it is determined that the classification result average value is greater than a preset classification threshold.
[0162] In the embodiment, a classification threshold (for example, 0.5) can be set in advance. If the classification result average value of the to-be-predicted user data is greater than the classification threshold, it indicates that the user corresponding to the to-be-predicted user data can be labeled with the first labeling value (for example, 1, indicating that the user is a potential user).
[0163] In an embodiment, the Internet of Things potential user identification apparatus 100 further includes:
[0164] The second labeling unit is configured to set a labeling value of the to-be-predicted user data as a second labeling value if it is determined that the classification result average value is less than or equal to the classification threshold.
[0165] If the classification result average value of the to-be-predicted user data is less than or equal to the classification threshold, it indicates that the user corresponding to the to-be-predicted user data can be labeled with the second labeling value (for example, the second labeling value is 0, indicating that the user is not a potential user).
[0166] The device realizes training of multiple LightGBM models based on the labeled sample set and the unlabeled sample set obtained based on the first data set and the second data set to form a classification model set, and simultaneously performs multi-model classification and averaging on to-be-predicted user data based on the classification model set to obtain a more accurate classification result.
[0167] The above-mentioned Internet of Things potential user identification device can be realized in the form of a computer program, which can run on a computer device as shown in Figure 4 .
[0168] Please refer to Figure 4 , Figure 4 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 is a server, and can also be a server cluster.
[0169] Referring to Figure 4 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a device bus 501, wherein the memory can include a storage medium 503 and an internal memory 504.
[0170] The storage medium 503 can store an operation device 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the Internet of Things potential user identification method.
[0171] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.
[0172] The internal memory 504 provides an environment for the running of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the Internet of Things potential user identification method.
[0173] The network interface 505 is configured to perform network communication, such as providing transmission of data information. Those skilled in the art can understand that Figure 4The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0174] The processor 502 is configured to run the computer program 5032 stored in the memory, so as to implement the method for identifying potential users of an Internet of Things disclosed in the embodiments of the present application.
[0175] Those skilled in the art can understand that Figure 4 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific structure of the computer device. In other embodiments, the computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. For example, in some embodiments, the computer device can only include the memory and the processor. In such embodiments, the structure and function of the memory and the processor are consistent with those of the memory and the processor in the embodiments shown in the figure, and will not be described here again. Figure 4
[0176] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0177] In another embodiment of the present application, a computer readable storage medium is provided. The computer readable storage medium can be a non-volatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the method for identifying potential users of an Internet of Things disclosed in the embodiments of the present application.
[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here. Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, each example has been described in the foregoing description in terms of its general functionality, without regard to its specific combination of hardware and software. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0179] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is merely logical functional division, and actual implementation can have another division, or units with the same function can be combined into one unit, for example, multiple units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, or can be electrical, mechanical or other form of connection.
[0180] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.
[0181] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or software functional unit.
[0182] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a background server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A method for identifying potential users of an Internet of Things, characterized in that, The method comprises the following steps: obtaining a first data set and a second data set; wherein the first type of user data in the first data set and the second type of user data in the second data set both comprise a user unique identification code; obtaining target first type of user data with the user unique identification code of the second type of user data in the first data set to form a labeled sample set, and obtaining an unlabeled sample set from the complement of the labeled sample set relative to the first data set; converting each piece of first type of user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and converting each piece of first type of user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy; performing labeling on each first type of user data in the unlabeled sample set according to a preset labeling strategy and the labeled sample set for a plurality of times to obtain a plurality of sets of labeling sets corresponding to the unlabeled sample set; obtaining the plurality of sets of labeling sets corresponding to the unlabeled sample set to respectively perform model training on a LightGBM model to be trained to obtain a plurality of LightGBM models respectively corresponding to the plurality of sets of labeling sets, and forming a classification model set by the plurality of LightGBM models respectively corresponding to the plurality of sets of labeling sets; obtaining to-be-predicted user data, and obtaining a to-be-predicted user feature vector corresponding to the to-be-predicted user data according to the field value conversion strategy, inputting the to-be-predicted user feature vector into each LightGBM model in the classification model set to perform operation, and obtaining a classification result corresponding to each LightGBM model to form a classification result set; obtaining an average value of the classification results in the classification result set as a classification result average value; and if it is determined that the classification result average value is greater than a preset classification threshold, setting a labeling value of the to-be-predicted user data as a first labeling value; wherein the first data set is enterprise business data, and the second data set is Internet of Things operator user data; the step of performing labeling on each first type of user data in the unlabeled sample set according to a preset labeling strategy and the labeled sample set for a plurality of times to obtain a plurality of sets of labeling sets corresponding to the unlabeled sample set comprises the following steps: obtaining a k value, and generating a kth random number with a value range of [0, 1]; wherein the initial value of k is 1, and the value range of k is [1, N3], and N3 is a preset total labeling times; obtaining a jth second user feature vector corresponding to a jth first type of user data in the unlabeled sample set; wherein the initial value of j is 1, and the value range of j is [1, N2], and N2 is the total number of first type of user data included in the unlabeled sample set; obtaining a vector similarity value between the jth second user feature vector and other second user feature vectors corresponding to other first type of user data in the unlabeled sample set, and obtaining a vector similarity value between the jth second user feature vector and corresponding first user feature vectors of each first type of user data in the labeled sample set to obtain a jth vector similarity set; obtaining target vector similarity values in a descending order of ranking values of vector similarity values in the jth vector similarity set and target user feature vectors corresponding to the target vector similarity values, to form a jth target user feature vector set; obtaining a jth positive example ratio in the jth target user feature vector set according to a preset positive example determination strategy, and performing numerical size comparison between the jth positive example ratio and a kth random number in a value range of [0, 1] to obtain a jth comparison result; if the jth comparison result is that the jth positive example ratio is greater than or equal to the random number, assigning a label value of the jth first type user data to 1; if the jth comparison result is that the jth positive example ratio is less than the random number, assigning the label value of the jth first type user data to -1; increasing j by 1 to update the value of j; if it is determined that j exceeds N2, saving the label values of the first type user data from the 1st to the N2th first type user data in combination with the corresponding second user feature vectors as a kth label set of the unlabeled sample set; increasing k by 1 to update the value of k; if it is determined that k exceeds N3, returning to execute the step of obtaining the value of k and randomly generating a kth random number in a value range of [0, 1]; if it is determined that k exceeds N3, obtaining the first label set of the unlabeled sample set and the N3th label set of the unlabeled sample set and saving them. The field value conversion strategy includes a text type field value conversion sub-strategy, a category type field value conversion sub-strategy and a numerical value type field value conversion sub-strategy; the text type field value conversion sub-strategy is used to convert the field value of the text type of the user data into a feature vector, the category type field value conversion sub-strategy is used to convert the field value of the category type of the user data into a feature vector, and the numerical value type field value conversion sub-strategy is used to convert the field value of the numerical value type of the user data into a feature vector. 2.The IoT potential user identification method of claim 1, wherein, The step of converting each piece of first type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy includes: 3.The IoT potential user identification method of claim 2, wherein, obtaining a ith first type user data in the labeled sample set, and obtaining a text type field value set, a category type field value set and a numerical value type field value set of the ith first type user data; wherein the initial value of i is 1 and the value range of i is [1, N1], and N1 is the total number of first type user data included in the labeled sample set; converting each text type field value in the text type field value set of the ith first type user data into a first field feature vector according to the text type field value conversion sub-strategy; converting each category type field value in the category type field value set of the ith first type user data into a second field feature vector according to the category type field value conversion sub-strategy; and converting each numerical value type field value in the numerical value type field value set of the ith first type user data into a third field feature vector according to the numerical value type field value conversion sub-strategy. convert each numerical field value in the numerical field value set in the i-th first type user data into a third field feature vector according to the numerical field value conversion sub-strategy; concatenate the first field feature vector, the second field feature vector and the third field feature vector corresponding to each field value in the i-th first type user data to obtain an i-th first user feature vector of the i-th first type user data; increment i by 1 to update the value of i; if it is determined that i does not exceed N1, return to execute the steps of obtaining the i-th first type user data in the labeled sample set, and obtaining the text field value set, the category field value set and the numerical field value set of the i-th first type user data; if it is determined that i exceeds N1, save the first first user feature vector to the N1th first user feature vector. 4.The IoT potential user identification method of claim 1, wherein, After obtaining the several groups of labeled sets corresponding to the unlabeled sample set, respectively training the LightGBM model to be trained to obtain the LightGBM model corresponding to the several groups of labeled sets, and composing the classification model set by the LightGBM model corresponding to the several groups of labeled sets, the method further comprises: optimizing each LightGBM model in the classification model set through five-fold cross-validation to update the classification model set. 5.The IoT potential user identification method of claim 1, wherein, After obtaining the average value of the classification results in the classification result set as the classification result average value, the method further comprises: if it is determined that the classification result average value is less than or equal to the classification threshold, setting the label value of the user data to be predicted as the second label value.
6. An Internet of Things potential user identification apparatus characterized by, The method comprises: a data set initial obtaining unit configured to obtain a first data set and a second data set; wherein the first type user data in the first data set and the second type user data in the second data set both comprise a user unique identification code; a data set classification unit configured to obtain target first type user data in the first data set having the user unique identification code of the second type user data to compose a labeled sample set, and obtain an unlabeled sample set from the complement of the labeled sample set with respect to the first data set; a user feature vector obtaining unit configured to convert each piece of first type user data in the labeled sample set into a corresponding first user feature vector according to a preset field value conversion strategy, and convert each piece of first type user data in the unlabeled sample set into a corresponding second user feature vector according to the field value conversion strategy; a data labeling unit configured to label each piece of first type user data in the unlabeled sample set according to a preset labeling strategy and the labeled sample set for several times to obtain several groups of labeled sets corresponding to the unlabeled sample set; a model set training unit configured to obtain the several groups of labeled sets corresponding to the unlabeled sample set, respectively train a LightGBM model to be trained, obtain the LightGBM model corresponding to the several groups of labeled sets, and compose a classification model set by the LightGBM model corresponding to the several groups of labeled sets. The classification result set obtaining unit is configured to obtain to-be-predicted user data, and obtain a to-be-predicted user feature vector corresponding to the to-be-predicted user data according to the field value conversion strategy, input the to-be-predicted user feature vector into each LightGBM model in the classification model set for operation, and obtain a classification result corresponding to each LightGBM model to form a classification result set; The classification result average value obtaining unit is configured to obtain an average value of the classification results in the classification result set as a classification result average value; and The first labeling unit is configured to set a labeling value of to-be-predicted user data as a first labeling value if it is determined that the classification result average value is greater than a preset classification threshold value. The first data set is enterprise business data, and the second data set is Internet of Things operator user data. The method comprises the following steps: obtaining a k value, and generating a kth random number in a value range of [0, 1]; wherein the initial value of k is 1, and the value range of k is [1, N3], and N3 is a preset total number of labeling times; obtaining a jth second user feature vector corresponding to jth first type user data in the unlabeled sample set; wherein the initial value of j is 1, and the value range of j is [1, N2], and N2 is a total number of first type user data included in the unlabeled sample set; obtaining a vector similarity value between the jth second user feature vector and other second user feature vectors corresponding to other first type user data in the unlabeled sample set, and obtaining a vector similarity value between the jth second user feature vector and first user feature vectors corresponding to first type user data in the labeled sample set, to obtain a jth vector similarity set; obtaining target vector similarity values in the jth vector similarity set whose descending order ranking values do not exceed a stored ranking threshold value, and target user feature vectors corresponding to the target vector similarity values, to form a jth target user feature vector set; obtaining a jth positive example proportion in the jth target user feature vector set according to a preset positive example determination strategy, and performing numerical value comparison between the jth positive example proportion and a kth random number in a value range of [0, 1] to obtain a jth comparison result; if the jth comparison result is that the jth positive example proportion is greater than or equal to the random number, setting a labeling value of the jth first type user data as 1; if the jth comparison result is that the jth positive example proportion is less than the random number, setting the labeling value of the jth first type user data as -1; increasing j by 1 to update the value of j; if it is determined that j does not exceed N2, returning to perform the step of obtaining a jth second user feature vector corresponding to jth first type user data in the unlabeled sample set; If it is determined that j exceeds N2, the label value of the first type of user data No. 1 is combined with the corresponding second user feature vector after the label value of the first type of user data No. N2 is combined with the corresponding second user feature vector, and then saved as the kth group of label sets of the unlabeled sample set; The value of k is updated by increasing k by 1; If it is determined that k does not exceed N3, the step of obtaining the value of k and randomly generating the kth group of random numbers with a value range of [0, 1] is returned to be executed; If it is determined that k exceeds N3, the first group of label sets of the unlabeled sample set and the N3th group of label sets of the unlabeled sample set are obtained and saved.
7. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the Internet of Things potential user identification method in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program causes the processor to execute the Internet of Things potential user identification method in any one of claims 1 to 5 when the processor executes the computer program.
Citation Information
Patent Citations
Target customer determination method and device
CN112835958A
Personnel selection method and device based on neural network and tree model, and equipment
CN113240213A
Information recommendation method and device based on machine learning, equipment and storage medium
CN113656694A