Learning apparatus, learning method, and learning program

The learning device enhances CV prediction accuracy by categorizing user information and calculating category intensity, addressing false positives and improving model precision across diverse contexts.

JP2024165686APending Publication Date: 2024-11-28LY CORP
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2023082077
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing models for predicting conversion values (CV) based on user attributes and behavior history suffer from decreased accuracy due to false positives, particularly when user information similarities lead to inaccurate predictions across various industries and products.

Method used

A learning device that extracts and classifies user information into categories, calculates the intensity of each category based on user behavior, and learns a classification model to enhance prediction accuracy by incorporating the strength of each category.

Benefits of technology

The solution suppresses the decrease in CV prediction accuracy by generating a classification model that provides highly accurate predictions even for users with similar feature amounts, improving learning efficiency and accuracy across different industries and products.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165686000001_ABST
    Figure 2024165686000001_ABST
Patent Text Reader

Abstract

To suppress deterioration in the accuracy of a CV prediction value based on a model.SOLUTION: A learning apparatus 100 extracts user information of a predetermined type from a plurality of pieces of user information related to a predetermined user. The learning apparatus 100 classifies the extracted user information of the predetermined type into a plurality of categories. The learning apparatus 100 calculates the intensity of each category based on a user's behavior using a classification result. The learning apparatus 100 learns a classification model for classifying the predetermined user on the basis of the plurality of pieces of user information and the intensity of each category.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] There are known techniques for estimating user behavior based on information about the attributes and behavioral history of users who use the Internet. For example, there is known a technique for calculating predicted values ​​of various conversions (hereinafter, abbreviated as "CV") such as purchases, using the similarity between a plurality of search actions derived based on the contents of the user's search actions (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-020939 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned technology has a problem that the accuracy of the CV prediction value based on the model may decrease. For example, in the past, a model was trained using information on the attributes and behavioral history of a user as learning data, and a CV prediction value was calculated based on the model. However, the CV prediction value calculated based on the model may include a prediction value related to a user who does not actually reach CV as a false positive, resulting in a decrease in the accuracy of the CV prediction value. [Means for solving the problem]

[0005] In order to solve the above problems and achieve the object, the learning device of the present invention is characterized by comprising an extraction unit that extracts a predetermined type of user information from a plurality of user information related to a predetermined user, a classification unit that classifies the predetermined type of user information extracted by the extraction unit into a plurality of categories, a calculation unit that calculates an intensity of each category based on user behavior using a result of classification by the classification unit, and a learning unit that learns a classification model that classifies the predetermined user based on the plurality of user information and the intensity of each category. Effect of the Invention

[0006] The present invention has the effect of suppressing a decrease in accuracy of a model-based CV prediction value. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing an overview of model learning and prediction according to this embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the learning device according to this embodiment. [Diagram 3] FIG. 3 is a table diagram showing an example of user information according to the present embodiment. [Figure 4] FIG. 4 is a table diagram showing an example of the number of CVs for each category according to the present embodiment. [Diagram 5] FIG. 5 is a table showing an example of predetermined types of categories and strength classifications according to the present embodiment. [Figure 6] FIG. 6 is a table illustrating an example of a comparison of CV scores according to this embodiment. [Figure 7] FIG. 7 is a table showing an example of CV prediction scores used in learning according to the modified example. [Figure 8] FIG. 8 is a flowchart showing an example of a model learning procedure according to this embodiment. [Figure 9] FIG. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the learning device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, the present embodiment will be described in detail with reference to the drawings. Note that the following description does not limit the learning device, learning method, and learning program according to the present embodiment. In addition, in the following embodiment, the same components are given the same reference numerals, and duplicated descriptions are omitted.

[0009] [1. Introduction] First, an overview of model learning according to this embodiment and CV prediction based on the model will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of model learning and prediction according to this embodiment.

[0010] Conventionally, CV prediction may be performed based on a model that uses information about users who use the Internet or services on the Internet (hereinafter, simply referred to as "user information") as explanatory variables and outputs a predetermined CV prediction value as a response variable. Note that in this embodiment, CV prediction refers to predicting the degree to which CV will occur as a result of user actions such as viewing or clicking on content, reaching a specified page (outcome point), completing a purchase, completing a request for information, etc.

[0011] However, it is known that the accuracy of CV prediction based on a model can decrease for various reasons. For example, as shown in (1-1) and (1-2) of Figure 1, when user A and user B have the same or similar attribute information or features, such as gender "male", age "25", occupation "office worker", and interest "automobiles", the accuracy of CV prediction can decrease. In such a situation, when CV prediction is performed based on user information, a false positive prediction such as "there is a possibility of CV occurrence" may be made even for users who would not normally generate CV due to similarity in user information.

[0012] On the other hand, it is known that the deterioration in accuracy of CV prediction caused by the above-mentioned false positives varies depending on the category such as the target industry and the type of product, and it is difficult to obtain uniformly high-accuracy CV prediction results for multiple targets with CV prediction based on a conventional model. Therefore, in order to improve the accuracy of CV prediction, it is necessary to perform tuning so that the accuracy of the model is improved for each category, but there are problems in that performing tuning each time is a heavy load and the accuracy is not sufficiently improved. Therefore, the learning device 100 according to the present embodiment provides a model learning technology that can suppress the deterioration in accuracy of CV prediction and obtain highly accurate prediction results.

[0013] In this embodiment, the "model" will be described as a "classification model" that classifies users based on user information and the strength of each category, and outputs a predetermined score such as a CV prediction score. However, the above-mentioned classification model is merely an example, and the model that the learning device 100 according to this embodiment learns is not limited to a classification model, and may be, for example, a learning model, a calculation model, or the like.

[0014] The above-mentioned "category" refers to a grouping in which user information is classified according to a predetermined classification, for example, industry categories such as automobiles and human resources. The above-mentioned "strength" refers to the frequency of user information acquired based on the behavior of users classified by category, and is an index indicating the degree of distribution of indices such as number of times, frequency, and ratio. In this embodiment, the strength includes the meaning of "strength of the connection between the user and the category" and "strength of the relationship between the user and the category" in addition to the meaning of "degree of strength". In this embodiment, the "strength" may be read as "frequency". In this case, the "frequency" includes the meaning of a numerical value indicating the relationship between the user and the category, such as "frequency (score) indicating the relationship between the user and the category" and "frequency (score) indicating the user's interest in the category".

[0015] As shown in FIG. 1, the learning device 100 learns a classification model that calculates a CV prediction value using “user information” including user attribute information, information about behavioral history, and the like, and “strength for each category” ((2) in FIG. 1).

[0016] Next, the learning device 100 outputs the trained classification model ((3) in FIG. 1). For example, the learning device 100 can output the classification model trained based on the above-described learning method to a prediction device 200 that performs CV prediction, etc. Then, the prediction device 200 performs CV prediction using the classification model trained by the learning device 100 ((4) in FIG. 1). Note that, if the learning device 100 has a function of performing CV prediction, it can perform CV prediction based on the trained classification model ((5) in FIG. 1).

[0017] The prediction device 200 outputs prediction results as shown in (6-1) and (6-2) of Fig. 1. For example, the prediction device 200 predicts the CV score (car purchase) for user A to be "0.9" ((6-1) of Fig. 1). On the other hand, the prediction device 200 predicts the CV score (car purchase) for user B to be "0.5" ((6-2) of Fig. 1).

[0018] In this way, the learning device 100 uses the "strength for each category" calculated based on the user information classified into a predetermined category in addition to the user information to learn the classification model, thereby making it possible to generate a classification model capable of predicting CV with higher accuracy than ever before. Therefore, the classification model learned by the learning device 100 according to this embodiment enables highly accurate CV prediction even between users with high similarity in features. Therefore, the learning device 100 provides the effect of enabling highly accurate CV prediction regardless of the target industry, product, etc.

[0019] 2. Configuration of the learning device Next, the configuration of the learning device 100 according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the learning device 100 according to the embodiment. As shown in Fig. 2, the learning device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in Fig. 2, the learning device 100 may also have an input unit (e.g., a touch panel, a keyboard, a mouse, etc.) that accepts various operations.

[0020] (Communication unit 110) The communication unit 110 is connected to a network via a wired or wireless connection, and transmits and receives information between the learning device 100 and an external device. The communication unit 110 is realized by a network adapter, a network interface card, or the like. For example, although not shown in FIG. 2, the learning device 100 can output a trained classification model to the prediction device 200 via the communication unit 110.

[0021] (Storage unit 120) The storage unit 120 stores and holds programs for the learning device 100 to function, various information for each functional unit of the control unit 130 to perform processing, and the like. The storage unit 120 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 further includes a user information storage unit 121, an intensity information storage unit 122, and a model storage unit 123. The following items will explain each of the functional units of the storage unit 120.

[0022] (User information storage unit 121) The user information storage unit 121 stores information (user information) about users of the Internet and services provided on the Internet. Specifically, the user information storage unit 121 stores, as the user information, attribute information such as demographic data and psychographic data of the user, history information such as purchase history, browsing history, search history, and the like.

[0023] For example, the user information storage unit 121 can store demographic data such as the user's age, sex, family structure, occupation, educational background, annual income, and life stage as the user information. The user information storage unit 121 can also store psychographic data such as the user's values, personality, interests, concerns, hobbies, tastes, inclinations, preferences, habits, and the like as the user information. The user information storage unit 121 can also store purchase history such as purchase date and time, purchased products, product genre, price, number of purchases, and purchase store as the user information. The user information storage unit 121 can also store browsing history such as viewed websites, browsing date and time, number of searches and frequency of searches within a predetermined period as the user information. The user information storage unit 121 can also store search history such as search date and time in keyword searches, search keywords, number of searches and frequency of searches within a predetermined period as the user information.

[0024] Here, an example of user information stored by the user information storage unit 121 will be described with reference to FIG. 3. FIG. 3 is a table diagram showing an example of user information according to this embodiment. As shown in FIG. 3, the user information storage unit 121 can store age, sex, family structure, occupation, educational background, annual income, residential area, interests, purchase history, browsing history, and the like, which are identified as items by "user identification information". For example, the user information storage unit 121 can store user identification information "User A", age "25", sex "male", family structure "1 person", occupation "company employee", educational background "graduated from XX University", annual income "5 million yen", residential area "Tokyo", interests "automobiles", purchase history "AAA", and browsing history "BBB".

[0025] It should be noted that the above content is merely an example, and the user information storage unit 121 can store any information that falls within the category of user information related to a specific user, without any limitations. The above-mentioned purchasing behavior item is described as "AAA" as an example, but may include purchase date and time, purchased product, product genre, price, number of purchases, purchase store, etc. On the other hand, the above-mentioned browsing history item is described as "BBB" as an example, but may include the viewed website, viewed date and time, number of views and viewing frequency within a specific period, etc.

[0026] (Strength information storage unit 122) Returning to Fig. 2, the description will be continued. The intensity information storage unit 122 stores information on the intensity for each category calculated by a calculation unit 134 described later. Specifically, the intensity information storage unit 122 can store an intensity based on the user's behavior for each item of a predetermined category, based on the user information classified into the predetermined category by a classification unit 133 described later.

[0027] Here, an example of the strength of each category stored by the strength information storage unit 122 will be described with reference to FIG. 4. FIG. 4 is a table diagram showing an example of the number of CVs for each category according to this embodiment. As shown in FIG. 4, the strength information storage unit 122 can store the strength of each category identified by predetermined identification information in a list format. In the example of FIG. 4, the strength information storage unit 122 can store "user identification information" as the predetermined identification information and "the number of CVs for each category from C1 to C30" as the strength for each predetermined category. For example, the strength information storage unit 122 can store information such as C2 being "8" and C27 being "16" for the number of CVs for each category identified by the user identification information "user A".

[0028] 4 is merely an example, and the contents and format of the strength for each category stored by the strength information storage unit 122 are not limited to this. For example, the strength information storage unit 122 can use a different type of identification information, such as "company identification information", instead of "user identification information", as the predetermined identification information. Furthermore, the strength information storage unit 122 can store different intensities, such as "predicted CVR (Conversion Rate)" or "index showing user interests", instead of "number of conversions", as the strength for each category.

[0029] 4, the types of categories are expressed as C1 to C30, etc., but this expression is merely an example, and the strength information storage unit 122 can store predetermined categories set in advance, such as classification of industry, business type, user's hobbies and preferences, etc. Also, the types of categories described above are 30, from C1 to C30, but are not limited to this, and may be, for example, less than 30 or more than 30.

[0030] Here, an example of categories and intensity classifications stored in the intensity information storage unit 122 will be described with reference to Fig. 5. Fig. 5 is a table diagram showing an example of predetermined types of categories and intensity classifications according to this embodiment.

[0031] As shown in Fig. 5, the strength information storage unit 122 can store the strength of each category for various types of user information. For example, as shown in Fig. 5, the strength information storage unit 122 can store the strength of each category for types of user information such as web content, search query, purchase history, clicked content, survey response results, visited place, and willingness to replace electronic devices.

[0032] Furthermore, the strength information storage unit 122 can store the strength of each category based on the type of category and the strength classification corresponding to the type of user information. For example, when the type of user information is "web content", the strength of each category can be stored with the category type being "web content category" and the strength classification being "number of accesses". Note that the correspondence between the category type and the strength classification for each type of other user information is as shown in FIG. 5, and therefore will not be described here.

[0033] (Model storage unit 123) Returning to Fig. 2, the description will be continued. The model storage unit 123 stores a predetermined model that outputs a predetermined CV predicted value as a response variable, using user information and the strength of each category as explanatory variables. For example, the model storage unit 123 can store, as the predetermined model, a classification model that classifies predetermined users based on multiple pieces of user information and the strength of each category.

[0034] (Control unit 130) The control unit 130 includes an acquisition unit 131, an extraction unit 132, a classification unit 133, a calculation unit 134, a learning unit 135, and an output unit 136. The control unit 130 is realized by a processor, a micro processing unit (MPU), a central processing unit (CPU), or the like, executing various programs stored in the storage unit 120 of the learning device 100 using a RAM as a working area. The control unit 130 is realized by an integrated circuit (IC), such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0035] (Acquisition part 131) The acquiring unit 131 acquires user information to be used for learning the classification model. For example, the acquiring unit 131 can acquire user information stored in advance in an external device or the like. Note that the method by which the acquiring unit 131 acquires the user information is not particularly limited, and for example, the acquiring unit 131 may acquire the user information by accepting user information input via a storage medium or the like.

[0036] (Extraction part 132) The extraction unit 132 extracts a predetermined type of user information from a plurality of pieces of user information related to a predetermined user. Specifically, the extraction unit 132 extracts the user information from the user information stored in the user information storage unit 121 based on a predetermined extraction condition.

[0037] As an example, a case will be described in which the learning device 100 learns a classification model used to calculate a CV prediction value with "viewing a website related to automobiles" as a conversion condition. In this case, the extraction unit 132 can extract age, sex, family structure, occupation, educational background, annual income, residential area, interests, purchase history, browsing history, etc. from the user information based on the user's identification information. Note that the above-mentioned extracted items are merely examples, and the extraction unit 132 can extract user information according to the target of CV prediction.

[0038] (Classification Department 133) The classification unit 133 classifies the user information of a predetermined type extracted by the extraction unit 132 into a plurality of categories. Specifically, the classification unit 133 classifies the user information extracted by the extraction unit 132 into categories according to the target of CV prediction.

[0039] For example, when the learning device 100 learns a classification model related to "calculation of CV prediction value of browsing of websites related to automobiles", the classification unit 133 can classify, by category, the "number of accesses" included in the "browsing history" extracted by the extraction unit 132. Note that the above-mentioned items to be classified are merely examples, and the classification unit 133 can classify, by category, information included in the user information extracted by the extraction unit 132, such as age, sex, family structure, occupation, educational background, annual income, residential area, interests, purchase history, etc.

[0040] (Calculation unit 134) The calculation unit 134 calculates the strength of each category based on the user's behavior, using the result of the classification by the classification unit 133. Specifically, the calculation unit 134 calculates the strength of each category based on the user information classified into categories by the classification unit 133.

[0041] For example, when the learning device 100 learns a classification model related to “calculating the predicted CV value of browsing of automobile-related websites,” the calculation unit 134 can calculate the strength of each category for the “number of accesses” classified by the classification unit 133 described above.

[0042] Here, the calculation of the strength of each category by the calculation unit 134 will be described with reference to Fig. 4 again. For example, when the number of conversions for each category shown in Fig. 4 is the "number of accesses", the calculation unit 134 can calculate the number of accesses (strength) for each category, such as C2 being "8" and C27 being "16" for the number of conversions for each category identified by the user identification information "user A". Note that the above-mentioned items to be classified are merely examples, and the calculation unit 134 can calculate the strength of each category based on information included in the user information classified by the classification unit 133, such as age, sex, family structure, occupation, educational background, annual income, residential area, interests, purchase history, etc.

[0043] (Learning Section 135) 2 again, the description will be continued. The learning unit 135 learns a classification model for classifying a predetermined user based on a plurality of pieces of user information and the strength of each category. Specifically, the learning unit 135 learns the classification model using the user information acquired by the acquisition unit 131 and the strength of each category calculated by the calculation unit 134 as learning data.

[0044] For example, the learning unit 135 can learn a classification model related to "calculating the predicted CV of browsing of automobile-related websites" by using "browsing history of automobile-related websites" as user information and "strength of each category based on the number of accesses to automobile-related websites" as the strength of each category.

[0045] Furthermore, the learning unit 135 may learn the classification model using learning data that has been weighted in a predetermined manner. Specifically, the learning unit 135 estimates the importance of each of a plurality of pieces of user information and the strength of each category. Then, the learning unit 135 learns the classification model using at least one of the plurality of pieces of user information whose importance satisfies a predetermined condition or the strength of each category.

[0046] For example, when the learning device 100 learns a classification model related to "calculation of a predicted CV value of browsing of a website related to automobiles", the learning unit 135 can weight specific user information (e.g., annual income, interests, etc.) so that the importance of the information is high. In this way, the learning unit 135 can improve the learning efficiency by weighting specific user information and learning the classification model based on user items that affect "calculation of a predicted CV value of browsing of a website related to automobiles".

[0047] Regarding the estimation of the importance, the learning unit 135 can estimate the importance of each of the multiple pieces of user information and the strength of each category based on ridge regression or lasso regression. Note that the ridge regression or lasso regression referred to here is a well-known technique, and therefore a detailed description thereof will be omitted.

[0048] Furthermore, the learning unit 135 can learn the classification model using the strength of each category that changes over time. Specifically, the learning unit 135 can learn the classification model using the strength of each category based on a predetermined period such as "three months" or a predetermined time such as "March." For example, when the learning device 100 learns a classification model related to "calculating the CV predicted value of browsing of a website related to automobiles," the learning unit 135 can learn more efficiently by using the strength of each category in March, when the number of automobiles sold is high.

[0049] The classification model may be trained by appropriately using various known techniques related to machine learning. For example, the classification model may be trained by using a technique related to machine learning with supervision, such as SVM (Support Vector Machine). The classification model may be trained by using a technique related to machine learning with unsupervised learning. The classification model may be trained by using a deep learning technique. The classification model may be trained by appropriately using various deep learning techniques, such as DNN (Deep Neural Network), RNN (Recurrent Neural Network), and CNN (Convolutional Neural Network). The above description of the classification model training is an example, and the classification model may be trained by a learning method appropriately selected according to obtainable information, etc.

[0050] (Output unit 136) The output unit 136 outputs the trained classification model trained by the training unit 135. For example, the output unit 136 can output the trained classification model to a prediction device 200 that performs CV prediction, etc. Note that, regarding the output of the classification model by the output unit 136, there is no particular limitation on the output destination or output format.

[0051] 3. Experimental Example From here, the fluctuation in accuracy of CV predicted values ​​due to learning of a classification model realized by the learning device 100 according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a table diagram showing an example of a comparison of CV scores according to this embodiment. Specifically, Fig. 6 is a table diagram showing the fluctuation in ROC_AUC for each predetermined industry and for each piece of learning data (feature amount) used to learn the classification model.

[0052] 6 indicates a classification of industries such as automobiles, consumer goods, human resources, etc. In the present embodiment, the industry is expressed as "industries A to H" as an example, but in reality, the present invention is not limited to this.

[0053] The "number of features" indicates the type of data used as training data. For example, the number of features "conventional" is training data using only user information. The number of features "conventional + strength per category" is training data using user information and the calculated strength per category. The number of features "strength per category" is training data using the strength per category calculated by the calculation unit 134. The numerical value written in the item "number of features" in FIG. 6 is the number of items or categories of user information used as training data, or the total value of the number of items and categories of user information.

[0054] "ROC_AUC" means the AUC (Area Under an ROC Curve) in a ROC (Receiver Operating Characteristic curve). AUC means the area under an ROC curve, and it can be said that the closer the AUC value is to 1, the higher the accuracy (i.e., in this embodiment, the lower the false positive rate). Note that "conventional", "conventional + strength for each category", and "strength for each category" in ROC_AUC are the same as those described above, so they will be omitted.

[0055] As shown in Fig. 6, the calculation result of the CV predicted value by the classification model trained by the learning device 100 according to the present embodiment using the strength of each category was more accurate (i.e., the false positive rate was lower) than the conventional technology. For example, for the industry "Industry A", the conventional ROC_AUC was "0.890", while the conventional + strength of each category was "0.925" and the strength of each category was "0.911", resulting in the ROC_AUC being closer to 1. The above results show a similar tendency in each industry except for industries C and D, and it can be said that the classification model generated by the learning device 100 according to the present embodiment has the effect of improving the accuracy of the CV predicted value.

[0056] In addition, regarding the number of features used as learning data, the number of features used in "conventional + strength of each category" and "strength of each category" is smaller than that in "conventional," so it can be said that the learning device 100 of this embodiment is capable of learning classification models more efficiently.

[0057] 4. Modifications From here, a modified example of the learning of the classification model realized by the learning device 100 according to the present embodiment will be described. As a modified example, the learning device 100 (extraction unit 132) extracts at least one of web content browsing history, search query, purchase history, clicked content, survey response results, visited places, and willingness to replace electronic devices as a predetermined type of user information. In the following items, an example of learning for each of the above user information will be individually described.

[0058] (Web content browsing history) The learning device 100 can calculate the strength of each category using the browsing history of web content as user information and learn a classification model. Specifically, the extraction unit 132 extracts the browsing history of web content as a predetermined type of user information. The classification unit 133 classifies the browsing history of web content into categories related to multiple web contents. The calculation unit 134 calculates the strength of each category using at least one of the number of accesses and the predicted CVR as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the multiple pieces of user information and the strength of each category related to web content.

[0059] (Search query) Furthermore, the learning device 100 can calculate the strength of each category by using a search query as user information, and learn a classification model. Specifically, the extraction unit 132 extracts a search query as a predetermined type of user information. The classification unit 133 classifies the search query into categories related to a plurality of search queries. The calculation unit 134 calculates the strength of each category by taking the number of inputs as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user, based on the plurality of pieces of user information and the strength of each category related to the search query.

[0060] (Purchase history) Furthermore, the learning device 100 can calculate the strength of each category using purchase history as user information and learn a classification model. Specifically, the extraction unit 132 extracts purchase history as a predetermined type of user information. The classification unit 133 classifies the purchase history into categories related to a plurality of purchase histories. The calculation unit 134 calculates the strength of each category using the number of purchases as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the plurality of user information and the strength of each category related to the purchase history.

[0061] (Clicked content) Furthermore, the learning device 100 can calculate the strength of each category using clicked content as user information and learn a classification model. Specifically, the extraction unit 132 extracts clicked content as a predetermined type of user information. The classification unit 133 classifies the clicked content into categories related to multiple clicked contents. The calculation unit 134 calculates the strength of each category using the number of clicks as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on multiple pieces of user information and the strength of each category related to the clicked content.

[0062] (Survey response results) Furthermore, the learning device 100 can calculate the strength of each category using the questionnaire response results as user information and learn a classification model. Specifically, the extraction unit 132 extracts the questionnaire response results as a predetermined type of user information. The classification unit 133 classifies the questionnaire response results into categories related to multiple questionnaire response results. The calculation unit 134 calculates the strength of each category using the number of responses as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the multiple pieces of user information and the strength of each category related to the questionnaire response results.

[0063] (Places visited) Furthermore, the learning device 100 can calculate the strength of each category using visited places as user information and learn a classification model. Specifically, the extraction unit 132 extracts visited places as a predetermined type of user information. The classification unit 133 classifies visited places into categories related to multiple visited places. The calculation unit 134 calculates the strength of each category using the number of visits as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the multiple pieces of user information and the strength of each category related to visited places.

[0064] (Willingness to replace electronic devices) Furthermore, the learning device 100 can calculate the strength for each category using the willingness to replace electronic devices as user information, and learn a classification model. Specifically, the extraction unit 132 extracts the willingness to replace electronic devices as a predetermined type of user information. The classification unit 133 classifies the willingness to replace electronic devices into categories related to the willingness to replace multiple electronic devices. The calculation unit 134 calculates the strength for each category using the willingness score as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the multiple user information and the strength for each category related to the willingness to replace electronic devices. Note that the willingness score here may be an index that indicates the possibility that a user will perform a predetermined purchasing behavior.

[0065] (Strength by other categories) In this embodiment, the learning device 100 can use the CV prediction score or the intensity of a predetermined user's interest as the intensity for each category. Specifically, the calculation unit 134 calculates the intensity for each category using at least one of the CV prediction index or the intensity of a predetermined user's interest as the intensity.

[0066] Here, the above-mentioned CV prediction index (CV prediction score) will be described with reference to FIG. 7. FIG. 7 is a table diagram showing an example of a CV prediction score used for learning according to a modified example. FIG. 7 shows a CV prediction score for each user and for each company. Note that the CV prediction score here is an index indicating the possibility of whether or not each user will achieve a predetermined conversion condition set for each target company. In other words, the CV prediction score can be rephrased as the degree of interest in the target company.

[0067] In the example shown in FIG. 7, four companies (Company A to Company D) in the same industry are shown. The conversion prediction score for each company is shown for each user. For example, the conversion prediction scores for User A are shown as "0.9" for Company A, "0.97" for Company B, "0.73" for Company C, and "0.77" for Company D. The conversion prediction scores for User B are shown as "0.9" for Company A, "0.86" for Company B, "0.78" for Company C, and "0.62" for Company D. The conversion prediction scores for User C are shown as "0.9" for Company A, "0.82" for Company B, "0.99" for Company C, and "0.86" for Company D. The conversion prediction scores for User D are shown as "0.9" for Company A, "0.85" for Company B, "0.85" for Company C, and "0.74" for Company D. As this shows, even within the same industry, users have different levels of interest in each company.

[0068] Therefore, by using the strength of each category, including the CV prediction score as described above, as learning data, the learning device 100 can learn a classification model while reflecting the degree of user interests that are not reflected in the user's attribute information.

[0069] 5. Processing Procedure Next, a procedure for learning a classification model by the learning device 100 according to an embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of a learning procedure for a classification model according to the present embodiment. Note that the steps described below may be performed in a different order, and some processes may be omitted.

[0070] First, the acquisition unit 131 acquires user information related to a predetermined user (step S101). For example, the acquisition unit 131 can acquire user information previously stored in an external device or the like. Next, the extraction unit 132 extracts a predetermined type of user information from the user information (step S102). For example, the extraction unit 132 extracts user information from the user information stored in the user information storage unit 121 based on preset extraction conditions.

[0071] The classification unit 133 classifies a predetermined type of user information into a plurality of categories (step S103). For example, the classification unit 133 classifies the user information extracted by the extraction unit 132 into categories according to the target of CV prediction. Next, the calculation unit 134 calculates the strength of each category based on the result of the classification (step S104). For example, the calculation unit 134 calculates the strength of each category based on the user information classified into categories by the classification unit 133.

[0072] The learning unit 135 learns a classification model for classifying a predetermined user based on a plurality of pieces of user information and the strength of each category (step S105). For example, the learning unit 135 learns the classification model using the user information acquired by the acquisition unit 131 and the strength of each category calculated by the calculation unit 134 as learning data.

[0073] If the predetermined learning end condition is not satisfied (No in step S106), the learning device 100 returns to the previous step and continues the process. Note that the predetermined learning end condition may be, for example, the number of times learning has been performed, whether or not an index indicating the accuracy of the model exceeds a threshold, etc.

[0074] On the other hand, if the predetermined learning end condition is satisfied (Yes in step S106), the output unit 136 outputs the classification model (step S107). Then, the learning device 100 ends the process.

[0075] 6. Effects As described above, the extraction unit 132 of the learning device 100 of this embodiment extracts a predetermined type of user information from a plurality of pieces of user information related to a predetermined user. Next, the classification unit 133 of the learning device 100 classifies the predetermined type of user information extracted by the extraction unit 132 into a plurality of categories. Next, the calculation unit 134 of the learning device 100 calculates the strength of each category based on the user's behavior using the result of the classification by the classification unit 133. Then, the learning unit 135 of the learning device 100 learns a classification model that classifies the predetermined user based on the plurality of pieces of user information and the strength of each category. Therefore, the learning device 100 according to this embodiment provides the effect of suppressing a decrease in the accuracy of a CV prediction value based on a model.

[0076] The learning unit 135 of the learning device 100 estimates the importance of each of the multiple pieces of user information and the strength of each category. The learning unit 135 can estimate the importance of each of the multiple pieces of user information and the strength of each category based on ridge regression or lasso regression. The learning unit 135 then learns the classification model using at least one of the multiple pieces of user information whose importance meets a predetermined condition or the strength of each category. As described above, the learning device 100 according to this embodiment can improve the learning efficiency and accuracy of the classification model by weighting the user information and the strength of each category that have a large influence on the CV prediction. Therefore, the learning device 100 enables learning of a classification model that enables CV prediction with higher accuracy than before.

[0077] The learning unit 135 of the learning device 100 learns a classification model using the strength of each category that changes over time. The behavior of a user may change over time, for example, due to seasonal changes, time-of-day changes, changes in behavior due to changes in the user's life stage, etc. Therefore, the learning device 100 according to this embodiment uses the strength of each category that changes over time, thereby enabling learning of a classification model that enables CV prediction with higher accuracy than a classification model that learns using information of only one point in a continuous time series.

[0078] The extraction unit 132 of the learning device 100 extracts a browsing history of web content as a predetermined type of user information. Next, the classification unit 133 classifies the browsing history of web content into a plurality of categories related to web content. Next, the calculation unit 134 calculates the strength of each category by using at least one of the access count and the predicted CVR as the strength. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the plurality of pieces of user information and the strength of each category related to web content. Therefore, the learning device 100 according to this embodiment enables learning of a classification model based on a browsing history such as a viewed web site, a viewing date and time, a viewing count and a viewing frequency within a predetermined period, etc. As a result, the learning device 100 can provide a classification model that makes predictions with higher accuracy than before.

[0079] The extraction unit 132 of the learning device 100 extracts a search query as a predetermined type of user information. Next, the classification unit 133 classifies the search query into categories related to a plurality of search queries. Next, the calculation unit 134 calculates the intensity of each category using the number of inputs as the intensity. Then, the learning unit 135 learns a classification model that classifies a predetermined user based on the plurality of user information and the intensity of each category related to the search query. Therefore, the learning device 100 according to this embodiment enables learning of a classification model based on the user's search words and the number of inputs for each search word, that is, the user's interests. As a result, the learning device 100 can provide a classification model that makes predictions with higher accuracy than ever before.

[0080] The extraction unit 132 of the learning device 100 extracts at least one of purchase history, clicked content, survey response results, visited locations, and willingness to replace electronic devices as a predetermined type of user information. Therefore, the learning device 100 according to this embodiment is not limited to the web content and search queries described above, and can calculate the strength of each category based on various user information. Therefore, even when the accuracy of CV prediction varies depending on categories such as the target industry and type of product, the learning device 100 can provide a classification model that predicts with higher accuracy than before.

[0081] The calculation unit 134 of the learning device 100 calculates the strength of each category by using at least one of the CV prediction index and the strength of a predetermined user's interest as the strength. Therefore, the learning device 100 according to this embodiment can calculate the strength of each category based on the strength of the CV prediction for each user and the index related to the user's interests in addition to the user information. Therefore, the learning device 100 can provide a classification model that makes predictions with higher accuracy than before, even when the accuracy of CV prediction for each target category varies for each user.

[0082] [7. Hardware Configuration] The learning device 100 according to the embodiment described above is realized by a computer 1000 having a configuration as shown in Fig. 9, for example. Fig. 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the learning device 100. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I / F (Interface) 1060, an input I / F 1070, and a network I / F 1080 are connected by a bus 1090.

[0083] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050 and programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as a RAM, that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device that stores data used by the arithmetic device 1030 for various calculations and various databases, and is realized by a ROM (Read Only Memory), a HDD, a flash memory, or the like.

[0084] The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). The input I / F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, and scanner, and is realized by a USB, for example.

[0085] The input device 1020 may be a device that reads information from an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory, etc. The input device 1020 may also be an external storage medium such as a USB memory.

[0086] The network I / F 1080 receives data from other devices via the network NW and sends the data to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network NW.

[0087] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0088] For example, when the computer 1000 functions as the learning device 100, the arithmetic device 1030 of the computer 1000 realizes the functions of the control unit 130 of the learning device 100 by executing a program loaded onto the primary storage device 1040.

[0089] [8. Other] Although the details of this embodiment have been described above, this embodiment is not limited to the above contents. The above-mentioned components include those that a person skilled in the art can easily imagine, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-mentioned components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the gist of the above-mentioned embodiment.

[0090] In addition, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0091] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0092] In addition, the above-described embodiments can be combined as appropriate to the extent that the processing content is not contradictory, and "section," "module," or "unit" can be read as "means," "circuit," etc. [Explanation of symbols]

[0093] 100 Learning Device 110 Communications Department 120 Storage section 121 User information storage unit 122 Strength information storage unit 123 Model Memory Section 130 Control section 131 Acquisition Department 132 Extraction part 133 Classification Department 134 Calculation section 135 Learning Department 136 Output section 200 Prediction Device 1000 Computers 1010 Output device 1020 Input Device 1030 Arithmetic equipment 1040 Primary storage 1050 Secondary storage 1060 Output I / F 1070 Input I / F 1080 Network Interface 1090 Bus NW Network

Claims

1. An extraction unit that extracts a predetermined type of user information from a plurality of pieces of user information related to a predetermined user; a classification unit that classifies the predetermined type of user information extracted by the extraction unit into a plurality of categories; a calculation unit that calculates a strength of each category based on a user's behavior using a result of the classification by the classification unit; a learning unit that learns a classification model for classifying the predetermined user based on the plurality of pieces of user information and the strength of each category; A learning device comprising:

2. The learning unit estimates importance of each of the plurality of pieces of user information and a strength of each of the categories; training the classification model using at least one of the plurality of pieces of user information whose importance satisfies a predetermined condition or the strength of each category; 2. The learning device according to claim 1 .

3. the learning unit estimates importance of each of the plurality of pieces of user information and the strength of each category based on ridge regression or lasso regression; 3. The learning device according to claim 2.

4. The learning unit learns the classification model using the strength of each category that changes over time.

3. The learning device according to claim 1 or 2.

5. The extraction unit extracts a browsing history of web content as the predetermined type of user information, The classification unit classifies the browsing history of the web content into a plurality of categories related to the web content, the calculation unit calculates a strength for each of the categories using at least one of an access count and a predicted CVR as the strength.

2. The learning device according to claim 1 .

6. The extraction unit extracts a search query as the predetermined type of user information, The categorizing unit categorizes the search queries into a plurality of categories related to search queries; The calculation unit calculates a strength for each of the categories using the number of inputs as the strength.

2. The learning device according to claim 1 .

7. the extraction unit extracts at least one of a purchase history, clicked content, a questionnaire response result, a place visited, and a willingness to replace an electronic device as the predetermined type of user information; 2. The learning device according to claim 1 .

8. The calculation unit calculates an intensity for each of the categories using at least one of an index of CV prediction or an intensity of interest of the predetermined user as the intensity.

8. A learning device according to claim 1, 2, 5, 6 or 7.

9. An extraction step of extracting a predetermined type of user information from a plurality of pieces of user information related to a predetermined user; a classification step of classifying the predetermined type of user information extracted by the extraction step into a plurality of categories; a calculation step of calculating a strength of each category based on the user's behavior using the result of the classification by the classification step; a learning process of learning a classification model for classifying the predetermined user based on the plurality of pieces of user information and the strength of each category; A learning method comprising:

10. An extraction step of extracting a predetermined type of user information from a plurality of pieces of user information related to a predetermined user; a classification step of classifying the predetermined type of user information extracted by the extraction step into a plurality of categories; a calculation step of calculating a strength of each category based on a user's behavior using a result of the classification step; a learning procedure for learning a classification model for classifying the predetermined user based on the plurality of pieces of user information and the strength of each category; A learning program characterized by causing a computer to execute the above.

Citation Information

Patent Citations

  • Ordering conversion rate prediction method and device

    CN110858366A

  • Information analysis device, information analysis method, and information analysis program

    JP2016062411A

  • Prediction device, prediction method, prediction program, generation device, generation method and generation program

    JP2019164738A

  • Estimation device, estimation method and estimation program

    JP2020135770A

  • Information processing device, information processing method, and information processing program

    JP2021012547A