Information processing device, information processing method, and information processing program

Abstract

To provide an information processing apparatus, an information processing method, and an information processing program that can determine the quality of models for estimating users' attribute information.SOLUTION: An information processing apparatus according to the present application comprises: an acquisition unit that acquires user information indicating information on users and action information including users' search histories; a model storage unit that stores a plurality of index calculation models that have learned the characteristics of the users based on the user information and the action information; a calculation unit that calculates indices for the specific characteristics for every user by using the plurality of index calculation models; a questioning unit that asks a question related to whether the users correspond to the specific characteristics to the users for whom the indices for the specific characteristics are calculated; and a creation unit that creates a graph indicating the relationship between the indices for the specific characteristics of the plurality of users and the ratio between right and wrong as to whether the plurality of users correspond to the specific characteristics.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Information distribution based on information acquired via a network is becoming increasingly common. For example, targeting distribution is being carried out in which attribute information indicating the attributes of users to whom information is to be distributed is registered in advance as distribution destination information, and advertisements corresponding to the attribute information of the users to whom the information is to be distributed are selectively distributed.

[0003] As a technology for grasping attribute information of users to whom advertisements are to be delivered, for example, Patent Document 1 below discloses a technology for calculating the true relevance between an advertisement and attribute information based on the apparent degree of relevance between the advertisement and attribute information and the average degree of relevance between multiple advertisements and attribute information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-1956 Summary of the Invention [Problem to be solved by the invention]

[0005] When estimating such user attribute information, there is a need to evaluate whether the user attribute information can be accurately estimated and to determine whether the estimation is successful. There is also a need for a technology to improve the accuracy of estimation of user attribute information.

[0006] In view of the above-mentioned problems, the present disclosure aims to provide an information processing device, an information processing method, and an information processing program capable of determining the quality of a model that estimates attribute information of a user. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objectives, the information processing device according to the present disclosure includes an acquisition unit that acquires user information indicating information about a user and behavioral information including the user's search history, a model storage unit that stores a plurality of index calculation models that have been trained to learn user characteristics based on the user information and the behavioral information, a calculation unit that calculates an index for a specific characteristic for each user using the plurality of index calculation models, a questioning unit that asks users who have calculated an index for the specific characteristic whether or not they correspond to the specific characteristic, and a generation unit that generates a graph showing the relationship between the index for the specific characteristic of a plurality of users and the percentage of correct answers regarding whether or not the specific characteristic of a plurality of users corresponds to the specific characteristic. [Effects of the Invention]

[0008] According to one aspect of the embodiment, it is possible to provide an information processing device, an information processing method, and an information processing program capable of determining the quality of a model that estimates attribute information of a user. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of information processing according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of information stored in the user information storage unit of the information processing device according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of information stored in the behavior information storage unit of the information processing device according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of information stored in the model storage unit of the information processing apparatus according to the embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of information stored in the content storage unit of the information processing device according to the embodiment. [Figure 8] FIG. 8 is a diagram showing a first example of a graph generated by the generation unit of the information processing device according to the embodiment. [Figure 9] FIG. 9 is a diagram showing a second example of a graph generated by the generation unit of the information processing device according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of a user terminal according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of a carrier terminal according to the embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of information processing according to the embodiment. [Figure 13] FIG. 13 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the information processing device, the information processing method, and the information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing device, the information processing method, and the information processing program according to the present application are not limited to the embodiments.

[0011] (Embodiment) 1-1. Example of information processing according to the embodiment First, an example of information processing according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of information processing according to an embodiment. First, an overview of the information processing according to the embodiment will be described, and then each process will be described in detail.

[0012] 1 shows an example in which an information processing device 100 acquires user information and behavioral information from multiple user terminals 200, inputs the acquired user information and behavioral information into multiple index calculation models, calculates an index for a specific feature, asks the user whether or not the specific feature applies, generates a graph showing the relationship between the index for the specific feature and the percentage of correct answers obtained from the answers to the question, trains a discriminant model on the features of the generated graph, and uses the discriminant model to determine whether the index calculation model is good or bad. Below, an example of information processing according to an embodiment will be described in detail step by step using FIG. 1.

[0013] First, the information processing device 100 receives content and request information input by the provider M1 from the provider terminal 300 (step S1). For example, the information processing device 100 receives content input from the provider terminal 300 by the provider M1 for the purpose of distribution to users with specific characteristics. Furthermore, the information processing device 100 receives request information regarding distribution of content input by the provider M1 to the provider terminal 300 from the provider terminal 300. Note that the request information here refers to information regarding the provider's request for content distribution, and may be information including, for example, "specific characteristics" that indicate the characteristics of users to whom the provider wishes to distribute content.

[0014] Next, the information processing device 100 acquires user information and behavioral information from multiple user terminals 200 (step S2). For example, as shown in Fig. 1, the information processing device 100 acquires user information and behavioral information from user terminals 200A to 200F of users U1 to U6. Note that users U1 to U6 and user terminals 200A to 200F shown in Fig. 1 are examples, and the information processing device 100 may acquire user information and behavioral information from user terminals 200 of more users.

[0015] Next, the information processing device 100 calculates an index for a specific feature using multiple index calculation models (step S3). Note that the index calculation model here refers to an index calculation model that has learned the relationship between user information, behavioral information, and indexes related to the user's features, and when the user information and behavioral information of the user are input, it outputs an index for the user's specific feature as a value between 0 and 1. The specific feature can also be referred to as user attribute information, and is a concept that indicates the user's hobbies, preferences, and characteristics, such as "I like glasses" or "I like books." The information processing device 100 inputs the user information and behavioral information acquired from the multiple user terminals 200 into multiple index calculation models that calculate indexes for the specific features, and calculates an index for the user's specific feature for each index calculation model.

[0016] Next, the information processing device 100 transmits question information to the plurality of user terminals 200 as to whether or not the user U corresponds to a specific characteristic (step S4). For example, the information processing device 100 may transmit question information including a question as to whether or not the user U corresponds to the specific characteristic "likes glasses" to the user terminals 200 of the plurality of users for which the index for the specific characteristic was calculated in step S3, and the plurality of user terminals 200 that have received the question information may display the question information on the output unit 230, thereby asking the users for which the index for the characteristic was calculated in step S3 as to whether or not they correspond to the specific characteristic.

[0017] Next, the information processing device 100 receives answer information from the plurality of user terminals 200 as to whether or not the specific characteristic applies (step S5). For example, the information processing device 100 may receive answer information from the user terminal 200 that is input by a user U, who has confirmed a question displayed on the user terminal 200 as to whether or not the specific characteristic applies, via the input unit 220 of the user terminal 200. In this case, the answer information that the user U inputs into the input unit 220 of the user terminal 200 includes "yes" or "no." As shown in FIG. 1, the information processing device 100 receives answer information input by the user U from the plurality of user terminals 200.

[0018] Next, the information processing device 100 generates a graph showing the percentage of correct answers for the index of a specific feature for each of the multiple index calculation models (step S6). For example, the information processing device 100 may calculate the percentage of correct answers indicating whether or not a user corresponds to a specific feature for each index value for the specific feature based on the response information received from the multiple user terminals 200, and generate a graph showing the relationship between the score value indicating the index for the specific feature and the calculated percentage of correct answers. As shown in FIG. 1, since there are multiple specific features, such as "likes glasses" or "likes books," the information processing device 100 generates multiple graphs for each specific feature. In the example shown in FIG. 1, graphs A to C are shown, with the indexes calculated by models A to C on the horizontal axis and the percentage of correct answers for the index on the vertical axis.

[0019] Next, the information processing device 100 causes the discriminant model to learn the features of the graph of the good index calculation model (step S7). For example, the information processing device 100 may assign a label indicating the quality of the index calculation model to a graph showing the index and the rate of correctness for a specific feature generated for each of the multiple index calculation models, and cause the discriminant model to learn the features of the graph of the good index calculation model by having the discriminant model learn the features of the graph of the good index calculation model. In this case, the discriminant model may learn the range in which the good index calculation model can be used by assigning a label indicating the range in which the good index calculation model can be used, and cause the discriminant model to learn the features of the graph of the good index calculation model by having the discriminant model ....

[0020] Next, the information processing device 100 outputs whether the index calculation model is good or bad and the range in which a good index calculation model can be used by inputting the graph into the trained discriminant model (step S8). For example, as shown in Fig. 1, the information processing device 100 may input the graph X generated in step S6 into the trained discriminant model in step S7, causing the discriminant model to output whether the index calculation model is good or bad and the range in which a good index calculation model can be used.

[0021] Next, the information processing device 100 calculates an index related to a specific feature of the user using a good index calculation model, and determines a target of content distribution (step S9). For example, it is assumed that the discriminant model determines that index calculation model A is a good model among a plurality of index calculation models. In this case, the information processing device 100 may calculate an index related to a specific feature of a plurality of users using index calculation model A, and determine users whose calculated indexes are equal to or greater than a predetermined value as a target of content distribution.

[0022] Next, the information processing device 100 distributes the content to the user terminals 200 of the users determined to be the distribution targets (step S10). For example, based on information about the specific characteristics of the users to whom the content is to be distributed received from the provider terminal 300, the information processing device 100 distributes the content received from the provider M1 to the user terminal 200E of user U5 and the user terminal 200F of user U6, whose indexes for the specific characteristics have been calculated to be high.

[0023] This enables the information processing device 100 to determine whether an index calculation model is good or bad. Therefore, it is possible to provide an information processing device that can determine whether a model that estimates user attribute information is good or bad. Furthermore, after determining whether an index calculation model is good or bad, an index for a specific feature of a user is calculated using the index calculation model that is determined to be a good index calculation model, and users who are estimated to have the specific feature can be estimated with high accuracy, and content can be delivered to users who are highly likely to have the specific feature. Therefore, it is possible to meet the targeted delivery needs of advertisement delivery businesses with high accuracy.

[0024] 1-2. Another example of information processing according to an embodiment 1 The information processing device 100 outputs the range of the usable index calculation model as a numerical range including a minimum value and a maximum value, using the value of the index related to the specific feature of the user.

[0025] This information processing will be explained step by step. First, the information processing device 100 executes the same processing as steps S1 to S7 shown in Fig. 1. The processing from step S1 to step S7 is the same as the processing described above, and therefore the explanation will be omitted.

[0026] Next, the information processing device 100 executes the process of step S8 shown in Fig. 1. In this case, the information processing device 100 outputs not only the quality of the index calculation model but also the range of usable index calculation models as a numerical range. For example, as shown in graph A in Fig. 1, when the index of the index calculation model is in the numerical range from 0.4 to 0.8, the percentage of correct answers declines to the right relative to the index, and a one-to-one correspondence is obtained between the index and the percentage of correct answers. In this case, the information processing device 100 outputs the range of usable index calculation models as a numerical range from a minimum value of 0.4 to a maximum value of 0.8.

[0027] Next, the information processing device 100 executes the same processes as those from step S9 to step S10 shown in Fig. 1. In step S9, the information processing device 100 determines the distribution target of the content by using an index in a numerical range determined to have high estimation accuracy from among the index calculation models.

[0028] This allows the information processing device 100 to grasp the numerical range of the index that is evaluated as having high accuracy among the index calculation models, thereby making it possible to estimate with high accuracy users who are likely to have a specific characteristic and to narrow down content delivery to users estimated to have the specific characteristic.

[0029] 1-3. Another example of information processing according to the embodiment 2 The information processing device 100 calculates a moving average of the ratio of correct and incorrect values ​​for the index calculated by the index calculation model that was determined to be incorrect, and if the calculated moving average is declining relative to the index, it re-determines that the index calculation model is good.

[0030] This information processing will be explained step by step. First, the information processing device 100 executes the same processing as steps S1 to S6 shown in Fig. 1. The processing from step S1 to step S6 is the same as the processing described above, and therefore the explanation will be omitted.

[0031] Next, the information processing device 100 calculates a moving average of the percentage of correct errors for the index calculated by the index calculation model determined to be bad, and if the calculated moving average is declining to the right with respect to the index, re-determines the index calculation model as a good index calculation model (step S8). That is, the information processing device 100 calculates a moving average of the percentage of correct errors for the index calculated by the bad index calculation model using the discrimination model for the index calculation model determined to be a bad index calculation model, and determines whether the calculated moving average of the percentage of correct errors is declining to the right with respect to the index. If the calculated moving average of the percentage of correct errors is declining to the right with respect to the index, the information processing device 100 re-determines the index calculation model as a good index calculation model.

[0032] Next, the information processing device 100 executes the same processes as those from step S9 to step S10 shown in Fig. 1. In step S9, the information processing device 100 calculates an index for a specific feature of the user using an index calculation model, and determines a target of content distribution.

[0033] As a result, the information processing device 100 can determine the quality of an index calculation model that has been determined to be a bad model by the discriminant model based on the moving average of the accuracy rate, making it possible to revise the evaluation of an index calculation model that has been determined to be a bad index calculation model due to an error in the discriminant model.

[0034] 1-4. Another example 3 of information processing according to the embodiment The information processing device 100 asks the user again, in a specific numerical range of the index, the percentage of correct answers regarding the specific feature of the user in the numerical range in which the percentage is unknown.

[0035] This information processing will be explained step by step. First, the information processing device 100 executes the same processing as steps S1 to S6 shown in Fig. 1. The processing from step S1 to step S6 is the same as the processing described above, and therefore the explanation will be omitted.

[0036] Next, the information processing device 100 identifies users whose percentage of correct answers for a specific feature is unknown from a graph showing the percentage of correct answers for an index for the specific feature (step S6-2). For example, in the example of graph A shown in FIG. 1, users whose index value is between 0.8 and 1.0 have an unknown percentage of correct answers. The information processing device 100 identifies users whose index has an unknown percentage of correct answers from the graph generated in step S6, and retransmits question information regarding whether the identified users correspond to the specific feature to the user terminal 200. The information processing device 100 receives the answer information transmitted from the user terminal 200 and regenerates a graph showing the percentage of correct answers for an index for the specific feature.

[0037] Next, the information processing device 100 executes the same processes as those from step S7 to step S10 shown in FIG.

[0038] As a result, the information processing device 100 can accurately grasp the estimation accuracy of the index calculation model by asking the user again if the percentage of correct answers for a specific feature is unknown. Therefore, it is possible to accurately evaluate the estimation accuracy of the index calculation model, calculate an index related to the specific feature of the user using an index calculation model with high estimation accuracy, and deliver content to users who are estimated to have the specific feature with a high probability.

[0039] [2. Information Processing System Configuration] Next, the configuration of the information processing system according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing system according to the embodiment. As shown in Fig. 2, the information processing system 1 includes an information processing device 100, a user terminal 200, and a provider terminal 300. Note that the information processing system 1 shown in Fig. 2 may be configured to include a plurality of information processing devices 100, a plurality of user terminals 200, and a plurality of provider terminals 300. The information processing device 100, the user terminal 200, and the provider terminal 300 are connected to each other via a predetermined communication network (network N) so as to be able to communicate with each other via wired or wireless communication.

[0040] The information processing device 100 may be, for example, a personal computer (PC), a workstation (WS), a computer with server functions, etc. The information processing device 100 performs processing based on information transmitted from the user terminal 200 and the provider terminal 300 via the network N.

[0041] The user terminal 200 is an information processing device used by a user. The user terminal 200 may be, for example, an information processing device such as a smartphone, a tablet terminal, a desktop PC, a notebook PC, a mobile phone, or a PDA (Personal Digital Assistant). In the example shown in FIG. 1, the user terminal 200 is a smartphone.

[0042] The business operator terminal 300 is an information processing device used by a business operator. The business operator terminal 300 may be, for example, an information processing device such as a smartphone, a tablet terminal, a desktop PC, a notebook PC, a mobile phone, or a PDA. In the example shown in Fig. 1, the business operator terminal 300 is a notebook PC.

[0043] 3. Configuration of Information Processing Device Next, the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the information processing device according to the embodiment.

[0044] 3, the information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in FIG. 3, the information processing device 100 may also include an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information processing device 100, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0045] (Regarding the communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC) etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information between the user terminal 200 and the provider terminal 300.

[0046] (Regarding the storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk, a solid state drive (SSD), an optical disk, etc. As shown in Fig. 5, the storage unit 120 has a user information storage unit 121, a behavior information storage unit 122, a model storage unit 123, and a content storage unit 124.

[0047] (Regarding the user information storage unit 121) The user information storage unit 121 stores information about a user, i.e., user information. The user information is, for example, information about a user that is provided to an information service provider when the user uses a predetermined information service. Here, an example of information stored in the user information storage unit 121 will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of information stored in the user information storage unit of the information processing device according to the embodiment.

[0048] In the example shown in FIG. 4, the user information storage unit 121 stores information relating to the items "user ID," "date of birth," "gender," and "occupation" in association with each other.

[0049] "User ID" is an identifier that identifies a user and is represented by a string of characters or a number. "Date of birth" is information about the user's date of birth linked to the "User ID". "Gender" is information about the user's gender linked to the "User ID". "Occupation" is information about the user's occupation linked to the "User ID".

[0050] That is, FIG. 4 shows that the date of birth of the user identified by the user ID "UID#1" is "date of birth #U1", the gender is "female", and the occupation is "occupation #U1".

[0051] The information stored in the user information storage unit 121 is not limited to information relating to the items "user ID," "date of birth," "gender," and "occupation," but may also store any other information related to the user.

[0052] (Regarding the behavioral information storage unit 122) The behavioral information storage unit 122 stores information indicating the behavior of a user, i.e., behavioral information. The behavioral information is information indicating the behavior of a user using a predetermined information service, which is generated as a result of the user's use of the predetermined information service. Here, an example of information stored in the behavioral information storage unit 122 will be described with reference to FIG. 5. FIG. 5 is a diagram showing an example of information stored in the behavioral information storage unit of the information processing device according to the embodiment.

[0053] In the example shown in FIG. 5, the behavior information storage unit 122 stores information relating to the items "user ID," "search history," "browsing history," "purchase history," "route search history," and "bulletin board posting history" in association with each other.

[0054] "User ID" is an identifier that identifies a user and is represented by a string of characters, a number, etc. "Search history" is information that includes the search query used by a user linked to a "User ID" for a search and the time of entry. "Browsing history" is information that includes the sites viewed by a user linked to a "User ID" and the time of viewing. "Purchase history" is information that includes the products or services purchased by a user linked to a "User ID" on an internet mail order site or a specified service contract site and the time of purchase. "Route search history" is information that includes the route search results and the search time of a user linked to a "User ID". "Bulletin board posting history" is information that includes the bulletin board posts by a user linked to a "User ID" and the time of posting.

[0055] That is, in Figure 5, the search history of the user identified by the user ID "UID#1" is "Search History #U1", the user's browsing history is "Browse History #U1", the purchase history is "Purchase History #U1", the route search history is "Route Search History #U1", and the message board posting history is "Message board posting history #U1".

[0056] The information stored in the behavioral information storage unit 122 is not limited to information relating to the items "user ID," "search history," "browsing history," "purchase history," "route search history," and "bulletin board posting history," but may also store any other information relating to the behavior of the user.

[0057] (Regarding the model storage unit 123) The model storage unit 123 stores a plurality of index calculation models that learn user characteristics based on user information and behavioral information and calculate an index for a specific characteristic when the user information and behavioral information are input. The model storage unit 123 may also store a discrimination model that determines whether the index calculation model is good or bad. Fig. 6 is a diagram showing an example of information stored in the model storage unit of the information processing device according to the embodiment.

[0058] In the example shown in FIG. 6, the model storage unit 123 stores information relating to the items "model ID" and "model data" in association with each other.

[0059] "Model ID" is an identifier that identifies a machine learning model and is represented by a character string, a number, or the like. "Model data" indicates the model data of the machine learning model. For example, the machine learning model may be a neural network.

[0060] 6, the model identified by the model ID "M#1" indicates the machine learning model M#1. Also, model data "MDT#1" indicates the model data of the machine learning model M#1.

[0061] Here, if the machine learning model is a neural network, the model data "MDT#1" includes various information, such as connection information on how the nodes included in each of the multiple layers that make up the neural network are connected to each other, and connection coefficients that are multiplied by the numerical values ​​input and output between the connected nodes.

[0062] The model storage unit 123 is not limited to storing information related to the items "model ID" and "model data," and may store information related to any other machine learning model.

[0063] (Regarding the content storage unit 124) The content storage unit 124 stores content input by the provider via the provider terminal 300. Fig. 7 is a diagram showing an example of information stored in the content storage unit of the information processing device according to the embodiment.

[0064] In the example shown in FIG. 7, the content storage unit 124 stores information relating to the items "business operator ID," "content ID," "content data," and "specific feature" in association with each other.

[0065] "Provider ID" is an identifier that identifies a provider and is expressed as a string or number. "Content ID" is an identifier that identifies content received from a provider and is expressed as a string or number. "Content data" is data on the content that a provider wishes to distribute. "Specific characteristics" is information that indicates the specific characteristics of a user that serve as criteria for selecting users to whom a provider wishes to distribute content.

[0066] That is, in Figure 7, the content data indicated by the content data "CD#1" is submitted to the operator terminal 300 by the operator indicated by the operator ID "M1" as content identified by the content ID "CT#1", specifying a specific characteristic of the user indicated by the specific characteristic "FT#1", and is then accepted by the reception unit 132 of the information processing device 100 and stored in the content storage unit 124.

[0067] It should be noted that the content storage unit 124 is not limited to information relating to the items "Provider ID," "Content ID," "Content Data," and "Specific Features," and may store any other information relating to any content.

[0068] (Regarding the control unit 130) Next, returning to Fig. 3, the control unit 130 will be described. The control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs stored in a storage device of the information processing device 100 using RAM as a work area. The control unit 130 may also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), for example.

[0069] As shown in FIG. 3, the control unit 130 includes an acquisition unit 131, a reception unit 132, a calculation unit 133, a question unit 134, a generation unit 135, a learning unit 136, a discrimination unit 137, a determination unit 138, and a distribution unit 139.

[0070] (Regarding the acquisition unit 131) The acquisition unit 131 acquires user information indicating information about a user and behavioral information indicating the user's behavior. Here, the user information is information about a user, such as information provided to an information service provider when the user uses a predetermined information service. The behavioral information is information indicating the user's behavior using the predetermined information service, which is generated as a result of the user's use of the predetermined information service. After acquiring the user information, the acquisition unit 131 stores the acquired user information in the user information storage unit 121. After acquiring the behavioral information, the acquisition unit 131 stores the acquired behavioral information in the behavioral information storage unit 122. Note that the acquisition unit 131 may acquire either or both of the user information and the behavioral information at each predetermined date and time, or may acquire either or both of the user information and the behavioral information every time the user accesses the information processing device 100.

[0071] Furthermore, the source from which the acquisition unit 131 acquires user information and behavioral information is not limited to the user terminal 200, but may be acquired from other server devices that provide various information services such as search services, shopping services, payment services, route search services, map provision services, travel services, restaurant introduction services, weather forecast services, schedule management services, news provision services, auction services, video content distribution services, and financial trading (stock trading, etc.) services, or may be acquired from external storage media.

[0072] (Regarding the reception unit 132) The reception unit 132 receives content input by a business operator to the business operator terminal 300. When the reception unit 132 receives content from the business operator terminal 300, it stores the received content in the content storage unit 124. The reception unit 132 also receives request information input by the business operator together with the content. When the reception unit 132 receives the request information, it stores the request information in the content storage unit 124 in association with the content input by the business operator.

[0073] (Regarding the calculation unit 133) The calculation unit 133 uses a plurality of index calculation models to calculate an index for a specific feature of the user for each index calculation model. When user information and behavioral information are input, the calculation unit 133 inputs the user information and behavioral information into an index calculation model that calculates an index for the specific feature, thereby calculating an index for the specific feature of the user for each index calculation model. Note that the index calculation model used to calculate the index is read from the model storage unit 123 and used, and the index for the specific feature of the user is calculated for each index calculation model. Furthermore, the number of index calculation models used by the calculation unit 133 to calculate the index for the specific feature of the user may be any number.

[0074] (Regarding Question 134) The questioning unit 134 asks a user for whom an index related to a specific feature has been calculated whether or not the user corresponds to the specific feature. That is, as shown in FIG. 1 , the calculation unit 133 calculates an index related to a specific feature for each user, and therefore the questioning unit 134 asks a user for whom the calculation unit 133 has calculated an index related to a specific feature whether or not the user has the specific feature. That is, the questioning unit 134 transmits question information to the user terminal 200 of the user for whom the calculation unit 133 has calculated an index related to the specific feature, and receives answer information input by the user who has confirmed the question information displayed on the user terminal 200. Here, the question information may include a message such as "Do you correspond to the specific feature?" or "Do you have the specific feature?" Furthermore, the answer information indicates an answer to the question information and may include an answer such as "yes" or "no."

[0075] (Regarding the generation unit 135) The generation unit 135 generates a graph showing the relationship between the index for a specific feature of a plurality of users and the percentage of correct answers regarding whether or not the specific feature of a plurality of users applies. For example, the generation unit 135 generates a graph in which the horizontal axis shows the numerical value of the index and the vertical axis shows the percentage of correct answers. Note that the index is expressed by a numerical value ranging from 0 to 1.0, and the percentage of correct answers is also expressed by a numerical value ranging from 0 to 1.0. The graph generated by the generation unit 135 will be described with reference to Figs. 8 and 9.

[0076] 8 is a diagram showing a first example of a graph generated by the generation unit of the information processing device according to the embodiment. As shown in FIG. 8, the generation unit 135 generates a graph in which the horizontal axis represents a score indicating an index related to a specific feature calculated by the calculation unit 133, and the vertical axis represents a percentage of correct answers indicating the percentage of "yes" answers included in the answer information to the question from the questioning unit 134. In the example of the graph shown in FIG. 8, the index is shown to range from 0.4 to 0.7, which indicates a range in which the percentage of correct answers has a one-to-one correspondence with the index related to the specific feature, i.e., the score. It can be said that the index calculation model that calculated the index shown in FIG. 8 can accurately estimate whether a user has a specific feature using the index related to the specific feature if it is within this range.

[0077] FIG. 9 is a diagram showing a second example of a graph generated by the generation unit of the information processing device according to the embodiment. As shown in FIG. 9, the generation unit 135 generates a graph in which the horizontal axis represents the index related to the specific feature calculated by the calculation unit 133 and the vertical axis represents the percentage of correct answers included in the answer information to the question from the questioning unit 134. In the example graph shown in FIG. 9, the range in which the percentage of correct answers shows a one-to-one correspondence with the index related to the specific feature is not clearly shown. In this case, it is not possible to accurately estimate whether or not a user has a specific feature using the index related to the specific feature. Therefore, an index calculation model that generates such a graph can be said to be a bad index calculation model.

[0078] (About learning unit 136) The learning unit 136 trains the discriminant model on the features of the graphs generated for each of the multiple index calculation models. For example, the learning unit 136 may train the discriminant model using data labeled "good" or "bad" for the graphs generated by the generation unit 135 as training data. In this case, the training data is image data of the graphs generated by the generation unit 135, and a machine learning model that performs image recognition is used as the discriminant model. The discriminant model may be realized by, for example, a convolutional neural network (CNN).

[0079] Furthermore, the learning unit 136 causes the discriminant model to learn the usable range in the graph generated for each of the multiple index calculation models. For example, the learning unit 136 may cause the discriminant model to learn image data, to which labels indicating the usable range as a numerical value are attached, for the graph generated by the generation unit 135. Furthermore, for example, the learning unit 136 may cause the discriminant model to learn image data, to which image data indicating the usable range by encircling it or image data indicating the usable range by using a different color from the rest, for the graph generated by the generation unit 135. The discriminant model in this case may similarly be realized by CNN.

[0080] (Regarding the discrimination unit 137) The discrimination unit 137 determines whether the index calculation model is good or bad based on the graph, and if the index calculation model is determined to be good, outputs a range of usable index calculation models. That is, the discrimination unit 137 determines whether the index calculation model is good or bad by inputting image data of the graph generated by the generation unit 135 into a discrimination model trained by the learning unit 136. The learning unit 136 trains the discrimination model using image data of each graph, to which image data of the index calculation model is assigned a label of "good" or "bad" as learning data. Therefore, by inputting the image data of the graph into the discrimination model, it is possible to determine whether the index calculation model that calculated the index used in the graph is good or bad. Furthermore, if the discrimination unit 137 determines that the index calculation model is "good," it displays a range of usable index calculation models in the image of the graph.

[0081] Furthermore, the discrimination unit 137 outputs the range of usable index calculation models as a numerical range including a minimum value and a maximum value using the values ​​of the indexes related to the specific characteristics of the user. By inputting the image data of the graph to a discrimination model that judges the quality of the index calculation model, the discrimination unit 137 outputs the usable range of the index calculation model as a numerical range in addition to the judgment result of "good" or "bad" of the index calculation model if it is judged to be "good".

[0082] (Regarding the decision unit 138) The determination unit 138 uses the index calculation model determined to be good to calculate an index for the specific feature of the user and determines the target of content distribution. That is, the determination unit 138 uses the index calculation model determined to be "good" by the discrimination unit 137 to calculate an index for the specific feature of the user, and determines users whose index value for the feature is equal to or greater than a predetermined value as the target of content distribution. Note that the specific feature in this case refers to the specific feature possessed by the user to whom the provider M desires content distribution, received from the provider terminal 300. That is, the determination unit 138 uses the index calculation model to estimate users who possess the specific feature possessed by the user to whom the provider M desires content distribution, and determines users to whom the content will be distributed based on the index.

[0083] (About Distribution Section 139) The distribution unit 139 distributes the content to the users determined as recipients of the content by the determination unit 138. For example, if the determination unit 138 determines users U5 and U6 as recipients of the content, the distribution unit 139 reads out the content input by the business operator from the content storage unit 124 and distributes it via the communication unit 110 to the user terminal 200E of user U5 and the user terminal 200F of user U6.

[0084] [4. User terminal configuration] Next, the configuration of the user terminal 200 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of the configuration of the user terminal according to the embodiment. As shown in Fig. 10, the user terminal 200 has a communication unit 210, an input unit 220, an output unit 230, and a control unit 240.

[0085] The communication unit 210 is realized by, for example, a NIC etc. The communication unit 210 is connected to a network N by wire or wirelessly, and transmits and receives various information to and from the information processing device 100 via the network N.

[0086] Various types of operation information are input from the user to the input unit 220. For example, the input unit 220 may accept various operations from the user via a display surface (e.g., the output unit 230) using a touch panel. The input unit 220 may also accept various operations from buttons provided on the user terminal 200 or a keyboard or mouse connected to the user terminal 200.

[0087] The output unit 230 is a display screen of a tablet terminal or the like realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. In other words, if the input unit 220 of the user terminal 200 is a touch panel, the display screen of the output unit 230 accepts input from the user and also outputs the input to the user. The output unit 230 may also be a speaker, and may output sound from the speaker.

[0088] The control unit 240 is realized, for example, by a CPU, an MPU, or the like executing various programs stored in the user terminal 200 using RAM as a work area. The control unit 240 may also be realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0089] As shown in FIG. 10, the control unit 240 includes a receiving unit 241 and a providing unit 242.

[0090] The reception unit 241 receives an answer to a question from a user. The reception unit 241 receives question information transmitted from the information processing device 100 via the communication unit 210, displays the question information on the output unit 230, and receives an answer to the question from the user via the input unit 220. Note that the answer information received by the reception unit 241 may include "yes" or "no" that is the user's answer to the question included in the question information.

[0091] The providing unit 242 provides the content distributed from the information processing device 100 to the user. For example, if the content distributed from the information processing device 100 is a video, the providing unit 242 provides the content to the user by causing the output unit 230 to output the video. Furthermore, for example, if the content distributed from the information processing device 100 is audio, the providing unit 242 may provide the content to the user by causing the output unit 230 to output the audio of the content. Furthermore, for example, if the content distributed from the information processing device 100 is text data, the providing unit 242 may provide the content to the user by causing the output unit 230 to display the text.

[0092] [5. Configuration of operator terminal] Next, the configuration of the operator terminal 300 according to the embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram showing an example of the configuration of the operator terminal according to the embodiment. As shown in Fig. 11, the operator terminal 300 includes a communication unit 310, an input unit 320, an output unit 330, and a control unit 340.

[0093] The communication unit 310 is realized by, for example, a NIC etc. The communication unit 310 is connected to a network N by wire or wirelessly, and transmits and receives various information to and from the information processing device 100 via the network N.

[0094] Various types of operation information are input from the business operator to the input unit 320. For example, the input unit 320 may accept various operations from the business operator via a keyboard or mouse connected to the business operator terminal 300. Alternatively, the input unit 320 may accept various operations from the business operator via a display surface (e.g., the output unit 330) of a touch panel.

[0095] The output unit 330 is a display screen realized by, for example, a liquid crystal display, an organic EL display, etc., and is a display device for displaying various information. When the input unit 320 of the business operator terminal 300 accepts various operations from the business operator via a touch panel, the display screen of the output unit 330 accepts input from the user and also outputs the information to the user.

[0096] The control unit 340 is realized, for example, by a CPU, an MPU, or the like executing various programs stored in the operator terminal 300 using RAM as a work area. The control unit 340 may also be realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0097] As shown in FIG. 11, the control unit 340 includes a receiving unit 341.

[0098] The reception unit 341 receives content from a business operator. The content may be, for example, data including text, audio, still images, and moving images advertising the business operator's products and services. For example, the reception unit 341 receives content from the business operator in association with a "specific characteristic" of a user. That is, the reception unit 341 receives from the business operator a business operator ID that identifies the business operator, data indicating the content, and the "specific characteristic" of the user to whom the business operator wishes to distribute the content.

[0099] [6. Information Processing Flow] Next, a procedure of information processing by the information processing device 100 according to the embodiment will be described with reference to FIG. 12. FIG. 12 is a flowchart illustrating an example of information processing according to the embodiment. For example, the information processing device 100 acquires user information indicating information about a user and behavioral information including the user's search history (step S101). Then, the information processing device 100 stores a plurality of index calculation models that have been trained to learn user characteristics based on the user information and the behavioral information (step S102). Then, the information processing device 100 calculates an index for a specific characteristic for each user using the plurality of index calculation models (step S103). Then, the information processing device 100 asks the user, whose index for the specific characteristic has been calculated, whether or not the specific characteristic applies to the user (step S104). Then, the information processing device 100 generates a graph showing the relationship between the index for the specific characteristic of the plurality of users and the percentage of correct answers regarding whether or not the specific characteristic applies to the plurality of users (step S105).

[0100] [7. Hardware Configuration] The information processing device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration as shown in Fig. 13, for example. Fig. 13 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected via a bus 1090.

[0101] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The primary storage device 1040 is a memory device, such as a RAM, that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device that stores data used by the arithmetic device 1030 for various calculations and various databases, and is realized by a ROM (Read Only Memory), an HDD (Hard Disk Drive), a flash memory, or the like.

[0102] The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (High Definition Multimedia Interface), etc. The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, scanner, etc., and is realized by a USB, etc.

[0103] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. The input device 1020 may also be an external storage medium such as a USB memory.

[0104] The network IF 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0105] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0106] For example, when the computer 1000 functions as the information processing device 100, the arithmetic unit 1030 of the computer 1000 realizes the functions of the control unit 130 of the information processing device 100 by executing a program loaded onto the primary storage device 1040.

[0107] [8. Composition and Effects] The information processing device 100 according to the present disclosure includes an acquisition unit 131 that acquires user information indicating information about a user and behavioral information including the user's search history, a model storage unit 123 that stores a plurality of index calculation models that have been trained to learn user characteristics based on the user information and the behavioral information, a calculation unit 133 that calculates an index for a specific characteristic for each user using the plurality of index calculation models, a questioning unit 134 that asks the user who has calculated the index for the specific characteristic whether or not the specific characteristic applies to the user, and a generation unit 135 that generates a graph showing the relationship between the index for the specific characteristic of a plurality of users and the percentage of correct answers regarding whether or not the specific characteristic applies to the plurality of users.

[0108] According to this configuration, it is possible to generate a graph showing the relationship between an index related to a specific feature of a user and the percentage of correct answers regarding whether or not the specific feature of the user applies. Therefore, by checking the generated graph, it is possible to understand whether or not the index calculated by the index calculation model is capable of predicting whether or not the user has the specific feature. Therefore, it is possible to provide an information processing device 100 that can determine the quality of a model that predicts user attribute information.

[0109] The information processing device 100 according to the present disclosure further includes a learning unit 136 that trains a discrimination model to learn the characteristics of the graphs generated for each of a plurality of index calculation models, and a discrimination unit 137 that determines whether the index calculation model is good or bad based on the graphs, and outputs a range of usable index calculation models when the index calculation model is determined to be good.

[0110] According to this configuration, it is possible to determine whether an index calculation model is good or bad by using a discrimination model that has learned the features of the graph generated for each index calculation model. Furthermore, if an index calculation model is determined to be good, it is possible to grasp the range of usable index calculation models. Therefore, it is possible to provide an information processing device 100 that can determine whether a model that estimates user attribute information is good or bad.

[0111] The discrimination unit 137 of the information processing device 100 according to the present disclosure outputs the range of usable index calculation models as a numerical range including a minimum value and a maximum value, using the values ​​of the indexes related to the specific characteristics of the user.

[0112] With this configuration, the range of usable index calculation models can be grasped as a numerical range including the minimum and maximum values ​​using the values ​​of the indexes related to the specific characteristics of the user, which makes it possible to clearly grasp the range of the indexes calculated by the index calculation model with high estimation accuracy.

[0113] The discrimination unit 137 of the information processing device 100 according to the present disclosure calculates a moving average of the ratio of correct and incorrect values ​​for the index calculated by the index calculation model that has been determined to be incorrect, and if the calculated moving average is declining to the right for the index, the discrimination unit 137 re-determines that the index calculation model is good.

[0114] According to this configuration, even if an index calculation model is judged to be invalid by the discriminant model, the quality of the index calculation model can be judged again by re-evaluating it using the moving average. Therefore, since an incorrect judgment of the discriminant model can be corrected, the quality of the index calculation model can be judged with high accuracy.

[0115] The questioning unit 134 of the information processing device 100 according to the present disclosure asks a new question to users whose percentage of correct answers regarding a specific user characteristic is unknown within a specific numerical range of the index.

[0116] According to this configuration, by asking a user whose numerical range of correct answers for a specific feature is unknown, it becomes possible to accurately grasp the estimation accuracy of the index calculation model. Therefore, it is possible to provide an information processing device 100 that can judge the quality of a model that estimates user attribute information.

[0117] The information processing method according to the present disclosure includes the steps of acquiring user information indicating information about a user and behavioral information including the user's search history, storing a plurality of index calculation models that have been trained to learn the user's characteristics based on the user information and the behavioral information, calculating an index for a specific characteristic for each user using the plurality of index calculation models, asking the users whose indexes for the specific characteristics have been calculated whether or not they correspond to the specific characteristic, and generating a graph showing the relationship between the indexes for the specific characteristics of the plurality of users and the percentage of correct answers regarding whether or not the plurality of users correspond to the specific characteristic.

[0118] According to this configuration, it is possible to generate a graph showing the relationship between an index related to a specific characteristic of a user and the percentage of correct answers regarding whether or not the specific characteristic of the user applies. Therefore, by checking the generated graph, it is possible to understand whether or not the index calculated by the index calculation model is capable of predicting whether or not the user has a specific characteristic. Therefore, it is possible to provide an information processing method capable of determining the quality of a model that predicts user attribute information.

[0119] In addition, the information processing program according to the present disclosure causes a computer to perform the following steps: acquiring user information indicating information about a user and behavioral information including the user's search history; storing a plurality of index calculation models that have been trained to learn the user's characteristics based on the user information and the behavioral information; calculating an index for a specific characteristic for each user using the plurality of index calculation models; asking the users whose indexes for the specific characteristics have been calculated whether or not they correspond to the specific characteristic; and generating a graph showing the relationship between the indexes for the specific characteristics of a plurality of users and the percentage of correct answers regarding whether or not the specific characteristic of a plurality of users corresponds to the specific characteristic.

[0120] According to this configuration, it is possible to generate a graph showing the relationship between an index related to a specific characteristic of a user and the percentage of correct answers regarding whether or not the specific characteristic of the user applies. Therefore, by checking the generated graph, it is possible to understand whether or not the index calculated by the index calculation model is capable of predicting whether or not the user has a specific characteristic. Therefore, it is possible to provide an information processing method capable of determining the quality of a model that predicts user attribute information.

[0121] The above describes the embodiments of the present application in detail based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0122] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit," etc. For example, the acquisition unit 131 can be read as acquisition means or acquisition circuit. [Explanation of symbols]

[0123] 100 Information processing device 110 Communications Department 120 Storage section 121 User information storage unit 122 Behavior information storage unit 123 Model Memory Unit 124 Content storage unit 130 Control Unit 131 Acquisition Department 132 Reception Department 133 Calculation Unit 134 Questions 135 Generation part 136 Learning Department 137 Discrimination part 138 Decision Section 139 Distribution Department 200 User terminals 210 Communications Department 220 Input section 230 Output section 240 Control Unit 241 Reception Department 242 Providing Department 300 Operator terminal 310 Communications Department 320 Input section 330 Output section 340 Control Unit 341 Reception Department N Network

Claims

1. an acquisition unit that acquires user information indicating information about a user and behavioral information including a search history of the user; a model storage unit that stores a plurality of index calculation models that have been trained based on the user information and the behavioral information; a calculation unit that calculates an index for a specific feature for each user using the plurality of index calculation models; a questioning unit that asks a user who has calculated an index relating to a specific feature whether or not the index corresponds to the specific feature; a generating unit that generates a graph showing the relationship between an index of a specific characteristic of a plurality of users and a rate of correct or incorrect answers regarding whether or not the specific characteristic of the plurality of users applies; Information processing device.

2. a learning unit that causes a discriminant model to learn features of the graphs generated for each of a plurality of index calculation models; a determination unit that determines whether the index calculation model is good or bad based on the graph, and outputs a range of usable index calculation models when the index calculation model is determined to be good. The information processing device according to claim 1 .

3. The discrimination unit outputs the range of usable index calculation models as a numerical range including a minimum value and a maximum value using the value of an index related to a specific feature of the user. The information processing device according to claim 2 .

4. The discrimination unit calculates a moving average of the rate of correctness for the index calculated by the index calculation model determined to be bad, and if the calculated moving average is declining to the right with respect to the index, re-determines the index calculation model as good. The information processing device according to claim 3 .

5. the questioning unit asks a new question to users whose percentage of correct answers regarding the specific feature of the user is unknown within a specific numerical range of the index; The information processing device according to any one of claims 1 to 4.

6. An information processing method executed by an information processing device, comprising: acquiring user information indicating information about the user and behavioral information including the user's search history; storing a plurality of index calculation models that have been trained based on the user information and the behavioral information; calculating an index for a specific feature for each user using the plurality of index calculation models; A step of asking a user who has calculated an index related to a specific feature whether or not the specific feature applies to the user; generating a graph showing the relationship between an index for a specific characteristic of a plurality of users and a percentage of correct answers regarding whether or not the specific characteristic applies to the plurality of users; An information processing method including:

7. acquiring user information indicating information about the user and behavioral information including the user's search history; storing a plurality of index calculation models that have been trained based on the user information and the behavioral information; calculating an index for a specific feature for each user using the plurality of index calculation models; A step of asking a user who has calculated an index related to a specific feature whether or not the specific feature applies to the user; generating a graph showing the relationship between an index for a specific characteristic of a plurality of users and a percentage of correct answers regarding whether or not the specific characteristic applies to the plurality of users; An information processing program that causes a computer to execute the above.

Images