Learning device, inference device, learning method, inference method, and program

Abstract

A training video information acquisition unit (102) of a learning device (10) acquires training video information which has been acquired by analyzing a training video advertisement for training. A training purchase information acquisition unit (103) acquires training purchase information pertaining to a purchase of a training viewer of the training video advertisement. A learning unit (104) trains a machine learning model on the basis of the training video information and the training purchase information. An inference video information acquisition unit (202) of an inference device (20) acquires inference video information which has been acquired by analyzing an inference video advertisement for inference. A model storage unit (201) stores the machine learning model which has been trained. An inference unit (203) infers the advertisement effect of the inference video advertisement on the basis of the inference video information and the machine learning model.

Description

Learning device, estimation device, learning method, estimation method, and program

[0001] The present disclosure relates to a learning device, an estimation device, a learning method, an estimation method, and a program.

[0002] Conventionally, techniques for estimating the advertising effectiveness of video advertisements have been known. For example, Patent Literature 1 (JP-A-2005-102666) describes a method in which behavioral data and physiological data of viewers who have viewed media content such as video advertisements are processed to obtain time-series emotional state data points, and a classification model that maps between the effect data and prediction parameters of the time-series emotional state data points outputs predicted effect data indicating the predicted effect of the media content. The prediction parameters in Patent Literature 1 are quantitative indicators of relative changes in the viewer's response to the video advertisement.

[0003] Special Publication No. 2020-501260

[0004] However, the classification model in Patent Document 1 outputs predicted effectiveness data by taking into account only data based on the behavior of viewers of video advertisements, and therefore the content of the video advertisements actually delivered is not sufficiently taken into account when estimating the advertising effectiveness of the video advertisements. As a result, the technology in Patent Document 1 sometimes does not provide sufficient accuracy in estimating the advertising effectiveness of video advertisements. This point is also true for other conventional technologies other than the technology in Patent Document 1.

[0005] One of the objectives of the present disclosure is to improve the accuracy of estimating the advertising effectiveness of video advertisements.

[0006] The learning device according to the present disclosure includes a training video information acquisition unit that acquires training video information acquired by analyzing a training video advertisement that is a video advertisement for training; a training purchase information acquisition unit that acquires training purchase information regarding purchases made by training viewers who are viewers of the training video advertisement; and a learning unit that learns a machine learning model that estimates the advertising effectiveness of an estimated video advertisement from estimated video information acquired by analyzing an estimated video advertisement that is a video advertisement for estimation, based on the training video information and the training purchase information.

[0007] The estimation device according to the present disclosure includes an estimated video information acquisition unit that acquires estimated video information acquired by analyzing an estimated video advertisement that is a video advertisement for estimation, a model memory unit that stores a machine learning model that has been trained based on training video information acquired by analyzing a training video advertisement that is a video advertisement for training and training purchase information regarding purchases by training viewers who are viewers of the training video advertisement, and an estimation unit that estimates the advertising effectiveness of the estimated video advertisement based on the estimated video information and the machine learning model.

[0008] According to the present disclosure, it is possible to improve the accuracy of estimating the advertising effectiveness of video advertisements.

[0009] FIG. 1 is a diagram showing an example of the hardware configuration of each of a learning device and an estimation device. FIG. 2 is a diagram showing an example of a training video advertisement distributed by a live distribution service. FIG. 3 is a diagram showing an example of an outline of each of a learning device and an estimation device. FIG. 4 is a diagram showing an example of functions implemented by each of a learning device and an estimation device. FIG. 5 is a diagram showing an example of a training database. FIG. 6 is a diagram showing an example of an estimation database. FIG. 7 is a diagram showing an example of a video advertisement database. FIG. 8 is a diagram showing an example of processing executed by a learning device. FIG. 9 is a diagram showing an example of processing executed in S10. FIG. 10 is a diagram showing an example of processing executed by an estimation device. FIG. 11 is a diagram showing an example of processing executed in S20. FIG. 12 is a diagram showing an example of functions implemented in a modified example.

[0010] [1. Hardware Configuration of the Learning Device and the Estimation Device] An example of an embodiment of a learning device, an estimation device, a learning method, an estimation method, and a program according to the present disclosure will be described. In this embodiment, an example is given in which the learning device and the estimation device are different devices, but the learning device and the estimation device may be the same device. In other words, a single device may function as both a learning device and an estimation device.

[0011] In this embodiment, the learning device learns a machine learning model that estimates the advertising effectiveness of video advertisements, which are advertisements that use videos. The estimation device performs estimation based on the trained machine learning model. Hereinafter, training (learning) video advertisements used in learning the machine learning model are referred to as training video advertisements. Performers in training video advertisements are referred to as training performers. Viewers of training video advertisements are referred to as training viewers. Video advertisements for estimation that are the subject of processing by the trained machine learning model are referred to as estimated video advertisements. Performers in estimated video advertisements are referred to as estimated performers. Viewers of estimated video advertisements are referred to as estimated viewers.

[0012] In this embodiment, when there is no need to distinguish between training video advertisements and estimated video advertisements, they may simply be referred to as video advertisements. When there is no need to distinguish between training performers and estimated performers, they may simply be referred to as performers. When there is no need to distinguish between training viewers and estimated viewers, they may simply be referred to as viewers. In addition, when there is no need to distinguish between terms such as training XX and estimated XX, they may simply be referred to as XX.

[0013] FIG. 1 is a diagram illustrating an example of the hardware configuration of each of a learning device and an estimation device. In this embodiment, an example of a live streaming system 1 including a learning device 10 and an estimation device 20 is described. In the example of FIG. 1, in addition to the learning device 10 and the estimation device 20, the live streaming system 1 also includes a server 30, a training performer device 40, a training viewer device 50, an estimated performer device 60, and an estimated viewer device 70. Each of the learning device 10, the estimation device 20, the server 30, the training performer device 40, the training viewer device 50, the estimated performer device 60, and the estimated viewer device 70 is connected to a network N such as the Internet or a LAN.

[0014] The learning device 10 is a computer that performs learning of a machine learning model. For example, the learning device 10 is a personal computer, a server computer, a tablet, or a smartphone. For example, the learning device 10 includes a control unit 11, a memory unit 12, a communication unit 13, an operation unit 14, and a display unit 15. The control unit 11 includes at least one processor. The memory unit 12 includes at least one of a volatile memory such as RAM and a non-volatile memory such as flash memory. The communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication. The operation unit 14 is an input device such as a touch panel. The display unit 15 is a liquid crystal or organic EL display.

[0015] The estimation device 20 is a computer that performs estimation based on a trained machine learning model. For example, the estimation device 20 is a personal computer, a server computer, a tablet, or a smartphone. For example, the estimation device 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The hardware configurations of the control unit 21, the storage unit 22, the communication unit 23, the operation unit 24, and the display unit 25 may be similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 14, and the display unit 15, respectively.

[0016] Server 30 is a server computer managed by the operator of a live streaming service that distributes videos to an unspecified number of people in real time. In this embodiment, the operator of the live streaming service also manages learning device 10 and estimation device 20, but other parties may also manage learning device 10 and estimation device 20. For example, server 30 includes a control unit 31, a storage unit 32, and a communication unit 33. The hardware configurations of control unit 31, storage unit 32, and communication unit 33 may be similar to those of control unit 11, storage unit 12, and communication unit 13, respectively.

[0017] The training performer device 40 is a computer of the training performer. For example, the training performer device 40 is a personal computer, a tablet, or a smartphone. For example, the training performer device 40 includes a control unit 41, a memory unit 42, a communication unit 43, an operation unit 44, and a display unit 45. The hardware configurations of the control unit 41, the memory unit 42, the communication unit 43, the operation unit 44, and the display unit 45 may be similar to those of the control unit 11, the memory unit 12, the communication unit 13, the operation unit 14, and the display unit 15, respectively. A photographing unit 46 is connected to the training performer device 40. The photographing unit 46 includes at least one camera. The photographing unit 46 may be included inside the training performer device 40.

[0018] The training viewer device 50 is a computer of the training viewer. For example, the training viewer device 50 is a personal computer, a tablet, or a smartphone. For example, the training viewer device 50 includes a control unit 51, a storage unit 52, a communication unit 53, an operation unit 54, and a display unit 55. The hardware configurations of the control unit 51, the storage unit 52, the communication unit 53, the operation unit 54, and the display unit 55 may be similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 14, and the display unit 15, respectively.

[0019] The estimated performer device 60 is a computer of the estimated performer. For example, the estimated performer device 60 is a personal computer, tablet, or smartphone. For example, the estimated performer device 60 includes a control unit 61, a storage unit 62, a communication unit 63, an operation unit 64, and a display unit 65, and is connected to a capture unit 66. The hardware configurations of the control unit 61, the storage unit 62, the communication unit 63, the operation unit 64, the display unit 65, and the capture unit 66 are similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 14, the display unit 15, and the capture unit 46, respectively. The capture unit 66 may be included within the estimated performer device 60.

[0020] The estimated viewer device 70 is a computer of the estimated viewer. For example, the estimated viewer device 70 is a personal computer, a tablet, or a smartphone. For example, the estimated viewer device 70 includes a control unit 71, a storage unit 72, a communication unit 73, an operation unit 74, and a display unit 75. The hardware configurations of the control unit 71, the storage unit 72, the communication unit 73, the operation unit 74, and the display unit 75 may be similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 14, and the display unit 15, respectively.

[0021] The programs stored in storage units 12, 22, 32, 42, 52, 62, and 72 may be supplied to learning device 10, estimation device 20, server 30, training performer device 40, training viewer device 50, estimated performer device 60, or estimated viewer device 70 via network N. Also, the programs stored in a computer-readable information storage medium may be supplied to learning device 10, estimation device 20, server 30, training performer device 40, training viewer device 50, estimated performer device 60, or estimated viewer device 70 via a reading unit (e.g., an optical disk drive or a memory card slot) that reads the information storage medium, or an input / output unit (e.g., a USB port) that inputs and outputs data to and from an external device.

[0022] Furthermore, the live streaming system 1 need only include at least one computer and is not limited to the example shown in Figure 1. For example, the live streaming system 1 may include a learning device 10, an estimation device 20, and a server 30, but may not include a training performer device 40, a training viewer device 50, an estimated performer device 60, or an estimated viewer device 70. In this case, the training performer device 40, the training viewer device 50, the estimated performer device 60, and the estimated viewer device 70 exist outside the live streaming system 1. Alternatively, for example, the learning device 10 and the estimation device 20 may not be included in the live streaming system 1, but may exist outside the live streaming system 1.

[0023] 2. Overview of the Learning Device and the Estimation Device Fig. 2 is a diagram showing an example of a training video advertisement distributed via a live distribution service. For example, a training performer advertises a product or service that is the subject of e-commerce in front of a filming unit 46. The training performer device 40 transmits the training video advertisement generated by the filming unit 46 to the server 30. The server 30 distributes the training video advertisement received from the training performer device 40 to the training viewer devices 50 of an unspecified number of training viewers in real time. A screen SC showing the training video advertisement is displayed on the display unit 55 of the training viewer device 50.

[0024] For example, when the training viewer selects button B1, the training viewer device 50 accesses a page showing details of the product or service introduced in the training video advertisement. The training viewer can purchase the product or service from the page. The details may be displayed on the screen SC. When the training viewer selects button B2, the training viewer device 50 accesses a page showing coupons for the product or service introduced in the training video advertisement. The training viewer can obtain a coupon for the product on the page. The coupon may be obtainable on the screen SC.

[0025] For example, a training viewer can input comments about a training video advertisement into an input form F. Comments input by the training viewer and other training viewers are displayed in a display area A on the screen SC. One of the goals of such a live streaming service is to maximize advertising effectiveness. Advertising effectiveness can also be referred to as conversion. For example, advertising effectiveness can be the number of sales of a product or service, sales revenue, the number of items added to a shopping cart, the number of items added to bookmarks, the number of users who added a store to their favorites from the video advertisement page, or a combination of these. Advertising effectiveness is thought to be influenced by various factors. In this embodiment, the learning device 10 trains a machine learning model that estimates advertising effectiveness in a live streaming service. The estimation device 20 performs estimation using the trained machine learning model.

[0026] 3 is a diagram illustrating an example of an overview of each of the learning device 10 and the estimation device 20. In this embodiment, the learning device 10 acquires training video information indicating characteristics of a training video advertisement by performing various analyses on the training video advertisement that has already been distributed. The training video information indicates characteristics that are thought to be causally related to the advertising effectiveness of the training video advertisement. Details of the method for acquiring training video information will be described later. For example, the learning device 10 analyzes facial expressions of training performers, responses by training performers, explanations by training performers, etc., to acquire multiple pieces of training video information.

[0027] In this embodiment, the learning device 10 acquires training viewer information related to the purchases of training viewers by tracking the behavior of the training viewers. The training viewer information corresponds to an index showing the advertising effectiveness of the training video advertisement. Details of the method for acquiring training purchase information will be described later, but for example, the training viewer information is acquired by tracking whether or not the training viewers have made a purchase. Although one piece of training viewer information is shown in FIG. 3, the learning device 10 may acquire multiple pieces of training viewer information.

[0028] In the example of Figure 3, the learning device 10 generates training data for the machine learning model M to learn based on multiple pieces of training video information and one piece of training viewer information. Details of the training data will also be described later. For example, the learning device 10 may perform similar processing on each of multiple training video advertisements to generate training data one after another. The learning device 10 learns the machine learning model M based on the training data. Details of the learning will also be described later. The learning device 10 transmits the trained machine learning model M to the estimation device 20. The estimation device 20 records the trained machine learning model M received from the learning device 10 in the memory unit 22.

[0029] For example, the estimation device 20 acquires estimated video information by performing various analyses on the estimated video advertisement in order to retroactively analyze the advertising effectiveness of the estimated video advertisement that has already been distributed. In this embodiment, the estimated video advertisement is distributed in the same manner as the training video advertisement described in FIG. 2. For example, the estimated performer operates the estimated performer device 60, films himself / herself with the filming unit 66, and introduces the product or service. The server 30 distributes the estimated video advertisement to the estimated viewer devices 70 of an unspecified number of estimated viewers. Details of the estimated video information will be described later, but the method of acquiring the estimated video information may be the same as the method of acquiring the training video information.

[0030] For example, the estimation device 20 inputs a plurality of pieces of estimated video information to the machine learning model M. The machine learning model M calculates features (embedded representations) based on the plurality of pieces of estimated video information. The machine learning model M estimates the advertising effectiveness of the estimated video advertisement based on the features. For example, the machine learning model M estimates the probability that an estimated viewer who has viewed the estimated video advertisement will purchase a product or service.

[0031] For example, the estimation device 20 acquires the estimation result output from the machine learning model M. The estimation device 20 can use the estimation result of the machine learning model M for any purpose. For example, the estimation device 20 may display the estimation result of the machine learning model M on the display unit 25. The estimation device 20 may transmit the estimation result of the machine learning model M to the estimated performer device 60 in order to provide feedback on the advertising effectiveness of the estimated video advertisement to the estimated performer.

[0032] As described above, the learning device 10 of this embodiment creates a machine learning model M with high accuracy in estimating advertising effectiveness by learning the machine learning model M based on training video information and training purchase information. The estimation device 20 can accurately estimate the advertising effectiveness of an estimated video advertisement by making estimations based on the machine learning model M trained by the learning device 10. Details of this embodiment will be described below.

[0033] 4 is a diagram showing an example of functions implemented by the learning device 10 and the estimation device 20. In FIG. 4, functions implemented by the server 30 are also shown.

[0034] [3-1. Functions Realized by the Learning Device] For example, the learning device 10 includes a data storage unit 100, a model storage unit 101, a training video information acquisition unit 102, a training purchase information acquisition unit 103, and a learning unit 104. The data storage unit 100 and the model storage unit 101 are realized by the storage unit 12. The training video information acquisition unit 102, the training purchase information acquisition unit 103, and the learning unit 104 are realized by the control unit 11.

[0035] [Data Storage Unit] The data storage unit 100 stores data necessary for training the machine learning model M. For example, the data storage unit 100 stores a training database DB1.

[0036] 5 is a diagram showing an example of a training database DB1. The training database DB1 is a database that stores training data to be learned by the machine learning model M. In this embodiment, each unit of data used when learning the machine learning model M is called training data. A collection of training data in which multiple pieces of training data are stored is called the training database DB1. All or part of the multiple pieces of training data stored in the training database DB1 are learned by the machine learning model M.

[0037] In the example of Figure 5, "No." is the record number of the training database DB1. In this embodiment, an example is taken of a case where one training data item is generated from one training video advertisement. That is, the training video advertisement and the training data have a one-to-one relationship. Therefore, "No." in Figure 5 can also be said to be information that can identify the training video advertisement. Note that multiple training data items may be generated from one training video advertisement, or one training data item may be generated from multiple training video advertisements.

[0038] For example, the training data includes an input portion that is input to the machine learning model M during learning, and an output portion that is to be output from the machine learning model M during learning. The input portion of the training data has the same format as the input data that is input to the machine learning model M during estimation. The input portion of the training data is sometimes called an explanatory variable. In this embodiment, the input portion of the training data is a plurality of pieces of training video information. The output portion of the training data has the same format as the output data that is output from the machine learning model M during estimation. The output portion of the training data corresponds to the correct answer during learning. The output portion of the training data is sometimes called a target variable. In this embodiment, an example is given in which the output portion of the training data is training purchase information.

[0039] The data storage unit 100 can store any data. The data stored in the data storage unit 100 is not limited to the training database DB1. For example, the data storage unit 100 may store a learning program that indicates a series of processes in learning the machine learning model M. The data storage unit 100 may store video data of training video advertisements downloaded by the learning device 10 from the server 30. The data storage unit 100 may store data accompanying the video data of training video advertisements (e.g., categories of products or services introduced in the training video advertisements, pre-entered descriptions, coupon information, or combinations thereof).

[0040] For example, the data storage unit 100 may store a program for natural language processing, which will be described later. The data storage unit 100 may store a dictionary database in which keywords used in natural language processing are stored. The data storage unit 100 may store a machine learning model M for natural language processing. The data storage unit 100 may store a program for image analysis processing, which will be described later. The data storage unit 100 may store a machine learning model M for image analysis processing.

[0041] [Model Storage Unit] The model storage unit 101 stores a machine learning model M. The machine learning model M includes a program portion that indicates processing such as calculation of feature quantities (embedded representation) of input data input to the model, and a parameter portion (e.g., weighting coefficients and biases) referenced by the program portion. The parameter portion of the machine learning model M is changed by learning by the learning unit 104. The model storage unit 101 stores a machine learning model M whose parameter portion has initial values. The machine learning model M whose parameter portion has initial values ​​is the machine learning model M before learning. The parameter portion with the initial values ​​is overwritten by processing by the learning unit 104.

[0042] The machine learning model M is a model that uses a machine learning technique. Various known techniques can be used for the machine learning itself. For example, the machine learning model M may be a model that uses any of supervised learning, semi-supervised learning, or unsupervised learning. In this embodiment, an example is given in which the machine learning model M is a neural network, but the machine learning model M may be a model that uses other techniques and is not limited to a neural network. For example, the machine learning model M may be a model that uses linear regression, a model that uses logistic regression, a model that uses a support vector machine, or a model that uses a decision tree.

[0043] [Training Video Information Acquisition Unit] The training video information acquisition unit 102 acquires training video information acquired by analyzing training video advertisements, which are video advertisements for training. In this embodiment, a case where a video advertisement that has been distributed in the past corresponds to the training video advertisement is taken as an example, but a video advertisement that has not yet been distributed may also correspond to the training video advertisement. Furthermore, a case where a video of a live streaming service corresponds to the training video advertisement is taken as an example, but the training video advertisement may be any video advertisement used for learning and is not limited to a video of a live streaming service. For example, a web advertisement, a television advertisement, an outdoor advertisement, a movie advertisement, or other advertisements may also correspond to the training video advertisement.

[0044] Analysis of training video advertisements is a process of extracting features related to training video advertisements. For example, voice analysis processing, natural language processing, image analysis processing, or a combination of these corresponds to the analysis of training video advertisements. Training video information indicates features extracted by analyzing training video advertisements. Hereinafter, an example of a feature acquired by the training video information acquisition unit 102 as training video information will be described. Note that the training video information acquisition unit 102 may acquire, as training video information, features that a person who creates the machine learning model M (e.g., an operator of a live streaming service) believes have some causal relationship with advertising effectiveness. The features acquired by the training video information acquisition unit 102 as training video information are not limited to the example in this embodiment.

[0045] In this embodiment, the training video information acquisition unit 102 acquires training video information acquired by performing natural language processing on training text acquired by analyzing the audio of the training video advertisement. For example, the audio of the training video advertisement is audio produced by a training performer. The audio of the training video advertisement may be audio contained in the training video advertisement and is not limited to audio produced by the training performer. For example, the audio of the training video advertisement may be audio produced by a staff member other than the training performer, an audience watching the training video advertisement on-site, a third party interviewed by the training performer, or an artificial audio prepared in advance.

[0046] The training text is text generated from the audio of the training video advertisement. For example, the training video information acquisition unit 102 generates the training text by performing audio analysis processing on the audio of the training video advertisement. The algorithm for the audio analysis processing itself may be a known algorithm. For example, the training information acquisition unit generates the training text from the audio of the training video advertisement based on a method using a hidden Markov model, a method using a predetermined waveform pattern, a machine learning method such as a neural network, or other methods. The training text may be generated by a computer other than the learning device 10. In this case, the training video information acquisition unit 102 acquires the training text generated by the other computer from that other computer, another computer, or an information storage medium.

[0047] For example, the training video information acquisition unit 102 acquires training video information by performing natural language processing on the training text. Natural language processing is a process by which a computer recognizes language used by humans. The natural language processing may be performed by a computer other than the learning device 10. In this case, the training video information acquisition unit 102 acquires the training text generated by the natural language processing performed by the other computer from the other computer, another computer, or an information storage medium.

[0048] For example, the natural language processing may be at least one of a sentiment analysis process that analyzes the emotions of the training performers who appear in the training video advertisement, a response detection process that detects responses by the training performers who appear in the training video advertisement, an explanation detection process that detects explanations about products or services featured in the training video advertisement, and a promotion detection process that detects promotions related to the training video advertisement.

[0049] For example, the training video information acquisition unit 102 acquires training video information related to the emotions of the training performers by performing a sentiment analysis process on the training text. The sentiment analysis process may be performed by a known method. For example, the training video information acquisition unit 102 may analyze the emotions of the training performers based on a rule approach that determines whether or not a keyword indicating an emotion is included in the training text. As another example, the training video information acquisition unit 102 may analyze the emotions of the training performers based on a machine learning approach that calculates features of the training text and outputs an emotion classification result.

[0050] For example, the training video information acquisition unit 102 acquires training video information indicating the analysis results of the emotions of the training performers through emotion analysis processing. Assuming that the four emotions of joy, anger, sadness, and happiness have been analyzed, the training video information acquisition unit 102 acquires training video information indicating the degree of the emotion of "joy" (e.g., the number or frequency of appearance of keywords indicating "joy"), the degree of the emotion of "anger" (e.g., the number or frequency of appearance of keywords indicating "anger"), the degree of the emotion of "sadness" (e.g., the number or frequency of appearance of keywords indicating "sadness"), and the degree of the emotion of "happiness" (e.g., the number or frequency of appearance of keywords indicating "happiness"). The training video information acquisition unit 102 may analyze emotions other than joy, anger, sadness, and happiness.

[0051] For example, the training video information acquisition unit 102 acquires training video information related to responses of training performers by executing a response detection process on the training text. The response detection process may be performed by a known method. For example, the training video information acquisition unit 102 may detect responses of training performers based on a rule approach that determines whether a keyword indicating a response is included in the training text. As another example, the training video information acquisition unit 102 may detect responses of training performers based on a machine learning approach that calculates features of the training text and outputs a classification result of the response.

[0052] The responses detected by the response detection process can also be called reactions. The responses detected by the response detection process can be responses from one training performer to another training performer's utterances. In this case, the responses can also be called exchanges between the training performers. The training video information acquisition unit 102 acquires training video information indicating the detection results of the responses of the training performers by the response detection process. For example, when a utterance such as "I see" is included in the training text, the training video information acquisition unit 102 detects the responses of the other training performers to the utterances of one training performer.

[0053] As another example, the response detected by the response detection process may be a response of the training performer to a comment input by the training viewer. When a statement such as "Thank you for your comment" is included in the training text, the training video information acquisition unit 102 detects the response of the training performer to the comment input by the training viewer. The training video information acquisition unit 102 may acquire training video information indicating the number of responses detected by the response detection process, the frequency of the responses, the type of response (e.g., what the response is in response to, or the specific content of the response), or a combination thereof.

[0054] For example, the training video information acquisition unit 102 acquires training video information related to the explanations of the training performers by performing an explanation detection process on the training text. The explanation detection process may be performed by a known method. For example, the training video information acquisition unit 102 may detect the responses of the training performers based on a rule approach that determines whether or not a keyword indicating a description of a product or service is included in the training text. As another example, the training video information acquisition unit 102 may detect the explanations of the training performers based on a machine learning approach that calculates features of the training text and outputs a classification result of the product or service description.

[0055] For example, the training video information acquisition unit 102 acquires training video information indicating the detection results of explanations by training performers through an explanation detection process. If sweets are introduced in a training video advertisement, the training video information acquisition unit 102 counts the number of appearances or frequency of appearances of keywords indicating the product itself, such as "chocolate," in the training text. The training video information acquisition unit 102 counts the number of appearances or frequency of appearances of keywords indicating the quality of the product, such as "delicious," in the training text. The training video information acquisition unit 102 acquires training video information indicating these counting results.

[0056] For example, the promotion detection process may be a keyword detection process that detects at least one of keywords related to the price of a product or service featured in the training video advertisement, keywords related to sales of the product or service featured in the training video advertisement, and keywords related to a description associated with the training video advertisement. The promotion detection process may also be referred to as a process that detects words that lead to the promotion of a product or service. The training video information acquisition unit 102 may detect the responses of the training performers based on a rule approach that determines whether keywords indicating the price of a product or service are included in the training text.

[0057] For example, the training video information acquisition unit 102 determines whether or not a keyword stored in a dictionary database storing keywords related to price (e.g., "1000 yen," "bargain," "discount," or "coupon") is included in the training text, and detects the keyword. The training video information acquisition unit 102 determines whether or not a keyword stored in a dictionary database storing keywords related to sales (e.g., "purchase" or "only a few left") is included in the training text, and detects the keyword.

[0058] The explanation associated with the training video advertisement is an explanation about the product or service introduced in the training video advertisement. For example, information that the training viewer can refer to as needed from the screen SC may correspond to the explanation associated with the training video advertisement. In the example of FIG. 2, the detailed explanation that the training viewer can refer to by selecting button B1 is an example of the explanation associated with the training video advertisement. Coupon information that the training viewer can refer to by selecting button B2 is an example of the explanation associated with the training video advertisement.

[0059] For example, the training video information acquisition unit 102 determines whether or not a keyword stored in a dictionary database that stores keywords describing a product or service introduced in the training video advertisement (e.g., "limited time offer," "delicious," or "topic") is included in the training text, and detects the keyword. Based on the keyword detection results, the training video information acquisition unit 102 acquires training video information indicating the number or frequency of keyword detection.

[0060] The promotion detection process is not limited to the above example. For example, the training video information acquisition unit 102 may detect promotions based on a machine learning approach that calculates features of training texts and outputs promotion estimation results, rather than a rule approach that uses keywords. The model used in the machine learning approach is assumed to have learned various words that are related to the sales promotion of products or services.

[0061] Furthermore, the training video information acquisition unit 102 can perform any natural language processing on the training text. The natural language processing performed by the training video information acquisition unit 102 is not limited to the example of this embodiment. For example, the training video information acquisition unit 102 may perform morphological analysis, syntactic analysis, semantic analysis, feature extraction, or a combination thereof as natural language processing on the training text, and acquire training video information based on the results of the natural language processing. The training video information acquisition unit 102 may perform natural language processing on the training text using a machine learning technique such as a transformer.

[0062] For example, the training video information acquisition unit 102 may acquire training video information related to facial expressions of training performers who appear in the training video advertisement, acquired by analyzing the video of the training video advertisement. The video of the training video advertisement includes multiple frames (still images). The training video information acquisition unit 102 acquires the training video information by performing image analysis processing on each frame. The image analysis processing may be performed by a computer other than the learning device 10. In this case, the training video information acquisition unit 102 acquires the training video information generated by the image analysis processing performed by the other computer from the other computer, another computer, or an information storage medium.

[0063] The facial expression of the training performer can also be said to represent the emotion of the training performer. Therefore, the training video information acquisition unit 102 may identify emotions similar to those obtained through emotion analysis processing by image analysis processing. The method of analyzing human facial expressions by image analysis processing may be a known method. For example, the training video information acquisition unit 102 may analyze the facial expression of the training performer in the training video advertisement based on template matching using template images that represent basic human facial expressions. The training video information acquisition unit 102 may analyze the facial expression of the training performer in the training video advertisement based on an arrangement pattern of feature points detected from the video of the training video advertisement. The training video information acquisition unit 102 may analyze the facial expression of the training performer in the training video advertisement based on a machine learning model in which various human facial expressions have been learned.

[0064] The training video information acquisition unit 102 may analyze features other than facial expressions of the training performers through image analysis processing. For example, the training video information acquisition unit 102 may analyze camera work, lighting brightness, gestures of the training performers, effects in the video, content of subtitles, or other features in the training video advertisement through image analysis processing. The training video information acquisition unit 102 may acquire training video information indicating these features.

[0065] Furthermore, in the present embodiment, an example is given in which the training video information acquisition unit 102 acquires multiple pieces of training video information, but the training video information acquisition unit 102 only needs to acquire at least one piece of training video information. The training video information acquisition unit 102 may acquire only one piece of training video information. For example, the training video information acquisition unit 102 may execute only one of the above-described methods and acquire only one piece of training video information.

[0066] [Training Purchase Information Acquisition Unit] The training purchase information acquisition unit 103 acquires training purchase information regarding purchases made by training viewers who watch the training video advertisement. A purchase by a training viewer is a purchase of a product or service introduced in a training video advertisement. A training viewer may make a purchase while watching a training video advertisement, or may make a purchase after watching a training video advertisement. A training viewer may make a purchase not during the live distribution of a training video advertisement, but by watching an archived training video advertisement after the live distribution has ended.

[0067] 2, the training viewer may select button B1 on the screen SC and then make a purchase from a page displayed on the training viewer device 50, or may make a purchase by an operation other than selecting button B1. For example, the training viewer may close the screen SC for the time being, select a link to a page for a product or service introduced in the training video advertisement, and then make a purchase from a page displayed on the training viewer device 50.

[0068] In this embodiment, the distribution unit 301 of the server 30 tracks the behavior of training viewers and generates training purchase information. The training purchase information acquisition unit 103 acquires the training purchase information from the server 30. The training purchase information may be generated by a computer other than the server 30. The training purchase information acquisition unit 103 may acquire the training purchase information from the other computer, another computer, or an information storage medium. The training purchase information acquisition unit 103 may acquire the training purchase information by generating the training purchase information itself.

[0069] In this embodiment, the training purchase information acquisition unit 103 acquires training purchase information indicating the proportion of training viewers who made a purchase among the training viewers who viewed the training video advertisement (total number of training viewers who made a purchase / total number of training viewers). The training purchase information may be any information related to purchases by the training viewers and is not limited to the example of this embodiment. For example, the training purchase information acquisition unit 103 may acquire training purchase information related to at least one of whether or not a purchase was made by the training viewer and information related to sales of a product or service introduced in the training video advertisement.

[0070] The training purchase information regarding whether or not a purchase has been made is information generated based on whether or not a training viewer has made a purchase. The above percentage is also an example of training purchase information regarding whether or not a purchase has been made, but the training purchase information regarding whether or not a purchase has been made may be other information. For example, the training purchase information regarding whether or not a purchase has been made may be information indicating whether or not each training viewer has made a purchase, the total number of training viewers who made a purchase, the total number of training viewers per unit time, or a combination of these. The unit time may be any length, for example, one minute or five minutes. For example, the server 30 generates the training purchase information by aggregating this information.

[0071] The information regarding sales of the products or services introduced in the training video advertisement is information generated based on the sales of the products or services. For example, the information regarding sales may be the number of sales (sales volume), the sales amount, the number of sales per unit time, the sales amount per unit time, or a combination thereof. For example, the server 30 generates the training purchase information by aggregating this information.

[0072] [Learning Unit] The learning unit 104 learns a machine learning model M that estimates the advertising effectiveness of an estimated video advertisement from estimated video information obtained by analyzing the estimated video advertisement, which is a video advertisement for estimation, based on training video information and training purchase information. The learning unit 104 generates training data based on the training video information and the training purchase information. In this embodiment, the learning unit 104 generates the training data so that the training video information is the input part of the training data and the training purchase information is the output part of the training data.

[0073] The learning unit 104 may use the training video information as the input part of the training data after performing some processing such as normalization or aggregation on the training video information, rather than using the training video information as the input part of the training data as is. Similarly, the learning unit 104 may use the training purchase information as the output part of the training data after performing some processing such as normalization or aggregation on the training purchase information, rather than using the training purchase information as the output part of the training data as is.

[0074] For example, the learning unit 104 learns the machine learning model M based on training data. In this embodiment, the learning unit 104 successively generates training data based on each of a plurality of training video advertisements and stores the training data in the training database DB1. The learning unit 104 learns the machine learning model M based on all or part of the plurality of training data stored in the training database DB1. Part of the training data may be used to verify the learned machine learning model M. Note that the learning unit 104 may generate training data based on only one training video advertisement, rather than multiple training video advertisements.

[0075] Learning is a process of adjusting the parameter portion of the machine learning model M. The learning method may be a known method adopted in known machine learning. For example, the learning unit 104 learns the machine learning model M based on a learning method such as backpropagation or gradient descent so that an output portion of training data is output when an input portion of training data is input. The learning unit 104 calculates a loss based on a known loss function and performs learning until the loss becomes small to a certain extent. The learning unit 104 may complete learning when learning based on a certain number of training data has been performed.

[0076] For example, when training video information is acquired by performing natural language processing on training text, at the time of estimation, similar data is input to the machine learning model M. Therefore, the learning unit 104 learns the machine learning model M that estimates advertising effectiveness from estimated video information acquired by performing processing similar to natural language processing on the training text on estimated text acquired by analyzing the audio of the estimated video advertisement.

[0077] The estimated text is text generated from the audio of the estimated video advertisement. In this embodiment, the process of estimation is performed by the estimation device 20, and therefore the generation of the estimated text is performed by the estimation device 20. The method of generating the estimated text may be the same as the method of generating the training text. For details of the method of generating the estimated text, "training" in the description of the method of generating the training text can be read as "estimation." In the description of the natural language processing of the estimated text, "training" can be read as "estimation."

[0078] For example, when training video information is acquired by analyzing a video of a training video advertisement, at the time of estimation, similar data is input to the machine learning model M. Therefore, the learning unit 104 learns the machine learning model M that estimates advertising effectiveness from estimated video information related to the facial expressions of the estimated performers who appear in the estimated video advertisement, which is acquired by analyzing the video of the estimated video advertisement.

[0079] The facial expression of the estimated performer is an expression identified by analyzing the video of the estimated video advertisement. In this embodiment, the estimation process is performed by the estimation device 20, and therefore the analysis of the facial expression of the estimated performer is performed by the estimation device 20. The method for analyzing the facial expression of the estimated performer may be the same as the method for analyzing the facial expression of the training performer. For details of the method for analyzing the facial expression of the estimated performer, "training" can be read as "estimation" in the description of the method for analyzing the facial expression of the training performer.

[0080] For example, when at least one of the training purchase information including whether or not a purchase has been made by an estimated viewer who is a viewer of an estimated video advertisement and information regarding sales of a product or service introduced in the estimated video advertisement is acquired, the learning unit 104 trains the machine learning model M that estimates the at least one of the information as the advertising effectiveness. The learning unit 104 acquires an output portion of the training data based on the at least one of the information. The learning method based on the training data is as described above.

[0081] [3-2. Functions Realized by the Estimation Device] For example, the estimation device 20 includes a data storage unit 200, a model storage unit 201, an estimated moving image information acquisition unit 202, and an estimation unit 203. The data storage unit 200 and the model storage unit 201 are realized by the storage unit 22. The estimated moving image information acquisition unit 202 and the estimation unit 203 are realized by the control unit 21.

[0082] [Data Storage Unit] The data storage unit 200 stores data necessary for estimation based on the machine learning model M. For example, the data storage unit 200 stores an estimation database DB2.

[0083] 6 is a diagram showing an example of the estimation database DB2. The estimation database DB2 is a database that stores data related to estimated video advertisements for allowing the machine learning model M to estimate advertising effectiveness. In this embodiment, an example is given in which video data for each of a plurality of estimated video advertisements is stored in the estimation database DB2. However, the estimation database DB2 may also store estimated video information generated from each of the plurality of estimated video advertisements by the estimated video information acquisition unit 202. The estimation database DB2 may also store data indicating advertising effectiveness estimated for the estimated video advertisements. Note that the video data is assumed to also include audio information.

[0084] For example, the estimation device 20 downloads, from the server 30, all or part of the video data of the video advertisement stored in a video advertisement database DB3 (described later), as video data of the estimated video advertisement. The estimation device 20 stores the downloaded video data in the estimation database DB2. The estimation device 20 may acquire the video data of the estimated video advertisement from a computer or information storage medium other than the server 30.

[0085] The data storage unit 200 can store any data. The data stored in the data storage unit 200 is not limited to the estimated database DB2. For example, the data storage unit 200 may store data incidental to the video data of the estimated video advertisement (for example, a category of a product or service introduced in the estimated video advertisement, a description input in advance, coupon information, or a combination thereof).

[0086] For example, the data storage unit 200 may store a program for natural language processing. The data storage unit 200 may store a dictionary database in which keywords used in natural language processing are stored. The data storage unit 200 may store a machine learning model M for natural language processing. The data storage unit 200 may store a program for image analysis processing. The data storage unit 200 may store a machine learning model M for image analysis processing.

[0087] [Model Storage Unit] The model storage unit 201 stores a machine learning model M that has been trained based on training video information acquired by analyzing a training video advertisement that is a video advertisement for training, and training purchase information related to purchases by training viewers who are viewers of the training video advertisement. The estimation device 20 acquires the machine learning model M that has been trained by the learning unit 104 from the learning device 10. The estimation device 20 records the machine learning model M acquired from the learning device 10 in the model storage unit 201.

[0088] [Estimated Video Information Acquisition Unit] The estimated video information acquisition unit 202 acquires estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation. For example, the estimated video information acquisition unit 202 acquires estimated video information of all or part of an estimated video advertisement whose video data is stored in the estimation database DB2. The estimated video advertisement from which the estimated video information acquisition unit 202 acquires estimated video information may be specified by a person operating the estimation device 20, or may be determined by a predetermined method.

[0089] In this embodiment, a case where a video advertisement that has been distributed in the past corresponds to an estimated video advertisement is taken as an example, but a video advertisement that has not yet been distributed may also correspond to an estimated video advertisement. Also, a case where a video of a live streaming service corresponds to an estimated video advertisement is taken as an example, but the estimated video advertisement may be any video advertisement that is the subject of advertising effectiveness estimation and is not limited to a video of a live streaming service. For example, a web advertisement, a television advertisement, an outdoor advertisement, a movie advertisement, or other advertisements may also correspond to an estimated video advertisement.

[0090] Analysis of an estimated video advertisement is a process of extracting features related to the estimated video advertisement. For example, voice analysis processing, natural language processing, image analysis processing, or a combination of these corresponds to analysis of an estimated video advertisement. The estimated video information indicates the features extracted by analyzing the estimated video advertisement. The estimated video information acquisition unit 202 may acquire the estimated video information based on the same acquisition method as the training video information. Therefore, for details of the process by which the estimated video information acquisition unit 202 acquires the estimated video information, "training" in the description of the training video information acquisition unit 102 can be read as "estimation."

[0091] For example, the estimated video information acquisition unit 202 may acquire estimated video information by performing a process similar to the natural language processing performed on the training text on estimated text acquired by analyzing the audio of the estimated video advertisement. The process may be at least one of the aforementioned sentiment analysis process, response detection process, explanation detection process, and promotion detection process. In these details, the word "training" in the description of the training video information acquisition unit 102 can also be read as "estimation." Similar to the natural language processing on the training text, any natural language processing may be performed on the estimated text and the natural language processing may be performed by a computer other than the estimation device 20.

[0092] For example, the estimated video information acquisition unit 202 may acquire estimated video information related to the facial expressions of the estimated performers who appear in the estimated video advertisement, acquired by analyzing the video of the estimated video advertisement. The video of the estimated video advertisement includes multiple frames (still images). The estimated video information acquisition unit 202 acquires the estimated video information by performing image analysis processing on each frame. Similar to the image analysis processing on the video of the training video advertisement, any image analysis processing may be performed on the video of the estimated video advertisement, and the image analysis processing may be performed by a computer other than the estimation device 20.

[0093] [Estimation unit] The estimation unit 203 estimates the advertising effectiveness of an estimated video advertisement based on estimated video information and the machine learning model M. For example, the estimation unit 203 inputs the estimated video information to the machine learning model M. The estimation unit 203 may not input the estimated video information as is, but may perform some processing such as normalization or aggregation, and then input the processed estimated video information to the machine learning model M. The machine learning model M calculates the feature amounts of the estimated video information input thereto based on the parameter portion adjusted by learning, and outputs an estimation result according to the feature amounts. The internal processing of the machine learning model M may be processing adopted in known machine learning techniques.

[0094] In this embodiment, an example is taken in which the output portion of the training data is training purchase information indicating the proportion of training viewers who have viewed the training video advertisement and have made a purchase, and therefore the estimation result output from the machine learning model M is similar information. For example, the machine learning model M outputs estimated purchase information indicating the proportion of estimated viewers who have viewed the estimated video advertisement and are estimated to make a purchase (the proportion of estimated viewers who are expected to make a purchase, i.e., the probability of purchase) as the estimation result. The estimation unit 203 acquires the estimation result output from the machine learning model M. The estimation unit 203 may store the estimation result of the machine learning model M in the estimation database DB2.

[0095] Note that if the training purchase information is the other information described above, the estimated purchase information will be information similar to the other information. For example, the machine learning model M may output, as the estimation result, information indicating whether each estimated viewer will make a purchase, the total number of estimated viewers who are estimated to make a purchase, the total number of viewers per unit time, or a combination thereof. As another example, the machine learning model M may output, as the estimation result, information regarding estimated sales for a product or service introduced in the estimated video advertisement. The information regarding sales may be the estimated number of sales (sales volume), sales amount, number of sales per unit time, sales amount per unit time, or a combination thereof, for the product or service.

[0096] 3-3. Functions Realized by the Server For example, the server 30 includes a data storage unit 300 and a distribution unit 301. The data storage unit 300 is realized by the storage unit 32. The distribution unit 301 is realized by the control unit 31.

[0097] [Data Storage Unit] The data storage unit 300 stores data necessary for the live streaming service. For example, the data storage unit 300 stores a video advertisement database DB3.

[0098] 7 is a diagram showing an example of the video advertisement database DB3. The video advertisement database DB3 is a database that stores data related to video advertisements in live streaming services. In this embodiment, the video advertisements whose data are stored in the video advertisement database DB3 can be at least one of training video advertisements and estimated video advertisements. The video advertisements whose data are stored in the video advertisement database DB3 may be used as both training video advertisements and estimated video advertisements.

[0099] For example, the video advertisement database DB3 stores a video advertisement ID capable of identifying a video advertisement, performer data related to the performers, product / service data related to the product or service introduced in the video advertisement, coupon data related to coupons for the product or service, video data of the video advertisement, and comment data related to viewer comments. In addition to currently distributed video advertisements, archives of previously distributed video advertisements may also be stored in the video advertisement database DB3. For example, data related to reaction stamps may also be stored in the video advertisement database DB3. The reaction stamp is a function that allows a viewer to react by selecting at least one of multiple stamps (e.g., a hard or thumbs-up) at any timing while watching a video advertisement. The reaction stamp is an example of a user's response (reaction).

[0100] For example, when a performer performs an operation to start distributing a new video advertisement, the distribution unit 301 issues a video advertisement ID for the video advertisement. The distribution unit 301 stores data such as performer data in the video advertisement database DB3 in association with the video advertisement ID. The performer data may indicate any information related to the performer, such as the performer's account, name, gender, age, other characteristics, or a combination thereof. The product / service data may indicate any information related to the product or service, such as the product or service category, the product or service title, a product or service description, a link to a page with detailed product or service information, or a combination thereof.

[0101] The coupon data may indicate any information related to a coupon for a product or service, such as a discount amount, a discount rate, a total number of coupons, coupon application conditions, coupon acquisition conditions, coupon acquisition status, or a combination thereof. The video data is generated by the distribution unit 301 recording a video advertisement. The video data may be in any data format. The video data may indicate only video and may not include audio. The video data may include basic information about the video advertisement, such as the video playback time.

[0102] The comment data is data indicating the content of the comment. For example, the comment data indicates the account of the viewer who posted the comment, text indicating the content of the comment, the date and time the comment was posted, or a combination of these. The viewer may also make a reaction other than a comment. In this case, data indicating the viewer's reaction is stored in the video advertisement database DB3. The video advertisement database DB3 may also store data regarding sales of products or services introduced in the video advertisement.

[0103] The data storage unit 300 can store any data. The data stored in the data storage unit 300 is not limited to the video advertisement database DB3. For example, the data storage unit 300 may store data indicating the purchasing status of a product or service introduced in each video advertisement. The data may indicate the accounts of viewers who purchased the product or service, the number of sales of the product or service, the sales amount, the number of sales per unit time, the sales amount per unit time, or a combination thereof. These data may be stored in the video advertisement database DB3. The video advertisement database DB3 may also store data regarding viewers who viewed a video advertisement, regardless of whether or not they posted a comment. Based on the data, information such as how many viewers viewed the video advertisement and how long each viewer viewed the video advertisement may be analyzed.

[0104] [Distribution Unit] The distribution unit 301 distributes video advertisements to viewers. The processing performed by the distribution unit 301 may be similar to processing employed in known live distribution services. The distribution unit 301 distributes video advertisements to each of the training viewer devices 50 and the estimated viewer devices 70. The distribution unit 301 accepts input of comments from each of the training viewer devices 50 and the estimated viewer devices 70. The distribution unit 301 updates the video advertisement database DB3 based on the content of communications with each of the training performer devices 40, the training viewer devices 50, the estimated performer devices 60, and the estimated viewer devices 70.

[0105] 4. Processing Executed by the Learning Device and the Estimation Device FIG. 8 is a diagram showing an example of processing executed by the learning device 10. The processing in FIG. 8 is executed by the control unit 11 operating in accordance with a program stored in the storage unit 12. Before executing the processing in FIG. 8, it is assumed that the learning device 10 has downloaded video data of a training video advertisement from the server 30 in advance. As shown in FIG. 8, the learning device 10 acquires training video information by analyzing the training video advertisement (S10).

[0106] 9 is a diagram showing an example of the processing executed in S10. The learning device 10 extracts the audio from the training video advertisement (S100). The learning device 10 converts the audio into text to obtain training text (S101). The learning device 10 performs natural language processing on the training text (S102). The learning device 10 analyzes the video of the training video advertisement and analyzes the facial expressions of the training performers (S103). The learning device 10 obtains training video information based on the results of each analysis (S104). Details of each process in S100 to S104 are as described above for the processing of the training video information acquisition unit 102.

[0107] Returning to FIG. 8 , the learning device 10 acquires training purchase information (S11). Details of the processing of S11 are as described above for the processing of the training purchase information acquisition unit 103. It is assumed that the learning device 10 has previously downloaded data necessary for acquiring training purchase information from the server 30. The learning device 10 trains the machine learning model M based on the training video information and the training purchase information (S12). Details of the processing of S12 are as described above for the processing of the learning unit 104. The learning device 10 transmits the trained machine learning model M to the estimation device 20 (S13), and the processing of FIG. 8 ends.

[0108] Fig. 10 is a diagram showing an example of processing executed by the estimation device 20. The processing in Fig. 10 is executed by the control unit 21 operating in accordance with a program stored in the storage unit 22. Before executing the processing in Fig. 10, the estimation device 20 is assumed to have downloaded video data of an estimated video advertisement from the server 30 in advance. As shown in Fig. 10, the estimation device 20 acquires estimated video information by analyzing the estimated video advertisement (S20).

[0109] FIG. 11 is a diagram showing an example of the processing executed in S20. The estimation device 20 extracts audio from the estimated video advertisement (S200). The estimation device 20 converts the audio into text to obtain estimated text (S201). The estimation device 20 performs natural language processing on the estimated text (S202). The estimation device 20 analyzes the video of the estimated video advertisement and analyzes the facial expressions of the estimated performers (S203). The estimation device 20 obtains estimated video information based on the results of each analysis (S204). Details of each of the processing steps S200 to S204 are as described above for the processing of the estimated video information obtaining unit 202.

[0110] 10 , the estimation device 20 performs estimation using the machine learning model M based on the estimated video information and the machine learning model M (S21), and the processing in Fig. 10 ends. Details of the processing in S21 are as described above as the processing of the estimation unit 203.

[0111] [5. Summary of the embodiment] The learning device 10 of the present embodiment trains the machine learning model M based on training video information and training purchase information. The learning device 10 allows the machine learning model M to learn the training video information and the purchases of training viewers, thereby enabling the machine learning model M to estimate the advertising effectiveness from the features of the estimated video advertisement, and thus creating a machine learning model M with high accuracy in estimating the advertising effectiveness. For example, the learning device 10 allows the machine learning model M to learn training video information that has a causal relationship with the advertising effectiveness, thereby enabling the machine learning model M to learn the features of the training video advertisement itself. As a result, the machine learning model M can estimate the advertising effectiveness from the features of the estimated video advertisement itself, thereby improving the accuracy in estimating the advertising effectiveness.

[0112] The learning device 10 also acquires training video information obtained by performing natural language processing on the training text. The learning device 10 trains a machine learning model M that estimates advertising effectiveness from estimated video information obtained by performing processing similar to natural language processing on the estimated text. The learning device 10 trains the machine learning model M on the training video information analyzed by natural language processing, thereby enabling the machine learning model M to estimate advertising effectiveness from the characteristics of estimated text, which is text converted from the audio of an estimated video advertisement. As a result, the learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness. For example, by converting audio into text by the learning device 10, analysis based on various characteristics becomes easier.

[0113] Furthermore, the natural language processing is at least one of a sentiment analysis process, a response detection process, an explanation detection process, and a promotion detection process. For example, the learning device 10 can use the sentiment analysis process to cause the machine learning model M to learn the causal relationship between the emotions of the training performer and the purchases of the training viewers. The learning device 10 can use the response detection process to cause the machine learning model M to learn the causal relationship between the responses of the training performer and the purchases of the training viewers. The learning device 10 can use the explanation detection process to cause the machine learning model M to learn the causal relationship between the explanation given by the training performer and the purchases of the training viewers. The learning device 10 can use the promotion detection process to cause the machine learning model M to learn the causal relationship between the promotion in the training video advertisement and the purchases of the training viewers. As a result, the learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness.

[0114] The promotion detection process is a keyword detection process that detects at least one of keywords related to the price of the product or service introduced in the training video advertisement, keywords related to the sales of the product or service introduced in the training video advertisement, and keywords related to the description associated with the training video advertisement. The learning device 10 can use these keywords to detect promotions that can be effectively utilized in estimating advertising effectiveness. As a result, the learning device 10 can improve the estimation accuracy of the machine learning model M that estimates advertising effectiveness from promotions in video advertisements. For example, the learning device 10 can detect keywords such as coupon acquisition rates as price-related keywords and acquire a wider range of information by analyzing parameters other than the words related to the product or service introduced in the video advertisement.

[0115] The learning device 10 also acquires training video information related to the facial expressions of the training performers who appear in the training video advertisement, which is acquired by analyzing the video of the training video advertisement. The learning device 10 trains a machine learning model M that estimates advertising effectiveness from estimated video information related to the facial expressions of the estimated performers who appear in the estimated video advertisement, which is acquired by analyzing the video of the estimated video advertisement. The learning device 10 can train the machine learning model M to learn the causal relationship between the facial expressions of the training performers and purchases by training viewers. As a result, the learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness. For example, the learning device 10 can analyze individual characteristics of the training performers that cannot be analyzed by natural language processing.

[0116] The learning device 10 also acquires training purchase information related to at least one of whether or not a training viewer has made a purchase and information regarding sales of a product or service introduced in the training video advertisement. The learning device 10 trains a machine learning model M that estimates, as advertising effectiveness, at least one of whether or not a purchase has been made by an estimated viewer who is a viewer of the estimated video advertisement and information regarding sales of a product or service introduced in the estimated video advertisement. The learning device 10 can train the machine learning model M to understand the causal relationship between the training video information and at least one of these. As a result, the learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness. For example, since unit prices vary depending on the product or service, if analysis is performed using only the sales amount as a parameter, products with high unit prices may be advantageous. In this regard, the learning device 10 can create a more accurate machine learning model M by analyzing various characteristics such as sales volume.

[0117] The estimation device 20 of this embodiment estimates the advertising effectiveness of the estimated video advertisement based on the estimated video information and the machine learning model M. The estimation device 20 can estimate the advertising effectiveness from the characteristics of the estimated video advertisement itself, thereby improving the accuracy of estimating the advertising effectiveness.

[0118] [6. Modifications] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present disclosure.

[0119] Fig. 12 is a diagram showing an example of functions realized in a modified example. As shown in Fig. 12, in the modified example described below, the learning device 10 includes a contribution calculation unit 105 and a training viewer information acquisition unit 106. The contribution calculation unit 105 and the training viewer information acquisition unit 106 are realized by the control unit 11. The estimation device 20 includes an estimated viewer information acquisition unit 204. The estimated viewer information acquisition unit 204 is realized by the control unit 21.

[0120] [6-1. Variation 1] For example, as described in the embodiment, the training video information acquisition unit 102 may acquire multiple pieces of training video information. The learning unit 104 may further learn the machine learning model M based on the multiple pieces of training video information. These processes are as described in the embodiment. When multiple pieces of training video information are learned by the machine learning model M, some training video information may contribute strongly to advertising effectiveness, while other training video information may not contribute much to advertising effectiveness. Therefore, the contribution of each piece of training video information to advertising effectiveness may be calculated.

[0121] The learning device 10 of the first modification includes a contribution calculation unit 105. The contribution calculation unit 105 calculates the contribution of each of the multiple pieces of training video information to the advertising effectiveness. The contribution is the degree to which the training video information contributes to the advertising effectiveness. In other words, the contribution is the degree to which the training video information contributes to the estimation of the machine learning model M. The contribution may also be called importance, score, impact, or other names. In the first modification, an example is given in which the contribution is expressed as a numerical value between 0 and 100, but the contribution may be expressed as a numerical value in other ranges, or may be expressed as letters or symbols instead of numerical values. In the first modification, a high numerical value indicated by the contribution means that the training video information contributes strongly to the advertising effectiveness.

[0122] The method for calculating the contribution may be a well-known method known in the field of machine learning. For example, the contribution calculation unit 105 calculates the contribution of each of the multiple training video information based on a calculation method such as random forest, gradient boosting, SHAP (Shapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), L1 normalization, or L2 normalization. For example, if a change in the value of a certain training video information significantly changes the estimation result of the machine learning model M, the contribution calculation unit 105 calculates the contribution of the certain training video information so that the contribution of the certain training video information is higher. For example, the contribution calculation unit 105 calculates the contribution of the certain training video information so that the greater the increase in the numerical value indicated by the estimation result of the machine learning model M relative to the increase in the numerical value of the certain training video information, the higher the contribution of the certain training video information.

[0123] The contribution degree calculation unit 105 records the contribution degree of each of the multiple pieces of training video information in the data storage unit 100. The contribution degree may be used for any purpose. For example, the learning device 10 may display the contribution degree of each of the multiple pieces of training video information on the display unit 15. The learning device 10 may output data indicating the contribution degree of each of the multiple pieces of training video information to another computer or an information storage medium. The learning unit 104 may select at least one piece of training video information with a relatively high contribution degree from the multiple pieces of training video information, and train a new machine learning model M based only on the selected at least one piece of training video information.

[0124] The learning device 10 of variant example 1 calculates the contribution of each of multiple pieces of training video information to advertising effectiveness. The learning device 10 can identify training video information that contributes to advertising effectiveness based on the contribution of each piece of training video information. For example, the learning device 10 can present which of multiple pieces of training video information contributes to advertising effectiveness to a creator of the machine learning model M, a performer, or other person. For example, performers can create future video advertisements with higher advertising effectiveness by referring to the contribution degree. For example, the learning device 10 can identify which elements of the training video information used in learning the machine learning model M are effective in improving advertising effectiveness based on the contribution degree, and create a machine learning model M that can estimate advertising effectiveness with higher accuracy.

[0125] [6-2. Modification 2] For example, the machine learning model M may learn not only the characteristics of the training video advertisement but also the characteristics of the training viewers. The learning device 10 of Modification 2 includes a training viewer information acquisition unit 106. The training viewer information acquisition unit 106 acquires training viewer information related to the training viewers. For example, the training viewer information acquisition unit 106 acquires the training viewer information from the server 30. The training viewer information acquisition unit 106 may generate the training viewer information based on data acquired from the server 30. The training viewer information acquisition unit 106 may generate the training viewer information based on data acquired from the training viewer device 50. The training viewer information acquisition unit 106 is only required to acquire training viewer information for at least one training viewer, and may also acquire training viewer information for each of a plurality of training viewers.

[0126] The training viewer information is information that indicates some characteristic of the training viewer. For example, the training viewer information is behavior of the training viewer, demographic information of the training viewer, or a combination thereof. For example, the training viewer information acquisition unit 106 may acquire training viewer information related to at least one of the age of the training viewer, the gender of the training viewer, the purchase history of the training viewer at a store, training comments entered by the training viewer in response to the training video advertisement, acquisition of coupons related to the training video advertisement by the training viewer, viewing status of the training video advertisement by the training viewer, access status by the training viewer, and search by the training viewer.

[0127] Demographic information of the training viewers, such as their ages and genders, may be stored in the data storage unit 300 of the server 30, or may be input by the training viewers when viewing the training video advertisement. The training viewer information acquisition unit 106 may acquire the training viewer information by acquiring the ages and genders stored in advance in the data storage unit 300 of the server 30, or by acquiring the ages and genders input by the training viewers.

[0128] The purchase history of the training viewers at a store is a history of online or offline purchases at the store. The purchase history can also be referred to as a history of the training viewers' use of the store. For example, the purchase history is information about the store used by the training viewers, information about the products or services purchased by the training viewers, the date and time when the training viewers used the store, or a combination of these. The purchase history of the training viewers at a store may be stored in the data storage unit 300 of the server 30, or may be stored in another computer or information storage medium. The training viewer information acquisition unit 106 may acquire the training viewer information by acquiring the purchase history of the training viewers at the store from the data storage unit 300, another computer, or an information storage medium.

[0129] For example, comment data indicating training comments is stored in the video advertisement database DB3. The training viewer information acquisition unit 106 acquires training comment information by acquiring the training comments from the video advertisement database DB3. The training viewer information acquisition unit 106 may perform natural language processing on the training comments and acquire training viewer information based on the execution result of the natural language processing. The natural language processing may be the same as the processing on the training text described in the embodiment. For example, the training viewer information acquisition unit 106 may perform natural language processing such as sentiment analysis processing or response detection processing on the training comments.

[0130] For example, coupon data indicating the acquisition status of coupons related to training video advertisements by training viewers is stored in the video advertisement database DB3. The training viewer information acquisition unit 106 acquires training viewer information by acquiring the coupon acquisition status from the video advertisement database DB3. The coupon acquisition status may be the total number of coupons acquired by training viewers, the total number per unit time, a time-series transition of the total number, or a combination thereof.

[0131] For example, the viewing status of training video advertisements by training viewers is stored in the video advertisement database DB3. The training viewer information acquisition unit 106 acquires training viewer information by acquiring the viewing status from the video advertisement database DB3. The viewing status of training video advertisements may be any information related to viewing by training viewers, and may be, for example, the viewing time by training viewers (the time that the training viewers actually viewed the video advertisement out of the playback time of the video advertisement), the ratio of the viewing time to the playback time, the time period when the training viewers viewed the video advertisement, or a combination of these.

[0132] For example, access status of training viewers is stored in the video advertisement database DB3. The training viewer information acquisition unit 106 acquires training viewer information by acquiring the access status from the video advertisement database DB3. The access status may be the access status for any page, for example, the access status for a page of a live streaming service or the access status for an e-commerce page. The access status may be the presence or absence of access to a certain page, the number of accesses, the frequency of accesses, the time period of accesses, or a combination thereof.

[0133] For example, search results by training viewers are stored in the video advertisement database DB3. The training viewer information acquisition unit 106 acquires training viewer information by acquiring access status from the video advertisement database DB3. The search by the training viewer may be a search in a live streaming service, e-commerce, or other service. The search by the training viewer may be a query entered at the time of the search, the time period when the search was performed, user behavior after the search, or a combination thereof.

[0134] Note that the training viewer information may be any characteristic related to the training viewer and is not limited to the above examples. For example, the training viewer information acquisition unit 106 may acquire, as the training viewer information, the elapsed time since the training viewer started using the live streaming service, products or services purchased by the training viewer via the live streaming service, the number of video advertisements viewed by the training viewer via the live streaming service, the annual income of the training viewer, the preferences of the training viewer, or other information. The training viewer information acquisition unit 106 may acquire multiple pieces of training viewer information.

[0135] The learning unit 104 of Modification 2 learns the machine learning model M that estimates the advertising effectiveness by further using estimated viewer information about the estimated viewer who is the viewer of the estimated video advertisement, based on the training viewer information. For example, the learning unit 104 learns the machine learning model M that estimates the advertising effectiveness by further using estimated viewer information about at least one of the age of the estimated viewer, the gender of the estimated viewer, the purchase history at a store by the estimated viewer, the estimated comment input by the estimated viewer for the estimated video advertisement, the acquisition of coupons related to the estimated video advertisement by the estimated viewer, the viewing status of the estimated video advertisement by the estimated viewer, the access status by the estimated viewer, and the search by the estimated viewer. The learning unit 104 may train the machine learning model M in the same manner when other estimated viewer information is acquired.

[0136] For example, the learning unit 104 generates the input portion of the training data based on the training video information and the training viewer information. The learning unit 104 may not use the training viewer information as the input portion of the training data as it is, but may perform some processing, such as normalization or aggregation, on the training viewer information before using it as the input portion of the training data. For example, the learning unit 104 may aggregate the training video information of each of multiple training viewers and use the aggregation result as the input portion of the training data. The learning unit 104 may aggregate the gender ratio or number of training viewers who viewed the training video advertisement and use the ratio or number of people as the input portion of the training data.

[0137] For another example, the learning unit 104 may compile the age distribution of training viewers who viewed a training video advertisement and use the distribution as an input part of the training data. The learning unit 104 may compile the ratio or number of training viewers who purchased a specific product and use the ratio or number as an input part of the training data. Similarly, the learning unit 104 may perform a compilation process or the like on other training viewer information and then use the information as an input part of the training data. Although the input part of the training data is different from that in the embodiment, the process in which the learning unit 104 causes the machine learning model M to learn the training data is as described in the embodiment.

[0138] The learning device 10 of Variation 2 learns a machine learning model M that estimates advertising effectiveness by further utilizing estimated viewer information about estimated viewers who are viewers of estimated video advertisements, based on training viewer information. The learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness by having the machine learning model M learn not only training video information but also training viewer information. For example, if there is a causal relationship between the characteristics of the training viewers and the advertising effectiveness (e.g., if the advertising is effective for a specific age group), the machine learning model M can estimate the advertising effectiveness according to the characteristics of the training viewers. For example, the learning device 10 can broaden the scope of learning by having the machine learning model M learn the characteristics of individual training viewers that cannot be learned from training purchase information alone.

[0139] The learning device 10 also acquires training viewer information related to at least one of the following: the age of the training viewer, the gender of the training viewer, the in-store purchase history of the training viewer, training comments entered by the training viewer in response to the training video advertisement, the acquisition of coupons related to the training video advertisement by the training viewer, the viewing status of the training video advertisement by the training viewer, the access status of the training viewer, and searches by the training viewer. The learning device 10 further uses estimated viewer information related to at least one of the items to train a machine learning model M that estimates advertising effectiveness. The learning device 10 can create a machine learning model M with high accuracy in estimating advertising effectiveness by having the machine learning model M learn not only the training video information but also the at least one item. For example, if there is a causal relationship between the at least one item and advertising effectiveness, the machine learning model M can estimate advertising effectiveness according to the characteristics of the training viewer.

[0140] [6-3. Modification 3] For example, in Modification 2, the training viewer information acquisition unit 106 may acquire training viewer information regarding training comments by performing at least one of an emotion analysis process for analyzing the emotions of training viewers, a comment classification process for classifying the training comments, and a keyword detection process for detecting keywords from the training comments.

[0141] The emotion analysis process may be the same as the emotion analysis process for the training text described in the embodiment. The training viewer information acquisition unit 106 analyzes the emotions of the training viewers, such as joy, anger, sadness, and happiness, by performing emotion analysis process for the training comments, and acquires training viewer information indicating the analysis results. In the description of the emotion analysis process for the training text in the embodiment, "training text" may be read as "training comments," and "training performers" may be read as "training viewers." Similar to the emotion analysis process for the training text, various emotion analysis processes may be performed on the training comments.

[0142] The comment classification process is a process of classifying the content of the training comments. The comment classification process can utilize a method used to classify the meaning of words in natural language processing. For example, the training viewer information acquisition unit 106 may classify the training comments based on a rule approach that determines whether the training comments contain keywords that correspond to each of multiple categories. As another example, the training viewer information acquisition unit 106 may classify the training comments based on a machine learning approach that calculates feature quantities of the training comments and outputs classification results. For example, in the comment classification process, the training comments are classified as either positive or negative. The training comments may also be classified into other categories.

[0143] The keyword detection process is a process of detecting keywords included in the training comments. The training viewer information acquisition unit 106 determines whether or not a keyword stored in a dictionary database storing keywords related to price (e.g., "cheap" or "expensive") is included in the training comment, and detects the keyword. The training viewer information acquisition unit 106 determines whether or not a keyword stored in a dictionary database storing keywords related to impressions of a product or service (e.g., "cute" or "cool") is included in the training comment, and detects the keyword.

[0144] The learning unit 104 performs learning of the machine learning model M that estimates advertising effectiveness by further utilizing estimated viewer information regarding the estimated comment obtained by executing at least one of a sentiment analysis process that analyzes the sentiment of the estimated viewer, a comment classification process that classifies the estimated comment, and a keyword detection process that detects keywords from the estimated comment. The learning unit 104 may aggregate the results of the sentiment analysis process, the comment classification process, and the keyword detection process, and generate an input portion of the training data based on the aggregation result (e.g., 100 positive comments, 50 negative comments, etc.). This point is also as described in Modification Example 2.

[0145] In addition, for details of the processing for estimated comments, "training" in the description of the processing for training comments can be read as "estimation." Similarly, for the natural language processing for estimated comments, "training" in the description of the natural language processing for training comments can be read as "estimation." Since the processing during estimation is performed by the estimation device 20, the processing for estimated comments is also performed by the estimation device 20.

[0146] The learning device 10 of Variation 3 acquires training viewer information related to the training comments by performing at least one of a sentiment analysis process, a comment classification process, and a keyword detection process on the training comments. The learning device 10 further utilizes the estimated viewer information acquired by performing at least one of a sentiment analysis process, a comment classification process, and a keyword detection process on the estimated comments to train a machine learning model M that estimates advertising effectiveness. The learning device 10 can create a machine learning model M with high estimation accuracy of advertising effectiveness by having the machine learning model M learn the analysis results of at least one of the sentiment analysis process, the comment classification process, and the keyword detection process. For example, the learning device 10 can further improve the estimation accuracy of the machine learning model M by performing an analysis based on information related to the responses of the training viewers.

[0147] [6-4. Variation 4] For example, as described in the embodiment, the training video information acquisition unit 102 may acquire multiple pieces of training video information. As described in Variations 2 and 3, the training viewer information acquisition unit 106 may acquire multiple pieces of training viewer information. In this case, the learning unit 104 learns a machine learning model M that estimates advertising effectiveness from multiple pieces of estimated video information and multiple pieces of estimated viewer information, based on the multiple pieces of training video information and the multiple pieces of training viewer information. These processes are as described in the embodiment, Variation 2, and Variation 3.

[0148] The learning device 10 of Modification 4 includes a contribution calculation unit 105. The contribution calculation unit 105 calculates the contribution of each of the plurality of training video information and each of the plurality of training viewer information to the advertising effectiveness. Among the functions of the contribution calculation unit 105, the configuration for calculating the contribution of each of the plurality of training video information may be the same as that of Modification 1. The contribution calculation unit 105 of Modification 4 calculates not only the contribution of each of the plurality of training video information, but also the contribution of each of the plurality of training viewer information. That is, the contribution calculation unit 105 calculates the contribution of each of the plurality of training viewer information, such as the emotions of the training viewers and the training comments of the training viewers, in order to analyze which of the plurality of training viewer information contributes to the advertising effectiveness.

[0149] The meaning of the contribution degree of training viewer information is the same as that of the contribution degree of training video information. Regarding the meaning of the contribution degree of training viewer information, "video" in the description of the contribution degree of training video information in Modification 1 can be read as "viewer." The contribution degree calculation unit 105 may calculate the contribution degree of training viewer information using a calculation method similar to that of the calculation method of the contribution degree of training video information. Regarding the calculation method of the contribution degree of training viewer information, "video" in the description of the calculation method of the contribution degree of training video information in Modification 1 can be read as "viewer." The contribution degree calculation unit 105 of Modification 4 records the contribution degree of each of the plurality of pieces of training video information and each of the plurality of pieces of training viewer information in the data storage unit 100. As in Modification 1, the contribution degree may be used for any purpose.

[0150] The learning device 10 of Variation 4 calculates the contribution of each of a plurality of pieces of training video information and each of a plurality of pieces of training viewer information to advertising effectiveness. The learning device 10 can identify the training video information and training viewer information that contribute to advertising effectiveness based on the contribution of each of the plurality of pieces of training video information and each of the plurality of pieces of training viewer information. For example, the learning device 10 can present to a creator of the machine learning model M, a performer, or other person which of the plurality of pieces of training video information and the plurality of pieces of training viewer information contribute to advertising effectiveness. For example, performers can create future video advertisements with higher advertising effectiveness by referring to the contribution degree. For example, the learning device 10 can identify which elements of the training video information and training viewer information used in learning the machine learning model M are effective in improving advertising effectiveness based on the contribution degree, thereby creating a machine learning model M that can estimate advertising effectiveness with higher accuracy.

[0151] [6-5. Variation 5] For example, in Variation 4, the learning unit 104 may select at least one of the plurality of training video information and the plurality of training viewer information based on the contribution rate of each of the plurality of training video information and the contribution rate of each of the plurality of training viewer information, and may learn a new machine learning model M based on the selected at least one.

[0152] The learning unit 104 selects at least one piece of training video information with a relatively high contribution degree from among the plurality of pieces of training video information. For example, the learning unit 104 selects at least one piece of training video information with a contribution degree equal to or greater than a threshold from among the plurality of pieces of training video information. The learning unit 104 may select a predetermined number of pieces of training video information from among the plurality of pieces of training video information in descending order of contribution degree. The learning unit 104 uses only the selected at least one piece of training video information from among the plurality of pieces of training video information in learning the new machine learning model M.

[0153] The learning unit 104 selects at least one piece of training viewer information with a relatively high contribution level from the plurality of pieces of training viewer information. For example, the learning unit 104 selects at least one piece of training viewer information with a contribution level equal to or greater than a threshold from the plurality of pieces of training viewer information. The learning unit 104 may select a predetermined number of pieces of training viewer information from the plurality of pieces of training viewer information in descending order of contribution level. The learning unit 104 uses only the selected at least one piece of training viewer information from the plurality of pieces of training viewer information in learning the new machine learning model M.

[0154] For example, the learning unit 104 generates an input portion of training data based on the selected at least one piece of training video information and the selected at least one piece of training viewer information. The learning unit 104 learns a new machine learning model M based on the training data. Although the training data is different, the learning method itself may be the same as that of the old machine learning model M (the machine learning model M created temporarily to calculate the contribution rate). The old machine learning model M here is the machine learning model M created by the method described in the embodiment and modifications 1 to 4.

[0155] The learning device 10 of Variation 5 selects at least one of the plurality of training video information and the plurality of training viewer information based on the contribution degree of each of the plurality of training video information and the contribution degree of each of the plurality of training viewer information, and learns a new machine learning model M based on the selected at least one. This allows the learning device 10 to reduce the amount of information input to the machine learning model M during estimation, thereby generating a machine learning model M that can reduce the processing load during estimation. Furthermore, training video information and training viewer information with relatively low contribution degrees are no longer learned by the machine learning model M, thereby improving the accuracy of the machine learning model M.

[0156] [6-6. Modification 6] For example, the estimation device 20 may perform estimation based on a machine learning model M generated by the methods described in Modifications 2 to 5. The machine learning model M in Modification 6 is further trained based on training viewer information about training viewers. The machine learning model M is as described in Modifications 2 to 5.

[0157] The estimation device of Variation 6 includes an estimated viewer information acquisition unit 204. The estimated viewer information acquisition unit 204 acquires estimated viewer information about estimated viewers who are viewers of the estimated video advertisement. The estimated viewer information acquisition unit 204 may acquire the estimated viewer information based on the same acquisition method as that for the training viewer information. Therefore, in the details of the process by which the estimated viewer information acquisition unit 204 acquires the estimated viewer information, "training" in the description of the training viewer information acquisition unit 106 can be read as "estimation."

[0158] For example, the estimated viewer information is behavior of the estimated viewer, demographic information of the estimated viewer, or a combination thereof. This information may be stored in the data storage unit 300 of the server 30. The estimated viewer information acquisition unit 204 may acquire estimated viewer information related to at least one of the age of the estimated viewer, the gender of the estimated viewer, the purchase history of the estimated viewer at a store, the estimated comment entered by the estimated viewer in response to the estimated video advertisement, the acquisition of a coupon related to the estimated video advertisement by the estimated viewer, the viewing status of the estimated video advertisement by the estimated viewer, the access status by the estimated viewer, and the search by the estimated viewer.

[0159] For example, the estimated viewer information acquisition unit 204 may acquire estimated viewer information regarding an estimated comment by performing at least one of a sentiment analysis process for analyzing the sentiment of an estimated viewer, a comment classification process for classifying the estimated comment, and a keyword detection process for detecting keywords from the estimated comment. The estimated viewer information acquisition unit 204 may acquire multiple pieces of estimated viewer information.

[0160] The estimation unit 203 of variant example 6 estimates advertising effectiveness further based on estimated viewer information. The estimation unit 203 inputs not only estimated video information but also estimated viewer information to the machine learning model M. Although the input to the machine learning model M differs from that of the embodiment, the processing executed by the machine learning model M is the same as that of the embodiment. The machine learning model M calculates features based on not only the estimated video information but also the estimated viewer information, and outputs an estimation result according to the features. The estimation unit 203 may aggregate the estimated viewer information and then input the aggregation result to the machine learning model M.

[0161] The estimation device 20 of Variation 6 estimates the advertising effectiveness further based on estimated viewer information. The estimation device 20 can improve the estimation accuracy of the advertising effectiveness by inputting not only estimated video information but also estimated viewer information into the machine learning model M. For example, if there is a causal relationship between the characteristics of the estimated viewer and the advertising effectiveness, the machine learning model M can estimate the advertising effectiveness according to the characteristics of the estimated viewer. For example, the estimation device 20 can improve the estimation accuracy by having the machine learning model M learn the characteristics of individual estimated viewers that cannot be estimated based on estimated purchase information alone.

[0162] [6-7. Other Modifications] For example, the modifications described above may be combined.

[0163] For example, the learning device 10 and the estimation device 20 may be the same device. The functions described as being realized by the learning device 10 may be shared among multiple computers. The functions described as being realized by the estimation device 20 may be shared among multiple computers. The functions described as being realized by the server 30 may be realized by the learning device 10 or the estimation device 20.

[0164] [7. Supplementary Note] For example, the learning device can also have the following configuration.

[0165] (1) A learning device including: a training video information acquisition unit that acquires training video information acquired by analyzing a training video advertisement that is a video advertisement for training; a training purchase information acquisition unit that acquires training purchase information related to purchases by training viewers who are viewers of the training video advertisement; and a learning unit that trains a machine learning model that estimates advertising effectiveness of an estimated video advertisement from estimated video information acquired by analyzing an estimated video advertisement that is a video advertisement for estimation based on the training video information and the training purchase information. (2) The training video information acquisition unit acquires the training video information acquired by performing natural language processing on training text acquired by analyzing audio of the training video advertisement, and the learning unit trains the machine learning model that estimates the advertising effectiveness from the estimated video information acquired by performing processing similar to the natural language processing on estimated text acquired by analyzing the audio of the estimated video advertisement. (3) The learning device according to (2), wherein the natural language processing is at least one of a sentiment analysis process that analyzes the sentiment of a training performer who is a performer in the training video advertisement, a response detection process that detects a response by the training performer who is a performer in the training video advertisement, a description detection process that detects a description of a product or service introduced in the training video advertisement, and a promotion detection process that detects a promotion related to the training video advertisement. (4) The learning device according to (3), wherein the promotion detection process is a keyword detection process that detects at least one of a keyword related to a price of a product or service introduced in the training video advertisement, a keyword related to sales of the product or service introduced in the training video advertisement, and a keyword related to a description associated with the training video advertisement.(5) The learning device according to any of (1) to (4), wherein the training video information acquisition unit acquires the training video information related to facial expressions of training performers who are performers in the training video advertisement, acquired by analyzing a video of the training video advertisement, and the learning unit trains the machine learning model that estimates the advertising effectiveness from the estimated video information related to facial expressions of estimated performers who are performers in the estimated video advertisement, acquired by analyzing a video of the estimated video advertisement. (6) The learning device according to any of (1) to (5), wherein the training purchase information acquisition unit acquires the training purchase information related to at least one of whether or not the training viewer has made a purchase and information on sales of a product or service introduced in the training video advertisement, and the learning unit trains the machine learning model that estimates at least one of whether or not the estimated viewer who is a viewer of the estimated video advertisement has made a purchase and information on sales of a product or service introduced in the estimated video advertisement, as the advertising effectiveness. (7) The learning device according to any of (1) to (6), wherein the training video information acquisition unit acquires a plurality of pieces of training video information, the learning unit performs learning of the machine learning model further based on the plurality of pieces of training video information, and the learning device further includes a contribution calculation unit that calculates a contribution of each of the plurality of pieces of training video information to the advertising effectiveness. (8) The learning device according to any of (1) to (7), wherein the learning device further includes a training viewer information acquisition unit that acquires training viewer information about the training viewers, and the learning unit performs learning of the machine learning model that estimates the advertising effectiveness by further utilizing estimated viewer information about estimated viewers who are viewers of the estimated video advertisement, based on the training viewer information.(9) The learning device described in (8), wherein the training viewer information acquisition unit acquires the training viewer information related to at least one of the age of the training viewer, the gender of the training viewer, the purchasing history of the training viewer at a store, the training comments input by the training viewer in response to the training video advertisement, the acquisition of a coupon related to the training video advertisement by the training viewer, the viewing status of the training video advertisement by the training viewer, the access status by the training viewer, and a search by the training viewer; and the learning unit further uses the estimated viewer information related to at least one of the age of the estimated viewer, the gender of the estimated viewer, the purchasing history of the estimated viewer at a store, the estimated comments input by the estimated viewer in response to the estimated video advertisement, the acquisition of a coupon related to the estimated video advertisement by the estimated viewer, the viewing status of the estimated video advertisement by the estimated viewer, the access status by the estimated viewer, and a search by the estimated viewer to learn the machine learning model that estimates the advertising effectiveness. (10) The learning device described in (9), wherein the training viewer information acquisition unit acquires the training viewer information regarding the training comments by performing at least one of a sentiment analysis process for analyzing the sentiment of the training viewer, a comment classification process for classifying the training comments, and a keyword detection process for detecting keywords from the training comments, and the learning unit further utilizes the estimated viewer information regarding the estimated comments acquired by performing at least one of a sentiment analysis process for analyzing the sentiment of the estimated viewer, a comment classification process for classifying the estimated comments, and a keyword detection process for detecting keywords from the estimated comments, to learn the machine learning model that estimates the advertising effectiveness.(11) The learning device according to any of (8) to (10), wherein the training video information acquisition unit acquires a plurality of pieces of training video information, the training viewer information acquisition unit acquires a plurality of pieces of training viewer information, the learning unit learns the machine learning model that estimates the advertising effectiveness from a plurality of pieces of estimated video information and a plurality of pieces of estimated viewer information based on the plurality of pieces of training video information and the plurality of pieces of training viewer information, and the learning device further includes a contribution calculation unit that calculates a contribution of each of the plurality of pieces of training video information and each of the plurality of pieces of training viewer information to the advertising effectiveness. (12) The learning device according to (11), wherein the learning unit selects at least one of the plurality of pieces of training video information and the plurality of pieces of training viewer information based on the contribution degree of each of the plurality of pieces of training video information and the contribution degree of each of the plurality of pieces of training viewer information, and learns a new machine learning model based on the selected at least one piece.

Claims

1. A learning device including: a training video information acquisition unit that acquires training video information acquired by analyzing a training video advertisement, which is a video advertisement for training; a training purchasing information acquisition unit that acquires training purchasing information regarding purchases by training viewers who are viewers of the training video advertisement; and a learning unit that trains a machine learning model that estimates the advertising effectiveness of an estimated video advertisement from estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation, based on the training video information and the training purchasing information.

2. The learning device described in claim 1, wherein the training video information acquisition unit acquires the training video information acquired by performing natural language processing on training text acquired by analyzing the audio of the training video advertisement, and the learning unit learns the machine learning model that estimates the advertising effectiveness from the estimated video information acquired by performing processing similar to the natural language processing on estimated text acquired by analyzing the audio of the estimated video advertisement.

3. The learning device of claim 2, wherein the natural language processing is at least one of a sentiment analysis process for analyzing the emotions of a training performer who appears in the training video advertisement, a response detection process for detecting a response by a training performer who appears in the training video advertisement, an explanation detection process for detecting an explanation regarding a product or service introduced in the training video advertisement, and a promotion detection process for detecting a promotion related to the training video advertisement.

4. The learning device described in claim 3, wherein the promotion detection process is a keyword detection process that detects at least one of keywords related to the price of a product or service introduced in the training video advertisement, keywords related to sales of a product or service introduced in the training video advertisement, and keywords related to a description associated with the training video advertisement.

5. A learning device as described in any of claims 1 to 4, wherein the training video information acquisition unit acquires the training video information relating to facial expressions of training performers who are performers in the training video advertisement, obtained by analyzing the video of the training video advertisement, and the learning unit learns the machine learning model that estimates the advertising effectiveness from the estimated video information relating to facial expressions of the estimated performers who are performers in the estimated video advertisement, obtained by analyzing the video of the estimated video advertisement.

6. A learning device as described in any of claims 1 to 4, wherein the training purchasing information acquisition unit acquires the training purchasing information regarding at least one of whether or not the training viewer has made a purchase and information regarding sales of a product or service introduced in the training video advertisement, and the learning unit learns the machine learning model that estimates at least one of whether or not the estimated viewer who is a viewer of the estimated video advertisement has made a purchase and information regarding sales of a product or service introduced in the estimated video advertisement as the advertising effect.

7. A learning device as described in any of claims 1 to 4, wherein the training video information acquisition unit acquires a plurality of pieces of training video information; the learning unit learns the machine learning model based further on the plurality of pieces of training video information; and the learning device further includes a contribution degree calculation unit that calculates the contribution degree of each of the plurality of pieces of training video information to the advertising effect.

8. A learning device as described in any of claims 1 to 4, further comprising a training viewer information acquisition unit that acquires training viewer information regarding the training viewer, and the learning unit further uses estimated viewer information regarding the estimated viewer who is a viewer of the estimated video advertisement based on the training viewer information to learn the machine learning model that estimates the advertising effectiveness.

9. The learning device according to claim 8, wherein the training viewer information acquisition unit acquires the training viewer information relating to at least one of the age of the training viewer, the gender of the training viewer, the purchasing history of the training viewer in a store, the training comment entered by the training viewer in response to the training video advertisement, the acquisition of a coupon related to the training video advertisement by the training viewer, the viewing status of the training video advertisement by the training viewer, the access status by the training viewer, and a search by the training viewer; and the learning unit further uses the estimated viewer information relating to at least one of the age of the estimated viewer, the gender of the estimated viewer, the purchasing history of the training viewer in a store, the estimated comment entered by the estimated viewer in response to the estimated video advertisement, the acquisition of a coupon related to the estimated video advertisement by the estimated viewer, the viewing status of the estimated video advertisement by the estimated viewer, the access status by the estimated viewer, and a search by the estimated viewer to learn the machine learning model that estimates the advertising effectiveness.

10. The learning device described in claim 9, wherein the training viewer information acquisition unit acquires the training viewer information regarding the training comments by executing, on the training comments, at least one of a sentiment analysis process for analyzing the sentiment of the training viewer, a comment classification process for classifying the training comments, and a keyword detection process for detecting keywords from the training comments; and the learning unit further utilizes the estimated viewer information regarding the estimated comment acquired by executing, on the estimated comment, at least one of a sentiment analysis process for analyzing the sentiment of the estimated viewer, a comment classification process for classifying the estimated comment, and a keyword detection process for detecting keywords from the estimated comment, to learn the machine learning model that estimates the advertising effectiveness.

11. A learning device as described in any of claims 8 to 10, wherein the training video information acquisition unit acquires a plurality of the training video information, the training viewer information acquisition unit acquires a plurality of the training viewer information, the learning unit learns the machine learning model that estimates the advertising effectiveness from a plurality of the estimated video information and a plurality of the estimated viewer information based on the plurality of the training video information and the plurality of the training viewer information, and the learning device further includes a contribution calculation unit that calculates the contribution of each of the plurality of the training video information and each of the plurality of the training viewer information to the advertising effectiveness.

12. The learning device described in claim 11, wherein the learning unit selects at least one of the plurality of training video information and the plurality of training viewer information based on the contribution degree of each of the plurality of training video information and the contribution degree of each of the plurality of training viewer information, and learns a new machine learning model based on the selected at least one.

13. An estimation device including: an estimated video information acquisition unit that acquires estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation; a model memory unit that stores a machine learning model that has been trained based on training video information acquired by analyzing a training video advertisement, which is a video advertisement for training, and training purchasing information regarding purchases of training viewers who are viewers of the training video advertisement; and an estimation unit that estimates the advertising effectiveness of the estimated video advertisement based on the estimated video information and the machine learning model.

14. The estimation device described in claim 13, wherein the machine learning model is further trained based on training viewer information regarding the training viewer, the estimation device further includes an estimated viewer information acquisition unit that acquires estimated viewer information regarding an estimated viewer who is a viewer of the estimated video advertisement, and the estimation unit estimates the advertising effectiveness further based on the estimated viewer information.

15. A learning method including: a training video information acquisition step of acquiring training video information acquired by analyzing a training video advertisement, which is a video advertisement for training; a training purchasing information acquisition step of acquiring training purchasing information regarding purchases of training viewers who are viewers of the training video advertisement; and a learning step of learning a machine learning model that estimates the advertising effectiveness of an estimated video advertisement from estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation, based on the training video information and the training purchasing information.

16. An estimation method including: an estimated video information acquisition step of acquiring estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation; and an estimation step of estimating the advertising effectiveness of the estimated video advertisement based on a machine learning model that has been trained based on training video information acquired by analyzing a training video advertisement, which is a video advertisement for training, and training purchasing information regarding purchases of training viewers who are viewers of the training video advertisement, and the estimated video information.

17. A program for causing a computer to function as: a training video information acquisition unit that acquires training video information acquired by analyzing a training video advertisement, which is a video advertisement for training; a training purchasing information acquisition unit that acquires training purchasing information regarding purchases by training viewers who are viewers of the training video advertisement; and a learning unit that learns a machine learning model that estimates the advertising effectiveness of an estimated video advertisement from estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation, based on the training video information and the training purchasing information.

18. A program for causing a computer to function as: an estimated video information acquisition unit that acquires estimated video information acquired by analyzing an estimated video advertisement, which is a video advertisement for estimation; and an estimation unit that estimates the advertising effectiveness of the estimated video advertisement based on a machine learning model that has been trained based on training video information acquired by analyzing a training video advertisement, which is a video advertisement for training, and training purchasing information regarding purchases by training viewers who are viewers of the training video advertisement, and the estimated video information.

Images