Estimation apparatus, estimation method, and estimation program

Abstract

To appropriately estimate a distribution surface involved in an implementation violation.SOLUTION: An estimation apparatus 10 according to an embodiment comprises an acquisition unit 12a, an estimation unit 12b, and an output unit 12c. The acquisition unit 12a acquires information related to an operation history of web content to be examined. The estimation unit 12b estimates whether or not the web content satisfies a predetermined condition from the information related to the operation history acquired by the acquisition unit 12a by using a model having learned characteristics of the operation history that occurs when the web content does not satisfy a predetermined condition. The output unit 12c outputs the result estimated by the estimation unit 12b.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an estimation device, an estimation method, and an estimation program. [Background technology]

[0002] In recent years, opportunities to view web advertisements have increased in various situations, such as various social media, search engines, and websites. One type of web advertisement is pay-per-click advertising. Pay-per-click advertising is characterized by the fact that the advertising fee varies depending on the number of clicks on the advertisement by the user. For this reason, the distribution surfaces on which the web advertisements are implemented may be implemented in a way that makes it easier to mis-tap (tapping the advertisement by mistake when you intend to close it) in order to increase the number of clicks.

[0003] Guidelines are set for the implementation of web advertisements, and whether or not the advertisements are implemented in accordance with the guidelines is judged by an examination. Therefore, any ad placement that is harmful to advertisers and site visitors and that is prone to accidental tapping will be judged as violating the implementation.

[0004] Conventionally, in ad screening, CTR (Click Through Rate), which indicates the ratio of the number of clicks on an ad to the number of times the ad is displayed, is used, and in some cases, delivery spaces that may have implementation violations are given priority in screening. Specifically, delivery spaces that have a relatively high CTR due to the addition of incorrect taps to normal taps are judged to have a lot of incorrect taps and may have implementation violations, and are given priority in screening. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] “(Example of violation of advertising implementation guidelines) What is an interstitial ad that does not cause stress to users?”, [online], [Searched on July 28, 2023], Internet〈https: / / marketing.yahoo.co.jp / blog / post / 2021083130174708.html〉 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the conventional technology, there are cases where it is not possible to appropriately estimate a delivery surface that is an implementation violation. For example, in the conventional technology, a delivery surface with a high CTR is estimated as a delivery surface that is an implementation violation, so that a delivery surface that is unlikely to have a high CTR and is implemented so that it is easy to erroneously tap cannot be estimated as a delivery surface that is an implementation violation. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objective, the estimation device of this embodiment is characterized by comprising an acquisition unit that acquires information regarding the operation history of web content to be reviewed, an estimation unit that estimates whether or not the web content satisfies specified conditions from the information regarding the operation history acquired by the acquisition unit using a model that has learned characteristics of the operation history that arise when the web content does not satisfy specified conditions, and an output unit that outputs the result estimated by the estimation unit. Effect of the Invention

[0008] According to the present invention, it is possible to appropriately estimate a distribution surface that is a mounting violation. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of a distribution surface that causes a mounting violation. [Diagram 2] FIG. 2 is a diagram illustrating an example of an estimation system according to the embodiment. [Diagram 3]FIG. 3 is a diagram illustrating an example of the configuration of an estimation device according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating a specific example of data stored in the storage unit of the estimation device according to the embodiment. [Diagram 5] FIG. 5 is a diagram illustrating the relationship between the characteristics of the operation history and the implementation violations on the distribution surface according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of features used in a model and the importance of each feature. [Figure 7] FIG. 7 is a flowchart illustrating an example of a processing procedure according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating the effect of the estimation process according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of an estimation device, an estimation method, and an estimation program according to the present application will be described in detail with reference to the drawings. Note that the estimation device, the estimation method, and the estimation program according to the present application are not limited to the embodiment.

[0011] 1. Introduction First, a description will be given of a delivery surface (web content) that is estimated to be an implementation violation by the estimation device of this embodiment. As described above, a pay-per-click advertisement has a feature that the advertising fee varies depending on the number of clicks on the advertisement by the user, so the advertisement may be implemented in a way that makes it easy to make an erroneous tap. A delivery surface that is easy to make an erroneous tap is determined to be a delivery surface that violates the implementation because it is harmful to users and advertisers.

[0012] Here, a description will be given of a delivery surface with implementation violation (a delivery surface that is easily tapped by mistake) with reference to Fig. 1. Fig. 1 is a diagram showing an example of a delivery surface with implementation violation. Fig. 1 shows diagrams (A) and (B) displaying the same advertisement, with Fig. 1 (A) being a delivery surface with implementation violation and Fig. 1 (B) being a delivery surface with OK implementation.

[0013] The delivery surface in Figure 1 (A) displays the ad frame and the "close button" adjacent to each other. Therefore, when the ad is displayed on a user's smartphone, even if the user intends to tap the "close button," if the tap position is slightly off, the user will end up tapping the ad instead. In other words, the delivery surface in Figure 1 (A) induces erroneous taps, causing stress to users and hindering accurate ad analysis, and is therefore judged to be a delivery surface that violates implementation standards.

[0014] On the other hand, the delivery surface in Figure 1(B) has a certain distance between the ad frame and the "close button." Therefore, compared to Figure 1(A), it is less likely to cause accidental taps, and is therefore judged to be an OK delivery surface to implement.

[0015] The estimation device according to this embodiment estimates a delivery surface such as that shown in FIG. 1A, which is harmful to users and advertisers, as a delivery surface that does not comply with the advertising implementation guidelines and is likely to be a violation of the implementation guidelines.

[0016] 2. Overall Structure Next, an estimation system according to the present embodiment will be described. Fig. 2 is a diagram showing an example of the estimation system according to the embodiment. As shown in Fig. 2, the estimation system according to the present embodiment is configured by connecting an estimation device 10 and an external terminal 20 to each other via a network N.

[0017] The estimation device 10 is, for example, a server device that stores a learning model used to estimate a distribution surface of an implementation violation. The estimation device 10 is connected to a network N, and acquires an operation history of a distribution surface to be inspected from the outside and outputs an estimation result to an external terminal 20.

[0018] The external terminal 20 is, for example, a PC (Personal Computer), a smartphone, etc. The external terminal 20 receives information on the distribution side estimated to be a mounting violation and the score calculated at that time from the estimation device 10 via the network N, and outputs the information on the screen.

[0019] The estimation device 10 acquires information about the operation history of the web content to be examined. Then, using a model that has learned the characteristics of the operation history that occur when the web content does not satisfy a predetermined condition, the estimation device 10 estimates whether the web content satisfies the predetermined condition from the acquired information about the operation history. Then, the estimation result is output.

[0020] First, the estimation device 10 acquires information on an operation history of the web content to be examined. For example, the estimation device 10 acquires, via the network N, an operation history of a user with respect to an advertisement displayed by the distribution surface to be examined and information obtained by analyzing the operation history.

[0021] Next, the estimation device 10 estimates whether or not the web content satisfies a predetermined condition from the acquired information on the operation history, using a model that has learned characteristics of the operation history that occur when the web content does not satisfy a predetermined condition. For example, the estimation device 10 estimates whether or not the UI (User Interface) of the distribution surface to be examined is likely to be an implementation violation from information obtained by analyzing the operation history for the distribution surface to be examined, using a model that has learned characteristics of the operation history, such as the above-mentioned CTR, CVR (Conversion Rate), time from advertisement display to click, and advertisement size.

[0022] After that, the estimation device 10 outputs the estimation result. For example, the estimation device 10 transmits the estimation result to the external terminal 20 via the network N, and causes the external terminal 20 to display information such as a distribution surface having a high possibility of an implementation violation and a score calculated for each of the distribution surfaces to be examined.

[0023] 3. Configuration of Estimation Device 10 Next, the configuration of the estimation device 10 shown in Fig. 2 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the estimation device according to the embodiment. As shown in Fig. 3, the estimation device 10 according to the embodiment includes a communication unit 11, a control unit 12, and a storage unit 13. Furthermore, the estimation device 10 and an external terminal 20 are connected to each other via a network N in a wired or wireless manner so as to be able to communicate with each other.

[0024] The communication unit 11 is realized by, for example, a network interface card (NIC) etc. The communication unit 11 is connected to a network N by wire or wirelessly, and transmits and receives information to and from an external terminal 20, an external device, etc. via the network N.

[0025] The storage unit 13 is realized by a storage device such as a RAM (Random Access Memory), a hard disk, etc. The storage unit 13 stores data and programs necessary for various processes by the control unit 12, and has an operation history information storage unit 13a, a model information storage unit 13b, and an estimation result storage unit 13c as units closely related to the present invention.

[0026] The operation history information storage unit 13a stores information obtained by analyzing the operation history of users on the distribution surfaces to be examined. For example, the operation history information storage unit 13a stores information obtained by analyzing the operation history of users on each distribution surface to be examined.

[0027] Here, with reference to (A) of Fig. 4, the features of the operation history and the values ​​measured by analyzing each feature stored in the operation history information storage unit 13a will be described. Fig. 4 is a diagram showing a specific example of data stored in the storage unit of the estimation device according to the embodiment. As shown in (A) of Fig. 4, the operation history information storage unit 13a is composed of, for example, a "feature" item and a "calculated value" item.

[0028] The "Features" item stores the features of the user's operation history that are used to learn a model used to estimate the distribution side of the implementation violation. The "Calculated Value" item stores a calculated value calculated by analyzing the corresponding feature.

[0029] In FIG. 4(A), "CVR", "tCVR", and "CTR" are shown as features. "CVR" is a feature calculated by CV / click, and "tCVR" is a feature that targets CV t seconds or more after a click. And "CVR: 0.15", "tCVR: 0.1", and "CTR: 0.2" are stored as the measured values ​​of each feature.

[0030] The operation history information storage unit 13a can store, in addition to the features shown in (A) of FIG. 4, features such as the bounce rate at the Nth second point, the image size of the advertisement, and the time from the advertisement display to the click, and can store a calculated value obtained by analyzing the operation history for each feature.

[0031] Returning to the description of Fig. 3, the model information storage unit 13b stores, for example, data of a model used in the estimation process of the delivery surface of a mounting violation by the estimation unit 12b described later. In addition, the model information storage unit 13b can store, for example, information on the type of features used in learning the model, and correct answer data and incorrect answer data described later.

[0032] The estimation result storage unit 13c stores the estimation result of whether or not the distribution surface to be examined satisfies a predetermined condition, which is estimated by the estimation unit 12b described later. For example, the estimation result storage unit 13c stores a score representing the possibility of an implementation violation, calculated for each distribution surface to be examined.

[0033] Here, with reference to (B) of Fig. 4, the score representing the possibility of an implementation violation calculated for each distribution surface to be examined and stored in the estimation result storage unit 13c will be described. As shown in (B) of Fig. 4, the estimation result storage unit 13c is composed of, for example, a "distribution surface" item and a "score" item.

[0034] The "Distribution surface" field stores a number assigned to identify the distribution surface being reviewed, while the "Score" field stores a calculated value between 0 and 1 that indicates the possibility of implementation violations for the corresponding distribution surface.

[0035] In (B) of Fig. 4, it is shown that the estimated results of "distribution surface 1: 0.9", "distribution surface 2: 0.8", and "distribution surface 3: 0.2" are stored. Here, the score value of "1" is the calculated value with the highest possibility of an implementation violation, indicating that "distribution surface 1" is highly likely to be an implementation violation and "distribution surface 3" is least likely to be an implementation violation.

[0036] Returning to the description of Fig. 3, the control unit 12 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like executing various programs stored in a storage device inside the estimation device 10 using a RAM as a working area. The control unit 12 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 12 has an acquisition unit 12a, an estimation unit 12b, and an output unit 12c.

[0037] The acquisition unit 12a acquires information about the operation history of the web content to be examined. For example, the acquisition unit 12a acquires information about the operation history of the user with respect to the advertisement implemented on the distribution surface to be examined from the terminal owned by the user via the network N. Here, the information about the operation history is an operation log used to calculate the characteristics of the operation history in the processing of the estimation unit 12b described later, and specifically, the information acquired includes the time required from the display of the advertisement until it was clicked, the page navigation history after the advertisement was clicked, and the like.

[0038] Then, for example, the estimation device 10 calculates operation history features such as CVR and CTR from the acquired information on the operation history, and stores them in the operation history information storage unit 13a. In addition, the acquisition unit 12a can directly acquire operation history features such as CVR and CTR calculated by an external device, and store them in the operation history information storage unit 13a.

[0039] The estimation unit 12b estimates whether the web content satisfies a predetermined condition from the information on the operation history acquired by the acquisition unit 12a, using a model that has learned the characteristics of the operation history that occurs when the web content does not satisfy a predetermined condition. Then, the estimation unit 12b stores the estimation result in the estimation result storage unit 13c.

[0040] For example, the estimation unit 12b uses a model stored in the model information storage unit 13b to estimate whether or not the distribution surface to be reviewed satisfies predetermined conditions from the calculated values ​​of the characteristics of the operation history stored in the operation history information storage unit 13a, and stores the estimation result in the estimation result storage unit 13c.

[0041] Here, the specified conditions are the UI configuration requirements of a delivery surface that is permitted for implementation, as described in advertising implementation guidelines, etc. (such as leaving a certain distance between the ad space and the "close button"), and a delivery surface that does not meet the UI configuration requirements is considered to be a delivery surface that violates the implementation.

[0042] In addition, the estimation unit 12b, for example, uses the learned model to evaluate and score the possibility of an implementation violation for each distribution surface from 0 to 1, thereby estimating the possibility of whether the distribution surface being reviewed satisfies a specified condition.

[0043] In addition, the estimation unit 12b estimates whether or not the web content satisfies a predetermined condition by using a model that has learned information indicating the time from when an advertisement implemented in the web content is displayed until an input to the advertisement is made as a feature of an operation history. For example, the estimation unit 12b estimates whether or not the distribution surface to be examined satisfies a predetermined condition by using a model that has learned information indicating the time from when an advertisement implemented in the distribution surface to be examined is displayed until a user clicks on the advertisement as a feature of an operation history in a user terminal.

[0044] Here, the relationship between the time from when an advertisement implemented on a distribution surface to be examined is displayed to when a user clicks on the advertisement and the implementation violation of the distribution surface will be described with reference to Fig. 5. Fig. 5 is a diagram showing the relationship between the characteristics of the operation history and the implementation violation of the distribution surface according to the embodiment.

[0045] Figure 5 shows graph (A) showing the time from advertisement display to click on a distribution surface with implementation violation, graph (B) showing the time from advertisement display to click on a distribution surface with OK implementation, and table (C) showing Q1 to Q3 (quartiles) of both graphs.

[0046] The vertical axis of the graph in Figure 5 shows the number of seconds at the 90% point when the logs of the time from the display of the ad to the click for each delivery surface are sorted in ascending order, and the horizontal axis of the graph in Figure 5 shows the delivery surface number when the number of seconds at the 90% point for each delivery surface are sorted in descending order.

[0047] As shown in (A) and (C) of Figure 5, the time from ad display to click on a delivery surface with implementation violation was "Q1: 7 seconds," "Q2: 4 seconds," and "Q3: 3 seconds," with little change in the number of seconds between Q1 and Q3. On the other hand, as shown in (B) and (C) of Figure 5, the time from ad display to click on a delivery surface with OK implementation was "Q1: 34 seconds," "Q2: 5 seconds," and "Q3: 2 seconds," with a difference of about 30 seconds between the number of seconds in Q1 and the number of seconds in Q2 and Q3.

[0048] In other words, it can be said that the time to ad click is less variable on the delivery media with implementation violations, and the time from displaying the ad to clicking tends to be shorter. This is because the delivery media with implementation violations, which have a lot of erroneous taps, tend to have a shorter time to click because the ad is often tapped without being read.

[0049] Therefore, the estimation unit 12b can accurately estimate whether or not the delivery surface satisfies the specified conditions by using a model that has been learned using the time from when the advertisement posted on the delivery surface to be reviewed is displayed to when the user clicks on the advertisement as a feature of the operation history.

[0050] Furthermore, the estimation unit 12b estimates whether the web content satisfies a predetermined condition by using a model trained on the operation history of web content determined to be an implementation violation as correct data and the operation history of web content not determined to be an implementation violation as incorrect data.

[0051] For example, the estimation unit 12b can estimate distribution surfaces that violate implementation by learning operation history features such as CTR, CVR, and the time from when an advertisement is displayed to when it is clicked, of a distribution surface that is determined to have an implementation violation as correct data, and operation history features of a distribution surface that is determined to have an OK implementation as incorrect data.

[0052] Here, the result of the judgment of implementation violation or implementation OK is determined by, for example, the final judgment made by an examiner on whether the configuration requirements of the UI of the distribution surface satisfy the guidelines. In addition, the distribution surface to be learned as the incorrect answer data described above may be a distribution surface selected randomly from among all distribution surfaces, in addition to using only the distribution surfaces that are judged to be OK for implementation.

[0053] Furthermore, the estimation unit 12b estimates whether or not the web content satisfies a predetermined condition by using a model that has learned the features of the operation history according to the importance calculated for each of the features of the operation history by the random forest.

[0054] For example, the estimation unit 12b uses a random forest, which is a type of machine learning algorithm, to compare the importance values ​​calculated for each of the features of the operation history that the model is made to learn, and then uses a model that has learned features with high importance to estimate whether the distribution surface to be reviewed satisfies specified conditions.

[0055] Here, the features of the operation history and the importance calculated for each of the features will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the features used in the model and the importance of each feature. In the example of Fig. 6, "CVR", "tCVR_1", and "CTR_all" are shown as features. "tCVR_1" is calculated only for CVs that occurred one second or more after a click, and "CTR_all" is the CTR calculated from all ad impressions and clicks on the target delivery surface.

[0056] In Figure 6, the importance of each feature is shown as "CVR: 0.00882321", "tCVR_1: 0.01682495", and "CTR_all: 0.06171473". Here, the importance values ​​are "CVR", "tCVR_1", and "CTR_all" in that order, so it can be said that the feature of "CTR_all" has the greatest impact on the classification process of the model among the three features.

[0057] In other words, among the three features shown in Fig. 6, whether or not "CTR_all" is learned has the greatest impact on the accuracy of the estimation process. Therefore, the estimation device 10 can perform highly accurate estimation processing with a small amount of learning by referring to the importance value of each feature and using a model in which only features with high importance are efficiently learned.

[0058] Furthermore, the estimation unit 12b estimates whether the web content satisfies a predetermined condition by using a model that has learned the characteristics of the operation history depending on the type of advertisement implemented in the web content. For example, the estimation unit 12b estimates whether the delivery surface to be examined satisfies a predetermined condition by using a model in which the learned characteristics of the operation history are changed depending on whether the advertisement implemented in the delivery surface to be examined is a video advertisement or an image advertisement.

[0059] Here, the features corresponding to the type of advertisement are selected by, for example, referring to the importance values ​​calculated by the random forest described above, so that features having high importance for each type of advertisement are selected.

[0060] In addition, the estimation unit 12b estimates whether the web content satisfies a predetermined condition by using a model that has learned the characteristics of the operation history according to the type of the web content. For example, the estimation unit 12b estimates whether the delivery surface satisfies a predetermined condition by using a model in which the characteristics of the operation history to be learned are changed according to the type of advertisement presentation mode of the delivery surface to be examined.

[0061] Here, the type of advertisement presentation mode refers to the type of advertisement implementation method, such as overlay advertisement or embedded advertisement, and the features corresponding to the type of advertisement presentation mode are selected by, for example, referring to the importance value calculated by the random forest described above, so that features with high importance for each type of advertisement presentation mode are selected.

[0062] Furthermore, the estimation unit 12b can also estimate whether the delivery surface satisfies specified conditions by using a model in which the characteristics of the operation history to be learned are changed depending on the type of device (smartphone, PC, etc.) on which the advertisement is displayed, in addition to the types of advertisements and types of advertisement presentation modes described above.

[0063] Returning to the description of Fig. 3, the output unit 12c outputs the result estimated by the estimation unit 12b. For example, the output unit 12c refers to the information stored in the estimation result storage unit 13c, transmits the estimation result to the external terminal 20, and causes the external terminal 20 to display information such as a distribution surface having a high possibility of an implementation violation and a score calculated for each of the distribution surfaces to be examined.

[0064] 4. Processing Procedure Next, the process of the estimation device 10 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of a process procedure according to the embodiment. In the example shown in Fig. 7, the acquisition unit 12a acquires information about an operation history in web content to be examined (S101).

[0065] Then, the estimation unit 12b estimates whether the web content satisfies a predetermined condition from the information on the operation history using a model that has learned the characteristics of the operation history (S102). After that, the output unit 12c outputs the estimated result (S103).

[0066] 5. Effects Here, the relationship between the score calculated by the model that has been trained as described above for the possibility of a distribution-side implementation violation and the final review result of whether the implementation is OK or an implementation violation will be described with reference to Fig. 8. Fig. 8 is a diagram showing the effect of the estimation process according to the embodiment.

[0067] Fig. 8 shows a graph (A) showing the score of each distribution surface for distribution surfaces with implementation violations, a graph (B) showing the score of each distribution surface for distribution surfaces with OK implementation, and a table (C) showing Q1 to Q3 (quartiles) for both graphs. The vertical axis of the graph in Fig. 8 shows the score value calculated for each distribution surface, and the horizontal axis of the graph in Fig. 8 shows the distribution surface number when the scores of each distribution surface are arranged in descending order.

[0068] As shown in (A) and (C) of Figure 8, the scores of each distribution surface in the distribution surface with implementation violation are "Q1: 0.994", "Q2: 0.994", "Q3: 0.980", and there is little change in the number of seconds from Q1 to Q3. On the other hand, as shown in (B) and (C) of Figure 5, the scores of each distribution surface in the distribution surface with OK implementation are "Q1: 0.994", "Q2: 0.985", "Q3: 0.717", and there is a large difference between the scores of Q1 and Q2 and the score of Q3.

[0069] In other words, there was less variation in scores for delivery surfaces with implementation violations, and the majority of delivery surfaces that were determined to be implementation violations were also calculated to have high scores by the model. Therefore, it can be said that there is a high probability that delivery surfaces that were calculated to have high scores by the model are implementation violations. Note that even among delivery surfaces that were determined to be implementation OK, there were delivery surfaces that were calculated to have high scores. This is because even delivery surfaces with a lot of erroneous taps may be considered implementation OK if certain conditions are met.

[0070] The estimation device 10 according to this embodiment includes an acquisition unit 12a, an estimation unit 12b, and an output unit 12c. The acquisition unit 12a acquires information about operation history in web content to be examined. The estimation unit 12b estimates whether the web content satisfies a predetermined condition from the information about the operation history acquired by the acquisition unit 12a, using a model that has learned characteristics of the operation history that occurs when the web content does not satisfy a predetermined condition. The output unit 12c outputs the result estimated by the estimation unit 12b.

[0071] As a result, the estimation device 10 uses a model that has learned the characteristics of the operation history on the delivery surface with the implementation violation to calculate a score that represents the possibility of an implementation violation for each delivery surface that is highly correlated with the delivery surface with the implementation violation, thereby achieving the effect of appropriately estimating the delivery surface with the implementation violation.

[0072] In addition, the estimation unit 12b of the estimation device 10 estimates whether or not the web content satisfies a predetermined condition by using a model that has learned information indicating the time from when an advertisement implemented in the web content is displayed to when an input is made to the advertisement as a feature of the operation history.

[0073] As a result, the estimation device 10 has the effect of being able to accurately estimate the delivery surface that is an implementation violation by having the model used in the estimation process learn the characteristics of the operation history regarding the time from when an advertisement is displayed to when input to the advertisement is made, which has a relatively large impact on the determination of the delivery surface that is an implementation violation.

[0074] Furthermore, the estimation unit 12b of the estimation device 10 estimates whether or not the web content satisfies a predetermined condition by using a model trained with an operation history for web content determined to be an implementation violation as correct data and an operation history for web content not determined to be an implementation violation as incorrect data.

[0075] As a result, the estimation device 10 has the effect of being able to appropriately estimate the delivery surface that has an implementation violation by having a model learn the operation history for the delivery surface, which is labeled as correct data or incorrect data depending on whether or not it is an implementation violation, and then performing an estimation process.

[0076] Furthermore, the estimation unit 12b of the estimation device 10 estimates whether or not the web content satisfies a predetermined condition by using a model that has learned the features of the operation history according to the importance calculated for each feature of the operation history by the random forest. As a result, the estimation device 10 refers to the importance value of each feature calculated by the random forest and uses a model that has efficiently learned only features with high importance, thereby achieving the effect of being able to perform highly accurate estimation processing with a small amount of learning.

[0077] Furthermore, the estimation unit 12b of the estimation device 10 estimates whether the web content satisfies a predetermined condition by using a model that has learned the characteristics of the operation history according to the type of advertisement implemented in the web content. As a result, the estimation device 10 has an effect of being able to accurately estimate the delivery surface that is an implementation violation by using a model that has learned the characteristics of high importance preferentially according to the type of advertisement.

[0078] Furthermore, the estimation unit 12b of the estimation device 10 estimates whether or not the web content satisfies a predetermined condition using a model that has learned the characteristics of the operation history according to the type of web content. This allows the estimation device 10 to use a model that has learned characteristics with higher importance preferentially according to the type of advertisement presentation mode of the delivery surface, thereby achieving the effect of being able to accurately estimate the delivery surface that is an implementation violation.

[0079] [6. Hardware Configuration] The estimation device 10 according to the embodiment described above is realized by, for example, a computer 1000 as shown in Fig. 9. Fig. 9 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I / F (Interface) 1060, an input I / F 1070, and a network I / F 1080 are connected via a bus 1090.

[0080] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic unit 1030 is realized by, for example, a CPU, an MPU, an ASIC, an FPGA, or the like.

[0081] The primary storage device 1040 is a memory device such as a RAM that primarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The secondary storage device 1050 may be an internal storage device or an external storage device.

[0082] Furthermore, the secondary storage device 1050 may be a removable storage medium such as a Universal Serial Bus (USB) memory or a Secure Digital (SD) memory card, etc. Also, the secondary storage device 1050 may be a cloud storage (online storage), a Network Attached Storage (NAS), a file server, etc.

[0083] The output I / F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a display, a projector, a printer, etc., and is realized by a connector conforming to a standard such as USB, DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface).

[0084] The input I / F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a keypad, a button, a scanner, etc., and is realized by, for example, a USB.

[0085] Furthermore, the output I / F 1060 and the input I / F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

[0086] The output device 1010 and the input device 1020 may be integrated into one device, such as a touch panel. In this case, the output I / F 1060 and the input I / F 1070 may also be integrated into one device, such as an input / output I / F.

[0087] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0088] The network I / F 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0089] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I / F 1060 and the input I / F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0090] [7. Other] Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. The above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments.

[0091] Furthermore, among the various processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods.

[0092] In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0093] Furthermore, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. For example, all or part of the functions of the estimation device 10 may be performed by the external terminal 20.

[0094] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents are not contradictory.

[0095] Furthermore, the above-mentioned "section, module, unit" can be read as "means" or "circuit", etc. For example, an acquisition section can be read as an acquisition means or an acquisition circuit. [Explanation of symbols]

[0096] 10 Estimation device 11 Communications Department 12 Control section 12a Acquisition part 12b Estimation part 12c output section 13 Storage section 13a Operation history information storage unit 13b Model information storage section 13c Estimation result storage unit 20 External Terminal

Claims

1. An acquisition unit that acquires information regarding an operation history of a web content to be inspected; an estimation unit that estimates whether the web content satisfies a predetermined condition from information about the operation history acquired by the acquisition unit, using a model that has learned characteristics of an operation history that occurs when the web content does not satisfy a predetermined condition; an output unit that outputs the result estimated by the estimation unit; An estimation device comprising:

2. The estimation unit estimates whether the web content satisfies a predetermined condition by using a model that has been learned using information indicating a time from when an advertisement implemented in the web content is displayed to when an input is made to the advertisement as a feature of the operation history.

2. The estimation device according to claim 1 .

3. The estimation unit estimates whether the web content satisfies a predetermined condition by using a model trained with an operation history for a web content determined to be an implementation violation as correct answer data and an operation history for a web content not determined to be an implementation violation as incorrect answer data.

2. The estimation device according to claim 1 .

4. The estimation unit estimates whether the web content satisfies a predetermined condition by using a model that has learned the features of the operation history according to the importance calculated for each of the features of the operation history by a random forest.

2. The estimation device according to claim 1 .

5. The estimation unit estimates whether the web content satisfies a predetermined condition by using a model that has learned characteristics of the operation history according to a type of advertisement implemented in the web content.

2. The estimation device according to claim 1 .

6. The estimation unit estimates whether the web content satisfies a predetermined condition by using a model that has learned features of the operation history according to a type of the web content.

2. The estimation device according to claim 1 .

7. An estimation method executed by an estimation device, comprising: An acquisition step of acquiring information regarding an operation history of the web content to be examined; an estimation step of estimating whether or not the web content satisfies a predetermined condition from information about the operation history acquired by the acquisition step, using a model that has learned characteristics of an operation history that occurs when the web content does not satisfy a predetermined condition; an output step of outputting the result estimated by the estimation step; The estimation method according to claim 1,

8. An acquisition step for acquiring information regarding an operation history of the web content to be reviewed; an estimation step of estimating whether the web content satisfies a predetermined condition from information about the operation history acquired by the acquisition step, using a model that has learned characteristics of an operation history that occurs when the web content does not satisfy a predetermined condition; an output step of outputting a result estimated by the estimation step; An estimation program for causing a computer to execute the above.

Images