Methods for providing advertising effectiveness

Abstract

Provided is an advertisement delivery method with which it is possible to accurately calculate the delivery count for an advertisement that produces a favorable impression, and deliver an appropriate advertisement at a timing when an individual favors the advertisement. The present invention comprises a grouping unit 20, a learning content delivery unit 200, a result collection analysis unit 300, a questionnaire delivery unit, an advertisement delivery unit 400, a per-group advertisement effect analysis unit 500, a per-individual advertisement delivery unit 600, and the like, and calculates the appropriate advertisement delivery count for improving the favorability of a company and reducing the advertisement cost on the basis of questionnaire reactions of grouped learners to an advertisement. On the basis of the appropriate advertisement delivery count, the count and timing at which a guidance advertisement is to be delivered to a learner having a prescribed attribute (i.e., a learner having high favor) are derived, and the guidance advertisement is delivered.

Description

[Technical Field]

[0001] This invention relates to a method for providing advertising effectiveness. [Background technology]

[0002] Advertising includes flyers, newspaper ads, and web ads. The advantage of web advertising is that user behavior can be collected as data, which can then be used to improve future ad delivery. The main metrics for the objectives of web advertising are "awareness," "conversion," and "acquisition." By measuring (or analyzing) various data on delivered ads, it is possible to measure the specific changes in the target audience's emotions (interest level, purchase intent, etc.) before and after they see the ad (this is called ad effectiveness measurement).

[0003] Then, if the results of the advertising effectiveness measurement exceed the target, the advertising is considered effective. However, if the results fall short of the target, the advertising needs to be reviewed.

[0004] Banner ads are a common example of advertising used to raise awareness of a company's name, products, and services among a wide range of users. Banner ads are purchased as advertising revenue, and there are various billing models, including period-based, click-based, and impression-based (number of views) models. Furthermore, if the goal is to "guide" users who are interested in the advertised product or category to your company's website, there are options such as search engine marketing (listing ads), sponsored articles, and social media ads.

[0005] Search engine advertising involves linking specific keywords to ads in advance, so that when a user performs a search, the ad is displayed in conjunction with those search terms. To improve cost-effectiveness, it's best to reduce the budget for underperforming ads early on and focus on high-performing ones. To analyze the response to an advertisement, you can conduct an A / B test by preparing multiple variations of the element you want to examine and observing the differences in user responses.

[0006] Numerous patent documents on measuring advertising effectiveness have been disclosed. For example, Patent Document 1 discloses that whether an advertisement was effective can be determined by the conversion rate (CVR), which shows the ratio of the number of times the advertisement was displayed over a predetermined period to the number of consumer purchase contracts for the advertised product. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2018-037070 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] However, the advertising effectiveness measurement described in Patent Document 1 is an advertising effectiveness measurement that is judged by the conversion rate (CVR), which shows the ratio of the number of times an advertisement is displayed (also called the number of times the advertisement is delivered) over a predetermined period to the number of consumer purchase contracts for the advertised product, and is primarily based on the number of purchase contracts relative to the number of deliveries. However, the more frequently an ad is broadcast or the longer the advertising period, the greater the cost burden on the company. Furthermore, it has been believed that increasing the frequency of ad delivery helps to establish a positive image of the ad. However, if ads are delivered indiscriminately, the company's image will not improve (or gain favorability). If this level of favorability can be analyzed, it will be possible to obtain highly accurate results in measuring the effectiveness of advertising. [Means for solving the problem]

[0009] The advertising distribution method according to the present invention is an advertising effectiveness provision method which involves an advertising effectiveness measurement site generating presentation lists containing multiple learning contents appropriate to the learner's level for learning on each learning day over the learning period and sequentially transmitting them via a communication network to the learner terminals of multiple learners, sequentially collecting response lists in which the learners input their response values ​​to the learning contents, analyzing the learners' progress based on the responses in these response lists, incorporating the advertiser's advertisement into this analysis result, and analyzing the effectiveness of this advertisement. The server of the aforementioned advertising effectiveness measurement site is (A) A step of generating a learner ID for each learner, and generating another ID different from this learner ID, and storing it in the memory unit in correspondence with the learner ID for each learner, (B) The steps of adding the learner's learner ID to each of the learner's presentation lists and sending them, and adding this learner ID to the response list and sending it back to be stored in the memory unit, (C) A step of analyzing the learner's level of achievement based on each response value in the accumulated response list, (D) Each time the learner's level of achievement is analyzed, the analysis result is read from the other ID corresponding to the learner ID attached to the returned response list, and the analysis result is associated with this other ID and stored in the storage unit. (E) The steps include: incorporating the advertiser's advertisement into the analysis results of the learner's achievement level, sending a questionnaire to the learner to select their impression of the advertiser, returning the impression selection results along with the learner ID, and storing them in the memory unit; (F) A step of statistically analyzing the accumulated results of the advertiser's impression selection to obtain attribute information of learners who have reached a favorable impression level, (G) A step of storing in the memory unit the other ID corresponding to the learner ID attached to the impression selection result and the statistically analyzed result in association with each other. It is characterized by performing the following. [Effects of the Invention]

[0010] As described above, according to the present invention, it is possible to analyze what effect the number of ad deliveries of an advertiser has achieved. Further, it is also possible to analyze what effect has been achieved on an individual basis. Furthermore, when the analysis result is provided to the outside, the analysis result is provided in such a manner that individuals cannot be identified. Therefore, the result can be safely utilized as big data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] [Figure 1] FIG. 1 is a schematic configuration diagram of the advertisement delivery system of the present embodiment. [Figure 2] FIG. 2 is a schematic configuration diagram when the advertisement delivery system of the present embodiment is configured with a plurality of servers. [Figure 3] FIG. 3 is a first sequence diagram of the present embodiment. [Figure 4] FIG. 4 is a second sequence diagram of the present embodiment. [Figure 5] FIG. 5 is a third sequence diagram of the present embodiment. [Figure 6] FIG. 6 is a fourth sequence diagram of the present embodiment. [Figure 7] FIG. 7 is a fifth sequence diagram of the present embodiment. [Figure 8] FIG. 8 is a sixth sequence diagram of the present embodiment. [Figure 9] FIG. 9 is an explanatory diagram of a REG-RANDID list. [Figure 10] FIG. 10 is a schematic configuration diagram of a learning content distribution unit 200. [Figure 11] FIG. 11 is an explanatory diagram of schedule conditions. [Figure 12] FIG. 12 is an explanatory diagram of a method for generating a schedule condition SHi. [Figure 13] FIG. 13 is an explanatory diagram of a content table. [Figure 14] FIG. 14 is a first explanatory diagram of a schedule table. [Figure 15] FIG. 15 is a second explanatory diagram of a schedule table. [Figure 16] FIG. 16 is an explanatory diagram of a presentation list. [Figure 17] This is a description of the delivery history information for the sent list. [Figure 18] This is an explanatory diagram of the list of recipients. [Figure 19] This is an explanatory diagram of the display screen on a learner's device. [Figure 20] This is an explanatory diagram of the learning results list for each learner. [Figure 21] This is an explanatory diagram of the analysis result graph. [Figure 22] This is an explanatory diagram for the display of ABNi information with advertisements, analyzed graphs for each learner. [Figure 23] This is a schematic diagram of the advertising distribution unit 400. [Figure 24] This is a schematic diagram of the survey distribution unit 100. [Figure 25] This is an explanatory diagram of the questionnaire text. [Figure 26] This is an explanatory diagram of the group-specific survey distribution history list. [Figure 27] This is an explanatory diagram of the survey response history list. [Figure 28] This is a diagram illustrating the processing performed by the survey distribution department. [Figure 29] This is a schematic diagram of the 500-unit advertising effectiveness analysis department for each group. [Figure 30] This is an explanation of the advertising effectiveness measurement results. [Figure 31] This is an explanatory diagram of the graphing section. [Figure 32] This is a schematic diagram of the 600 individual ad delivery units. [Figure 33] This is a diagram illustrating the processing of the individual ad delivery unit 600. [Figure 34] This is an explanatory diagram illustrating the relationship between the advertising period and the trend in favorability ratings. [Figure 35] This is an explanatory diagram of an example of an advertisement for promotional purposes according to this embodiment. [Modes for carrying out the invention]

[0012] This embodiment assumes the use of a scheduling method (Japanese Patent Publication No. 3764456, Japanese Patent Publication No. 4391474) that identifies a group of learning content according to an individual's attributes (age, learning level, etc.: individual attributes), delivers predetermined content from this group to each individual at a certain time, and collects response data to that content for each individual (for example, selection buttons such as "completely useless," "useless," "needs improvement," and "good"). Furthermore, in this embodiment, the viewer of the content can be anyone, but will be described as a learner who studies using the scheduling method described above. Furthermore, while advertisements are from companies, banks, cram schools, organizations, schools, medical institutions, local governments, government agencies, etc. (hereinafter also referred to as advertisers or business owners), in this embodiment, we will explain them as bank advertisements. Companies, banks, cram schools, organizations, schools, medical institutions, local governments, government agencies, etc. will also be referred to as external organizations. Furthermore, the learning content may include kanji, English, idioms, law, medicine, physics, pharmacology, etc., but in this embodiment, it will be explained using English words. Furthermore, student devices are owned by individual schools and cram schools, and these are sometimes lent out to students for use. In such cases, this website and the schools or cram schools have contracts for content access that last for several years.

[0013] Figure 1 is a schematic diagram of the advertising effectiveness measurement system of this embodiment. As shown in Figure 1, the advertising effectiveness measurement system of this embodiment includes a learner information memory unit 10 that stores learner information SGi of learners who are members of this site (advertising effectiveness measurement site), a grouping unit 20 (program), and a group-specific learner information memory unit 30 that stores the grouped learner information SGi.

[0014] Furthermore, as shown in Figure 1, the system includes a learning content distribution unit 200 (program), a performance data collection and analysis unit 300 (program), a questionnaire distribution unit (program), an advertising distribution unit 400 (program), a group-specific advertising effectiveness analysis unit 500 (program), and an individual advertising distribution unit 600 (program), among others. These programs are stored in ROM, and the CPU reads them into RAM and executes them.

[0015] This configuration allows for the measurement of advertising effectiveness based on survey responses to advertisements from grouped learners, in order to determine the appropriate number of ad deliveries that improve a company's favorability and reduce advertising costs.

[0016] Furthermore, based on this optimal ad delivery frequency, the system determines the number and timing of deliveries of promotional ads to learners with specific attributes (learners with high favorability ratings), and delivers these ads accordingly. This reduces wasted ad delivery and lowers the advertising costs for companies. This will be explained in more detail later.

[0017] The aforementioned learner information SGi (also called learner attribute information) consists of learner ID (also called learner information identification code), password, name, address, region, telephone number, email address, name of the organization to which the learner belongs (e.g., school name), age, grade level, gender, learner level (e.g., Eiken Grade 1, Grade 2, High School 1st year, Junior High School 2nd year, etc.), desired occupation, preferences, and schedule name (schedule conditions), etc.

[0018] The learner ID and password (collectively referred to as the account) for learner information SGi are generated in advance by, for example, the learner content distribution unit 200 and notified to the learner.

[0019] In this embodiment, the learner information SGi stored in the learner information memory unit 10 will be described as students from the Faculty of Engineering, Faculty of Education, Faculty of Letters, Faculty of Science, Faculty of Medicine, Faculty of Pharmacy, Faculty of Agriculture, etc., of XX University, with each faculty representing 200 students. For example, the student information SGi for the Faculty of Engineering is stored in the memory unit 10a for the Faculty of Engineering, the student information SGi for the Faculty of Education is stored in the memory unit 10b for the Faculty of Education, and the student information SGi for the Faculty of Letters is stored in the memory unit 10c for the Faculty of Letters. These students are studying using the scheduling method described above.

[0020] The grouping unit 20 reads the input ad delivery target attribute information CGi [for example, fourth year of university, level (for example, Eiken Grade 2), school name, faculty, desired job type, preferences, advertiser (bank name), etc.] and the number of groups (7 groups, for example, A, B, ... G), and generates group-specific memories (memory for group A learner information 30a, memory for group B learner information 30b, ... G learner information memory 30g) in the group-specific learner information memory unit 30, and assigns an identification code for the group name to each.

[0021] Then, from each of the learner information memory units 10 (learner information memory unit 10a for the Faculty of Engineering, learner information memory unit 10b for the Faculty of Education, learner information memory unit 10c for the Faculty of Letters, ...: population), only the predetermined information (hereinafter also called sample learner information) from the learner information SGi corresponding to the advertisement target attribute information CGi is randomly read out for each of the number of people selected (for example, 10 people), and stored in each of the respective memories 30 (memory 30a for group A learner information, memory 30b for group B learner information, ...: memory 30g for group G learner information) (for example, 70 people each: also called sample group).

[0022] Group A is also known as the baseline group. The learning content distribution unit 200 distributes one set of learning content (multiple: presentation list) to the learner's device each time the learner logs in from that device (tablet, smartphone, or PC).

[0023] Specifically, the learning content distribution unit 200 generates a presentation list TRi (on a daily basis) for the learning period GTi (for example, 4 months, 6 months, 1 year, 2 years, or 3 years, etc.) based on the schedule table STi in the schedule table memory 210 generated by the scheduling method described above, and sends it to the learner's terminal.

[0024] The content table stores the content CNi (question and answer) associated with the schedule name SI of the schedule condition Shi. The questions are either test questions or drill questions.

[0025] Furthermore, the schedule table STi is a schedule table (for example, 60 weeks' worth) generated based on the schedule conditions SHi, which are a combination of timing conditions that indicate when to learn and presentation conditions that indicate how many times (including how to learn) content CNi with what attributes to learn, all of which are stored in the schedule condition memory 228.

[0026] Within this schedule table STi, one day's worth of information is generated as a presentation list TRi, which is sent to the learner's terminal when they log in from that terminal. This learning performance (good, slightly good, bad, very bad) is then associated with the learner information SGi (specifically, linked to the learner ID) and stored. Furthermore, the learning content distribution unit 200 generates a unique learner ID (REG) using a random number generator (not shown). - It generates an ID (also called an ID) and a unique RAND - The ID (also called the ID for analysis result information) is associated with this learner ID and stored in a memory for the REG-RANDID list, which is not shown in the diagram.

[0027] When a learner logs in from a learner terminal, the performance collection and analysis unit 300, if the learner has been studying for a certain period of time (for example, more than one week) (even just one day), calculates the regression line Y of the learner's performance in the learner information SGi, generates performance analysis result graph information consisting of this regression line Y and a performance accumulation bar, etc., and transmits it to the learner terminal for display.

[0028] The advertising distribution unit 400 stores the company's advertising distribution period CTn and the target attribute information CGi (for example, Eiken Grade 2, 4th year university student, region, gender, age, school name, etc.).

[0029] Then, when the learner logs in from their terminal, if the learner's performance analysis results graph information is stored, a command (also called a control code) is sent to send an advertisement according to the pre-stored advertising distribution schedule of XX Bank (or a company, organization, local government, etc.) and to incorporate and display it in the performance analysis results graph information. Furthermore, the ad delivery unit 400 stores an ad delivery schedule that initially delivers ads to users who are not interested, and then transitions to ads that encourage them to purchase products. This makes it possible to generate revenue in a way that is not possible with current media. In today's media landscape, the only option is to distribute advertisements simultaneously, meaning that if viewers become tired of an ad, the distribution to the entire audience must be stopped. However, in this embodiment, for example, the same advertisement distributed to first-year university students is then distributed to second-year students with modified content (an advertisement related to the advertisement distributed to first-year students), and then the advertisement distributed to first-year students in the following year is used again to distribute the advertisement repeatedly. In other words, it is possible to change the target audience systematically.

[0030] The survey distribution unit 100 stores the survey distribution conditions Ahj. For example, it stores groups A through G as the target audience for the survey, and also stores the survey distribution period. However, for group A, a survey is sent one week before the advertisement distribution (it could also be two weeks or one month before: also called the first survey timing), for example, to XX Bank.

[0031] Furthermore, the timing for the second survey for Group B, ..., Group G will be as follows: after the survey is distributed (mass distribution) to the learner terminals of Group A, Group B will receive the survey mass distribution after, for example, one week has passed; Group C will receive the survey mass distribution after, one week has passed, following the mass distribution to Group B; ..., Group G will receive the survey mass distribution after, one week has passed, following the distribution to Group F. After distributing to Group G, Group A will receive the survey mass distribution again one week later.

[0032] The group-specific advertising effectiveness analysis unit 500 stores the number of times each advertisement has been delivered up to the time each questionnaire is distributed for each group. Each time a response to each questionnaire is received, it calculates the ANOVA value (significant difference (p)) for each group based on the type of response and the number of deliveries, etc., and uses this as the result of the analysis of favorability based on the number of ad deliveries (also called the number of views).

[0033] Then, the Group-Specific Advertising Effectiveness Analysis Unit 500 calculates the average value for the advertising delivery period of XX Bank's advertisements over the learning period GTi (6 months) for all groups based on the trend of the ANOVA values ​​(significant difference (p)) for each group.

[0034] The individual ad delivery unit 600 generates a learner-specific favorability trend curve Li for each group and calculates the appropriate number of ad deliveries CPi for each learner. Then, the appropriate timing for delivering the promotional advertisement is determined from the learner-specific favorability trend curve Li, and the promotional advertisement for XX Bank is delivered by the advertising delivery unit 400 around the time the current time reaches this appropriate timing. Furthermore, on the scheduled delivery date, the advertisement will be displayed when the learner actually engages in learning.

[0035] This approach means that whether or not students engage in learning depends on their own will, so ads won't be delivered exactly as planned. However, by setting a delivery schedule in advance, it's possible to loosely control the frequency with which students see specific ads. While it's not possible to strictly control the timing of ad viewing (like with microstepping), it is possible to loosely synchronize it, allowing for retrospective analysis of image data to compare changes over time.

[0036] Each of the above components may be implemented on a server-by-server basis. For example, as shown in Figure 2, an advertising distribution system can be created that connects learner terminals 1a (1a, ...1i, ...), an advertising distribution site 900, a corporate terminal 700, and a communication network 800.

[0037] The advertising distribution site 900 consists of a web server 50, a server that is the learning content distribution unit 200, a server that is the performance data collection and analysis unit 300, a server that is the survey distribution unit 100, a server that is the advertising distribution unit 400, a server that is the group-specific advertising effectiveness analysis unit 500, a server that is the individual advertising distribution unit 600, a database server 1000, and the like.

[0038] The advertising delivery system configured as described above will be explained using the sequence diagrams in Figures 3 to 8. In these sequence diagrams, each server is collectively referred to as the site. However, the web server (the server for connecting to the network) will be described separately.

[0039] Learning is performed on a per-presentation list TRi (which contains multiple content CNi), and users can log in to the site multiple times a day to obtain presentation lists TRi (distribution lists) and learn from them. However, in this embodiment, the number of presentation lists TRi (distribution lists) learned per day will be described as a few (i.e., several times).

[0040] Furthermore, the difference between a distribution list and a presentation list (TRi) lies in whether or not learner information (SGi, also known as learner attribute information) is associated with it.

[0041] Furthermore, learning in this embodiment refers to the process of, for example, showing (displaying or viewing) an English word (question) and its answer on a terminal, and then selecting and sending a response ("completely wrong," ... "good"). The tests provided on this website also constitute learning in this embodiment.

[0042] Furthermore, this learning process involves sending a presentation list (TRi) consisting of multiple content items (CNi, for example, English vocabulary and answers) to the learner's device upon login.

[0043] Sometimes, learning is done for one presentation list TRi per day, while other times it's done for 3 to 5 presentation lists TRi (consisting of multiple content CNi). For example, if 5 presentation lists TRi are learned in one day, this site has a rule that prevents further learning on that day.

[0044] As shown in Figure 3, the learner terminal 1a (1a, ...1i, ...) and the web server of the advertising distribution site 900 communicate with each other to obtain learner basic information SKJi (d10a, ...d10i) and register it with the database server 1000 (d12, d14).

[0045] This learner basic information SKJi (SKJ1, SKJ2, ...) includes name, address, telephone number, email address, name of organization to which the learner belongs (e.g., school name and department), age, grade level, gender, learner's learning level (e.g., Eiken Grade 1, Grade 2), etc., and may also include favorite books, favorite foods, clothing, preferred environment (including natural environment), interests, etc. Name, address, telephone number, and email address are collectively referred to as personally identifiable information.

[0046] Further, a person in charge of the advertisement distribution site 900 operates the site-side terminal 1200 to associate a schedule name SI (I=A, B, ...) for learning according to the level of the learner basic information SKJi, a learner ID (unique code), and a password with the learner basic information SKJi. For example, a homeroom teacher of a school may operate the teacher's own terminal to access the site and perform this association.) The learner ID (unique code) and the password are generated by the computer of the present site.

[0047] On the other hand, the learning content distribution unit 200 (server) processes the learner ID (REG - also referred to as ID), a unique RAND - ID (also referred to as analysis result information ID) is generated, and is stored in a REG-RANDID list memory (not shown) in association with the learner ID (d24). This is referred to as a REG-RANDID list in the present embodiment (see FIG. 9). As shown in FIG. 9, the REG-RANDID list includes date and time, learner IDa1 (REG - ID), learner ID for advertisement effect measurement (RAND - ID) is a set of associated records. Note that REG - sub-REG with different IDs for each ID - a plurality of IDs are generated, and RAND - sub-RAND with different IDs for each ID - a plurality of IDs may be generated and associated with each other.

[0048] With this configuration, even if the schools are different or even at a cram school (even if the learner goes to another cram school), as long as the school or cram school has a contract with the present site, the content learning history can be managed with the same ID. Then, the learning content distribution unit 200 generates a schedule table STi and the like (d27).

[0049] FIG. 10 is a schematic configuration diagram of the learning content distribution unit 200. Note that FIG. 10 also shows the configuration of a grade collection and analysis unit 300 described later.

[0050] As shown in Figure 10, the learning content distribution unit 200 includes a schedule condition memory 228 that stores the schedule condition SHi, a schedule assignment unit 204, a content table memory 220, a schedule table generation unit 230, a presentation list generation unit 240, a presentation list memory 250 (file), a transmission list distribution unit 260 (also called the presentation list distribution unit), a content list memory 232, and the like.

[0051] The schedule condition memory 228 stores multiple schedule conditions SHi. These schedule conditions SHi are associated with schedule names SI (I=A, B, ...).

[0052] Before explaining the process for generating the schedule table STi, we will describe the schedule (schedule name SI and schedule condition SHi) in this embodiment.

[0053] The schedule in this embodiment is defined by a schedule condition SHi which is a pair of timing conditions (when to learn) that define what learning conditions (presentation conditions) will be repeatedly generated at what intervals (intervals) within a learning period GTi (e.g., 1 year) to reach a certain learning level (e.g., Eiken Grade 2), and presentation conditions [difficulty level and number of presentations of content CNi, tasks (programs that define how to show or display the content), conditions on how many content CNi can be allocated to one learning session, etc.] (see Figures 11(a) and 11(b)). Such a schedule is stored in the schedule condition memory 228.

[0054] This schedule condition SHi associates individual learner information SGi (preferably the learner ID) with content CNi, etc. As shown in Figure 11(b), it consists of content CNi (number, question, and answer), learner information (labeled as "individual" in Figure 11(b): learner ID), timing conditions, presentation conditions (including tasks), and schedule name SI, etc. Tasks will be discussed later.

[0055] Next, the method for generating the schedule condition SHi (generated by a schedule condition generation unit, which is not shown in the diagram) will be explained using Figure 12.

[0056] The schedule conditions SHi differ for each level, but the following is an example. These schedule conditions SHi are associated with schedule names SI (I=A, B, ...).

[0057] Figure 12(a) shows the schedule SA. Schedule SA is a timing condition that repeatedly performs learning of the same content CNi (one piece of content CNi incorporated into the presentation list TRi) every 5 days.

[0058] The difficulty level may also be indicated by the number of repetitions (number of repetitions in a single learning session). Figure 12(b) shows the Schedule SB. The Schedule SB is a timing condition in which the same content CNi is repeatedly learned (one piece of content CNi incorporated into the presentation list TRi) every three days (three times daily).

[0059] The schedule condition SHi is specifically defined using event cycle units and presentation units, as shown in Figures 12(a) and 12(b). The event cycle unit is a unit for distributing events (content CNi) and is defined on a time axis.

[0060] For example, in the case of an event cycle unit that generates an event (learning) once every five days, the length is defined on a five-day time axis (the first fixed period). The presentation unit has a length equivalent to the minimum period (one day).

[0061] Such event cycle units are generated over the learning period of GTi, and each event cycle unit is assigned a number (identification code). Then, the event cycle units are specified sequentially (by number). And for each event cycle unit specified, the presentation unit is specified sequentially (by number).

[0062] In other words, the position of this presentation unit is the timing for learning the content CNi of the presentation list TRi, and the learning date (number of times) is defined by the event cycle unit number and the presentation unit number, so this is called the timing condition.

[0063] Then, for each designated presentation unit, presentation conditions are sequentially incorporated, including a difficulty level corresponding to the level [(a combination of level, content type, easy content CNi or difficult content CNi or slightly difficult difficulty levels (e.g., 0.3 or 0.9) or 0.6), or any of these)] and the number of presentations (number of repetitions), etc.

[0064] Then, each time the next event cycle unit is specified, the presentation units of the previous event cycle unit are sequentially specified, and each time this is done, the same presentation conditions incorporated in the previous event cycle unit are incorporated into the presentation units that take, for example, a 5-day interval (second fixed period: first fixed period ≤ second fixed period) from the beginning of this presentation unit.

[0065] The schedule table generation unit 230 performs this processing and generates the schedule conditions SHi in the schedule condition memory 228. The presentation unit may be the condition unit described in Japanese Patent Publication No. 3764456 and Japanese Patent Publication No. 4391474.

[0066] Furthermore, the content table will be explained before the generation of the schedule table STi (see Figure 13). The content table memory 220 is provided separately for each learner's level (e.g., first year of university, second year of university, third year of university, fourth year of university) and each type of content.

[0067] Then, the schedule allocation unit 204 shown in Figure 10 sequentially specifies the content CNi in the content table and reads the schedule name SI associated with this content CNi.

[0068] Then, the schedule condition SHi of the loaded schedule name SI is read from memory and assigned to this content CNi. Specifically, as shown in Figure 13(a), the content table consists of a content number, content CNi (word (question and answer), difficulty level (0.1 or 0.2, ..., or number of presentations)), content type, and content data CDi (CD1, CD2, ...) which includes schedule conditions SHi, schedule name SI, etc.

[0069] Content data CD1-CD3 are examples where schedule SB is assigned. More difficult content is assigned schedule SC. Note that T indicates test material, and D indicates drill material.

[0070] In other words, the content table, as shown in Figure 13(b), consists of a content number, content CNi (word question: answer), difficulty level, schedule condition SHi, etc.

[0071] In other words, the content table is a table that stores content data consisting of pairs of content type (English, Kanji, idioms), difficulty level (English Junior High School: difficult level, medium level, easy level), content CNi, content number, and schedule condition SHi.

[0072] Next, we will provide further explanation regarding the generation process (d27) of the schedule table STi in Figure 3.

[0073] (Generation of the schedule table STi) As shown in Figure 10, the schedule table generation unit 230 sequentially specifies the schedule name SI (SA, SB, ...: associated with schedule condition SHi) in the schedule condition memory 228, as shown in Figure 3 (d23).

[0074] Then, as shown in Figure 3, the schedule table generation unit 230 in Figure 10 selects all content data CDi having the specified schedule name SI (SA or SB, ...) for each specification, and for each specification, it sequentially counts the number of repetitions (1 or 2, 3, ...) contained in the specified content data CDi using a counter (not shown), and registers the counted content CNi (number) in the content list memory 232 (d24).

[0075] Then, the schedule conditions SHi(SH1, SH2, ...) of the schedule condition memory 228 associated with the specified schedule name SI are specified, the event cycle units of these schedule conditions SHi(SH1, SH2, ...) are specified sequentially, and for each specification, the presentation units contained therein are specified, and the content CNi based on the presentation conditions hj(hj1, hj2, ...) incorporated therein is read from the content table 220 (d25).

[0076] Then, the schedule table generation unit 230 in Figure 10 generates a schedule table STi (ST1, ST2, ...) as shown in Figure 3 (d27) and stores it in the schedule table memory 210 (d29).

[0077] Specifically, the schedule table generation unit 230 reads the number of the specified event cycle unit (first week, first month, second month, etc.), the number of the specified presentation unit (first time (first day), second time (second day), etc.), and the schedule conditions SHi contained in the specified content data CDi (for example, timing condition is once a day: presentation condition is difficult or timing condition is once every 5 days: presentation condition is medium, etc.).

[0078] Then, the date (event occurrence) is the timing condition mentioned above, which indicates when to learn the pair of event cycle unit numbers (1st week, 1st month, or 2nd month, etc.) and presentation unit numbers (1st time (1st day), 2nd time (2nd day), etc.).

[0079] Then, the number of content items per schedule is divided by the period during which all entered learned content is available (for example, 1 month: 24 days). Then, based on a random selection from the content list or a predetermined rule, a large number of content CNi are loaded for each learning session in each schedule, and a schedule table STi is generated by sequentially assigning them to the event occurrence dates (month and day) (see Figures 14 and 15).

[0080] It consists of "M01D01", "M01D02", ... (where "M" indicates the month, "D" indicates the day, and "01" means the first instance of, for example, month X, day Y: i.e., corresponding to a timing condition), a schedule name (B or C, A or D, ...), and the number of repetitions (R01 or R02, ...R05, ...: R means repetition: the same word is limited to 5 times).

[0081] In Figure 15, the number of repetitions is indicated next to the content number, representing the number of items shown during each learning session in the schedule (for example, in Figure 15, the total number of repetitions is 28). In Figure 15, EXP-COND is the schedule condition SHi, and SC-COND is the content number. SCHETYPE is the schedule condition SHi.

[0082] This process stores the schedule table STi for each schedule in the schedule table memory 210. Alternatively, a schedule table STi may be generated by combining any two of the schedule names SI (I=A, B, ...).

[0083] Then, the presentation list generation unit 240 in Figure 10 generates the presentation list TRi as shown in Figure 3 (d30) and stores it in the presentation list memory 250 (d32). These memories are generated on the database server 1000.

[0084] (Presentation list TRi) The presentation list generation unit 240 then sequentially specifies the schedule table STi, and for each specification, it sequentially generates presentation lists TRi (first time (TR1), second time (TR2), ...: for the learning period) in the presentation list memory 250 (see Figure 16). Note that each presentation list TRi is assigned a number (identification number) corresponding to the month and day or the number of times or occurrences, but in this explanation, the numbers will be described in lowercase.

[0085] As shown in Figure 16, the presentation list TRi consists of a serial number, an English word (Q in Figure 16), its code, a Japanese translation (labeled A in Figure 16), a schedule name (labeled SZ in Figure 16), a presentation list number (e.g., TR1), and tasks (programs) not shown. The schedule name is associated with schedule conditions. In other words, since the schedule conditions include the learner ID, the presentation list is generated for each learner (based on the learner ID). The presentation list number is TR1 for the first presentation, TR2 for the second presentation, and TRn for the nth presentation.

[0086] This means that the system is now ready to receive logins from learner devices. Next, let's return to Figure 3 for explanation. As shown in Figure 3, the transmission list distribution unit 260 waits for a login from the learner terminal 1a (d34). Please note that the presentation list will not be delivered to the learner's device unless they log in. In other words, even if the schedule is set to study every day, the learning will not take place unless the learner logs in.

[0087] The learner operates learner terminal 1a to log in to the site (by selecting an icon) (d40, d42). The login ID is the learner ID. The transmission list distribution unit 260 searches the learner information memory unit 10 for learner information SGi (learner ID) that has this learner ID, in conjunction with this login (including account, date and time). If learner information (learner ID) with this learner ID is registered, it performs the distribution process of the presentation list for that learner (also called the distribution of the transmission list) (d44).

[0088] (Distribution of presentation list) Further explanation regarding the distribution process of presentation lists (distribution of sending lists) will be provided. The transmission list distribution unit 260 (distribution unit for one day's worth of presentation lists) shown in Figure 9 searches whether transmission list distribution history information Rhi (learner ID, presentation list distribution count value Tki, presentation list number Tri, date and time, transmission list distribution history information number rhi, etc.: see Figure 17) containing the retrieved learner information SGi (learner ID) exists in the transmission list history memory 320A. Figure 17 shows that the transmission list distribution history information Rhi is generated for each learner ID (labeled as IDa1 in Figure 17) and year / month / day, with IDa1 being labeled as Rh11, Rh12, ...

[0089] Additionally, the data for another learner (labeled IDa2) is listed as Rh21 and Rh22. On the other hand, if it does not exist, it is determined that this is the learner's first access, and the presentation list TR1 for the first time (day one) is searched from the presentation list memory 250 among the presentation lists TRi (see Figure 16) that have the learner ID included in this access information (also called login information) (d43). The transmission list distribution unit 260 then generates the presentation list TR1, the tasks and learner IDs associated with the schedule conditions of this presentation list, and other information as a learner-specific transmission list STRi (STR1), and stores it in the presentation list transmission history memory 320A (d45).

[0090] As shown in Figure 18, the learner-specific transmission list STRi consists of a serial number, an English word, a content code, a Japanese translation, a task code, a presentation list number TRi, a schedule name SI, etc. Furthermore, the learner-specific transmission list STRi is associated with the learner-specific transmission list number STri, the date and time, and the learner ID as headers. The learner-specific transmission list number STri is generated by a random number generator (not shown). Furthermore, each task code (D, T, K, ...) is associated with a program corresponding to that task code. Then, a send list STRi for each learner is sent to the learner's terminal via the web server (d46, d47).

[0091] At this time, the presentation list distribution counter counts the number of times the presentation list has been distributed (presentation list distribution count value Tki = Tki + 1). Furthermore, the transmission list distribution unit 260 stores the presentation list number Tri (Tr1), learner ID, date and time, presentation list distribution count value Tki, and transmission list distribution history information number of each learner's transmission list STRi (STR1) as transmission list distribution history information Rhi (see Figure 16) in the presentation list transmission history memory 320A (d48). In the case of communication using cookie information and session information, the transmission list delivery history information Rhi corresponds to the session information, and it is preferable that the session ID be the transmission list delivery history information number rhi. The cookie information preferably consists of the learner ID, the learner terminal identification code, and the session ID (the learner-specific transmission list number STri and the transmitted flag). In other words, the memory 320A for presentation list transmission history stores not only the learner-specific transmission list STRi, but also learner-specific transmission list distribution history information Rhi (Rh1, Rh2, ...Rhn) as shown in Figures 17(a) and (b). Figure 17 shows, as an example, the transmission list distribution history information (also called presentation list distribution history information) for learners a1 and a2. Learner a1's learner ID is denoted as IDa1, and learner a2's ID is denoted as IDa2. The first transmission list distribution history information (presentation list distribution history information) is denoted as Rh1, and subsequent entries are denoted as Rh2, ...

[0092] On the other hand, the transmission list distribution unit 260 (the daily presentation list distribution unit) reads the transmission list distribution count value TKi (presentation list distribution count value) of the transmission list distribution history information Rhi that has the logged-in learner ID. Then, the read transmission list distribution count value TKi (presentation list distribution count value) is updated (TKi = TKi + 1), and the presentation list TRi corresponding to this updated learning session (learning date) is read from the presentation list memory 250. Then, as described above, the transmission list distribution unit 260 transmits the presentation list TRi (TRi=TRi+1: for example, TR2), the tasks and learner IDs associated with the schedule conditions of this presentation list, and other information to the learner-specific transmission list STRi (STRi=STRi+1: for example, STR2) via the web server again to the learner terminal (with a learner terminal identification code added), as shown in d46 and d47.

[0093] At this time, as explained above, the memory 320A for the presentation list transmission history stores the transmission list distribution history information Rhi (Rhi = Rhi + 1: for example, Rh2). The transmission list distribution unit 260 performs this processing over the learning period GTi. In other words, the transmission list distribution history information (presentation list distribution history information) is saved as a history for each learner and for each learning date (year, month, and day).

[0094] On the other hand, as shown in Figure 3, the learner terminal 1a downloads the learner-specific transmission list STRi that has been sent, and sequentially displays the content CNi of this list based on the task for learning (see Figure 19) (d50). Then, it determines whether a response button has been selected for the content CNi (d52), and if no response button has been selected for the content CNi, it returns to d50 and sequentially displays the content CNi for learning. The display is based on the task. For example, a timer may be displayed, then items (good, slightly good, ...) may be displayed, and if an item is selected, the answer included in the learner-specific transmission list may be displayed, and the message may be sent by selecting the send button.

[0095] Let me now explain Figure 19. As shown in Figure 19(a), the problem consists of a problem content field ga (for example, Emphasis), a response measurement bar gb, and an answer display button gc (after thinking for a few seconds, the answer display button gc is selected), etc.

[0096] Furthermore, selecting the aforementioned answer display button gc (response button) displays a self-assessment screen for the level of mastery of the content CNi (words) shown in Figure 1917(b) (self-assessment items Fm: completely useless, useless, need a little more, good).

[0097] For the self-assessment item Fm, select the appropriate word by comparing the displayed meaning with your own implicit memory (however, do not try to memorize words you don't know at all; just look at the answer and move on).

[0098] Based on the task, the learner terminal 1a stores a set of information (collectively referred to as content learning result information (response)) for each selection of the self-assessment button Fm(gc) by the learner, including the learner-specific transmission list STRi number STri, reaction time, response selection number, and current time Tni. This set of content learning result information is called the learner-specific learning result list SARi (also simply called the response list).

[0099] Then, as shown in Figure 3, the learner terminal 1a (computer) displays a content learning result submission button (not shown) after learning content CNi based on the task (or even during the learning process) and sends the learner-specific learning result list SARi (also simply called the response list) to the site (d53, d56). At this time, cookie information is attached and sent.

[0100] The learner-specific learning results list SARi (file), as shown in Figure 20, is a collection of records consisting of a serial number, English word, content code, Japanese translation, learner-specific transmission list number STri, presentation list number Tri, schedule name, learner ID, response selection number hai, the time taken to respond to this content CNi (hereinafter referred to as content response time Thai), and task code (simply referred to as task). The content response time (Thai) is listed as, for example, 0.5 seconds, 1.0 seconds, 5.0 seconds, 0.8 seconds, 2.0 seconds, or 1.5 seconds, etc.

[0101] Furthermore, the learner terminal sends the learner-specific learning result list SARi (file) by associating it with the learner-specific transmission list number STri from the learner-specific transmission list STRi received in d46, the date and time (time from the learner terminal), the learner ID, the learner-specific learning result list number SAri, and learner terminal identification information as headers. The learner-specific learning result list number SAri is generated by a random number generator (not shown).

[0102] In other words, the date and time, the learner ID, the learner-specific learning result list number SAri, and the learner-specific transmission list number STri (learner terminal identification information) are transmitted as cookie information. The learner-specific learning result list number SAri and the learner-specific transmission list number STri correspond to the session ID.

[0103] Meanwhile, the performance data collection and analysis unit 300 of the site saves the learning result history memory 320B each time it receives a learning result list SARi for each learner (d58), and performs performance analysis processing. As shown in Figure 10, the performance data collection and analysis unit 300 consists of a performance analysis unit 310, a reaction collection unit 330, a performance analysis result distribution unit 340, and the like. As shown in Figure 3, the response collection unit 330 of the performance collection and analysis unit 300 of the site shown in Figure 10 receives this learner-specific learning result list SARi and stores it in the learning result reception history memory 320B (d58).

[0104] Then, as shown in Figure 3, the performance analysis unit 310 performs the analysis (d60). This analysis process is explained below.

[0105] (Analysis process) The performance analysis result distribution unit 340 shown in Figure 10 receives the analysis result request information (including account (learner ID), schedule name SI, and date and time) when a login is made (access may also be made by the site's analysis administrator).

[0106] The performance analysis result distribution unit 340 then searches the presentation list transmission history memory 320A for transmission list distribution history information Rhi (see Figure 17) that has the learner ID included in the analysis result request information RAYi. Then, it reads the presentation list distribution counter value TKi of the transmission list distribution history information Rhi with the earliest date and time among these presentation list transmission history memory 320A, and determines whether this presentation list distribution counter value TKi represents a count value of, for example, the 5th time (or the 5th day if learning is done daily) or more.

[0107] If the count value is for the 5th time (or the 5th day if learning is done daily) or more, this analysis result request (including learner ID, schedule name SI, date and time, and analysis result request flag) is output to the performance analysis unit 310 for analysis.

[0108] Then, the performance analysis result distribution unit 340 reads the analysis result information (see Figure 21) containing this analysis result request (learner ID, schedule name SI, year, month, day, and time) from the analysis result memory 321 and sends it to the logged-in learner terminal 1a (d63).

[0109] Specifically, the performance analysis unit 310, as shown in Figure 10, reads all of the learner-specific learning result list SARi (see Figure 20) containing the learner ID (IDai) included in the analysis result request information RAYi stored in the learning result history memory 320B (d61), calculates the performance (standard deviation) for each learning session up to the present, finds a regression line (see Figure 21), calculates this regression line as the trend up to the current login, and saves it in the analysis result memory 321 (d62).

[0110] As shown in Figure 21, the graph ANi of this analysis result has the learning cycle (number of learning iterations) on the horizontal axis and the average of the self-assessment values ​​of multiple content CNi on the vertical axis. It also shows the regression line Yi (Yi = ax + b: also called the approximation line). This shows that the learning performance has improved. The generation of the graph ANi of the analysis result will be described later.

[0111] More specifically, the performance analysis unit 310 generates a graph ANi of the analysis results for each learner, which corresponds the average of the response values ​​(performance (Fm)) to the problems in the content CNi of the learning result list SARi for all learners whose learner IDs are included in the analysis result request information RAYi, to the number of learning sessions (year, month, day), and for example, calculates a regression line Yi (Yi = ax + b). This regression line Yi is also called the learner-specific performance trend line.

[0112] Then, the system generates learner-specific analysis information (including learner-specific analysis information number) in the analysis result memory 321, which associates the graph ANi of the analysis results for each learner, the line showing the progress of each learner's performance, and the analysis result request information RAYi. This information is then output to the performance analysis result distribution unit 340, and the graph of the analysis results is distributed to the learner's terminal.

[0113] Next, we will explain the advertising distribution process of a company using the sequence diagram in Figure 4. In this embodiment, the company will be described as a bank (it could also be a university, local government, etc.). If the "Request Analysis Results" button (not shown) on the screen is selected, the learner terminal 1a sends the analysis result request information RAYi to the site (d74, d76). If the "Request Analysis Results" button (not shown) is not selected, the analysis results and advertisements will not be sent to the learner's terminal. In other words, the learner will not view the advertisements.

[0114] As shown in Figure 10, the analysis result distribution unit 340, upon receiving the analysis result request information RAYi, determines whether a predetermined number of learner-specific learning result lists SARi, each containing the learner ID included in the analysis result request information RAYi, have been stored in the learning result history memory 320B (d80).

[0115] If it is determined that the data has not been stored for a predetermined number of times, a message is sent to the logged-in learner terminal 1a informing it that the analysis result request information cannot yet be received (d81).

[0116] The determination of d80 mentioned above is, specifically, the same as that of d60.

[0117] If it is determined that the number is greater than or equal to a predetermined number, the performance analysis unit 340, as described above, outputs this analysis result request information RAYi to the performance analysis unit 310 (d82).

[0118] Then, the performance analysis result distribution unit 340 (Figure 10) of the performance collection and analysis unit 300 shown in Figure 1 outputs this learner-specific analysis information and the analysis result request information RAYi to the advertisement distribution unit 400 in Figure 1, which generates banner advertisement information BNi (including company code and banner advertisement image number), learner-specific performance trend line Yi, learner-specific graph ANi, and learner-specific analysis graph advertisement information ABNi including the analysis result request information RAYi (d86), and transmits it to the learner terminal (d88, d90).

[0119] Furthermore, the learner-specific analysis information (including the learner-specific performance graph ANi and the learner-specific performance trend line) and the analysis result request information RAYi [analysis result request code and learner-specific transmission list number STri] are collectively referred to as the advertisement request command information.

[0120] The learner terminal receives learner-specific analysis graph advertisement information ABNi and displays it as shown in Figure 22 based on the control codes contained in this learner-specific analysis graph advertisement information ABNi (d92, d93). Furthermore, the reaction collection unit 330 shown in Figure 10 performs a learner-specific reaction result history generation process each time it receives a learner-specific learning result list SARi (d94). The learner-specific response result history generation process, each time it receives a learner-specific learning result list SARi, records the learner IDs included in this learner-specific learning result list SARi (see Figure 20) as REG. - Read as ID, this REG - Learner ID for measuring advertising effectiveness corresponding to the ID (RAND - Read the ID from the REG-RANDID list. Then, the learned ID for measuring advertising effectiveness (RAND - Associate the learner-specific learning results list SARi with the ID (however, the learner ID is REG - (Excluding the ID). Then, the learner information SGi with the learner ID from the learner-specific learning result list SARi is searched from memory 10 (10a, 10b, or 10c).

[0121] Then, the organization name (e.g., school name), age, grade level, gender, region (e.g., XX prefecture, XX city, XX town), and learner level (e.g., Eiken Grade 1, Grade 2, 1st year high school, 2nd year junior high school, etc.) contained in the searched learner information SGi are associated as learner basic attribute information, and these are stored in memory 333 as learner-specific response result collection history information (excluding personally identifiable information such as name, address, telephone number, email address, etc.).

[0122] The learner response result collection history information includes the date, time, organization name (e.g., school name), age, grade level, gender, region (e.g., XX prefecture, XX city, XX town), name, learner level (e.g., Eiken Grade 1, Grade 2, High School 1st year, or Junior High School 2nd year, etc.), and REG - It consists of an ID and a list of learning results for each learner (SARi, but excluding the header (at least the learner ID)). Therefore, if this learner-specific response result collection history information is transmitted from this site to an external source (such as an external research institution's server or an advertiser's server), even if this learner-specific response result collection history information is obtained and unauthorized access is gained to this site, it will not be possible to identify who is doing what because it does not include the learner ID. Furthermore, if the learner-specific response result collection history information is transmitted to an external research institution, it will be possible to provide analysis results tailored to that research institution.

[0123] The configuration of the aforementioned advertising distribution unit 400 will be explained using Figure 23. Its processing sequence will be explained using Figure 5. As shown in Figure 23, the ad delivery unit 400 includes an ad schedule generation unit 420, an ad embedded delivery unit 460, an ad delivery history generation unit 480, an ad memory 470, an ad delivery history memory 490, and the like.

[0124] The ad schedule generation unit 420 generates an ad delivery schedule according to the learning schedule conditions (difficulty level). For example, if the learning schedule condition is to study every day (one presentation list), an ad delivery schedule will be generated that delivers ads every day.

[0125] The ad embedding and distribution unit 460 determines whether it has received an ad request command from the analysis result distribution unit 340 (d102 in Figure 5).

[0126] Then, if the ad embedding and distribution unit 460 determines that it has received an ad request command, it reads the company's ad (banner) stored in the ad memory 470, and generates learner-specific analysis graph ad-attached information ABNi by embedding this ad into the learner-specific analysis information (including learner-specific performance graph ANi and learner-specific performance trend line), and sends it to the learner terminal that has logged in.

[0127] As a result, the learner's terminal that has logged in will display an advertisement BNi (banner) and an analysis result graph ANi, as shown in Figure 22.

[0128] Furthermore, the ad embedding and distribution unit 460 adds the ad delivery code, date, and time to the number (identification code) of the learner-specific analysis graph ad attached information ABNi, and outputs this to the ad delivery history information generation unit 480 as learner-specific analysis graph ad delivery identification information [banner ad information BNi, learner-specific performance trend line Yi, learner-specific performance graph ANi, analysis result request information RAYi (analysis result request code, learner-specific transmission list number STri (including presentation list TRi number, learner information ID, and schedule name SI)].

[0129] In other words, the learner-specific ad delivery identification information consists of a learner ad delivery information identification code, a group name (e.g., A), a learner ID, an ad number ADi (Ct1, Ct2, ...), an ad delivery flag, an ad delivery time, and a learner-specific analysis result request information identification code (number), etc.

[0130] Furthermore, the ad number ADi (also called the ad identification code) is the ad number ADi included in the ad information (company name, ad number ADi, ad image, registration date, and ad delivery period for each company) in the ad memory 470.

[0131] The ad delivery history generation unit 480 stores the learneder-specific analysis graph ad delivery identification information in the ad delivery history memory 490 (also referred to as the ad delivery history list) each time it is input.

[0132] (Survey distribution) Next, we will explain the distribution of questionnaires. Figure 24 is a schematic diagram of the questionnaire distribution unit 100 shown in Figure 1. This questionnaire distribution unit 100 will be explained using the sequence diagrams in Figures 6 and 7.

[0133] The survey distribution unit 100 aims to measure the effects obtained from exposure to advertisements, for example, by defining favorability as factors such as familiarity, novelty, familiarity, and the degree to which the advertisement content is retained. It distributes surveys containing questions related to these factors. It also enables the measurement of evaluations of the advertiser company, the advertisement itself, and the degree to which keywords representing the company's characteristics in the advertisement are retained.

[0134] As shown in Figure 24, the survey distribution unit 100 includes a survey distribution order setting unit 110, a survey distribution condition memory 120, a survey distribution queue definition unit 130, a survey transmission / reception unit 150, a survey information memory 180, a calendar timer 160, a survey distribution history memory 170, and the like.

[0135] The survey distribution order setting unit 110 outputs the number of groups entered (7 groups, 10 groups, etc.) and the group attributes to the grouping unit 20 (which may include the date, time, and period), and groups them as described above (A, B, ...G). Note that grouping may be performed in advance before the survey distribution process. Group attributes, in the case of schools, include, for example, fourth-year university student, level (e.g., Eiken Grade 2), school name, faculty, gender, region, schedule name SI (schedule conditions), age, desired occupation, etc. The schedule name SI (schedule conditions) is optional. For those not affiliated with schools, the attributes include occupation, age, gender, desired occupation, current occupation, etc. Furthermore, the survey distribution order setting unit 100 stores the entered survey distribution order as survey distribution condition Ahj in the survey distribution condition memory 120.

[0136] The order in which the surveys are distributed is entered as follows: for example, Group A → Group B → ... Group G. Then, for example, during the distribution period, Group A (also called the baseline group) is entered as having the survey distributed one week (7 days prior: also called the specified period prior) before the advertisement is distributed.

[0137] Group B is entered, for example, 7 days after Group A's ad delivery (after the 7th learning), Group C is entered, for example, 5 days after Group B's ad delivery (after the 5th time), ... Group G is entered, for example, 5 days after Group F's ad delivery (after the 5th time).

[0138] Furthermore, starting five days after the survey for Group G is distributed, the input will be in the order of Group A → B → ... G. This information is compiled into a table or formula and stored in the survey distribution condition memory 120 as the survey distribution condition Ahj for the XX Bank survey.

[0139] The survey distribution queue definition unit 130 generates one pre-advertisement distribution queue Qa and seven post-advertisement distribution queues Qb. The reason for using seven queues is that after distributing the survey in the order of A → B → ... G, the survey is also distributed to group A to collect responses and analyze the results for group A. If the number of groups increases, these distribution queues are increased.

[0140] Then, the survey distribution queue definition unit 130 sequentially reads the survey distribution order from the survey distribution condition memory 120 in accordance with the setting of the survey distribution condition Ahj, and generates one pre-advertisement distribution queue Qa for the group designated as the pre-advertisement distribution group (Group A: also called the standard group).

[0141] Next, set the identification code and survey delivery conditions (one week before ad delivery for group B) for this group (group A). ​​Also, set the identification codes and survey delivery conditions (five days after ad delivery for the previous group) for the groups designated as post-ad delivery surveys (B, C, ..., G) in the post-ad delivery survey delivery queue Qb.

[0142] The survey distribution order update unit 140, based on the survey distribution condition Ahj, sets identification codes and their distribution conditions in the post-advertisement distribution queue Qb in the order of A, B, ... after G, if the survey has been distributed up to group G.

[0143] Furthermore, the survey distribution queue definition unit 130 reads the distribution date and time of the survey distribution conditions for group B (for example, October 1, 2024, 00:00:01) when setting the identification code (group A) and distribution conditions (one week before the distribution of advertisements for group B) for the survey distribution queue Qa used before advertisement distribution, compares it with the current time Tni of the calendar timer 160, and determines whether it is one week ago.

[0144] If it is determined that it is one week prior, the identification code for the pre-advertisement survey distribution queue Qa (Group A) is output to the survey transmission / reception unit 150.

[0145] Furthermore, after outputting the identification code for the pre-advertisement survey distribution queue Qa (Group A) to the survey transmission / reception unit 150, it is determined whether the current time Tni has reached the survey distribution conditions (for example, B is 1 week after A's distribution, C is 5 days after B's distribution, ... or G is 5 days after F's distribution, A is 5 days after G's distribution, B is 5 days after A's distribution, ...) of the first queue group (for example, B or C, ... G or A) of the post-advertisement survey distribution queue Qb. If it has reached these conditions, the identification code (B, ... G in that order, A, B, ... in that order, ...: group identification code) is output to the survey transmission / reception unit 150.

[0146] The questionnaire sending / receiving unit 150 determines, upon login from a learner's terminal, whether the learner's terminal has completed a predetermined number of learning sessions (for example, 5 or 10 or more). If the learner has completed the predetermined number of sessions, it sends a message to the learner's terminal instructing them to answer the questionnaire (even just once is acceptable).

[0147] Additionally, the identification codes of the distribution groups (A, B, C, ..., or G) from the survey distribution queue definition unit 130 are temporarily stored (overwritten).

[0148] Next, the login process will be explained using the sequences shown in Figures 6 and 7. In this embodiment, learners are contracted to answer questionnaires, and they are also committed to logging into the site daily to study. In particular, learners are provided with a regression line incorporating advertisements that shows whether their grades are improving and how much more they need to study to reach their target grades, so learners answer the questionnaires.

[0149] As shown in Figure 6, the survey distribution order setting unit 110 outputs the number of groups entered (7 groups, 10 groups, etc.) and the group attributes to the grouping unit 20 (which may include the date, time, and period) and groups them as described above (d120). On the other hand, as shown in Figure 6, the learner operates the learner terminal 1a to log in to the site (including the learner's account) (d121).

[0150] The survey sending / receiving unit 150 compares this login account with the learner information SGi in memory 30 (A, B, ...G) to determine the logged-in user (d122). The survey sending / receiving unit 150 determines whether the login user identified by the survey distribution queue definition unit 130 as matching the identification code of the pre-advertisement distribution survey queue Qa (identification code for group A) (d124).

[0151] If a match is found (for example, learners in Group A), the system determines that the learner was one week prior to ad delivery and loads the survey information stored in memory 180 (regarding brand awareness: very good, ..., very bad: survey text shown in Figure 25) (d132). The survey transmission / reception unit 150 then generates learner-specific survey text distribution information (d130) for this survey text, which includes the login account (learner ID, ...), the learner-specific survey text distribution information number, and the date and time, etc.

[0152] Then, the survey transmission / reception unit 150 sends this learner-specific survey text distribution information (with a number: Ami) to the learner's terminal that has logged in (d136). The information distributed for each learner's questionnaire consists of the group name (group ID), learner ID, date, time, advertisement number ADi, and learner-specific questionnaire distribution information number Ami, etc. At this time, the survey transmission / reception unit 150 stores learner-specific survey text distribution history information (see Figure 26) in the survey distribution history memory 170, indicating that learner-specific survey text distribution information has been distributed (d133). In addition, when saving the learner-specific survey text distribution history information, the survey transmission / reception unit 150 generates a learner-specific survey text distribution history information number Api and includes it in the storage. If a group-specific questionnaire distribution history list (file) containing the group name and date has not been generated, this file will be generated and stored. The information in the records of this file is referred to as the learner-specific questionnaire text distribution history information. In Figure 26, the learner ID is indicated as IDai(IDa1, IDa2, ...).

[0153] The learner's terminal receives the survey text information for each learner and displays the survey text on the screen (see Figure 25). do (d138).

[0154] The questionnaire text in Figure 25 consists of a score and a sentence, each associated with a number (identification code). For example, a score of 5 (very good) is associated with an identification code, and a score of 1 (very bad) is associated with an identification code. The learner operates their learner terminal to select one of the options (d140).

[0155] Specifically, participants select one of the following items (questionnaire response items: very good, somewhat good, neither good nor bad, somewhat bad, very bad) as shown in Figure 25. Although scores are shown in Figure 25, it is not necessary to display scores.

[0156] Furthermore, each of these items is assigned an identification code (number) (also known as the questionnaire response selection value ASi). At this time, the time taken to select an item (called the questionnaire response item selection time ARti) is measured.

[0157] The learner terminal transmits the selected identification code (questionnaire response selection value ASi), the learner-specific questionnaire distribution information number Ami (code), the questionnaire response item selection time ARti, the transmission time (questionnaire transmission time ASti), the learner ID, and the learner-specific questionnaire text distribution information number Ami, etc., to the site as learner-specific questionnaire response information (d142).

[0158] The survey transmission / reception unit 150 then receives this learner-specific survey response information (survey response selection value ASi, learner-specific survey distribution information number Ami, survey response item selection time ARti, transmission time (survey transmission time ASti), learner ID, date and time, and learner-specific survey text distribution information number Ami), and searches the survey distribution history memory 170 for a survey response result history list (file) containing the learner ID, group name, and date included in this information. The survey transmission / reception unit 150 then sequentially stores the received learner-specific survey response information (survey response selection value ASi, learner-specific survey distribution information number Ami, survey response item selection time ARti, transmission time (survey transmission time ASti), learner ID, date and time, and learner-specific survey text distribution information number Ami) in the searched survey response result history list (file) as learner-specific survey response result history information.

[0159] At this time, a learner-specific questionnaire response result history information number Aqi is generated and stored. If a survey response history list (file) has not been generated, a form for the survey response history list will be generated in the survey distribution history memory 170. Figure 27 shows a list of the survey response history. As shown in Figure 27, the survey response result history list consists of the group name, date (October 1, 2025), time, learner ID, survey response selection value ASi (1 or 2, ... or 5), learner-specific survey delivery information number Ami, survey response item selection time ARti, transmission time (survey transmission time ASti), learner ID, date and time, learner-specific survey text delivery information number Ami, advertisement number ADi, survey response result history information number Aqi, etc.

[0160] On the other hand, as shown in Figure 6, if a mismatch is determined in d124, it is determined that the login is not from a learner in Group A, and the system waits for a new queue to be set up from the survey distribution queue definition unit 130.

[0161] Next, we will explain the case of learners in groups other than Group A using Figure 7. On the other hand, if the survey distribution queue definition unit 130 sets the identification code for the pre-advertisement distribution survey queue Qa (identification code for group A) in the survey transmission / reception unit 150, and then the identification code for the post-advertisement distribution survey queue Qb (B or C, ... or G or A) is set in the survey transmission / reception unit 150 (for example, one week after the survey was delivered to A), and as shown in Figure 7, a login is made from the learner terminal (d151), the survey transmission / reception unit 150 determines whether the group of the logged-in learner (B or C, ... or G or A) matches the identification code for the post-advertisement distribution survey queue Qb (B or C, ... or G or A) set in the survey distribution queue definition unit 130 (d153).

[0162] If a match is found, the survey text (text: see Figure 25) stored in the survey information memory 180 is delivered to the logged-in learner terminal (d155), and the survey text delivery history information is stored in the survey delivery history memory 170 by group.

[0163] Then, the learner terminals of each group (B or C, ... or G or A) display this questionnaire text and, upon selecting a questionnaire response item, return the questionnaire response transmission time ASti, questionnaire text delivery information identification code, questionnaire response item identification code, and questionnaire response item selection time ARti as learner-specific questionnaire response information (d157).

[0164] The survey transmission / reception unit 150 receives this learner-specific survey response information and receives this learner-specific survey response information (survey response selection value ASi, learner-specific survey distribution information number Ami, survey response item selection time ARti, transmission time (survey transmission time ASti), learner ID, date and time, and learner-specific survey text distribution information number Ami). As described above, it sequentially stores this learner-specific survey response result history information in a survey response result history list (file) that contains the learner ID, group name, and date.

[0165] Then, it is determined whether the survey distribution period has ended (d168), and if not, the process returns to d151. If it is determined that the period has ended, the advertising effectiveness measurement processing unit, described later, is activated (d170).

[0166] In other words, the questionnaire distribution unit 100 performs the processing shown in Figure 28. Figure 28 is illustrated as an example where learners in groups A to G operate their learner terminals and complete a learning session once every five days, for ease of understanding. It would be preferable to explain it as learning once a day, but in this embodiment, it is explained as every five days.

[0167] For example, let's explain that learners in each group have been studying according to Schedule SI for a six-month period, starting from May 1st (or three months or six months prior). The timing of the learning sessions will be indicated as Gai(Ga1, Ga2, ...) as shown in Figure 28(a). Also, let's explain that the number of content items in the presentation list TRi for each learning session is approximately 100.

[0168] Furthermore, the timing of ad delivery is shown in Figure 28(b). Group A consists of CMa1, CMa2, .... Group B consists of CMB1, CMB2, .... Group C consists of CMC1, CMC2, ....

[0169] Group D consists of CMd1, CMd2, .... Group E consists of CMe1, CMe2, ....

[0170] The F group consists of CMf1, CMf2, ... The G group consists of CMg1, CMg2, ...

[0171] The distribution of the questionnaire is shown in Figure 28(c). It is written as follows: Group A: AAa, AAb2, ... Group B: ABa, ABb, ... Group C: ACa, ACb, ... Group D: ADa, ADb, ... Group E: AEa, AEb, ... Group F: AFa, AFb, ... Group G: AGa, AGb, ...

[0172] In other words, a group (Group A: the baseline group) is selected to take a survey before the advertisement is delivered. Within Group A, a survey is delivered to each learner's device a predetermined time before the advertisement is delivered (for example, 7 days prior) (AAa), and their responses to the survey are collected. For Group A, since the advertisement has not yet been delivered, their responses to the survey are those of a survey conducted before the advertisement was delivered.

[0173] Then, other groups (B, C, D, ..., G) are sequentially designated, and for each designated group (B or C, ..., the questionnaire is distributed at 7-day intervals starting from the distribution timing of the questionnaire for the designated group) to collect responses.

[0174] In Figure 28, for Group B, responses to a survey conducted immediately after the advertisement was delivered are collected (ABa). For Group C, the results of the survey responses during the period in which the three advertisements CMc1, CMc2, and CMc3 were delivered will be collected.

[0175] For Group D, the results of the survey responses during the period in which five advertisements (CMd1, CMd2, CMd3, CMd4, and CMd5) were delivered will be collected (ACa). For Group E, the results of the survey responses during the period in which the six advertisements CMe1, CMe2, CMe3, CMe4, CMe5, and CMe6 were delivered will be collected (AEa).

[0176] For Group F, survey responses will be collected for the period during which eight advertisements (CMf1, CMf2, CMf3, CMf4, CMf5, CMef6, CMf7, and CMef8) were delivered (AFa). For Group G, since it has been 1.5 months since the advertisements were delivered, more survey responses from the advertising campaigns will be collected (AGa).

[0177] On the other hand, after the questionnaire is distributed to groups A through G, group A is considered the first group to receive the ad, so the response results collected from group A after the questionnaire distribution will be higher than those collected from group G (AAb). Subsequent groups B, ..., G will then receive even higher response results (see ACb). Let me add some further explanation.

[0178] In other words, you determine the advertising effectiveness measurement period PLi (for example, 6 months, 3 months, 1 year, or 2 years), and also determine the group (Group A: the baseline group) to survey before the ad is delivered. Then, for each learner terminal within Group A, a questionnaire is delivered (AAa) a predetermined time before the delivery of advertisements to any of the other groups (for example, Group B) (for example, 7 days before (or earlier)), and responses to that questionnaire are collected (the collection of responses is also referred to as AAa). In this embodiment, the predetermined time before the delivery of advertisements (for example, 7 days before) is referred to as the response result collection reference period LTo (corresponding to the predetermined period before).

[0179] In this embodiment, the response result collection base period LTo is sequentially added to the response result collection base period LTo for each group, and for each of these sequentially added periods (hereinafter referred to as the group-specific response result collection period LTo (also called a predetermined period)), questionnaires are distributed for multiple advertisements delivered during the group-specific response result collection period LTo (LTi = LTo + LTo), and the responses are collected. For Group A, since the advertisements had not yet been delivered, their responses to the survey reflect responses to a survey conducted before the advertisements were delivered. In other words, they were surveyed about their favorability towards the company (although some were unaware of the company whose advertisements would be delivered) when the survey was conducted.

[0180] Then, other groups (B, C, D, ..., G) are sequentially designated, and for each designated group (B or C, ..., ...), the questionnaire is simultaneously distributed (also called spray distribution) to all learner terminals within that group (A or B, ..., or G) at 7-day intervals starting from the distribution timing of the questionnaire for that group (ABa, ACa, ..., AGa), and responses from these learner terminals are collected (response collection is also indicated as ABa, ACa, ..., AGa).

[0181] In Figure 28, for Group B, the responses of each participant to a questionnaire immediately after a single simultaneous advertisement distribution (CMb1 in Figure 28(b)) were collected (ABa in Figure 28(c)). For Group C, the results of the questionnaires were collected during the period in which advertisements were simultaneously delivered to all learner terminals within Group C in each of the three phases: CMc1, CMc2, and CMc3 (see Figure 28(b)) (ACa in Figure 28(c)).

[0182] For Group D, the results of the questionnaire responses were collected for each of the five periods in which the advertisement was simultaneously delivered to all learner terminals within Group D (see Figure 28(b)) (ADa in Figure 28(c)). For Group E, the results of the questionnaire responses were collected for each of the six periods (see Figure 28(b)) in which the advertisement was delivered to all learner terminals within Group E (AEa in Figure 28(c)).

[0183] For Group F, the results of the questionnaires were collected during the period when advertisements CMf1, CMf2, CMf3, CMf4, CMf5, CMef6, CMf7, and CMef8 (see Figure 28(b)) were simultaneously delivered to all learner terminals within Group F (AFa in Figure 28(c)). Similarly, the same distribution will be made to all learner devices within Group G. Since it has been 1.5 months since the advertisement was delivered to Group G, a larger number of responses to the advertisement delivery results survey will be collected (AGa in Figure 28(c)).

[0184] On the other hand, after the questionnaire is distributed to groups A through G, group A is considered the first group for new ad distribution. Therefore, the response results collected from group A after the questionnaire distribution will include more ad distribution responses than group G (AAb in Figure 28(c)). Subsequent groups B, ..., G will then collect response results for an even larger number of ad distributions (see ABb, ACb, ... in Figure 28(c)).

[0185] Next, the group-based advertising effectiveness analysis unit 500 shown in Figure 1 will be explained using Figures 29 and 8. Figure 29 is a schematic diagram of the group-based advertising effectiveness analysis unit 500. The group-specific advertising effectiveness analysis unit 500 consists of a significance calculation unit 520, a graphing unit 540, and the like.

[0186] The group-specific advertising effectiveness analysis unit 500 performs statistical significance calculation processing (d200). The statistical significance calculation process is performed by the statistical significance calculation unit 520. The significance calculation unit 520 reads the input calculation period (for example, from [Year] [Month] [Day] to [Year] [Month] [Day]: 2 months to 6 months, 8 months, 1 year, 2 years, ...).

[0187] By inputting this calculation period, the list of survey response results history for each group (see Figure 27) spanning this calculation period in the survey distribution history memory 170 is sequentially specified.

[0188] Then, for each of these group-specific (A, B, ..., G) lists of survey response results (see Figure 27), the advertising delivery count value Km (m = A, B, ..., G) for the 7-day learner-specific survey response result history information within that list is calculated (KA, KB, ..., KG). In other words, we are calculating the actual number of people who viewed the advertisement over a 7-day period for each group (A, B, ..., or G). Note that in this embodiment, the sample size within each group is 70 people, but even with a schedule condition of daily learning, some people may not actually learn every day. Additionally, calculate the sum of the ad delivery count values ​​in Km for KA, KB, ...KG, which is ΣKm.

[0189] Furthermore, for each group-specific (A, B, ..., G) list of survey response results history (see Figure 27), the average of the survey response selection values ​​ASi included in the learner-specific survey response result history information for that group every 7 days (using KA, KB, ..., KG), the group-level average ΣKm of the survey response selection values ​​ASi for each group during the calculation period, and the total sum ΣK for all groups are used. Then, using this information, the standard deviation distribution every 7 days for each group and the standard deviation distribution over the calculation period are determined, the significant difference p for each group is calculated, and this is associated with the survey response history list of the group names in the survey distribution history memory 170. This association is preferably made as shown in Figure 30 (referred to as advertising effectiveness measurement result information in this embodiment). The group-specific advertising effectiveness analysis unit 500 transmits this advertising effectiveness measurement result information to external research institutions or companies.

[0190] The graphing unit 540 generates a graph in which the average value of the questionnaire response selection value ASi (or the average of the standard deviations) is plotted on the vertical axis and the group name is plotted on the horizontal axis, with the average value for each group over the calculation period assigned to this coordinate system (see Figure 31). In other words, calculate the average values ​​for group A and group B, or ... or G, and then calculate the difference between each average value.

[0191] For example, if we assume there is "no difference" between the mean values ​​of Group A and Group B, we can calculate the probability that a difference in the mean values ​​measured from the samples will occur.

[0192] The probability is calculated assuming that the data for group A and group B follow a normal distribution.

[0193] The regression analysis revealed that viewers carefully observed the text and logos in advertisements, and that these elements tended to be retained in their memory. However, it also showed that even with increased ad views, there was little correlation with increased viewer sensitivity, trust, novelty, or purchase intent towards the advertisements.

[0194] Looking at the trend in average scores for each viewing group, there was a tendency for the advertising to be more effective in groups with fewer ad views than in group A, which had zero views. However, in some questions, the advertising was less effective in groups with moderate views and more effective in groups with many views. This suggests that as the number of ad deliveries increases, the advertising effect tends to fluctuate in a wave-like manner.

[0195] These results suggest that distributing large quantities of advertisements indiscriminately can diminish their impact, and focusing on this aspect in the future can contribute to the effective management of advertising. Note that in Figure 31, significant differences (p) are observed between A and B, and between B and C (the results of the significance test show p<0.5).

[0196] Alternatively, the standard deviation can be calculated using the response time (ARti) of each group to determine the p-value (for example, responses between 0.5 seconds and 1.3 seconds), and the optimal learner can be determined using this p-value and the p-value obtained using the aforementioned response selection value (ASi). Next, as shown in Figure 8, individual ad delivery processing is performed (d400). Individual ad delivery processing is performed by the individual ad delivery unit 600.

[0197] As shown in Figure 32, the individual ad delivery unit 600 consists of an appropriate ad frequency calculation unit 643, an individual ad delivery timing calculation unit 644, an ad embedding unit 647, an attribute setting unit 646, and the like.

[0198] The attribute setting unit 646 allows the site administrator to input ad distributor attribute information and ad distribution period, which are used to deliver effective ads to individual learners. Advertiser attribute information includes the ad number ADi, the period, and details such as the student's status (e.g., fourth year of university), level (e.g., Eiken Grade 2), school name, faculty, gender, region, age, and desired job type. The user is also prompted to input the number of groups (for example, 7 groups) and group names (A, B, ...G). The group names could be, for example, Group A (ads delivered once a day), Group B (ads delivered once every two days), ...G (ads delivered once every seven days). The attribute setting unit 646 then generates an ad delivery table (not shown) over the ad delivery period, with Group A receiving ads daily, Group B receiving ads every two days, Group C receiving ads every three days, ...G receiving ads every seven days.

[0199] The attribute setting unit 646 outputs this ad distributor attribute information and the ad distribution table to the ad schedule generation unit 420 of the ad distribution unit 400 shown in Figure 23 for grouping. The advertising schedule generation unit 420 shown in Figure 32 reads the advertising distribution table and generates memories for groups A, B, ... G in the group-specific advertising distributor memory unit (not shown). In other words, memory is generated for ad delivery group A (daily), ad delivery group B (every two days), ad delivery group C (every three days), ... and ad delivery group G (every seven days) (a total of 7). The advertising schedule generation unit 420 then reads, for example, 70 people at a time randomly from each memory 10 of the learner information memory unit 10, and saves them as advertising target attribute information CGi, including the ad distributor attribute information such as fourth-year university student, level (e.g., Eiken Grade 2), school name, faculty, gender, region, age, and desired occupation. Then it activates the appropriate ad frequency calculation unit 643.

[0200] As a result, each time the ad embedding distribution unit 460 receives an ad request command from the analysis result distribution unit 340, it generates learner-specific analysis graph ad-attached information ABNi, which incorporates the company's advertisement into the learner-specific analysis information (including the learner-specific performance graph ANi and the learner-specific performance trend line), and sends it to the learner terminal that made the analysis request. Furthermore, the ad delivery history generation unit 480 stores the identified information of the delivered ad in the ad delivery history memory 490 (ad delivery history list) each time it is input to the analyze graph of the delivered ad for each learner. An ad delivery history list is created for each learner ID. This list contains information (referred to as learner ad delivery history information) consisting of the ad delivery date, ad delivery time, learner ID, ad number, company name, and ad delivery flag. Each record in this list is the learner ad delivery history information.

[0201] The appropriate ad frequency calculation unit 643 activates the survey distribution unit 100 shown in Figure 1, for example, when the ad distribution period is reached (or even before). As described above, the survey distribution unit 100 reads the survey information (regarding brand awareness, very good, ..., very bad: survey text shown in Figure 25) one week before ad distribution for group A and sends it to the logged-in learner terminal. Then, the system receives the learner-specific survey response information (survey response selection value ASi, learner-specific survey distribution information number Ami, survey response item selection time ARti, transmission time (survey transmission time ASti), learner ID, date and time, and learner-specific survey text distribution information number Ami), and generates a survey response result history list containing the learner ID, group name, and date.

[0202] Then, as each subsequent week passes, questionnaires are collected in the order of Group B, Group C, ..., Group G, and the results are sequentially accumulated in a list as learner-specific questionnaire response history information, again in the order of A, ..., G. Based on this list of survey response results, the group-specific advertising effectiveness analysis unit 500 will perform the statistical significance calculation process as described above. In other words, information on the results of measuring advertising effectiveness over the calculation period is required, and this is graphed to determine the significant difference p in the number of ad deliveries for Group A, Group B, ..., Group G.

[0203] The appropriate ad count calculation unit 643 determines the most effective appropriate ad count based on the ad effectiveness measurement results and graphs. In this embodiment, the appropriate ad count is calculated as the number of learning sessions in which the average was, for example, 4.5 to 5.0. In other words, the appropriate advertising frequency calculation unit 643 uses the ○○ unit, ○○ unit, etc., to perform the processing shown in Figure 33. In Figure 33, we assume that each learner studies once a day throughout the learning period (Figure 33(a)). In Figure 33(a), the learning timings are Ga1, Ga2, ... (only Ga1 is shown).

[0204] The advertisements are delivered as follows: Group A receives an advertisement every two days (CMa1, CMa2, ...), Group B receives an advertisement every five days (CMb1, CMb2, ...), Group C receives an advertisement every day (CMc1, CMc2, ...), Group D receives an advertisement every seven days (CMd1, CMd2, ...), Group E receives an advertisement every nine days (CMe1, CMe2, ...), Group F receives an advertisement every 14 days (CMf1, CMf2, ...), and Group G receives an advertisement every 25 days (CMg1, CMg2, ...) (see Figure 33(a)). Meanwhile, the distribution of questionnaires and the collection of responses are shown in Figure 33(c). Group A is labeled AAa, AAb2, ..., and responses from AAa were collected by distributing the questionnaire one week before CMa1.

[0205] The questionnaire distribution and response collection for Group B are indicated as ABa, ABB, ... Group C is written as ACa, ACb, ..., Group D is written as ADa, ADb, ..., Group E is written as AEa, AEb, ..., Group F is written as AFa, AFb, ..., Group G is written as AGa, AGb, ...

[0206] In other words, the timing of ad delivery is varied for each ad, and the response (favorability) is collected in the same way as described above. Based on these results, the optimal number of ad appearances is determined. The individual ad delivery timing calculation unit 644, shown in Figure 32, sequentially searches the memory unit 10 for learner information that has the appropriate number of ads (for example, once every three times) from the appropriate ad count calculation unit 643 and attribute information (average of 4.5 or higher) of the group (for example, B) from which the appropriate number of ads was obtained. For each of the searched learner information (hereinafter referred to as each learner for whom a promotional ad is delivered), it sets the appropriate number of ads in the ad schedule generation unit 420 and delivers them to calculate the timing for delivering promotional ads from the company to each learner.

[0207] The individual ad delivery timing calculation unit 644 searches the ad delivery history memory 490 (ad delivery history list) which contains the learner ID of the referral ad delivery learner. Then, the number of learner ad delivery history entries in the searched ad delivery history list is used as the current number of ad deliveries to that learner. In other words, this ad delivery count indicates how many days the ad has been delivered. Human liking, as shown in Figure 34, can be explained by plotting liking on the vertical axis with the horizontal axis representing the date and year. Lithuanian liking is a wave-like curve Li that, after seeing something likable (something of interest) for the first time (once or several times), remains high for a while, then declines, rises slightly again after seeing it again, then declines again, gradually rises after seeing an advertisement, and then becomes even more likable afterward.

[0208] In Figure 33, the initial viewing period is represented as Lpa (e.g., 5 days), the period during which favorability remains high for a while before declining is Lpb (e.g., 2 months), the period of slight increase again is Lpc (e.g., 10 days), the subsequent decline is Lpd (e.g., 1.5 months), the period during which favorability gradually increases after seeing the ad is Lpe (e.g., 10 days), and the period during which favorability increases even further is Lpf (e.g., 15 days). The ad delivery period Lai is also shown as 6 months. The appropriate period for delivering promotional ads is from the time LPE (approximately 4 months after ad delivery) to Lpf.

[0209] The individual ad delivery timing calculation unit 644 activates the ad embedding unit 647 when it reaches Lpf. The banner ad embedding unit 647 reads the promotional ad (banner) from memory 645 and outputs it to the ad distribution unit 400. The promotional ad is, for example, an advertisement for a job fair at XX Bank, as shown in Figure 35.

[0210] Therefore, learners are more likely to click on this promotional advertisement and attend the job fair at XX Bank. Therefore, XX Bank can reduce its advertising costs. Furthermore, the advertisements used to attract customers can be repeatedly distributed; for example, the same advertisement distributed to first-year university students could be changed and distributed to second-year students, and then the same advertisement distributed the previous year could be used to distribute to first-year students in the following academic year.

[0211] The aforementioned task (also called the display control task) is a program that defines how the content CNi is displayed and how the user's response is obtained. For example, it is a display control task that displays the word "apple," and when a button is selected, displays the answer (apple in Japanese) and prompts the user to self-evaluate (input a response).

[0212] Tasks include K (e.g., first-year high school drills), U (e.g., first-year high school tests), O (questions), X (instruction), F (familiarity assessment), R (recognition tests), etc. [Explanation of Symbols]

[0213] 10. Memory section for learner information 20 Grouping section 30. Memory section for learner information per group. 100 Survey Distribution Department 200 Learning Content Distribution Department 300 Performance Data Collection and Analysis Department 400 Advertising Distribution Department 500 Group-Specific Advertising Effectiveness Analysis Department 600 Individual Ad Delivery Department

Claims

1. An advertising effectiveness measurement method is provided which, over the course of a learning period, an advertising effectiveness measurement site generates a presentation list containing multiple learning contents appropriate to the learner's level on each learning day of the learning period and transmits it sequentially via a communication network to the learner terminals of multiple learners, sequentially collects a response list in which the learners input their response values ​​to the learning contents, analyzes the learners' level of achievement based on the responses in these response lists, incorporates the advertiser's advertisement into this analysis result, and analyzes the effectiveness of this advertisement. The server of the aforementioned advertising effectiveness measurement site is (A) A step of generating a learner ID for each learner, and generating another ID different from this learner ID, and storing it in the memory unit in correspondence with the learner ID for each learner, (B) The steps of adding the learner's learner ID to each of the learner's presentation lists and sending them, and adding this learner ID to the response list and sending it back to be stored in the memory unit, (C) A step of analyzing the learner's level of achievement based on each response value in the accumulated response list, (D) Each time the learner's level of achievement is analyzed, the analysis result is read from the other ID corresponding to the learner ID attached to the returned response list, and the analysis result is associated with this other ID and stored in the storage unit. (E) The steps include: incorporating the advertiser's advertisement into the analysis results of the learner's achievement level, sending a questionnaire to the learner to select their impression of the advertiser, returning the impression selection results with the learner ID, and storing them in the memory unit; (F) A step of statistically analyzing the accumulated results of selecting the advertiser's impression to obtain attribute information of learners who have reached a favorable impression level, (G) A step of storing in the memory unit the other ID corresponding to the learner ID attached to the impression selection result and the statistically analyzed result in association with each other, A method for providing advertising effectiveness, characterized by performing the following actions.

2. The server of the aforementioned advertising effectiveness measurement site is (H) A step of sending the other ID and the statistically analyzed results together, excluding the learner ID, to the advertiser or an external organization. The method for providing advertising effects according to claim 1, characterized by performing the following.

3. The aforementioned memory unit stores learner information for multiple learners, including their personal identification information and learner attribute information such as their learning level and the organization to which they belong, with the learner ID attached. Step (E) above is, (E1) A step of incorporating the advertisement into the analysis results of the multiple learners based on the advertisement delivery schedule and sending it, (E2) The step of grouping the multiple learner information into a predetermined number of groups, (E3) Using one of the grouped groups as a reference group, the step of sending a questionnaire regarding the advertiser's favorability to learner terminals within this reference group before a predetermined period of time for sending the advertisement, (E4) After each predetermined period, the next group other than the standard group is sequentially designated, and after the last group is designated, the standard group is designated after the predetermined period has elapsed. (E5) The advertising effectiveness provision method according to claim 1, characterized in that each time the next group is designated, the questionnaire is sent to the learner terminals within that designated group and the results of the impression selections are accumulated, and when the group of the criteria is designated again, the questionnaire is sent and the results of the impression selections are accumulated.

4. The method for providing advertising effectiveness according to claim 1, characterized in that the advertisement and questionnaire are distributed simultaneously.

5. The aforementioned presentation list is, It is generated based on schedule conditions that combine learner attribute information for each learner, timing conditions indicating when they will study, and presentation conditions indicating what content they will study and how many times. The analysis of the learners' achievement levels is as follows: The method for providing advertising effectiveness according to claim 1, characterized in that the presentation list is distributed to multiple learner terminals, the content contained therein is trained on them, responses are collected, and an approximate straight line of performance is obtained.

Images