Service provision system

Abstract

To provide an advertisement-related service provision system and user-side advertisement equipment, which allow for minimizing manual work for collating an advertisement with manuscript data thereof.SOLUTION: An advertisement-related service provision system provided herein is configured to: receive advertisement data; receive manuscript data used to create the advertisement; determine consistency between the advertisement and the manuscript data on the basis of the received advertisement data and manuscript data to identify points of the advertisement to be corrected; and reflect a learning result of machine learning performed to determine consistency between the advertisement and the manuscript data so as to determine consistency.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to an advertisement-related service system that provides services related to advertisements such as flyers, and to a user-side advertisement facility. More specifically, the present invention relates to an advertisement-related service system that determines and notifies parts of an advertisement that need proofreading, and to a user-side advertisement facility that can be connected to the Internet on the advertisement-related service system side that provides services related to advertisement creation. [Background technology]

[0002] One such advertising-related service providing system is one that compares advertisements such as flyers with manuscript data and finds any parts that do not conform to the manuscript (eg, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-133734 A Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 has the drawback that manual work is required to compare advertisements such as flyers with manuscript data, resulting in high labor costs and other expenses.

[0005] The present invention has been devised in view of the above-mentioned circumstances, and its object is to provide an advertising-related service providing system and user-side advertising equipment that can minimize the manual work required to match an advertisement with its manuscript data. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an advertisement-related service providing system for providing a service related to advertisement creation, comprising: an advertisement data receiving means for receiving advertisement data; manuscript data receiving means for receiving manuscript data used in creating the advertisement; a proofreading portion determining means for determining consistency between the advertisement and the manuscript data based on the advertisement data received by the advertisement data receiving means and the manuscript data received by the manuscript data receiving means, and determining portions of the advertisement that need proofreading; The proofreading portion determination means determines the consistency by reflecting a learning result obtained by a proofreading machine learning means performing machine learning for determining the consistency between the advertisement and the manuscript data.

[0007] With this configuration, the learning results obtained by the proofreading machine learning means are reflected in the determination of the consistency between the advertisement and the manuscript data, and the parts of the advertisement that need to be proofread are identified, thereby minimizing human work as much as possible.

[0008] Preferably, the advertisement data includes data of a plurality of advertisement products; The manuscript data includes data specifying advertisement content for each of a plurality of products to be advertised, The machine learning performed by the calibration machine learning means includes value set classification identification machine learning for classifying and identifying the advertisement data received by the advertisement data receiving means for each single value set, The calibration part determination means A value group classification identification process that classifies and identifies the advertisement data by single value groups by reflecting the learning results of the value group classification identification machine learning; a document corresponding portion identification process for identifying which portion of the advertisement content in the document data corresponds to each of the single value sets identified by the value set classification identification process; A consistency determination process may be performed in which each single value set identified by the value set classification identification process is matched with the advertising content identified by the manuscript corresponding portion identification process to determine the consistency.

[0009] Preferably, the consistency determination process may determine the consistency between the document data and a character image of each single value set identified by the value set-by-value set classification identification process.

[0010] Preferably, the machine learning performed by the proofreading machine learning means includes text data conversion machine learning for converting character images of each single value set classified and identified by the value set classification identification process into text data, The consistency determination process may convert the character image into text data by reflecting a learning result obtained by the text data conversion machine learning, and then determine the consistency between the text data and the manuscript data.

[0011] Preferably, the machine learning performed by the calibration machine learning means further includes product feature learning for learning the features of the product based on data of an original image of an actual product corresponding to the advertised product; The method further includes: storing an association between the product characteristics learned by the product characteristic learning and identification information for identifying the product, and storing the association between the product characteristics learned by the product characteristic learning and identification information for identifying the product; the manuscript data is data in which the advertisement content is associated with the identification information of each of a plurality of products to be advertised, The manuscript corresponding portion identification process may identify identification information stored in the correspondence storage means in correspondence with the product features used to identify the advertised product, and identify the advertising content associated with the identification information from the manuscript data.

[0012] Preferably, the proofreading machine learning means performs machine learning specialized for each group by performing machine learning based on data grouped by predetermined attributes; The proofreading portion determination means may determine the consistency by reflecting a result of machine learning specialized for each group by the proofreading machine learning means.

[0013] In another aspect of the present invention, there is provided a user-side advertising device connectable to an advertisement-related service providing system that provides a service related to advertisement creation, comprising: an advertisement providing means for providing an advertisement to the advertisement-related service providing system; a manuscript data providing means for providing manuscript data used in creating the advertisement to the advertisement-related service providing system; a proofreading portion determining means provided on the advertisement-related service providing system side determines the consistency between the advertisement and the manuscript data based on the advertisement data provided by the advertisement providing means and the manuscript data provided by the manuscript data providing means, and determines the portions of the advertisement that need proofreading, and a receiving means receives a service based on the determination result from the advertisement-related service providing system side; The proofreading portion determination means determines the consistency by reflecting a learning result obtained by a proofreading machine learning means performing machine learning for determining the consistency between the advertisement and the manuscript data.

[0014] With this configuration, the learning results obtained by the proofreading machine learning means are reflected in the determination of the consistency between the advertisement and the manuscript data, and the parts of the advertisement that need to be proofread are identified, and services based on the results of this determination can be enjoyed, thereby enabling users to receive services that reduce labor costs by eliminating human work as much as possible. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an overall configuration of a service providing system. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of a service providing system and an advertisement-related service providing system. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of a user terminal. [Figure 4] (A) is a flowchart showing the processing of an advertiser's user terminal, (B) is a flowchart showing the advertising effectiveness measurement processing, and (C) is a flowchart showing the main routine of big data machine learning processing. [Diagram 5] 1A is a flowchart showing a subroutine program of machine learning for predicting price fluctuations, and FIG. 1B is a flowchart showing a subroutine program of machine learning for generating optimized advertisements. [Figure 6] FIG. 1A is a flowchart showing a subroutine program of supervised learning for generating optimized advertisements, and FIG. 1B is a flowchart showing a subroutine program of reinforcement learning for generating optimized advertisements. [Figure 7] 13 is a flowchart of a price fluctuation prediction process. [Figure 8] FIG. 2 is a diagram showing the overall configuration of a machine learning means for proofreading. [Figure 9] 13 is a flowchart showing a main routine of a flyer proofreading portion determination process. [Figure 10] This is an explanatory diagram explaining the flow of the process of identifying proofreading points on a flyer, where (A) is a diagram showing the flyer, (B) is a diagram showing multiple value sets on the flyer being identified for each single value set, (C) is a diagram showing single value sets being extracted, (D) is a diagram showing product images and character images being separated, and (E) is a diagram showing the AI ​​data created by converting character images into text data using OCR and comparing it with the manuscript data to output the discrepancies. [Figure 11] (A) is a flowchart showing a subroutine program for reference storage unit selection processing, (B) is a flowchart showing a subroutine program for single value set identification processing, (C) is a flowchart showing a subroutine program for product image identification processing, and (D) is a flowchart showing a subroutine program for target product image extraction processing. [Figure 12] (A) is a diagram showing a subroutine program for corresponding label extraction processing, (B) is a flowchart showing a subroutine program for character position identification processing, (C) is a flowchart showing a subroutine program for AI data creation processing using OCR, and (D) is a flowchart showing a subroutine program for word classification processing. [Figure 13](A) is a flowchart showing a subroutine program for comparing crawler collected data with manuscript data, (B) is a flowchart showing a subroutine program for comparing AI data with manuscript data, and (C) is a flowchart showing a subroutine program for storing price data. [Figure 14] 13 is a flowchart showing a process of a user terminal of an advertisement production company. [Figure 15] 1A is a flowchart showing a subroutine program for learning processing using training data in the practical phase, and FIG. 1B is a diagram showing trained models grouped and stored in the trained storage section using training data in the practical phase. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The service providing system, advertisement-related service providing system, user side equipment, and user side advertisement equipment in this embodiment will be described in detail with reference to the drawings. Hereinafter, database will be referred to as DB, and artificial intelligence as AI. The service providing system in this embodiment utilizes machine learning by AI, and the overall system configuration will be described with reference to FIG.

[0017] This service providing system has the function of having the machine learning means 34 perform machine learning using training data that utilizes big data, inputting the big data into the trained model 50 generated thereby, predicting price fluctuations and conducting futures trading based on the price fluctuation predictions, and even generating optimized advertisements and providing (transmitting) them to advertisers 39 and advertisement creation companies 31.

[0018] There are three methods for inputting the collected big data and storing it in the big data DB 36: inputting the flyers 27 and the manuscript data 24 used to create them provided by the advertising production company 31, inputting data collected by the crawler 37 crawling around the Internet (e.g., weather, natural disasters, stock prices, exchange rates, interest rates, economic trends, income fluctuations, geopolitical risks, etc.), and inputting data for measuring advertising effectiveness provided by the advertiser 30 (e.g., POS data, number of times flyer coupons are used, number of inquiries about flyers, etc.).

[0019] Advertisement creation company 31, which has received a request from advertiser 30 to create an advertisement, creates manuscript data 24 for creating a flyer 27 as an example of an advertisement, and creates the flyer 27 based on the manuscript data 24. When this manuscript data 24 and the data of the flyer 27 are input into advertisement-related service providing means 35, advertisement-related service providing means 35 uses AI to match the data of manuscript data 24 and flyer 27, finds parts of flyer 27 that differ from manuscript data 24, and provides these to advertisement creation company 31 as advertisement proofreading points.

[0020] By being provided with the advertisement proofreading parts, advertisement creation company 31 has an incentive to provide flyer 27 and its manuscript data 24 to advertisement-related service providing means 35, and will take the initiative in providing flyer 27 and its manuscript data 24 to advertisement-related service providing means 35. In this way, the data related to the advertisement provided by advertisement creation company 31, particularly the product data including the price of each product (product name, capacity, price, sales limit, etc.) is accumulated in big data DB 36 as price data 41.

[0021] This service providing system is provided with an advertising effectiveness measuring means 38. The advertising effectiveness measuring means 38 measures advertising effectiveness based on effectiveness data provided by the advertiser 30 (for example, POS data, the number of times a flyer coupon is used, the number of inquiries about the flyer, etc.), and provides the results of the effectiveness measurement to the advertiser 30. As a result, the advertiser 30 is provided with an incentive to provide effectiveness data, and will take the initiative in providing such data. The provided effectiveness data 42 is stored in the big data DB 36.

[0022] The crawler 37 periodically crawls around the Internet to collect various data required for predicting price fluctuations and creating optimized advertisements. The collected data is stored in the big data DB 36 as crawler collected data 40.

[0023] For example, when the machine learning means 34 performs supervised learning, a large amount of data set consisting of input information (vector x) and correct answer information y extracted from various data stored in the big data DB 36 is input to the machine learning means 34 as training data. The machine learning means 34 performs supervised learning based on the training data to generate a trained model 50. In the case of supervised learning, a function ci(x) (ci:x→y) that maps an input x to a correct answer y is learned, so that the trained model includes the function ci. Note that the machine learning performed by the machine learning means 34 is not limited to supervised learning, and may be any type of learning, such as unsupervised learning such as model estimation and pattern mining (data mining), semi-supervised learning, reinforcement learning, deep learning, etc., which are intermediate methods between supervised learning and unsupervised learning.

[0024] By inputting input data extracted from the big data DB 36 into the generated trained model 50, it becomes possible to predict, for example, price fluctuations as a regression problem of supervised learning. This prediction result is provided to a user (e.g., an advertiser 30 or an advertisement creation company 31) as a cloud service. Furthermore, various services (e.g., futures trading, etc.) that utilize the price fluctuation prediction result may be provided.

[0025] Next, the hardware configuration of the service providing system and the advertisement-related service providing system will be described with reference to Fig. 2. The services provided by the service providing system and the advertisement-related service providing system are provided to users (advertisers 30, advertisement creation companies 31, etc.) from a cloud 43 via the Internet. For this purpose, a data center 44 in which a large number of servers 14 are installed is provided, and terminals (personal computers, smartphones, etc.) installed at the advertisers 30, advertisement creation companies 31, etc. are connected to the data center 44 via the Internet so that information can be communicated.

[0026] The server 14 is composed of a CPU (Central Processing Unit) 10 as the control center, a RAM (Random Access Memory) 9 that functions as a work area for the CPU 10, a ROM (Read Only Memory) 11 that stores data and programs, a storage means such as an HDD (hard disk drive) 12, an input operation unit 7 such as a display and a keyboard, a communication unit 5, a display unit 6, an interface 8, a bus 13, and various other hardware.

[0027] The server 14 uses a general von Neumann-type computer, but a neural net processor (NNP) may also be used. The NNP chip is equipped with a large number of "artificial neurons" modeled on real neurons, and each neuron communicates with each other in a network. A quantum computer that employs the "quantum annealing method" may also be used. This can significantly reduce the time required for optimization calculations in machine learning.

[0028] 3 shows the hardware configuration of the user terminal 74. In a plurality of stores 73 (only one is shown in the drawing) operated by the advertiser 30, products handled by the advertiser 30 are displayed and sold. Each store 73 is equipped with a POS (Point Of Sale) system, which collects POS data such as the price of each product, the number of items sold, the date and time of sale, as well as the weather at the time of sale, the customer's age group, sex, purchased products, point management, etc., and accumulates them in the user terminal 74 of each store 73.

[0029] Each user terminal 74 of the advertiser 30 is connected by an in-house LAN (Local Area Network) 84. The user terminal 74 of each store 73 is connected to the in-house LAN 84 via the Internet or the like. POS data accumulated in the user terminal 74 of each store 73 is transmitted to the user terminal 74 of the advertiser 30 via the Internet and the in-house LAN 84. The user terminal 74 of the advertiser 30 tally up the transmitted POS data and periodically transmits it to the data center 44 as effectiveness data for measuring advertising effectiveness.

[0030] Each user terminal 74 of the advertisement production company 31 is also connected by an in-house LAN 84. The in-house LAN 84 is connected to the data center 44 via the Internet or the like. The user terminal 74 of the advertisement production company 31 transmits manuscript data for creating flyers to the data center 44, and also transmits data of the flyers created based on the manuscript data to the data center 44.

[0031] 3 shows the hardware configuration of the user terminal 74 of each store 73, the user terminal 74 of the advertiser 30, and the user terminal 74 of the advertisement production company 31. This hardware configuration is the same as the hardware configuration of the server 14 shown in FIG. 2, so a repeated explanation will be omitted here.

[0032] Next, referring to FIG. 4(A), the process of the user terminal 74 of the advertiser 30 will be described. In step S (hereinafter simply referred to as "S") 150, it is determined whether or not POS data has been received from the user terminal 74 of each store 73. If not, control proceeds to S152, but if received, a process of classifying the received POS data and storing it as effect data is performed in S151. Next, in S152, it is determined whether or not it is time to transmit the effect data to the data center 44. If the time has not yet come, control proceeds to S155, but if the time has come, a process of transmitting the stored effect data to the server 14 of the data center 44 is performed (S153a). In the server 14, the transmitted effect data is received and stored as effect data 42 in the big data DB 36 in S153b of the advertisement effect measurement process.

[0033] Next, the user terminal 74 of the advertiser 30 erases the stored effect data in S154. Next, in S155, it is determined whether or not an optimized advertisement or a price fluctuation forecast has been received, and if not, control returns to S150. On the other hand, if an optimized advertisement has been transmitted from the server 14 in S48 described below, S155 determines YES, the received optimized advertisement is stored in S156, and control returns to S150. Also, if a price fluctuation forecast has been transmitted from the server 14 in S58 described below, S155 determines YES, the received price fluctuation forecast is stored in S156, and control returns to S150.

[0034] Next, the contents of the advertising effect measurement process by the advertising effect measurement means 38 described above will be described with reference to FIG. 4(B). S1 performs a process of setting each value set classified by the single value set identification process as gj. Here, j is the j-th example in the training data and takes the values ​​1, 2, 3, . . . , J. J is the total number of value sets. A value set is composed of "a product image + a text image such as a price and a product name" on a flyer (see value sets 99a, 99b, 99c, 99d in FIG. 10(B)). The "single value set identification process" is a process of classifying and identifying a plurality of value sets listed on a flyer by single value set, and will be described later with reference to FIG. 11(B).

[0035] Next, in S2, an initial value "1" is set for j. Next, in S3, effect data 42 related to gj is read from the big data DB 36. At this stage, since j=1, effect data related to the value set of g1 is read. For example, if g1 is the price set of advertiser A's product chicken breast (see value set 99d in Fig. 10(B)(C)), effect data for chicken breast sent from advertiser A (POS data, number of times flyer coupons were used, number of inquiries about the flyer, etc.) is read.

[0036] Next, in S4, the number of units sold, sales, number of flyer coupon usages, and number of flyer inquiries sent by the advertiser are divided by their respective standard values ​​to calculate the values. For example, in the case of g1 (chicken breast), this standard value is the current average value of the number of units sold, sales, number of flyer coupon usages, and number of flyer inquiries for g1 (chicken breast) for that advertiser A.

[0037] Next, in S5, a process is performed to calculate an advertising effectiveness score tj for gj by adding up the calculated values. In S6, the advertising effectiveness (tj) for gj is output from the advertising effectiveness measurement means 38 and provided to the corresponding advertiser 30.

[0038] In S7, 1 is added to the current j, and in S8, it is determined whether j has exceeded J. If j has not yet exceeded J, control returns to S3. Each time the loop of S3→S4→S5→S6→S7→S8→S3 is circulated, j is incremented by "1" in S7, and j is incremented to 1, 2, 3, ... The value set gj is also updated to g1, g2, g3, ... Then, when j exceeds the total number of value sets J, S8 determines YES, and this advertising effectiveness measurement process returns.

[0039] Next, the big data machine learning process by the machine learning means 34 will be described with reference to Fig. 4(C). Machine learning for predicting price fluctuations is performed in S9, and machine learning for generating an optimized advertisement is performed in S10.

[0040] The flowchart of the subroutine program of the machine learning for price fluctuation prediction in S9 will be explained based on Fig. 5(A). In S15, a process is performed to define each product type as kj. Here, j is the j-th product type among all product types and takes the value of 1, 2, 3, ..., J. J is the total number of product types.

[0041] Next, in S16, j is set to an initial value of 1. In S17, training data {(xji, yji)} is generated for product type kj, which consists of a data set in which past crawler collected data is vector xji and the price at that time is correct answer information yji. Here, i is the i-th example in the training data and takes the value of 1, 2, 3, . . . , I. I is the total number of training data.

[0042] Next, in S18, supervised learning is performed using the training data {(xji, yji)} to learn a function cj(x) (cj: x→y) that maps an input x to a correct answer y as a regression problem.

[0043] In S19, 1 is added to the current j, and in S20, it is determined whether j has exceeded J. If j has not yet exceeded J, control returns to S17. Each time the loop of S17 → S18 → S19 → S20 → S17 is repeated, j is incremented by "1" in S19, and j is incremented to 1, 2, 3, ... The product type kj is also updated to k1, k2, k3, ... Then, when j exceeds the total number J of product types, S20 determines YES, and the machine learning for price fluctuation prediction returns.

[0044] Next, a flowchart of the subroutine program of machine learning for generating optimized advertisements in S10 will be described with reference to FIG. 5(B). In S25, a process is performed to define each product type to be learned as kj. Here, j is the j-th product type among all product types, and takes the value of 1, 2, 3, . . . , J. J is the total number of product types. Next, an initial value of "1" is set to j in S26. In S27, it is determined whether or not supervised learning has been completed for product type kj. At this stage, since j=1, it is determined whether or not supervised learning has been completed for product type k1. If supervised learning has not yet been completed, supervised learning for generating optimized advertisements is performed in S28.

[0045] On the other hand, if supervised learning has already been completed, reinforcement learning for optimized advertisement generation is performed by S29. In other words, in machine learning for optimized advertisement generation, supervised learning is performed first, and reinforcement learning is performed for product types for which supervised learning has been completed. This makes it possible to improve the efficiency of machine learning for optimized advertisement generation. Reinforcement learning is a mechanism in which an agent placed in a certain environmental state acquires a policy that maximizes the cumulative reward from the initial state to the goal based on the reward given when the agent selects an action. In reinforcement learning, learning proceeds through interaction between a software agent (hereinafter referred to as "agent"), which is a type of AI, and the environment. Here, an agent is a type of AI, and is software that behaves autonomously with a certain degree of judgment ability and acts persistently while communicating with users and software. When an agent performs a certain action a on the environment, the state s of the environment changes and a certain target state is reached, and a reward r is given to the agent. The agent learns a function that takes the state s as input and outputs action a with the aim of maximizing this reward r.

[0046] Reinforcement learning progresses by repeating the following simple steps over time: 1. An agent receives an observation o from the environment (or directly, the state of the environment s) and returns an action a to the environment based on a policy π. 2 Based on the action a received from the agent and the current state s, the environment changes to the next state s', and based on that transition, it returns to the agent the next observation o' and a number (scalar quantity) called the reward r, which indicates the quality of the previous action. 3. Time progression: t←t+1 Here, ← represents an assignment operation.

[0047] After execution of step S28 or S29, control proceeds to S30, where 1 is added to the current j, and in S31, it is determined whether j has exceeded J. If j has not yet exceeded J, control returns to S27. Each time the loop of S27→S28 or S29→S30→S31→S27 is circulated, j is incremented by "1" in S30, and j is incremented to 1, 2, 3, . The product type kj is also updated to k1, k2, k3, . Then, when j exceeds the total number J of product types, S31 determines YES, and the machine learning for generating optimized advertisements returns.

[0048] Next, a flowchart of the subroutine program of supervised learning for generating optimized advertisements in S28 will be described with reference to Fig. 6(A). In S36, a process is performed to extract all prices Pji listed on flyers 27 with high scores among the past advertising effectiveness scores tj calculated by the advertising effectiveness measurement process (see Fig. 4(B)) for a product type kj and the predicted prices pji at that time. Here, i is the i-th extracted data among all extracted data, and takes the value of 1, 2, 3, ..., I. I is the total number of extracted data. Specifically, a "high-scoring flyer" is a flyer with an advertising effectiveness score in, for example, the top 10% of the past advertising effectiveness scores tj.

[0049] Next, in S37, training data {(xji, yji)} is generated for kj, which is a data set with pji as input information (vector xji) and Pji as correct answer information yji. Pji is the price listed in the flyer with high score for product type kj (see S36), and is suitable as correct answer information.

[0050] Next, in S38, supervised learning is performed using {(xji, yji)}, and a function cj(x) (cj: x → y) that maps an input x to a correct answer y as a regression problem is learned. When supervised learning for generating an optimized advertisement for a product type kj is completed, for example, a completion flag is associated with kj and stored. By determining in the above-mentioned S27 whether or not this completion flag is stored, it is determined whether or not kj has completed supervised learning.

[0051] Next, a flowchart of the subroutine program of reinforcement learning for generating an optimized advertisement in S29 will be described with reference to FIG. 6(B). In S46, a process is performed to calculate a reward r for a product type kj listed in the flyer 27 of the optimized advertisement provided to the advertiser 30, based on the advertising effectiveness score tj calculated by the advertising effectiveness measurement process (see FIG. 4(B)). The reward r is controlled so that the higher the advertising effectiveness score tj, the larger the reward r. Next, in S47, the optimal policy π * The process of calculating the price Pi for action a according to is performed. TD learning stands for Temporal Difference Learning, which estimates the Q value using a model-free method. The state at time t is s t In general, the optimal policy is π * (s t ):Choose a t * ifQ * (s t ,a t * ) Next, in S48, an optimized advertisement with the product price set to Pj is generated and transmitted (provided) to the advertiser 30 or the advertisement creation company 31.

[0052] Next, the contents of the price fluctuation prediction process will be explained with reference to FIG. 7. In S55, each product type is defined as kj. Here, j is the j-th product type among all product types, and takes the value of 1, 2, 3, . . . , J. J is the total number of product types. In S56, j is set to an initial value of "1". In S57, the current effect data 42 is read from the big data DB 36 and substituted into the function cj(x) to predict the price Pj. Here, the function cj(x) is a function for predicting price fluctuations learned in S18 described above.

[0053] Next, the predicted price pj is transmitted by S58. Specifically, this transmission is to transmit the price pj to the advertiser or advertisement creation company that handles the product type kj. Note that instead of transmitting one by one by S58, the price pj may be tallied for each product type kj by S58, and the tallied results may be sent to the advertiser or advertisement creation company. In S59, 1 is added to the current j, and in S60, it is determined whether j has exceeded J. If j has not yet exceeded J, control returns to S57. Every time the loop of S57 → S58 → S59 → S60 → S57 is circulated, j is incremented by "1" by S59, and j is incremented by 1, 2, 3, ... The product type kj is also updated to k1, k2, k3, .... Then, when j exceeds the total number J of product types, S60 determines YES, and this price fluctuation prediction process returns.

[0054] Next, machine learning performed by the proofreading machine learning means 33 in advance so that the advertising-related service providing means 35 can determine the proofreading parts of the flyer 27 will be described with reference to Fig. 8. Product feature learning data 87 in which a large number of original product images (images of actual products) 25 are associated with labels such as the JAN codes of the products are input to the proofreading machine learning means 33, and product feature learning 22 is performed based on the product feature learning data 87. In the case of packaged products, a JAN code is assigned and affixed, and the product is identified using the JAN code as a label, but in the case of non-packaged products such as salmon fillets, for example, a JAN code is not assigned, and a house code uniquely assigned by the entity (vendor) providing the advertising-related service is used as a label for identifying the product.

[0055] A large amount of flyer 27 data is input to the proofreading machine learning means 33, and based on the flyer 27 data, flyer feature learning 23, advertising value set boundary learning 3, product image identification learning 4, character position learning 28, OCR learning 16, and word classification learning 39 are performed. The original product images 25 are image data of the actual products of various products listed in the flyer 27, and also include image data of similar products. The more similar products there are, the more features can be extracted.

[0056] Product feature learning 22 is for learning the features of actual products because the same product image on a flyer may be viewed from different angles or in different colors, and learns the features of each product using a Neural Network (NN) or a Deep Neural Network (DNN). The learning results, learned product features 20, are stored in the learning stage learned storage unit 21. The learned product features 20 are stored by associating the features of each product with the label of the product (JAN code, house code, etc.).

[0057] The flyer feature learning 23 learns the feature amount of the flyer from the image data of the input flyer 27, and uses NN or DNN for learning. The learning result, the learned flyer feature amount 1, is stored in the learning stage learned storage unit 21.

[0058] The advertising price set boundary learning 3, product image identification learning 4, character position learning 28, OCR learning 16, and word classification learning 39 use supervised learning similar to that in Figures 5(A) and 6(A), but the correct answer is a label classification problem. In addition, since machine learning is performed from a large amount of image data, CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and LSTM (Long Short-Term Memory) are used to minimize the artificial burden of humans extracting features and inputting them as input vectors x. CNN is a convolutional neural network, a type of forward propagation type artificial deep neural network. It is a model that is widely used in image and video recognition. RNN is a recurrent neural network, an algorithm that has achieved high results in the field of natural language processing. LSTM is a model or architecture for time series data that appeared in 1995 as an extension of RNN, and makes it possible to utilize short-term memory for a long period of time.

[0059] The advertising value set boundary learning 3 is a machine learning method for identifying the boundaries between multiple value set images published on a flyer and extracting a single value set image from the multiple value set images, and uses CNN, RNN, and LSTM. The advertising value set boundary learned model 2, which is the learning result of the advertising value set boundary learning 3, is stored in the learning stage learned storage unit 21.

[0060] The commodity image identification learning 4 is a machine learning for identifying commodity images from a cut-out single-value set image, and uses CNN or the like. A single-value set image includes a commodity image and a text image (character image) such as the price and name of the commodity, and the commodity image identification learning 4 is a machine learning for identifying a commodity image in such a single-value set image. A commodity image identification trained model 18, which is a learning result of this commodity image identification learning 4, is stored in the learning stage trained storage unit 21.

[0061] Character position learning 28 is a machine learning method for identifying the position of a text image (character image) relative to a product image based on a cut-out single-value pair image, and uses CNN. A character position trained model 19, which is the learning result of this character position learning 28, is stored in the learning stage trained storage unit 21.

[0062] OCR learning 16 is machine learning that learns the font of characters used in flyers 27, designs with different character sizes and thicknesses, character strings (one line) of different lengths, character spacing, line spacing, background color (with noise), etc., and uses LSTM or CNN. The OCR trained model 17, which is the learning result of this OCR learning 16, is stored in the learning stage trained storage unit 21.

[0063] Word classification learning 39 is machine learning to identify which category the contents of each character string identified by OCR belongs to, such as price, capacity, product name, manufacturer name, etc., and a word classification trained model 32, which is the learning result of this word classification learning 39, is stored in the learning stage trained storage unit 21. AI data is created using the OCR trained model 17 and the word classification trained model 32, as described below (see FIG. 12(C)). The manuscript data 24 is Excel data in which items are divided into categories (see "label, product name, meter, price" in FIG. 10(E)), and by using this word classification trained model 32, it is possible to compare the manuscript data 24 and the AI ​​data for each item.

[0064] Next, the flyer proofreading part determination process performed by the advertisement-related service providing means 35 will be described with reference to FIG. 9. In S65, it is determined whether or not a flyer image has been received. If the image data of the flyer is transmitted to the data center 44 and input to the advertisement-related service providing means 35 in S183 described later, then S65 determines YES. Instead of transmitting the image data of the flyer, the advertisement creation company 31 may send the flyer itself to the entity (vendor) providing the advertisement-related service. In that case, the entity (vendor) providing the advertisement-related service scans the image of the flyer page, or converts the data typeset on a personal computer into a PDF, and transmits the image data to the server 14 for input to the advertisement-related service providing means 35.

[0065] If image data of flyer 27 provided by advertisement creation company 31 is input to advertisement-related service providing means 35, S65 judges YES and S67 judges whether manuscript data 24 has been accepted. If manuscript data 24 transmitted (provided) from advertisement creation company 31 is input to advertisement-related service providing means 35 in S181 described later, S67 judges YES and control proceeds to S68.

[0066] On the other hand, if the result of S65 is NO or if the result of S67 is NO, control proceeds to S66, where crawler processing is performed. In this crawler processing, a crawler periodically patrols the Internet to collect information on products handled, business hours, etc., from the homepage of the advertiser 30. The collected information is collated with the input manuscript data 24, and differences are extracted. For example, if the advertiser 30 is a drug store A, information on products handled (e.g., second-class and third-class drugs) is collected from the homepage of the drug store A and collated with the manuscript data 24 and image data of the flyer 27 of the drug store A. As a result, if a first-class drug is included in the image data of the manuscript data 24 or the flyer 27, it is determined that the manuscript data 24 or the flyer 27 is incorrect (see FIG. 13(A)).

[0067] In S68, a reference storage unit selection process is performed. In this embodiment, two types of storage units for trained models are provided: the aforementioned training stage trained storage unit 21 and the trained storage unit 29 using training data in the practical stage. In S68, it is determined which of the storage units the trained model stored in should be used.

[0068] In S69, a single value pair identification process is performed, in S70, a product image identification process is performed, in S71, a target product image extraction process is performed, in S72, a corresponding label search process is performed, in S73, a character position identification process is performed, in S74, an AI data creation process by OCR is performed, and in S75, a word classification identification process is performed. Each of these processes will be described in detail later.

[0069] Next, in S76, M is decremented by 1. This M is the number of divisions defined in S89 in the single value set identification process in Fig. 11(B), and is the number of divisions into which the multiple value sets listed in the leaflet 27 received in S65 are divided into single value sets.

[0070] Next, in S77, it is determined whether M=0. If it is not yet 0, control returns to S71, and the loop of S71→S72→S73→S74→S75→S76→S77→S71 is repeated. Each time this loop is repeated, M is decremented by 1 in S76, and when M=0, control proceeds to S78. In other words, the processing of S71 to S75 is repeated for the number of value pairs listed on the flyer received in S65, and when the processing of S71 to S75 has been completed for all value pairs, the processing proceeds to S78.

[0071] In S78, the crawler collected data is compared with the manuscript data, in S79, the AI ​​data is compared with the manuscript data, and in S80, the proofread portion is output and provided to the client, advertisement creation company 31. Next, in S81, the price data is stored, and then control returns to S65.

[0072] The main part of this flyer proofreading part discrimination process will be described with reference to Fig. 10. Referring to Fig. 10(A), a plurality of price sets are listed on flyer 27 provided by advertisement production company 31. Fig. 10 shows four types of price sets: price set 99a for tomatoes, price set 99b for green peppers, price set 99c for domestic beef loin, and price set 99d for chicken breast.

[0073] From the image of the flyer 27 in which the multiple value sets are posted, the advertising value set boundaries are determined and each single value set is identified (see FIG. 10(B)). Next, the single value set (product frame) 99d is cut out (see FIG. 10(C)). The image of this cut out single value set 99d is separated into a product image portion 97 and a character image portion 98 (see FIG. 10(D)).

[0074] Next, the separated character image portion 98 is converted into text data using OCR to generate AI data, and the AI ​​data is compared (matched) with the manuscript data 24 to identify and output inconsistent areas (proofreading areas) (see Figure 10(E)).

[0075] Next, a flow chart of the subroutine program of the reference storage unit selection process of S68 will be described with reference to FIG. 11(A). In S83, a process is performed to determine the business type, region, and season from the received flyer image. Specifically, the insertion date listed on the flyer is read by OCR to recognize the date and determine the season, the address of the store listed on the flyer is read by OCR to identify the region, and the business type is identified by identifying the products and determining the number of products using the labels (JAN codes, house codes, etc.) extracted by the corresponding label extraction process of FIG. 12(A). Specifically, the business type is identified by identifying the types of multiple products listed on the flyer from labels such as JAN codes and house codes, determining the number of products by type (number of products) and determining which type of product the flyer is mainly about. Specific examples of business types are food supermarkets, GMS (general merchandise stores), drug stores, home centers (including general discount stores), liquor discounts, real estate, etc. Real estate includes new single-family homes, used single-family homes, new condominiums, and used condominiums.

[0076] Next, in S84, it is determined whether or not a trained model corresponding to the received business type, region, and season is stored in the trained storage unit 29 based on training data in the practical stage. In the trained storage unit 29 based on training data in the practical stage, as shown in Fig. 15(B), the data is grouped into a three-dimensional table consisting of business type, region, and season (spring, summer, fall, winter), and a trained model is stored for each group. The generation process of the trained models stored for each group will be described first with reference to Fig. 15(A).

[0077] FIG. 15(A) is a flowchart of the learning process using training data in the practical stage. Various trained models that have completed the learning stage are utilized in the practical stage (FIGS. 9 to 13) where advertising-related services are actually provided, but errors rarely occur even in the practical stage. This is because in the practical stage, flyers 27 and their manuscript data 24 are provided from special business formats or special regions, and special character fonts or special expressions that were not handled in the learning stage may be encountered, causing errors in such cases. In addition, special expressions may be used for each season. Therefore, in order to respond to the particularities of business formats, regions, and seasons, in the practical stage, groups are formed according to business formats, regions, and seasons, and specialized machine learning is performed for each group.

[0078] In S129, it is determined whether the number of errors found in the practical stage has reached a predetermined number R, and the process waits until the predetermined number is reached. When the advertising-related service providing means 35 notifies the advertising creation company 31 of the advertisement proofreading portion and the advertising creation company 31 attempts to proofread the leaflet 27 based on the notification, if the error is not actually found, a notice (complaint) indicating that the notified proofreading portion was not actually found to be an error is sent to the advertising-related service provider, making it possible to find the error in the practical stage. However, if an error occurs in which a mismatch between the leaflet 27 and the manuscript data 24 is overlooked, the advertising creation company 31 is not notified of the advertisement proofreading portion, so that a notice (complaint) from the advertising creation company 31 is not sent to the advertising-related service provider, making it difficult to find the error in the practical stage. As a means of solving this problem, it is possible to pay an appropriate compensation to the advertiser 30 or the like who finds the error in the leaflet 27, thereby creating an incentive for notifying the advertising-related service provider of the discovery of the error in the leaflet 27.

[0079] When the number of errors reaches a predetermined number R, control proceeds to S130, where the value set part where the error occurred is grouped into a three-dimensional table consisting of business type, region, and season. This process is performed by specifying the business type, region, and season using the method described in S83 above.

[0080] Next, in S131, correct answer information yi is created for each group. Here, i is the i-th correct answer information among all correct answer information, and takes the value of 1, 2, . . . I. I is the total number of correct answer information. More specifically, this correct answer information creation process determines in which process the error occurred, that is, in which process of the single value set identification process, the product image identification process, the target product image extraction process, the corresponding label search process, the character position identification process, the word classification identification process, or the AI ​​data creation process by OCR, the error occurred, and creates correct answer information yi for machine learning corresponding to the process in which the error occurred. For example, if an error occurs in the character position identification process, correct answer information yi for character position learning 28 is created.

[0081] Next, in S132, the input data of the erroneous portion for each group is designated as xi. Here, i is the i-th input data among all the input data, and takes the value of 1, 2, . . . , I. I is the total number of input data. Training data {(xi, yi)} for supervised learning is generated from a data set of this input data xi and correct answer information yi. For example, if this training data {(xi, yi)} is generated when an error occurs in the character position identification process, it is used as the training data {(xi, yi)} for retraining the character position learned model 19.

[0082] In S133, a process is performed in which one example (i-th example) {(xi, yi)} is reused multiple times (e.g., 20 times) as different examples, and this reuse is performed for all i to create augmented training data. In S134, supervised learning is performed on data obtained by adding this extra training data to the existing training data used in the learning stage as training data for the group, and a process is performed in which a function c(x) (c: x → y) that maps an input x to a correct answer y is learned for each group. As a result, for a group in which an error has occurred, the learned model of the group is retrained using extra training data in which the correct answer information yi for the error and the input data xi for the erroneous part have been reused multiple times (e.g., 20 times), which has the advantage of allowing machine learning to be performed with a greater influence of training data generated based on errors that have occurred in the practical stage.

[0083] Another method of performing machine learning with a larger influence of training data generated based on errors made in the practical stage is to thin out existing training data by the number I of training data {(xi, yi)} generated in S132, and add the training data {(xi, yi)} generated in S132 to the thinned existing training data, and perform supervised learning using the resulting data as training data for the group. This method also has the advantage of being able to perform machine learning with a larger influence of training data generated based on errors made in the practical stage.

[0084] Next, in S135, a process of updating the learned model of each group to a function c(x) is performed, and the learned models having the updated function c(x) are classified by group and stored in the learned storage unit 29 based on the training data in the practical stage (see FIG. 15(B)). More specifically, the machine learning specialized for each group retrains the learned model used in the process in which an error occurred among the single value set identification process, the product image identification process, the target product image extraction process, the corresponding label search process, the character position identification process, the word classification identification process, or the AI ​​data creation process by OCR, so the learned models classified and stored by group include the retrained flyer feature learned model 1, the advertising value set boundary learned model 2, the product image identification learned model 18, the character position learned model 19, the OCR learned model 17, and the word classification learned model 32.

[0085] Returning to Fig. 11(A), if S84 judges YES, control proceeds to S86, where the learned storage unit 29 based on the training data in the practical stage is selected. As a result, advanced processing is possible using a learned model that has been retrained with the training data in the practical stage and can handle the particularities of the business type, region, and season. On the other hand, if S87 judges NO, control proceeds to S85, where the learning stage learned storage unit 21 is selected.

[0086] Next, a flowchart of the subroutine program of the single value set identification process of S69 will be described with reference to Fig. 11(B). A process of reading out the advertising value set boundary learned model 2 from the learned storage unit selected in S85 or S86 is performed in S87. Next, based on the read out advertising value set boundary learned model 2, single value set classification identification is performed in S88 to classify the multiple value sets published in the leaflet 27 received in S65 into single value sets. Next, a process of setting the number of classifications identified as M is performed in S89, and this single value set identification process returns.

[0087] Next, a flowchart of the subroutine program of the commodity image identification process of S70 will be described with reference to Fig. 11(C). A process of reading out the learned flyer feature 1 data and the learned model 18 for commodity image identification from the learned storage unit selected in S85 or S86 is performed in S95. Next, a process of identifying a commodity image from an image published in the flyer 27 accepted in S65 is performed in S96 based on the read-out learned flyer feature 1 data and the learned model 18 for commodity image identification.

[0088] Next, a flowchart of the subroutine program of the target product image extraction process of S71 will be described with reference to Fig. 11(D). A process of reading out the trained model 18 for commodity image identification from the trained storage unit selected in S85 or S86 is performed in S98. Next, a process of sequentially extracting target product images from the single-value set images classified in S88 based on the trained model for commodity image identification read out in S99 is performed.

[0089] Next, a flowchart of the subroutine program of the S72 corresponding label extraction process will be described with reference to Fig. 12(A). A process of comparing the learned product feature 20 data stored in the learned storage unit selected in S85 or S86 with the target product image extracted in S99 and selecting a product with a high similarity is performed in S104. Next, a process of reading out the label (see Fig. 8) stored in correspondence with the feature of the selected product is performed in S105.

[0090] Next, a flowchart of the subroutine program of the character position identification process of S73 will be described with reference to Fig. 12(B). Based on the character position trained model 19 stored in the trained storage unit selected in S85 or S86, a process of discriminating the single-value set image identified in S88 into a product image and a character image and identifying the character image portion is performed in S108.

[0091] Next, a flowchart of the subroutine program of the AI ​​data creation process by OCR in S74 will be described with reference to FIG. 12(C). A process of reading out the OCR trained model 17 from the trained storage unit selected in S85 or S86 is performed in S111. Next, a process of converting the character image portion identified in S108 into text data by OCR using the OCR trained model 17 is performed in S112. Next, a word classification identification process is performed in S113. Next, a process of creating a table (AI data) that associates the label read out in S105 with the text data word-classified in S113 is performed in S114.

[0092] Next, a flowchart of the subroutine program of the word classification identification process of S75 will be described with reference to Fig. 12(D). A process of reading out the word classification trained model 32 from the trained storage unit selected in S85 or S86 is performed in S116. A process of classifying the text data into words using the word classification trained model 32 is performed in S117.

[0093] Next, a flow chart of a subroutine program for the process of comparing the crawler collected data with the manuscript data in S78 will be described with reference to Fig. 13(A). In S125, the crawler collected data is compared with the manuscript data 24. In S126, it is determined whether the two match as a result of the comparison. If they match, the process of comparing the crawler collected data with the manuscript data returns, but if they do not match, control proceeds to S127, where all data of the mismatched parts (data and labels of the mismatched parts) are output as proofreading parts and provided to the client, the advertisement creation company 31. In addition to or instead of the process of comparing the crawler collected data with the manuscript data, a process of comparing the crawler collected data with the data of the leaflet 27 may be performed.

[0094] Next, a flowchart of the subroutine program for the comparison process between the AI ​​data and the manuscript data 24 in S79 will be described with reference to Fig. 13(B). In S120, a process is performed to compare data corresponding to the same label in the AI ​​data and the manuscript data 24. As a result of the comparison, a determination is made in S121 as to whether or not the two match. If they match, control proceeds to S123, but if they do not match, control proceeds to S122, where the data and label of the mismatched part are stored as a proofread part, and then control proceeds to S123.

[0095] In S123, it is determined whether or not comparison of all the AI ​​data has been completed, and if not, control returns to S120. By going around this loop of S120 → S121 → S122 or S123 → S120, if comparison of all the AI ​​data has been completed, a YES determination is made in S123, control moves to S124, and a process is performed in which all the proofread points (data and labels of the mismatched points) stored in S122 are output and provided (transmitted) to the client, advertisement creation company 31.

[0096] Next, a flowchart of the subroutine program for the storage process of price data shown in S81 will be described with reference to Fig. 13(C). In S128, a process is performed to store product data (product name, capacity, price, sales limit, etc.) including the price of each product other than the proofreading part together with the date in the big data DB 36. This makes it possible to read and use the price data 41 stored in the big data DB 36 in, for example, S17 and S36.

[0097] Next, the control process by the user terminal 74 of the advertisement production company 31 will be described with reference to Fig. 14. In S180, it is determined whether or not manuscript data for creating a leaflet is to be transmitted, and if it is not yet to be transmitted, control proceeds to S182. If manuscript data has been created and is to be transmitted, the result of S180 is YES, and the manuscript data is transmitted to the data center 44 in S181.

[0098] Next, in S182, it is determined whether or not to send a flyer, and if not yet, control proceeds to S184, but if it is to be sent, image data of the flyer is sent to the data center 44 in S183. Note that the manuscript data sent in S181 and the image data of the flyer sent in S183 may be sent simultaneously instead of being sent separately. Next, in S184, it is determined whether or not the proofread portion of the flyer has been received, and if not, control returns to S180. On the other hand, if the proofread portion is sent from the server 14 of the data center 44 in S124, YES is determined in S184, and the proofread portion is stored in S185.

[0099] Next, in S186, it is determined whether other data has been received. "Other data" is, for example, the optimized advertisement transmitted in S48 or the price fluctuation prediction data transmitted in S58.

[0100] Modifications of the present embodiment described above are listed below.

[0101] (Variation 1) In the above embodiment, the advertising-related service providing means 35 finds the corrections in the flyer 27 and provides them to the advertising production company 31, but this is not limited to this, and the service may also be one in which an electronic flyer is created based on an advertising request from an advertiser 30 and the electronic flyer is searchable and viewable on the web.

[0102] (Variation 2) In the above embodiment, a flyer 27 was shown as an example of an advertisement, but the advertisement may be a paper medium other than a flyer, such as a pamphlet, or even a non-paper medium, such as a web advertisement.

[0103] (Variation 3) The data stored in the big data DB 36 may be grouped by a predetermined attribute (for example, a three-dimensional table consisting of business type, region, and season (spring, summer, fall, winter)) in the same way as the learned storage unit 29 using training data in the practical stage. Then, the machine learning means 34 may perform machine learning specialized for each group, and each learned model that is the learning result may be grouped and stored by attribute (for example, the above-mentioned three-dimensional table). The learned models 50 specialized for each group may be used to perform price fluctuation predictions, futures trading, and provide optimized advertisements specialized for each group.

[0104] (Variant 4) Although supervised learning has been shown as a machine learning method for predicting price fluctuations (see FIG. 5(A)), instead of or in addition to this, reinforcement learning may be used in which an agent is rewarded depending on how accurate its price fluctuation predictions are, and price fluctuation predictions may also be made by data mining.

[0105] (Variation 5) Deep Q-Network may be adopted as the reinforcement learning for generating optimized advertisements (S29). Deep Q-Network is the application of deep learning technology to function approximation in reinforcement learning. The greatest feature of Q-Learning in reinforcement learning is that if samples (s,a,r,s') can be obtained infinitely many times from all pairs of (s,a), the optimal value function Q will always be obtained regardless of the order in which they are given. * The advantage of this approach is that it can obtain (s,a). Creating a table function Q(s,a) for all states and actions would result in a huge amount of data to be processed, so function approximation is used for Q(s,a). Deep reinforcement learning is the application of deep learning techniques to this function approximation. Therefore, deep reinforcement learning is also a type of reinforcement learning, and the term "reinforcement learning" is a broad concept that includes "deep reinforcement learning".

[0106] (Variant 6) The advertising-related service providing means 35 described above discovered the parts of the flyer 27 that differed from its manuscript data 24 and needed to be proofread, and notified the advertising production company 31 of this. However, it would also be possible to provide the advertising production company 31 with data on a constructed flyer in which the parts of the flyer 27 that differ from its manuscript data 24 have been proofread to have the correct content.

[0107] (Variation 7) In the above-described embodiment, the optimized advertisement was an advertisement for a product price Pi that maximizes the advertising effectiveness, but in addition to or instead of that, it may be an advertisement created by predicting best-selling products.

[0108] The subject matter disclosed in the above-described embodiments will now be described.

[0109] [Theme 1] One service provision system that accumulates price data as big data and provides services using that big data involves having information providers provide market information on fluctuations in various trading prices, and then predicting price fluctuations based on that information (for example, JP 2001-22849 A).

[0110] The technique described in Patent Document 1 has the disadvantage that it is difficult to collect information from information providers, and in order to give information providers an incentive to provide information, the method of having them provide the information for a fee has to be adopted.

[0111] The present subject matter 1 has been devised in view of the above circumstances, and its purpose is to provide a service providing system capable of giving an incentive to information providers for providing information.

[0112] This subject 1 can be expressed, for example, as the following items:

[0113] (Item 1) A service providing system that accumulates at least price data (e.g., product data including the price of each product (product name, volume, price, sales limit, etc.)) as big data (e.g., S128) and provides a service using the big data, A data receiving means (e.g., S65, S67) for receiving at least price data (e.g., product data including the price of each product (product name, capacity, price, sales limit, etc.)); a big data storage means (e.g., big data DB36, S128) that stores the price data (e.g., price data 41) received by the data receiving means as big data; A service providing means (e.g., S48, S55) that provides a service (e.g., price fluctuation prediction, creation and provision of optimized advertisements, futures trading) that reflects the learning results of machine learning means (e.g., machine learning means 34, S9, S10) that performed machine learning based on the big data accumulated by the big data accumulation means; An advertisement-related service providing means (e.g., advertisement-related service providing means 35, S65 to S80) for providing a service related to advertisement creation, The advertisement-related service providing means provides a service (e.g., S65 to S80) relating to the creation of advertisements reflecting the price data received by the data receiving means to a provider of the price data (e.g., advertisement creation company 31), in a service providing system.

[0114] According to this service providing system, information providers are provided with a service relating to the creation of advertisements that reflect price data, and thus receive an incentive to provide information, and therefore try to take the initiative in providing information.

[0115] (Item 2) The big data storage means also stores data on fluctuation factors that cause price fluctuations (for example, weather, natural disasters, stock prices, exchange rates, interest rates, economic trends, income fluctuations, and geopolitical risks as crawler collected data 40) as big data, Further comprising a prediction means (e.g., S55 to S60) for predicting price fluctuations based on the price data and the fluctuation factor data accumulated by the big data accumulation means, The prediction means predicts the price fluctuation by reflecting a learning result obtained by the machine learning means performing machine learning (e.g., S15 to S20) based on the price data and the fluctuation factor data (e.g., S57); 2. The service providing system according to item 1, wherein the service providing means provides a service reflecting the price fluctuation prediction by the prediction means (e.g., price fluctuation prediction, creation and provision of optimized advertisements).

[0116] According to this service providing system, information providers can receive services that reflect forecasts of price fluctuations.

[0117] (Item 3) The machine learning means performs machine learning (e.g., S25 to S31, S36 to S38, S46, S47) for creating an optimized advertisement based on the big data stored by the big data storage means, and creates an advertisement by reflecting the learning result (e.g., S48). 3. The service providing system according to item 1 or 2, wherein the service providing means provides the advertisement created by the advertisement creating means (eg, S48).

[0118] According to this service providing system, it is possible to obtain advertisements that are created by reflecting the results of machine learning for creating optimized advertisements.

[0119] (Item 4) The system further includes an advertisement effectiveness measurement means (e.g., advertisement effectiveness measurement means 38, S1 to S8) for measuring the effectiveness of the advertisement created by the advertisement creation means, The machine learning for creating the optimized advertisement is a reinforcement learning in which an agent is given a reward according to the measurement results by the advertising effectiveness measurement means, and the agent learns a strategy for maximizing the accumulation of the reward (e.g., S46, S47).

[0120] According to this service providing system, it is possible to provide advertisements with high advertising effectiveness as a result of reinforcement learning.

[0121] (Item 5) The system further includes an advertisement effectiveness measurement means (e.g., advertisement effectiveness measurement means 38, S1 to S8) for measuring the effectiveness of the advertisement created by the advertisement creation means, The machine learning for creating the optimized advertisement is a service provision system described in item 3, which generates training data (e.g., {(xji, yji)}) using as input information the price predicted by a prediction means based on the price data accumulated by the big data accumulation means and fluctuation factor data that is a cause of price fluctuation, and using as correct answer information advertising content whose effectiveness is measured to be at least a predetermined level by the advertising effectiveness measurement means (e.g., S37), performs supervised learning using the training data, and learns a function (e.g., cj(x) (cj:x→y)) that maps an input to a correct answer as a regression problem (e.g., S36 to S38).

[0122] According to this service providing system, it is possible to provide advertisements with high advertising effectiveness as a result of supervised learning.

[0123] (Item 6) A user-side facility (e.g., a user terminal 74 connected to an internal LAN 84 of an advertiser 30, a user terminal 74 connected to an internal LAN 84 of an advertisement creation company 31) that can be connected to the Internet to a service providing system (e.g., a data center 44) that accumulates at least price data (e.g., product data including the price of each product (product name, volume, price, sales limit, etc.)) as big data (e.g., S128) and provides a service using the big data, A data providing means (e.g., S181, S183) for providing at least price data (e.g., product data including the price of each product (product name, volume, price, sales limit, etc.)) to the service providing system; a service receiving means (e.g., S155, S187) for receiving, from the service providing system, a service (e.g., optimized advertisement, price fluctuation prediction) reflecting a learning result obtained by a machine learning means based on the accumulated big data, in which the price data provided by the data providing means is accumulated as big data by a big data accumulation means provided on the service providing system side; and an advertisement-related service receiving means (e.g., S184) for receiving a service related to advertisement creation from an advertisement-related service providing means provided on the service providing system side; The advertisement-related service receiving means receives services (eg, S65 to S80) relating to the creation of advertisements that reflect the price data provided by the data providing means.

[0124] According to this service provision system, price data providers can receive services relating to the creation of advertisements that reflect the price data provided by the data provision means, so that they are given an incentive to provide price data and tend to take the initiative in providing price data.

[0125] [Theme 2] One advertising-related service providing system that identifies and notifies parts of an advertisement that need proofreading is one that compares advertisements such as flyers with their manuscript data and finds parts that do not conform to the manuscript (for example, JP 2004-133734 A).

[0126] However, this patent has the drawback of requiring manual work to match the advertisement with its manuscript data.

[0127] The present subject matter 2 has been devised in view of the above-mentioned circumstances, and its purpose is to provide an advertising-related service providing system that can reduce, as much as possible, the manual work required for matching advertisements with their manuscript data.

[0128] This topic 2 can be expressed, for example, as the following items:

[0129] (Item 1) An advertisement-related service providing system that provides a service related to advertisement creation, An advertisement data receiving means (e.g., S65) for receiving data of advertisements (e.g., flyers 27, pamphlets, web advertisements, etc.); manuscript data receiving means (e.g., S67) for receiving manuscript data (e.g., manuscript data 2) used in creating the advertisement; a proofreading portion determining means (e.g., S68 to S79) for determining consistency between the advertisement and the manuscript data based on the advertisement data received by the advertisement data receiving means and the manuscript data received by the manuscript data receiving means, and determining the portions of the advertisement that need proofreading; The proofreading part determination means determines the consistency by reflecting the learning results (e.g., learned flyer feature 1, advertising price set boundary learned model 2, learned product feature 20, trained model 18 for product image recognition, character position learned model 19, OCR trained model 17, word classification trained model 32) performed by a proofreading machine learning means (e.g., proofreading machine learning means 33) in machine learning for determining the consistency between the advertisement and the manuscript data (e.g., flyer feature learning 23, advertising price set boundary learning 3, product feature learning 22, product image recognition learning 4, character position learned model 28, OCR trained model 17, word classification trained model 32) (e.g., S87, S88, S95, S96, S98, S99, S104, S105, S108, S116, S117, S111, S112, S116, S117), advertising-related service providing system.

[0130] According to this advertising-related service provision system, the consistency between an advertisement and manuscript data is determined by reflecting the learning results obtained by the proofreading machine learning means performing machine learning to determine the consistency between the advertisement and manuscript data, thereby minimizing the manual work of matching the advertisement with the manuscript data.

[0131] (Item 2) The advertisement data includes data of a plurality of advertisement products (for example, value sets 99a, 99b, 99c, and 99d shown in FIG. 10(B)), The manuscript data includes data specifying the advertisement contents for each of the multiple products to be advertised (for example, the product name, manufacturer, price, etc., as shown in FIG. 10(E)), The machine learning performed by the calibration machine learning means includes machine learning for classifying and identifying each value set (e.g., leaflet feature learning 23, advertising value set boundary learning 3) for classifying and identifying the advertising data received by the advertising data receiving means for each single value set, The calibration part determination means A value group classification identification process (e.g., S87 to S89) that reflects the learning result by the value group classification identification machine learning and classifies and identifies the advertisement data for each single value group; a manuscript corresponding portion identification process (e.g., S95, S96, S98, S99, S104, S105, S108) for identifying which portion of the advertisement content in the manuscript data corresponds to each of the single value sets identified by the value set classification identification process; The advertising-related service providing system described in item 1 performs a consistency determination process (e.g., S111 to S114, S116, S117, S120 to S123) that matches each of the single value sets identified by the value set-by-value set classification identification process with the advertising content identified by the manuscript corresponding portion identification process to determine the consistency.

[0132] According to this advertising-related service provision system, it is determined which part of the advertising content in the manuscript data each single-value pair corresponds to, and the determined advertising content is matched with each single-value pair to determine consistency, thereby enabling an accurate determination of consistency.

[0133] (Item 3) The advertising-related service providing system described in item 2, wherein the consistency determination process determines the consistency between the character image of each single value set identified by the value set classification identification process and the manuscript data (e.g., S111 to S114, S116, S117, S120 to S123).

[0134] According to this advertising-related service providing system, it is possible to determine whether or not a character image of a single value set matches manuscript data.

[0135] (Item 4) The machine learning performed by the proofreading machine learning means includes text data conversion machine learning (e.g., character position learning 28, OCR learning 16) for converting character images of each single value set classified and identified by the value set classification identification process into text data, The advertising-related service providing system described in item 3, wherein the consistency determination process converts the character image into text data by reflecting the learning results of the text data conversion machine learning, and then determines the consistency between the text data and the manuscript data (e.g., S111 to S114, S116, S117, S120 to S123).

[0136] According to this system for providing advertising-related services, after converting character images into text data, the consistency between the text data and manuscript data is judged, making it possible to judge good consistency between the two data.

[0137] (Item 5) The machine learning performed by the calibration machine learning means further includes product feature learning (e.g., product feature learning 22) for learning the features of the product based on data of an original image of an actual product corresponding to the advertised product, The system further includes a correspondence storage unit (e.g., a learned product feature amount 20 in the learning stage learned storage unit 21) that stores the product feature learned by the product feature learning in association with identification information for identifying the product (e.g., a label such as a JAN code or a house code), The manuscript data is data in which the advertising content (e.g., product name, manufacturer, price, etc.) of each of the multiple products to be advertised is associated with the identification information (e.g., label) of the product (e.g., FIG. 10(E)), The manuscript corresponding portion identification process identifies identification information (e.g., a label) that is associated with the product features used to identify the advertised product and stored in the correspondence storage means, and identifies the advertising content (e.g., product name, manufacturer, price, etc.) that is associated with the identification information from the manuscript data (e.g., manuscript data 24) (e.g., S120), in an advertising-related service providing system described in any one of items 2 to 4.

[0138] According to this advertising-related service providing system, the advertisement content associated with the identification information for identifying a product is identified from the manuscript data and consistency is determined, so that the manuscript data can be accurately identified.

[0139] (Item 6) The calibration machine learning means performs machine learning specialized for each group (e.g., S129 to S135) by performing machine learning based on data grouped by a predetermined attribute (e.g., business type, region, season (spring, summer, fall, winter)); The proofreading part determination means determines the consistency by reflecting the results of machine learning specialized for each group by the proofreading machine learning means (e.g., S84, S86).

[0140] According to this advertising-related service provision system, consistency determination processing is performed by reflecting the results of machine learning specialized for each group grouped according to specified attributes, making it possible to utilize the results of machine learning that match the attributes of the group in the consistency determination processing.

[0141] (Item 7) A user-side advertising facility (e.g., a user terminal 74 connected to an in-house LAN 84 of an advertisement creation company 31) that can be connected to an advertisement-related service providing system (e.g., a data center 44) that provides a service related to advertisement creation, An advertisement providing means (e.g., S183) for providing an advertisement to the advertisement-related service providing system; A manuscript data providing means (e.g., S181) for providing manuscript data used in creating the advertisement to the advertisement-related service providing system; a proofreading portion determining means provided on the advertisement-related service providing system side determines the consistency between the advertisement and the manuscript data based on the advertisement data provided by the advertisement providing means and the manuscript data provided by the manuscript data providing means, and determines the portions of the advertisement that need proofreading, and receives a service based on the determination result from the advertisement-related service providing system side (e.g., S184); The proofreading part determination means determines the consistency by reflecting the learning results (e.g., learned flyer features 1, advertising value set boundary trained model 2, trained product features 20, trained model 18 for product image recognition, character position trained model 19, OCR trained model 17, word classification trained model 32) performed by the proofreading machine learning means to determine the consistency between the advertisement and the manuscript data (e.g., S87, S88, S95, S96, S98, S99, S104, S105, S108, S116, S117, S111, S112, S116, S117).

[0142] With this user-side advertising equipment, the learning results obtained by the proofreading machine learning means are reflected in the determination of the consistency between the advertisement and manuscript data, and the parts of the advertisement that need to be proofread are identified, and the user can receive services based on the results of this determination, thereby enabling the user to receive services that reduce labor costs by eliminating human work as much as possible.

[0143] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0144] 1 Flyer feature trained model, 2 Advertising price set boundary trained model, 3 Advertising price set boundary learning, 4 Training for product image recognition, 14 Server, 16 OCR training, 17 OCR trained model, 18 Trained model for product image recognition, 19 Character position trained model, 20 Trained product features, 22 Product feature training, 23 Flyer feature training, 24 Manuscript data, 27 Flyer, 28 Character position training, 29 Trained storage unit using training data in the practical phase, 32 Word classification trained model, 33 Machine learning for advertising-related services, 34 Machine learning means, 35 Advertising-related service provision means, 50 Trained model.

Claims

1. A service providing system for providing a service using big data, A big data storage means for storing price data and fluctuation factor data that is a cause of price fluctuation as big data; a service providing means for predicting price fluctuations using an AI model that has been machine-learned by a machine learning means based on the price data and the fluctuation factor data stored by the big data storage means, and providing a service that reflects the prediction; The service providing system includes an advertisement creating means for creating an advertisement that reflects the prediction, and provides the advertisement created by the advertisement creating means.

2. A service providing system as described in claim 1, further comprising an advertising effectiveness measuring means for measuring the effectiveness of the advertisement created by the advertising creation means.

3. The service provision system described in claim 2, wherein the advertisement creation means generates training data using the predicted price as input information and advertising content whose effectiveness has been measured to be at or above a predetermined level by the advertising effectiveness measurement means as correct answer information, performs supervised learning using the training data, and creates advertisements using an AI model that has performed supervised learning to learn a function that maps an input to a correct answer as a regression problem.

4. The service provision system described in claim 2, wherein the advertisement creation means creates advertisements using an AI model that has undergone reinforcement learning in which an agent is given a reward according to the measurement results by the advertising effectiveness measurement means, and the agent learns a strategy for maximizing the accumulation of the reward.

Images