Relevance estimation system, relevance estimation method, and program

The relevance estimation system addresses the challenge of estimating conversion query relevance to services, enhancing the effectiveness of search-linked advertising by accurately formatting and registering conversion queries.

JP7591536B2Active Publication Date: 2024-11-28RAKUTEN GROUP INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2022104460
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-06-29
Publication Date
2024-11-28
Estimated Expiration
2042-06-29

AI Technical Summary

Technical Problem

Existing technologies fail to estimate the relevance between conversion queries and the services where conversions occur, leading to noise and reduced effectiveness in search-linked advertising.

Method used

A relevance estimation system that includes an acquisition unit, a learning unit, and a processing execution unit to estimate the relevance between conversion queries and services using a model, allowing for appropriate formatting and registration of conversion queries.

Benefits of technology

Enables accurate estimation of the relationship between conversion queries and services, improving the relevance of search-linked advertisements and reducing reputational risk by ensuring appropriate display and registration of conversion queries.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591536000001
    Figure 0007591536000001
  • Figure 0007591536000002
    Figure 0007591536000002
  • Figure 0007591536000003
    Figure 0007591536000003
Patent Text Reader

Abstract

To estimate the relation between a conversion query and a service with generation of a conversion.SOLUTION: An acquisition unit (201) of a relation estimation unit (1) acquires a conversion query in which a search linkage advertisement has caused a conversion in a predetermined service. A relation estimation unit (205) estimates the relation between a conversion query and a predetermined service on the basis of a learning model which has learned a training query related to the predetermined service. A processing execution unit (204) executes predetermined processing on the basis of the relation.SELECTED DRAWING: Figure 11
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a relevance estimation system, a relevance estimation method, and a program. [Background technology]

[0002] Conventionally, search-based advertising is known that displays search results corresponding to a query entered by a user, together with advertisements corresponding to the query. A conversion query that has generated a conversion in a search-based advertising is very useful information since it can be used for various purposes. For example, the conversion query can be used as an index or display keyword in the search-based advertising, or can be used in marketing to analyze user trends.

[0003] Some conversion queries have low relevance to the service in which the conversion occurred. Such conversion queries are not useful information, and may become noise instead. For example, Patent Document 1 describes a technology for extracting keywords that enhance the appeal of content by calculating an evaluation value as a keyword associated with content. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-62275 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 can extract keywords that enhance the appeal of content, but cannot estimate the relevance between the keywords and the service. Furthermore, the technology of Patent Document 1 is only intended to extract keywords from content, and does not assume any consideration of conversion queries that have caused conversions in search-linked advertising. Conventional technologies have not been able to estimate the relevance between a conversion query and a service that has caused a conversion.

[0006] One objective of this disclosure is to estimate the relevance of a conversion query to the service where the conversion occurred. [Means for solving the problem]

[0007] The relevance estimation system according to the present disclosure includes an acquisition unit that acquires conversion queries in which conversions have occurred in a specified service due to search-related advertising, a relevance estimation unit that estimates a relevance between the conversion queries and the specified service based on a learning model in which training queries that are relevant to the specified service have been learned, and a processing execution unit that executes a specified process based on the relevance. Effect of the Invention

[0008] According to the present disclosure, it is possible to estimate the relevance between a conversion query and a service in which a conversion occurred. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of an overall configuration of a query shaping system. [Diagram 2] FIG. 13 is a diagram illustrating an example of a process in which a search-related advertisement is displayed on a search service and a conversion occurs on an EC service. [Diagram 3] FIG. 13 is a diagram illustrating an example of a flow for shaping a conversion query. [Figure 4]FIG. 2 is a diagram illustrating an example of functions realized in the first embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of a product database. [Figure 6] FIG. 13 is a diagram illustrating an example of a conversion database. [Figure 7] FIG. 1 illustrates an example of a method for training a query shaping model. [Figure 8] FIG. 2 is a diagram illustrating an example of an advertisement database. [Figure 9] FIG. 4 is a flow chart showing an example of processing executed in the first embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of a relevance estimation model in the second embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of functions realized in the second embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of processing executed in the second embodiment. [Figure 13] FIG. 11 is a diagram illustrating an example of functions in a modified example of the first embodiment. [Figure 14] FIG. 13 is a diagram showing an example of query shaping in Modification Example 1-2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [1. First embodiment] A first embodiment, which is an example of an embodiment according to the present disclosure, will be described. In the first embodiment, a case where an operator of an EC (Electronic Commerce) service uses a query shaping system will be described as an example. The query shaping system can also be applied to services other than the EC service. Examples of application to other services will be described in modified examples described later.

[0011] [1-1. Overall configuration of the query reformatting system] 1 is a diagram showing an example of the overall configuration of a query shaping system. For example, the query shaping system 1 includes an EC server 10 and an advertiser terminal 20. Outside the query shaping system 1, there exist a search server 30 and a user terminal 40. Each of the EC server 10, the advertiser terminal 20, the search server 30, and the user terminal 40 can be connected to a network N such as the Internet or a LAN.

[0012] The EC server 10 is a server computer in an EC service. For example, the EC server 10 includes a control unit 11, a storage unit 12, and a communication unit 13. The control unit 11 includes at least one processor. The storage unit 12 includes a volatile memory such as a RAM, and a non-volatile memory such as a flash memory. The communication unit 13 includes at least one of a communication interface for wired communication and a communication interface for wireless communication.

[0013] The advertiser terminal 20 is a computer of an advertiser that posts search-linked advertisements in a search service. In the first embodiment, a case will be described in which the operator of an EC service is the advertiser, but the advertiser may be another party such as a store that opens a shop in the EC service or a product manufacturer. For example, the advertiser terminal 20 is a personal computer, a smartphone, a tablet terminal, or a wearable terminal.

[0014] For example, the advertiser terminal 20 includes a control unit 21, a memory unit 22, a communication unit 23, an operation unit 24, and a display unit 25. The physical configurations of the control unit 21, the memory unit 22, and the communication unit 23 are similar to those of the control unit 11, the memory unit 12, and the communication unit 13, respectively. The operation unit 24 is an input device such as a touch panel or a mouse. The display unit 25 is a liquid crystal display or an organic EL display.

[0015] The search server 30 is a server computer in a search service. In the first embodiment, a search service for web pages is taken as an example, but the search service may be a service for searching other search targets such as content, products, services, or documents. For example, the search server 30 includes a control unit 31, a storage unit 32, and a communication unit 33. The physical configurations of the control unit 31, the storage unit 32, and the communication unit 33 are similar to those of the control unit 11, the storage unit 12, and the communication unit 13, respectively.

[0016] The user terminal 40 is a user's computer. For example, the user terminal 40 is a personal computer, a smartphone, a tablet terminal, or a wearable terminal. For example, the user terminal 40 includes a control unit 41, a storage unit 42, a communication unit 43, an operation unit 44, and a display unit 45. The physical configurations of the control unit 41, the storage unit 42, the communication unit 43, the operation unit 44, and the display unit 45 are similar to those of the control unit 11, the storage unit 12, the communication unit 13, the operation unit 24, and the display unit 25, respectively.

[0017] The programs stored in the storage units 12, 22, 32, and 42 may be supplied via the network N. Each computer may include at least one of a reading unit (e.g., a memory card slot) that reads a computer-readable information storage medium and an input / output unit (e.g., a USB port) for inputting and outputting data to and from an external device. For example, a program stored in an information storage medium may be supplied via at least one of the reading unit and the input / output unit.

[0018] Furthermore, the query shaping system 1 is not limited to the example of FIG. 1 as long as it includes at least one computer. For example, the query shaping system 1 may include only the advertiser terminal 20 without including the EC server 10. In this case, the EC server 10 exists outside the query shaping system 1. For example, the query shaping system 1 may include only the EC server 10 without including the advertiser terminal 20. In this case, the advertiser terminal 20 exists outside the query shaping system 1, and the EC server 10 has the same functions as the advertiser terminal 20.

[0019] [1-2. Overview of the Query Formatting System] In the first embodiment, an operator of an EC service requests a search service to place a search-related advertisement. A search-related advertisement is an advertisement displayed together with search results in a search service. A search-related advertisement is an advertisement related to a query. A query is a search condition entered by a user. A query is sometimes called a keyword. Various types of search engines, queries, and search-related advertisements themselves used in a search service are available. A query can be entered in any language, such as Japanese, Chinese, or English.

[0020] FIG. 2 is a diagram showing an example of a flow in which a search-based advertisement is displayed on a search service and a conversion occurs on an EC service. A conversion is a result of a search-based advertisement. In the first embodiment, a case will be described in which a conversion corresponds to the purchase of a product, but a conversion may be a state in which a target action is taken, and is not limited to the purchase of a product. For example, a conversion may be selecting a search-based advertisement, displaying a specific web page, downloading specific data, or adding a product to a shopping cart.

[0021] For example, when the user terminal 40 accesses the search server 30, a top screen SC1 of the search service is displayed on the display unit 45. The user inputs a query into an input form F10 on the top screen SC1. When the user selects a button B11, the search server 30 executes a search for web pages and a search-related advertisement based on the query entered by the user. When the user terminal 40 receives the results of these executions from the search server 30, a search result screen SC2 is displayed on the display unit 45.

[0022] For example, the search result screen SC2 includes a display area A20 for displaying search-related advertisements and a display area A21 for displaying search results. If there are no search-related advertisements that match the query, the search-related advertisements are not displayed in the display area A20. The search-related advertisements include a link to an EC service. The link includes, as arguments, the query and an advertisement ID that can identify the search-related advertisement. The link arguments enable the operator of the EC service to identify which search-related advertisement was selected by which query.

[0023] In the first embodiment, it is assumed that a search service performs morphological analysis on a query input by a user. In the search service, the query input by the user is divided into morphemes, unnecessary words are removed, and then a search is performed. The operator of the EC service is different from the operator of the search service. For this reason, it is assumed that the operator of the EC service cannot grasp what kind of morphological analysis is performed on the search service side.

[0024] For example, a query divided into morphemes and a link to an EC service are displayed in the display area A20. The query included in the argument of the link is the query divided into morphemes. The input form F22 displays the query before being divided into morphemes. However, internally in the search service, a search may be performed based on the query displayed in the display area A20, not on the input form F22. The user can also perform a search by inputting a query again into the input form F22.

[0025] For example, when a user selects a search-based advertisement in the display area A20, the user terminal 40 accesses the EC server 10 and a product screen SC3 is displayed on the display unit 45. The product screen SC3 is a web page linked to by the search-based advertisement. When a user adds a product to a shopping cart in the EC service and completes the ordering procedure, an order completion screen SC4 is displayed on the display unit 45. The product ordering procedure itself may be the same as that of known EC services.

[0026] In the first embodiment, a conversion occurs when a user selects a search-based advertisement, accesses an EC server, and completes an order for a product. Hereinafter, a query that has caused a conversion is referred to as a conversion query. In the display area A20, the conversion query is displayed in a state in which it is divided into morphemes, which may cause a user to feel uncomfortable. Furthermore, the conversion query displayed in the display area A20 does not look good, which may reduce the appeal of the search-based advertisement.

[0027] For example, it is conceivable that the search service side displays the conversion query before being divided into morphemes (the conversion query in the same state as the input form F22) in the display area A20. However, even if the operator of the EC service requests such a modification, there is a possibility that the operator of the search service will not comply with the modification. Even if the operator of the search service agrees to the modification, the burden will be on the operator of the search service.

[0028] In this regard, if the operator of the EC service can obtain the conversion query before it is divided into morphemes, the operator can register the conversion query in a database dedicated to advertisers in the search service to appropriately display the display area A20. However, since the conversion query after it is divided into morphemes is included in the link argument, the operator of the EC service cannot obtain the conversion query in an appropriate state. Therefore, the query shaping system 1 shapes the conversion query divided into morphemes so that it is returned to its original state.

[0029] FIG. 3 is a diagram showing an example of a flow for shaping a conversion query. In the first embodiment, a unit of words included in a conversion query is called a token. A token is a word divided by a separator such as a space or a symbol. In the example of FIG. 3, a conversion query Q1 input by a user includes tokens T10 to T12. Separators exist between tokens T10 and T11 and between tokens T11 and T12.

[0030] For example, the conversion query Q2 after being divided into morphemes includes tokens T20 to T27. Separators exist between tokens T20 and T21, between tokens T21 and T22, between tokens T22 and T23, between tokens T23 and T24, between tokens T24 and T25, between tokens T25 and T26, and between tokens T26 and T27. The operator of the EC service obtains the conversion query Q2.

[0031] For example, detokenization is performed on the conversion query Q2. Detokenization is the process of deleting all or some of the separators. In the example of FIG. 3, all the separators in the conversion query Q2 are deleted. As a result, the tokens T20 to T27 are all combined. The detokenized conversion query Q3 includes only one token T30.

[0032] In the first embodiment, a query shaping model M1 using a machine learning technique is prepared. The query shaping model M1 is trained to perform appropriate tokenization on the detokenized conversion query Q3. Tokenization is the insertion of at least one separator. In the example of FIG. 3, the conversion query Q4 shaped by the query shaping model M1 is the same as the conversion query Q1.

[0033] As described above, the query shaping system 1 performs detokenization on the conversion query Q2 after being divided into morphemes, and then performs tokenization using the query shaping model M1 to obtain the conversion query Q1. This makes it possible to appropriately shape the conversion query in search-linked advertising. Details of the query shaping system 1 will be described below. Note that, when it is not necessary to refer to FIG. 3, the reference symbols of the conversion queries Q1 to Q4 will be omitted.

[0034] [1-3. Functions realized in the first embodiment] FIG. 4 is a diagram illustrating an example of functions realized in the first embodiment.

[0035] [1-3-1. Functions realized by the EC server] For example, the EC server 10 includes a data storage unit 100 and an order processing unit 101. The data storage unit 100 is realized by the storage unit 12. The order processing unit 101 is realized by the control unit 11.

[0036] [Data storage section] The data storage unit 100 stores data necessary for providing EC services. For example, the data storage unit 100 stores a product database DB1 and a conversion database DB2.

[0037] Fig. 5 is a diagram showing an example of product database DB1. Product database DB1 is a database in which various information related to products in an EC service is stored. For example, product database DB1 stores a store ID, product ID, product title, product attribute, product price, detailed product description, and search keywords. Product database DB1 may store any information related to products, and may also store information other than that shown in Fig. 5.

[0038] The store ID is an example of store identification information capable of identifying a store in the EC service. The store identification information may be other information such as a store name, a phone number, or an email address. The product ID is an example of product identification information capable of identifying a product. The product identification information may be other information such as a product name or a JAN code. The title, attributes, price, detailed description, and keywords are used as indexes for searches in the EC service. A user can search for products by entering a query within the EC service, rather than within the search service.

[0039] Fig. 6 is a diagram showing an example of the conversion database DB2. The conversion database DB2 is a database in which various information related to conversions occurring in an EC service is stored. For example, the conversion query database stores an advertisement ID, a product ID, a user ID, a conversion query divided into morphemes, and the date and time of the conversion. The conversion database DB2 may store any information related to the conversion, and may also store information other than that shown in Fig. 6.

[0040] The advertisement ID is an example of advertisement identification information capable of identifying a search-based advertisement. The advertisement identification information may be any information capable of identifying a search-based advertisement in some way, and may be information called by a name other than the advertisement ID. The conversion database DB2 stores the advertisement ID of the search-based advertisement that caused the conversion. The advertisement ID is included in an argument of the link of the search-based advertisement. The EC server 10 can identify which search-based advertisement was selected by acquiring the advertisement ID included in the argument.

[0041] For example, the conversion database DB2 stores the product ID of the product ordered by the conversion, the user ID of the user who ordered the product, the conversion query included in the argument of the link of the search-linked advertisement, and the current date and time when the conversion occurred. In the first embodiment, every time a conversion occurs, the order processing unit 101 described later creates a new record in the conversion database DB2 and stores various information related to the conversion.

[0042] The data stored in the data storage unit 100 is not limited to the above examples. For example, the data storage unit 100 may store a user database that stores various information related to users who have registered as members of an EC service. The user database stores information such as user IDs, passwords, names, addresses, telephone numbers, email addresses, product viewing histories, and product purchase histories.

[0043] [Order Processing Unit] The order processing unit 101 executes order processing for accepting product orders from users. The order processing itself can utilize processing of known EC services. For example, when a user places a product in a shopping cart and completes the ordering procedure, the order processing unit 101 executes a settlement process required for settling the product and a shipping process required for shipping the product. The settlement process is a process for making a payment using a payment method such as a credit card designated by the user. The shipping process is a process for sending information about the product ordered by the user to the store. As described above, when a conversion occurs, the order processing unit 101 updates the conversion database DB2.

[0044] [1-3-2. Functions realized by the advertiser terminal] For example, the advertiser terminal 20 includes a data storage unit 200, an acquisition unit 201, a detokenization execution unit 202, a query shaping unit 203, and a processing execution unit 204. The data storage unit 200 is realized by the storage unit 12. The acquisition unit 201, the detokenization execution unit 202, the query shaping unit 203, and the processing execution unit 204 are realized by the control unit 11.

[0045] [Data storage section] The data storage unit 200 stores data necessary for query shaping. For example, the data storage unit 200 stores a trained query shaping model M1. The query shaping model M1 includes a program portion that executes a query shaping process and a parameter portion that is adjusted by learning. For example, the parameters are weights and biases. The parameters may be any parameters that correspond to a machine learning method used as the query shaping model M1.

[0046] In the first embodiment, a query shaping model M1 using Seq2Seq (Sequence to Sequence) is taken as an example. Seq2Seq includes an encoder to which input data to be processed is input, and a decoder to acquire features output from the encoder and output output data according to the features. The features are information on the features of the input data. In the first embodiment, a case where the features are expressed by a multidimensional vector will be described, but the features may be expressed in other formats such as an array or a single numerical value.

[0047] 7 is a diagram showing an example of a learning method of the query shaping model M1. For example, training data of the query shaping model M1 is a pair of a training query corresponding to input data and a training query corresponding to output data. The training query corresponding to the input data is the training query before shaping. The training query corresponding to the output data is the training query that has been shaped. The pairs included in the training data may be prepared depending on the type of shaping to be performed by the query shaping model M1.

[0048] In this embodiment, the detokenized conversion query is input to the query shaping model M1, and the tokenized conversion query is output from the query shaping model M1, so that the pairs included in the training data correspond to these pairs. In the example of Fig. 7, the training query corresponding to the input data is a training query that includes multiple tokens and is detokenized to include only one token. The training query corresponding to the output data is a training query that includes the multiple tokens.

[0049] For example, the advertiser terminal 20 adjusts parameters of the query shaping model M1 so that when a training query corresponding to the input data is input, a training query corresponding to the training output data is output, thereby making the query shaping model M1 learn the training data. The learning itself can use various algorithms used in machine learning techniques, and for example, the backpropagation method or the gradient descent method may be used. The learning of the query shaping model M1 may be performed by another computer instead of the advertiser terminal 20.

[0050] In the first embodiment, a query actually input in an EC service is used as a training query. This query does not have to be a conversion query. For example, the advertiser terminal 20 acquires a query actually input in an EC service from the EC server 10. The advertiser terminal 20 generates training data based on this query. If this query contains a separator in an appropriate position, a pair of the query after detokenization and this query becomes the training data. If this query does not contain a separator in an appropriate position, the operator of the EC service may manually reshape it as appropriate.

[0051] Note that a query actually input in a search service, rather than an EC service, may be used as the training query. This query may not be a conversion query. For example, the advertiser terminal 20 acquires a query actually input in a search service from the search server 30. The advertiser terminal 20 may generate training data based on this query. The data storage unit 200 may also store other data. For example, the data storage unit 200 may store a conversion database DB2.

[0052] In addition, the query shaping model M1 can use various machine learning methods other than Seq2Seq. The query shaping model M1 may be a model of supervised learning, semi-supervised learning, or unsupervised learning. For example, it may be a model using a neural network or a model called Transformer. The query shaping model M1 may be a model of a machine learning method. In addition, the query shaping model M1 may be a Seq2Seq model that uses an RNN such as LSTM or GRU as an encoder and decoder.

[0053] [Acquisition Department] The acquisition unit 201 acquires a conversion query in which a conversion related to a search-related advertisement has occurred. In the first embodiment, since the conversion query is stored in the conversion database DB2, the acquisition unit 201 acquires the conversion query to be shaped from the conversion database DB2. When the conversion query is stored in a database other than the conversion database DB2 or in a computer other than the EC server 10, the acquisition unit 201 acquires the conversion query from the other database or computer.

[0054] In the first embodiment, the search-related advertisement is an advertisement for making a user access an EC service different from the search service in which the search-related advertisement is displayed. For example, the acquisition unit 201 acquires a conversion query in which a conversion occurs in the EC service. The EC service is an example of the other service. The other service may be any service other than a search service and is not limited to an EC service. For example, the other service may be a travel reservation service, an online flea market service, a video distribution service, a payment service, a financial service, or a communication service.

[0055] In the search service of the first embodiment, the conversion query input at the time of search is changed, so the acquisition unit 201 acquires the changed conversion query changed in the search service. Changing the conversion query means adding a separator, a letter, a number, or other symbol to the conversion query, or deleting a separator, a letter, a number, or other symbol from the conversion query. Moving some words in the conversion query may correspond to changing the conversion query.

[0056] In the first embodiment, since the search service performs morphological analysis, the conversion query is divided into morphemes and a separator is added, which corresponds to the change in the conversion query. Therefore, the description of the morphological analysis can be read as any other change. Various methods can be used for the morphological analysis itself. For example, a morphological analysis tool such as MeCab, JUMAN, or JANOME may be used. The morphological analysis tool is not limited to Japanese, and may be a tool compatible with other languages ​​such as Chinese or English.

[0057] [Detokenization Execution Department] The detokenization execution unit 202 executes detokenization on the conversion query. For example, the detokenization execution unit 202 executes detokenization by deleting at least a part of the separators included in the changed conversion query. In the first embodiment, the detokenization execution unit 202 executes detokenization by deleting all the separators included in the changed conversion query. However, it is also possible to delete only a part of the separators included in the changed conversion query. In the example of FIG. 3, the detokenization results in a state in which only one token remains.

[0058] [Query Formatter] The query shaping unit 203 shapes the conversion query based on the query shaping model M1. The query shaping model M1 is an example of a learning model in which training data related to query shaping is learned. Therefore, the description of the query shaping model M1 can be read as the learning model. The training data may be created according to a machine learning technique used as the learning model.

[0059] For example, the query shaping unit 203 shapes the modified conversion query so that the modified conversion query returns to a state at the time of input. The state at the time of input is a state before modification by the search service. The query shaping unit 203 inputs the modified conversion query to the trained query shaping model M1. The query shaping model M1 calculates a feature amount of the modified conversion query, and outputs a conversion query according to the calculated feature amount.

[0060] For example, the output conversion query may be the same or substantially the same as the conversion query entered in the search service. The substantially same content means that the tokens and separators differ by about 1 to 3 characters. Also, for example, the output conversion query may be natural content with unnecessary separators that were included in the unnatural conversion query entered in the search service removed, or natural content with necessary separators that were not included in the unnatural conversion query entered in the search service added.

[0061] In the search service of the first embodiment, the conversion query input during the search is modified by executing morphological analysis, so the query reforming unit 203 reforms the modified conversion query so that the morphemes included in the modified conversion query are restored to the state at the time of input. For example, the query reforming unit 203 reforms the modified conversion query so that a token divided into multiple morphemes by the morphological analysis is returned as one token. In other words, the query reforming unit 203 reforms the modified conversion query so as to delete the separator inserted by the morphological analysis.

[0062] In the first embodiment, since the training query is a query actually input in the EC service, the query shaping unit 203 shapes the conversion query based on the query shaping model M1 in which the relationship between the training query before the change and the changed training query, which is obtained by changing the training query actually input in the EC service, is learned as training data. The learning method for making the query shaping model M1 learn the query actually input in the EC service as the training query is as described above.

[0063] In the first embodiment, the query shaping unit 203 performs tokenization on the conversion query on which detokenization has been performed based on the query shaping model M1, thereby shaping the executed conversion query. For example, the query shaping unit 203 performs tokenization on the conversion query that has been detokenized into one token by inserting a separator at an appropriate position. The appropriate position is determined by the query shaping model M1.

[0064] [Processing execution section] The processing execution unit 204 executes a predetermined process based on the reformulated conversion query reformed by the query reforming unit 203. The predetermined process may be a process according to the use of the reformulated conversion query. For example, the predetermined process may be a recording process for recording the reformulated conversion query in the data storage unit 200, or an output process for outputting the reformulated conversion query to the outside.

[0065] In the first embodiment, the process execution unit 204 executes, as the predetermined process, a registration process for registering the reformulated conversion query in a search service. For example, like an advertisement database DB3 described later, an index and a display keyword are registered in the search service for each search-related advertisement. The search service searches for the search-related advertisement based on the index, and displays the display keyword together with the search-related advertisement.

[0066] For example, the process execution unit 204 executes, as the registration process, a process of registering the reformulated conversion query in a search service as a display keyword. The process execution unit 204 executes the registration process by transmitting the reformulated conversion query to the search server 30. Upon receiving the reformulated conversion query, the search server 30 registers the received reformulated conversion query in the advertisement database DB3 as a display keyword in association with the operator of the EC service.

[0067] The reformulated conversion query may be registered in association with the search advertisement that was the source of the conversion, or in association with another search advertisement. For example, the operator of the EC service may specify the search advertisement to be associated with the reformulated conversion query. The process execution unit 204 may execute, as a predetermined process, a registration process for registering the reformulated conversion query as an index of the search advertisement.

[0068] [1-3-3. Functions realized by the search server] For example, the search server 30 includes a data storage unit 300 and a search unit 301. The data storage unit 300 is realized by the storage unit 32. The search unit 301 is realized by the control unit 31.

[0069] [Data storage section] The data storage unit 300 stores data necessary for providing a search service. For example, the data storage unit 300 stores an advertisement database DB3.

[0070] Fig. 8 is a diagram showing an example of the advertisement database DB3. The advertisement database DB3 is a database in which various information related to search-based advertisements is stored. For example, the advertisement database DB3 stores an advertisement ID, an advertiser ID, actual data of the search-based advertisement, an index, and display keywords. The advertisement database DB3 may store any information related to the search-based advertisements, and may store information other than that shown in Fig. 8.

[0071] The advertiser ID is an example of advertiser identification information that can identify an advertiser, and the advertiser identification information may be other information such as the advertiser's name, telephone number, or email address. In the first embodiment, it is assumed that a search-based advertisement of an advertiser other than the operator of the EC service is desired to be posted. For this reason, information on the search-based advertisements of various advertisers is stored in the advertisement database DB3. The actual data is data such as a character string indicating the content of the advertisement, an image, and a linked URL. In the first embodiment, it is assumed that the argument of the linked URL includes a query divided into morphemes and an advertisement ID.

[0072] An index is information to be compared with a query. Any information can be used as an index. For example, part of a product title, product attributes, keywords extracted from a detailed description of a product, a character string specified by an operator of an EC service, or a combination of these may be used as an index. A display keyword is a character string used when displaying a search-related advertisement in the display area A20. If no display keyword is specified as in FIG. 8, the query divided into morphemes is displayed in the display area A20. If a display keyword is specified, the display keyword is displayed in the display area A20.

[0073] The data stored in the data storage unit 300 is not limited to the above example. The data storage unit 300 can store any data. For example, the data storage unit 300 may store a webpage database required for searching webpages. For example, the webpage database stores URLs and indexes of various webpages.

[0074] [Search section] When a user inputs a query, the search unit 301 executes a search for web pages and a search-based advertisement based on the query. These searches themselves may be executed based on a known search engine. For example, the search unit 301 executes a search based on the query input by the user, an index stored in the web page database, and an index stored in the advertisement database DB3. The search unit 301 acquires search results for web pages and search results for search-based advertisements. The search unit 301 transmits display data for a search result screen SC2 showing the search results to the user terminal 40.

[0075] The display data may be in any format, for example, HTML data if the search result screen SC2 is to be displayed on a browser. If the search result screen SC2 is to be displayed on an application dedicated to the search service, the data may be in a format supported by the application. In addition, the search unit 301 can transmit display data of various screens of the search service to the user terminal 40.

[0076] [1-3-4. Functions realized on user terminals] For example, the user terminal 40 includes a data storage unit 400, a display control unit 401, and an operation reception unit 402. The data storage unit 400 is realized by the storage unit 42. The display control unit 401 and the operation reception unit 402 are realized by the control unit 41.

[0077] [Data storage section] The data storage unit 400 stores data necessary for providing search services and EC services. For example, the data storage unit 400 stores a browser or a dedicated application for displaying each screen.

[0078] [Display control section] The display control unit 401 causes the display unit 45 to display various screens.

[0079] [Operation reception section] The operation reception unit 402 receives various operations from the operation unit 44 .

[0080] [1-4. Processing performed in the first embodiment] Fig. 9 is a flow diagram showing an example of a process executed in the first embodiment. This process is executed by the control units 11, 21, 31, and 41 operating in accordance with the programs stored in the storage units 12, 22, 32, and 42. Hereinafter, the process when a conversion occurs by the user will be described. For this reason, the point at which an input by the user occurs is called a conversion query.

[0081] As shown in Fig. 9, the search server 30 executes a process between the user terminal 40 and the search server 30 to display a top screen SC1 of the search service (S1). When the process of S1 is executed, the user inputs an arbitrary character string as a conversion query into the input form F10 and selects the button B11. The user terminal 40 transmits the conversion query input by the user to the search server 30 (S2). Upon receiving the conversion query, the search server 30 executes a morphological analysis using a morphological analysis tool (S3).

[0082] The search server 30 executes a search for web pages and a search for search-related advertisements based on the conversion query divided into morphemes and the advertisement database DB3 (S4). The search server 30 generates display data for a search result screen SC2 based on the results of the search executed in S4 and transmits the display data to the user terminal 40 (S5). In S5, the search server 30 includes the conversion query divided into morphemes in S3 and the advertisement ID of the search-related advertisement as arguments in the link to the search-related advertisement.

[0083] When the user terminal 40 receives the display data, it displays the search result screen SC2 on the display unit 45 (S6). When the user selects the search-based advertisement, the user terminal 40 executes a process with the EC server 10 to display a product screen SC3 (S7). The EC server 10 can obtain the conversion query after being divided into morphemes in S3 using an argument included in the link of the search-based advertisement. The EC server 10 executes a process with the user terminal 40 to accept an order for a product and display an order completion screen SC4 (S8). In S8, the EC server 10 stores the conversion query in the conversion database DB2.

[0084] The advertiser terminal 20 executes a process for acquiring a conversion query to be subjected to query shaping between the EC server 10 (S9). The conversion query to be subjected to query shaping may be manually specified. All or part of the conversion queries stored in the conversion database DB2 may be acquired. The advertiser terminal 20 shapes the conversion query acquired in S9 based on the trained query shaping model M1 (S10). The advertiser terminal 20 transmits the shaped conversion query to the search server 30 (S11). Upon receiving the shaped conversion query, the search server 30 stores the shaped conversion query in the advertisement database DB3 as a display keyword (S12), and this process ends. Here, the display keyword corresponds to a search query that serves as a trigger in search-linked advertisement.

[0085] The query shaping system 1 of the first embodiment shapes the conversion query based on the query shaping model M1. The query shaping system 1 executes a predetermined process based on the shaped conversion query. This allows the conversion query in the search-linked advertisement to be shaped into an appropriate state. Therefore, the conversion query can be put into an appropriate state and then the predetermined process can be executed, making it easier to achieve the purpose of using the conversion query. For example, when the conversion query is used for the purpose of preventing the display of the search-linked advertisement displayed in the display area A20 from looking unnatural as in the first embodiment, the display can be made more natural.

[0086] In addition, the query shaping system 1 acquires a conversion query in which a conversion occurs in an EC service different from a search service in which a search-related advertisement is displayed. This makes it possible to shape the conversion query appropriately even if a conversion occurs across services.

[0087] In addition, the query shaping system 1 executes a registration process for registering the shaped conversion query in the search service as a predetermined process. This allows the conversion query shaped appropriately to be registered in the search service even if conversion occurs across services.

[0088] In addition, the query shaping system 1 executes, as a registration process, a process of registering the shaped conversion query in a search service as a display keyword. This allows the search service to display search-related advertisements in a natural way.

[0089] In addition, the query shaping system 1 shapes the modified conversion query so that the modified conversion query returns to the state at the time of input in the search service. This allows the conversion query to be restored to its original state even if it is modified to an unnatural state in the search service.

[0090] In addition, the query shaping system 1 shapes the modified conversion query so that the morphemes included in the modified conversion query are restored to the state at the time of input. This allows the conversion query to be restored to its original state even if the conversion query is divided into morphemes by the search service.

[0091] Furthermore, the query shaping system 1 shapes the conversion query based on the query shaping model M1 that has learned, as training data, the relationship between a changed training query that is a changed version of a training query actually input in an EC service and the training query before the change. This allows the conversion query to be shaped by the query shaping model M1 that has learned by effectively utilizing information held by the operator of the EC service.

[0092] In addition, the query shaping system 1 shapes the executed conversion query by performing tokenization on the executed conversion query on which detokenization has been performed based on the query shaping model M1. This allows the query shaping to be performed after making the conversion query into the same state of one token by detokenization regardless of the state of the conversion query, thereby improving the accuracy of query shaping for the conversion query.

[0093] [2. Second embodiment] Next, a second embodiment, which is another embodiment of the query shaping system 1, will be described. For example, a conversion query can be used as an index for search-based advertising or for marketing purposes, in addition to being registered as a display keyword. In this regard, in search-based advertising, if there is no association between the conversion query and the product for which the conversion has occurred, the conversion query may be inappropriate for the above-mentioned purposes and may become noise.

[0094] For example, suppose that a conversion query entered by a user hits a search-based advertisement associated with an index that is completely different in meaning but similar in wording. The user may accidentally become interested in the product of the search-based advertisement and a conversion may occur. In this case, the conversion query is likely not very useful. Therefore, the query shaping system 1 is configured to estimate the relevance between the conversion query and the EC service based on a relevance estimation model using machine learning.

[0095] Fig. 10 is a diagram showing an example of a relevance estimation model in the second embodiment. In the second embodiment, training data including words related to EC services is learned by the relevance estimation model M2. In Fig. 10, words included in the training data are described as training queries, and the training queries are words used as virtual queries. The training queries do not need to be queries that are actually input.

[0096] For example, the training query may be various words such as a query actually input in an EC service, a title of a product, an attribute of a product, a word included in a detailed description of a product, and a keyword specified as an index of a product. The training data is a pair of the training query as input data and information on relevance. Based on this training data, learning of the relevance estimation model M2 is executed so as to estimate whether or not a word included in the input data is related to the EC service.

[0097] For example, when a conversion query is input, the relevance estimation model M2 calculates a feature amount of the conversion query. The relevance estimation model M2 outputs whether or not the conversion query is related to an EC service based on the calculated feature amount. In the second embodiment, a case will be described in which a first value indicating that the conversion query is related to an EC service or a second value indicating that the conversion query is not related to an EC service is output, but instead of these two values, a score indicating the degree of relevance may be output.

[0098] For example, the advertiser terminal 20 registers a conversion query that is estimated to be related to an EC service in a search service. The advertiser terminal 20 does not register a conversion query that is estimated to be unrelated to an EC service in a search service even if a conversion occurs. As described above, in the second embodiment, relevance is estimated using the relevance estimation model M2. Details of the second embodiment will be described below. In the second embodiment, explanations of the same points as in the first embodiment will be omitted.

[0099] [2-1. Functions realized in the second embodiment] 11 is a diagram showing an example of functions realized in the second embodiment. In the second embodiment, the description of functions similar to those in the first embodiment will be omitted. In the second embodiment, in addition to the functions described in the first embodiment, the advertiser terminal 20 includes a relevance estimation unit 205. The relevance estimation unit 205 is realized by the control unit 11.

[0100] The data storage unit 200 of the second embodiment stores the relevance estimation model M2. The relevance estimation model M2 includes a program portion that executes the process of relevance estimation and a parameter portion that is adjusted by learning. For example, the parameters are weights and biases. The parameters may be any parameters that correspond to the machine learning method used as the relevance estimation model M2. In the second embodiment, since binary classification is performed, a machine learning method capable of binary classification may be used.

[0101] In the second embodiment, an example of the relevance estimation model M2 using a neural network is given. For example, when training data on words related to EC services are input, the advertiser terminal 20 executes learning of the relevance estimation model M2 so as to output a first value indicating that the EC service is related. For example, a backpropagation method or a gradient descent method may be used in the learning. The learning of the relevance estimation model M2 may be executed by another computer instead of the advertiser terminal 20.

[0102] The relevance estimation model M2 may use various machine learning methods other than neural networks. In the second embodiment, it is sufficient to estimate the presence or absence of relevance or the probability thereof, so any machine learning method used for some classification in the field of natural language processing may be used. For example, a machine learning method for estimating the genre of a document may be used. The relevance estimation model M2 may be any of supervised learning, semi-supervised learning, and unsupervised learning models. The machine learning method exemplified in the description of the query shaping model M1 in the first embodiment may be used in the relevance estimation model M2.

[0103] In the second embodiment, the acquisition unit 201 also acquires a conversion query in which a conversion occurs in an EC service due to search-based advertising. The EC service is an example of a predetermined service. Therefore, the description of the EC service can be replaced with the predetermined service. In the second embodiment, the predetermined service is a service other than the search service that displays the search-based advertising. Examples other than the EC service are as described in the first embodiment.

[0104] The relevance estimation unit 205 estimates the relevance between the conversion query and the EC service based on the relevance estimation model M2. The relevance is the correlation between the conversion query and the EC service. For example, the number of hits when searching an EC service with a conversion query is one index of the relevance. The higher the relevance, the more hits there are, and the lower the relevance, the fewer hits there are. The relevance is expressed by a binary value indicating whether or not there is relevance, or a score indicating the strength of the relevance. The score is not binary, but has an intermediate value. For example, the higher the score, the stronger the relevance.

[0105] The relevance estimation model M2 is an example of a learning model in which a training query that is related to an EC service is learned. Therefore, the description of the relevance estimation model M2 can be read as a learning model. The training query may be any query that can be used as training data. The relevance estimation model M2 is trained to output that the training query is related to an EC service when training data that is a training query is input.

[0106] For example, the relevance estimation unit 205 estimates the relevance based on the relevance estimation model M2 that has learned a training query that is actually input to a search service and has relevance to an EC service. In the second embodiment, as in the first embodiment, a case will be described in which morphological analysis is performed on the search service side. For this reason, it is assumed that query shaping must be performed in order to use the query as training data for the relevance estimation model M2.

[0107] In the second embodiment, the reformulated conversion queries, which have been subjected to query reforming by the query reforming unit 203 described in the first embodiment, are used as training data for the relevance estimation model M2. Since some of these conversion queries are not relevant, the irrelevant reformulated conversion queries are manually excluded by the operator of the EC service.

[0108] The training data for the relevance estimation model M2 may be a training query manually created by an administrator of an EC service, instead of data actually input in a search service. The query shaping unit 203 of the second embodiment may shape a conversion query estimated to be relevant based on the query shaping model M1. A conversion query estimated to be unrelated may not be subject to query shaping.

[0109] The process execution unit 204 of the second embodiment executes a predetermined process based on the relevance estimated by the relevance estimation unit 205. The predetermined process may be any process, as in the first embodiment. For example, the predetermined process may be a process of recording a conversion query estimated to be relevant in the data storage unit 200, or a process of outputting a conversion query estimated to be relevant to the outside. The predetermined process may be a process of subjecting a conversion query estimated to be relevant to query shaping.

[0110] For example, the process execution unit 204 executes, as the predetermined process, a registration process for registering a conversion query estimated to be relevant in a search service. The process execution unit 204 executes, as the registration process, a process for registering a conversion query that has been formatted by the query formatting unit 203 and that is estimated to be relevant in a search service. The process execution unit 204 differs from the first embodiment in that it targets conversion queries estimated to be relevant, but is similar in other respects. When a relevance score is output, a score equal to or greater than a threshold corresponds to relevance.

[0111] [2-2. Processing performed in the second embodiment] Fig. 12 is a diagram showing an example of the process executed in the second embodiment. The process of S21 to S28 in Fig. 12 is similar to the process of S1 to S8 in the first embodiment.

[0112] The advertiser terminal 20 acquires a conversion query for which relevance is to be estimated from the EC server 10 (S29). The advertiser terminal 20 estimates the relevance between the conversion query acquired in S29 and the EC service based on the trained relevance estimation model M2 (S30). If it is estimated that there is no relevance (S30:N), this process ends. On the other hand, if it is estimated that there is relevance (S30:Y), the processes of S31 to S33, which are similar to the processes of S10 to S12 in the first embodiment, are executed.

[0113] The query shaping system 1 of the second embodiment estimates the relevance between the conversion query and the EC service based on the relevance estimation model M2. The query shaping system 1 executes a predetermined process based on the relevance. This allows the relevance between the conversion query and the EC service to be estimated. As a result, the predetermined process can be executed based on the conversion that is related to the EC service, making it easier to achieve the purpose of using the conversion query. For example, it is possible to prevent a conversion query that is not related to the EC service from being displayed as part of a search-related advertisement in the display area A20. For example, it is also possible to prevent a search-related advertisement that is not related to the EC service from being displayed, thereby reducing exposure of the EC service to reputation risks.

[0114] In addition, the EC service is a service different from the search service that displays search-related advertisements, and the query shaping system 1 can estimate the relevance between the conversion query and the EC service even if the conversion occurs across services.

[0115] In addition, the query shaping system 1 executes a registration process of registering a conversion query estimated to be related to a search service as a predetermined process. This allows a conversion query related to an EC service to be registered to a search service even if a conversion occurs across services.

[0116] In addition, the query shaping system 1 executes a process of registering a shaped conversion query estimated to be relevant to a search service as a registration process. This allows a conversion query that is relevant to an EC service and has been shaped appropriately to be registered to the search service even if a conversion occurs across services.

[0117] Furthermore, the query shaping system 1 estimates relevance based on a relevance estimation model M2 that has learned training queries that are actually input in a search service and have relevance to other services. The relevance can be estimated by the relevance estimation model M2 that is created by effectively utilizing information that is actually input in a search service.

[0118] [3. Modifications] The present disclosure is not limited to the first and second embodiments described above, and may be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0119] [3-1. Modifications of the First Embodiment] Fig. 13 is a diagram showing an example of functions in a modified example related to the first embodiment. In modified example 1-2 described later, the advertiser terminal 20 includes a division execution unit 206. The division execution unit 206 is realized by the control unit 11. In the modified example related to the first embodiment, detokenization does not need to be performed, so Fig. 13 does not show the detokenization execution unit 202.

[0120] [Variation 1-1] For example, if the operator of the EC service knows what kind of morphological analysis is performed on the search service side, the training data for the query shaping model M1 may be generated by reproducing the morphological analysis performed on the search service side. In the modification 1-1, it is assumed that the operator of the EC service has acquired the same morphological analysis tool as the search service.

[0121] The query shaping unit 203 of the modified example 1-1 shapes the modified conversion query based on the query shaping model M1 in which the relationship between the modified training query, in which the training query is modified in response to a change in the conversion query in the search service, and the training query before the change is learned as training data. In the modified example 1-1, as in the first embodiment, the morphological analysis of the conversion query corresponds to the change to the conversion query. The training query before the morphological analysis is performed may be acquired in the same manner as in the first embodiment.

[0122] For example, in Modification 1-1, the advertiser terminal 20 performs morpheme analysis of the training query based on the same morpheme analysis tool as the search service. The advertiser terminal 20 acquires the training query after being divided into morphemes as a modified training query. The advertiser terminal 20 acquires a pair of the training query after being divided into morphemes and the training query before being divided into morphemes as training data, and causes the query shaping model M1 to learn the training data. The query shaping model M1 in Modification 1-1 may be similar to the query shaping model M1 in Modification 1-3.

[0123] In the modified example 1-1, the method of acquiring training data is different from that of the first embodiment, but the learning method of the query shaping model M1 is the same as that of the first embodiment. The process of shaping the conversion query using the trained query shaping model M1 is also the same as that of the first embodiment. In addition, when the search service side performs other changes to the conversion query than the morphological analysis, it is assumed that the operator of the EC system has acquired a change tool for performing the other changes. The training data may be generated based on the change tool.

[0124] The query shaping system 1 of variant example 1-1 improves the accuracy of query shaping by shaping the modified conversion query based on a query shaping model M1 that has been learned from training data that reproduces modifications of conversion queries in a search service.

[0125] [Variation 1-2] For example, in the first embodiment, a case has been described in which detokenization is performed and then tokenization is performed, thereby performing query shaping, but the method of query shaping is not limited to the example of the first embodiment. Query shaping may be performed in any manner so long as the conversion query is shaped into an appropriate form. In Modification 1-2, an example will be given in which, contrary to the first embodiment, detokenization is performed after tokenization is performed.

[0126] FIG. 14 is a diagram showing an example of query shaping in Modification 1-2. The query shaping system 1 in Modification 1-2 includes a division execution unit 206. The division execution unit 206 divides the conversion query into a predetermined division unit. In the example of FIG. 14, a case where the division unit is one character will be described, but the division unit may be any number of characters that is determined in advance, and may be two or more characters. The division execution unit 206 divides the conversion query into characters and inserts separators between the characters.

[0127] The query shaping model M1 of the modified example 1-2 is trained on training data including a training query divided into character units. In the example of Fig. 14, the training data includes the truth or falsity of separators between characters included in the training query. For example, the truth or falsity of the separators may be manually assigned by an operator of the EC service, or the separators included in the training query before division may be set to true, and the separators not included in the training query before division may be set to false.

[0128] When the divided training query is input, the query shaping model M1 outputs the truth or falsity of the separator in the training query. The parameters of the query shaping model M1 are adjusted so that the output truth or falsity becomes the correct truth or falsity included in the training data. For example, the query shaping unit 203 shapes the divided conversion query by estimating the suitability of division for the divided conversion query divided for each division unit based on the query shaping model M1.

[0129] 14, some of the pseudo-inserted separators are estimated to be false, so the query reforming unit 203 reforms the conversion query by leaving only the separators estimated to be true. That is, the query reforming unit 203 reforms the conversion query by deleting the separators estimated to be false from among the separators included in the conversion query after division into division units.

[0130] The query shaping system 1 of variant example 1-2 divides the conversion query into division units, so that regardless of the state of the conversion query, it can perform query shaping by making it into the same state of one token, thereby improving the accuracy of query shaping for the conversion query.

[0131] [Variation 1-3] For example, the query shaping model M1 may perform query shaping by a method other than tokenization as in the first embodiment or judging the truth or falsity of a separator as in Modification 1-2. In Modification 1-3, it is assumed that the query shaping model M1 has learned training data that is a pair of a training query before shaping and a training query after shaping.

[0132] The training query after reformulation is a query in an optimal state in which morphological analysis and the like have not been performed. The training query before reformulation may be a query after morphological analysis has been performed by some morphological analysis tool. For example, as in Modification Example 1-1, the training query before reformulation may be obtained by performing morphological analysis by a morphological analysis tool other than the same morphological analysis tool as that used by the search service.

[0133] For example, the advertiser terminal 20 executes learning of the query shaping model M1 such that, when a training query before shaping included in the training data is input to the query shaping model M1, the shaped training query included in the training data is output from the query shaping model M1. The query shaping unit 203 of the modification 1-3 shapes the conversion query by inputting a conversion query to the query shaping model M1 in which the relationship between the training query before shaping and the shaped training query is learned as training data, and acquiring the shaped conversion query output from the query shaping model M1.

[0134] For example, the query shaping unit 203 inputs a conversion query to the query shaping model M1. The query shaping model M1 calculates a feature amount of the input conversion query, and outputs a shaped conversion query according to the calculated feature amount. In the first embodiment, the shaped conversion query is output after detokenization is performed, but in the modified example 1-3, the conversion query divided into morphemes is input as it is to the query shaping model M1 without detokenization.

[0135] The query shaping system 1 of the modification 1-3 shapes the conversion query based on the query shaping model M1 in which the relationship between the training query before shaping and the training query after shaping is learned as training data. This makes it possible to shape the conversion query into an appropriate state.

[0136] [Variation 1-4] For example, the query shaping model M1 may have learned position information regarding the positions of tokens included in the training query. The position information itself may be any information, for example, a BIO (Begin Inside Outside) tag or a BIOES (Begin Inside Outside End Single) tag. In Modification 1-4, a case where the position information is a BIOES tag will be described. The training data is a pair of the training query and position information in which a B tag is assigned to the first character of each token and an E tag is assigned to the last character when the training query is appropriately tokenized.

[0137] The query shaping unit 203 of the modified example 1-4 inputs a conversion query to the query shaping model M1 in which location information on the location of a token included in a training query is learned as training data, and shapes the conversion query based on the location information output from the learning model. The query shaping model M1 assigns a B tag to a character estimated to be the first character of a token in the input conversion query, and an E tag to a character estimated to be the last character of the token. An I tag may be assigned to an intermediate character, and an O tag may be assigned to a separator. The query shaping unit 203 shapes the conversion query so that a character assigned with a B tag to a character assigned with an E tag becomes one token.

[0138] The query shaping system 1 of the modification 1-4 shapes the conversion query based on the query shaping model M1 in which position information related to the positions of tokens included in the training query is learned as training data. This makes it possible to shape the conversion query into an appropriate state.

[0139] [Other Modifications of the First Embodiment] For example, in the first embodiment, a case has been described in which the conversion query is changed on the search service side and the query shaping is performed on the EC service side, but the conversion query may not be changed on the search service side. For example, a user may enter a conversion query without any spaces or with a space in an unnatural position. In this case, the query shaping unit 203 may perform the same process as in the first embodiment to shape the unnatural conversion query entered by the user.

[0140] When shaping an unnatural conversion query input by the user, the conversion query input by the user is displayed as is in the display area A20. When a display keyword is associated with a search-linked advertisement, the display keyword is displayed in the display area A20. Even if the user inputs an unnatural conversion query, the natural display keyword shaped by the query shaping unit 203 can be displayed in the display area A20.

[0141] [3-2. Modifications of the second embodiment] A modification of the second embodiment will be described.

[0142] [Variation 2-1] For example, in the second embodiment, a case has been described in which a training query actually input in a search service is learned by the relevance estimation model M2, but a training query actually input in an EC service may be learned by the relevance estimation model M2. Although the second embodiment differs from the second embodiment in that the training query used as training data is one actually input in a search service or one actually input in an EC service, the learning method of the relevance estimation model M2 itself is similar to that of the second embodiment.

[0143] The relevance estimation unit 205 of the modified example 2-1 estimates relevance based on the relevance estimation model M2 in which a training query actually input in the EC service has been learned. Although the training query used in learning the relevance estimation model M2 is different from that in the second embodiment, the process of estimating relevance based on the trained relevance estimation model M2 is the same as in the second embodiment. It is assumed that the training query actually input in the EC service is stored in the data storage unit 100 as a usage log of the EC service.

[0144] The query shaping system 1 of the modification 2-1 estimates relevance based on a learning model that has learned training queries that are actually input in an EC service. This allows the relevance estimation model M2 to learn training queries that are actually input in an EC service, thereby improving the accuracy of relevance estimation.

[0145] [Variation 2-2] For example, the relevance estimation unit 205 may estimate the relevance based on the relevance estimation model M2 in which a first training query related to an EC service and a second training query not related to an EC service are trained. That is, the relevance estimation model M2 may train not only the first training query related to an EC service but also the second training query not related to an EC service.

[0146] The second training query that is not related to the EC service may be manually prepared by the operator of the EC service. The output corresponding to the second training query is a second value indicating that the query is not related to the EC service. The learning method based on the first training query is as described in the second embodiment. For example, when the second training query is input to the relevance estimation model M2, the relevance estimation model M2 is trained so that the relevance estimation model M2 outputs that the query is not related.

[0147] The query shaping system 1 of variant example 2-2 improves the accuracy of relevance estimation by estimating relevance based on a relevance estimation model M2 that has been trained on a first training query that is related to EC services and a second training query that is not related to EC services.

[0148] [Variation 2-3] For example, in the second embodiment, a case has been described in which it is estimated whether or not the conversion query has some relevance to the EC service. However, even if the conversion query has some relevance to the EC service, it may not be related to the converted product. For this reason, the relevance between the conversion query and the converted product for which the conversion has occurred may be estimated. The meaning of the relevance is as described in the second embodiment. For example, when a conversion query is entered in an EC service, the more products that are hit in the search, the higher the relevance.

[0149] The relevance estimation unit 205 of the modified example 2-3 estimates the relevance between a conversion query and a conversion product for which a conversion has occurred in an EC service, based on the relevance estimation model M2 in which training queries that are relevant to training products in an EC service have been learned. The training product is an example of a training item. The conversion product is an example of a conversion item. Therefore, the places where it is written as a training product or a conversion product can be read as a training item or a conversion item, respectively.

[0150] The items are not limited to products, and may be anything that is the subject of search-linked advertising. For example, in a travel reservation service, accommodation facilities correspond to the items. For example, in a video distribution service, content such as videos or music corresponds to the items. This embodiment differs from the second embodiment in that the subject of relevance estimation is not the entire EC service, but a conversion product, but the general mechanism of the learning method of the relevance estimation model M2 and the processing during estimation are similar.

[0151] For example, the training data of the relevance estimation model M2 includes, as input data, a training query and product information on a product suitable as a conversion product (e.g., a title, an attribute, all or part of a detailed description, a product name, or other keywords). The training data includes, as output data, a first value indicating relevance. The relevance estimation model M2 is trained so that, when a combination of a training query and product information is input, the first value indicating relevance is output. The relevance estimation unit 205 inputs a combination of a conversion query and product information on a conversion product to the relevance estimation model M2 and estimates relevance by obtaining an output from the relevance estimation model M2.

[0152] The query shaping system 1 of the modification 2-3 estimates the relevance between a conversion query and a conversion product for which a conversion has occurred in an EC service, based on the relevance estimation model M2 in which training queries that are related to training products in an EC service have been learned. This makes it possible to estimate whether or not a combination of a conversion query and a conversion product is appropriate, thereby improving the accuracy of estimating the relevance.

[0153] [Variation 2-4] For example, in the modified example 2-3, a case was described in which a relevance between a conversion query and a conversion product was estimated, but more specifically, a relevance between a conversion query and an attribute of the conversion product may be estimated. In the modified example 2-4, the product information described in the modified example 2-3 indicates an attribute. An attribute is a classification of a product. An attribute may also be called a category or genre.

[0154] The relevance estimation unit 205 of the modified example 2-4 estimates the relevance between the conversion query and the second attribute of the conversion product based on the relevance estimation model M2 in which the training query having a relevance with the first attribute of the training product is learned. The learning method of the relevance estimation model M2 is the same as that of the modified example 2-3. The product information in the description of the modified example 2-3 may be replaced with the attribute. The relevance estimation unit 205 estimates the relevance by inputting a combination of the conversion query and the attribute of the conversion product to the relevance estimation model M2 and acquiring an output from the relevance estimation model M2.

[0155] The query shaping system 1 of the modification 2-4 estimates the relevance between the conversion query and the second attribute related to the conversion item based on the relevance estimation model M2 in which the training query having the relevance with the first attribute related to the training item is learned. This makes it possible to estimate whether the combination of the conversion query and the attribute of the conversion item is appropriate, thereby improving the accuracy of estimating the relevance.

[0156] [Variation 2-5] For example, the predetermined process executed by the process execution unit 204 is not limited to the example described in the second embodiment. The process execution unit 204 may execute, as the predetermined process, a display process for displaying conversion queries based on relevance. In Modification 2-5, the process execution unit 204 takes as an example a display process for displaying information related to relevance (e.g., the presence or absence of relevance or a score) on the display unit 25 of the advertiser terminal 20, but may execute a display process on a computer other than the advertiser terminal 20. The process execution unit 204 may execute a display process for sorting and displaying a plurality of conversion queries based on a score indicating the relevance.

[0157] The query shaping system 1 of the modification 2-5 executes a display process of displaying a conversion query based on the relevance as a predetermined process, which makes it easier for an operator of an EC service to understand the relevance between the conversion query and the EC service.

[0158] [Other Modifications of the Second Embodiment] For example, the query shaping system 1 of the second embodiment is an example of a relevance estimation system. Therefore, the parts describing the query shaping system 1 of the second embodiment can be read as the relevance estimation system. The second embodiment does not need to have the query shaping function described in the first embodiment. Therefore, the relevance estimation system may include only the acquisition unit 201, the process execution unit 204, and the relevance estimation unit 205, without including the detokenization execution unit 202 and the query shaping unit 203.

[0159] [3-3. Other Modifications] For example, the above-described modifications may be combined.

[0160] For example, the query shaping system 1 may be applied to services other than EC services. The other services linked to the search service may be any services, for example, a travel reservation service, a video distribution service, a financial service, a communication service, or a payment service. The other services may be services that post search-related advertisements in the search service and generate some kind of conversion. The search-related advertisements may be advertisements within the EC service. In this case, the search-related advertisements are displayed when searching for products in the EC service.

[0161] For example, the functions described as being realized by each of the EC server 10, the advertiser terminal 20, the search server 30, and the user terminal 40 may be realized by another computer or may be shared among a plurality of computers. For example, the data stored in the data storage units 100, 200, 300, and 400 may be stored in a database server not shown in FIG.

[0162] [4. Notes] For example, the relevance estimation system may be configured as follows. (1) An acquisition unit that acquires a conversion query in which a conversion occurs in a predetermined service due to a search-related advertisement; a relevance estimation unit that estimates a relevance between the conversion query and the predetermined service based on a learning model in which a training query having a relevance to the predetermined service is learned; a process execution unit that executes a predetermined process based on the association; A relevance estimation system comprising: (2) The predetermined service is a service other than a search service that displays the search-related advertisement. The relevance estimation system according to (1). (3) the process execution unit executes, as the predetermined process, a registration process of registering the conversion query estimated to be relevant in the search service; (2) A relevance estimation system. (4) The relevance estimation system further includes a query shaping unit that shapes the conversion query based on a learning model in which training data related to query shaping has been learned; The process execution unit executes, as the registration process, a process of registering, in the search service, the conversion query that is estimated to be relevant and that has been reformulated by the query reforming unit. (3) A relevance estimation system according to the above. (5) the relevance estimation unit estimates the relevance based on the learning model in which the training query, which is actually input in the search service and has a relevance to the other service, has been learned; A relevance estimation system according to any one of (2) to (4). (6) The relevance estimation unit estimates the relevance based on the learning model in which the training query actually input in the other service has been learned. A relevance estimation system according to any one of (2) to (5). (7) The relevance estimation unit estimates the relevance based on the learning model in which a first training query having a relevance to the predetermined service and a second training query having no relevance to the predetermined service are learned. A relevance estimation system according to any one of (1) to (6). (8) The relevance estimation unit estimates the relevance between the conversion query and a conversion item for which a conversion has occurred in the specified service, based on the learning model in which the training query having a relevance with a training item in the specified service has been learned. A relevance estimation system according to any one of (1) to (7). (9) The relevance estimation unit estimates the relevance between the conversion query and a second attribute related to the conversion item based on the learning model in which the training query having a relevance with a first attribute related to the training item is learned. (8) A relevance estimation system according to the above. (10) the process execution unit executes, as the predetermined process, a display process of displaying the conversion query based on the relevance. A relevance estimation system according to any one of (1) to (9). [Explanation of symbols]

[0163] 1 Query shaping system, N Network, 10 EC server, 11, 21, 31, 41 Control unit, 12, 22, 32, 42 Memory unit, 13, 23, 33, 43 Communication unit, 20 Advertiser terminal, 24, 44 Operation unit, 25, 45 Display unit, 30 Search server, 40 User terminal, M1 Query shaping model, M2 Relevance estimation model, 100, 200, 300, 400 Data storage unit, 101 Order processing unit, 201 Acquisition unit, 202 Detokenization execution unit, 203 Query shaping unit, 204 Processing execution unit, 205 Relevance estimation unit, 206 Division execution unit, 300 Data storage unit, 301 Search unit, 400 Data storage unit, 401 Display control unit, 402 Operation reception unit, A20, A21 Display area, DB1 Product database, DB2 Conversion database, DB3 advertising database, SC1 top screen, SC2 search results screen, SC3 product screen, SC4 order completion screen.

Claims

1. An acquisition unit that acquires a conversion query in which a conversion occurs in a predetermined service due to a search-related advertisement; a relevance estimation unit that estimates a relevance between the conversion query and the predetermined service based on a learning model in which a training query having a relevance to the predetermined service is learned; a process execution unit that executes a predetermined process based on the association; A relevance estimation system comprising:

2. The predetermined service is a service other than a search service that displays the search-related advertisement. The relevance estimation system according to claim 1 .

3. the process execution unit executes, as the predetermined process, a registration process of registering the conversion query estimated to be relevant in the search service; The relevance estimation system according to claim 2 .

4. The relevance estimation system further includes a query shaping unit that shapes the conversion query based on a learning model in which training data related to query shaping has been learned; The process execution unit executes, as the registration process, a process of registering, in the search service, the conversion query that is estimated to be relevant and that has been reformulated by the query reforming unit. The relevance estimation system according to claim 3 .

5. the relevance estimation unit estimates the relevance based on the learning model in which the training query, which is actually input in the search service and has a relevance to the other service, has been learned; The relevance estimation system according to any one of claims 2 to 4.

6. The relevance estimation unit estimates the relevance based on the learning model in which the training query actually input in the other service has been learned. The relevance estimation system according to any one of claims 2 to 4.

7. the relevance estimation unit estimates the relevance based on the learning model in which a first training query having a relevance to the predetermined service and a second training query having no relevance to the predetermined service are learned; The relevance estimation system according to any one of claims 1 to 4.

8. The relevance estimation unit estimates the relevance between the conversion query and a conversion item for which a conversion has occurred in the specified service, based on the learning model in which the training query having a relevance with a training item in the specified service has been learned. The relevance estimation system according to any one of claims 1 to 4.

9. The relevance estimation unit estimates the relevance between the conversion query and a second attribute related to the conversion item based on the learning model in which the training query having a relevance with a first attribute related to the training item is learned. The relevance estimation system according to claim 8 .

10. the process execution unit executes, as the predetermined process, a display process of displaying the conversion query based on the relevance. The relevance estimation system according to any one of claims 1 to 4.

11. A computer comprising: An acquisition step of acquiring a conversion query in which a conversion occurs in a predetermined service due to a search-related advertisement; a relevance estimation step of estimating a relevance between the conversion query and the predetermined service based on a learning model in which training queries having relevance to the predetermined service are learned; a process execution step of executing a predetermined process based on the relevance; A relevance estimation method that performs

12. An acquisition unit that acquires a conversion query in which a conversion occurs in a predetermined service due to a search-related advertisement; a relevance estimation unit that estimates a relevance between the conversion query and the predetermined service based on a learning model in which a training query having a relevance to the predetermined service is learned; a processing execution unit that executes a predetermined process based on the correlation; A program that makes a computer function as a

Citation Information

Patent Citations

  • Extractor, and method and program for extraction

    JP2016062275A

  • Method and apparatus for pushing information

    JP2018106658A

  • Retrieval device, retrieval method, and retrieval program

    JP2018147045A

  • Information processing apparatus, information processing method and information processing program

    JP2019056962A

  • Associating device, associating method, and associating program

    JP2020119463A