Search extension generation system, search extension generation method, search extension generation program, and method for generating a natural language processing model

Abstract

By suggesting appropriate search keywords that align with user needs, we can mitigate the decline in advertising performance metrics. [Solution] The search support server 11 receives an input word set containing one or more words entered through a search request and a search associated with the user's user account, acquires the search word set and advertising performance data including advertising performance metric values, inputs the input word set, the search word set, the advertising performance metric values ​​associated with the search word set and a prompt including instructions to a natural language processing model to acquire a recommended word set that is predicted to have high advertising performance metric values, and sends data to the user terminal to display a search results page containing the recommended word set.

Description

Technical Field

[0001] This disclosure relates to a search expansion generation system, a search expansion generation method, a search expansion generation program, and a method for generating a natural language processing model.

Background Art

[0002] As an advertising targeting technology, behavioral targeting using the search history of a website is used. For example, a search-linked advertisement is a method of displaying an advertisement related to a search query entered by a user in a search box on a user terminal. Patent Document 1 describes accumulating a conversion query, which is a token obtained by morphological analysis of a search query. The search server performs an advertisement search based on the conversion query and an advertisement database.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, a user does not always enter a search query that meets their potential desires. If a user does not enter an appropriate keyword, the search results may not meet the user's needs. Therefore, a new search experience that meets the user's needs has been desired.

Means for Solving the Problems

[0005] This disclosure provides a search extension generation system that solves the above problems. The search extension generation system comprises a memory for storing a plurality of instructions and at least one processor, wherein the at least one processor executes the plurality of instructions to obtain an input word set entered for a search, wherein the input word set includes one or more words entered by a search associated with the user's user account, and obtains advertising performance data, wherein the advertising performance data includes a search word set and advertising performance indicator values, wherein the search word set is a word set that includes one or more words entered by past searches, and the advertising performance indicator values ​​are indicator values ​​that show the performance of advertisements displayed in conjunction with searches using the search word set, and is generated by inputting prompts to a natural language processing model. The system is configured to obtain a recommended word set, wherein the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction statement, wherein the instruction statement is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value; and to send data to a user terminal for displaying a search results page, wherein the search results page includes the recommended word set for the user, the search results for the input word set, and advertisements related to the input word set.

[0006] This disclosure provides a search extension generation method that solves the above problems. The search extension generation method is a method that can be executed by at least one processor, and includes obtaining an input word set entered for a search, wherein the input word set includes one or more words entered by a search associated with the user's user account; obtaining advertising performance data, wherein the advertising performance data includes a search word set and advertising performance indicator values, wherein the search word set is a word set that includes one or more words entered by past searches, and the advertising performance indicator values ​​are indicator values ​​that show the performance of advertisements displayed in conjunction with searches using the search word set; and obtaining a recommended word set generated by inputting a prompt to a natural language processing model. The present invention provides a natural language processing model in which a prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction statement, wherein the instruction statement is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value, and transmits data to a user terminal for displaying a search results page, wherein the search results page includes the recommended word set for the user, the search results for the input word set, and advertisements related to the input word set.

[0007] This disclosure provides a search extension generation program that solves the above problems. The search extension generation program involves at least one processor executing the plurality of instructions to obtain an input word set entered for a search, wherein the input word set includes one or more words entered by a search associated with the user's user account, and obtaining advertising performance data, wherein the advertising performance data includes a search word set and advertising performance indicator values, wherein the search word set is a word set that includes one or more words entered by past searches, and the advertising performance indicator values ​​are indicator values ​​that show the performance of advertisements displayed in conjunction with searches using the search word set, and a recommended word set generated by inputting prompts to a natural language processing model. The process involves obtaining a set of words, wherein the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction, wherein the instruction is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value; and sending data to the user terminal to display a search results page, wherein the search results page includes the recommended word set for the user, the search results for the input word set, and advertisements related to the input word set.

[0008] This disclosure provides a method for generating a natural language processing model. The method for generating a natural language processing model is a method for generating a natural language processing model, wherein the natural language processing model is configured to generate one or more recommended word sets when one input word set is input, the generation method includes at least one processor acquiring advertising performance data, wherein the advertising performance data includes a plurality of search word sets and advertising performance metric values ​​related to each of the search word sets, wherein each search word set includes at least one word, and each advertising performance metric value is a metric value indicating the performance of an advertisement displayed in conjunction with a search using the corresponding search word set, acquiring user behavior history data, wherein the behavior history data includes the search history of multiple searches performed in the user account, acquiring a pre-trained model for natural language processing, inputting the advertising performance data into the pre-trained model to allow the pre-trained model to learn the relationship between the search word sets and the advertising performance metric values, and inputting the behavior history data into the pre-trained model to allow the pre-trained model to learn the user's behavioral tendencies. [Effects of the Invention]

[0009] According to this disclosure, by suggesting appropriate keywords that meet the user's needs, a new search experience can be provided to the user. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows the schematic configuration of the information processing system according to the first embodiment. [Figure 2] This figure shows an example of an advertising performance table for the same embodiment. [Figure 3] This figure shows an example of an action history table in the same embodiment. [Figure 4] This figure shows an example of an advertising table in the same embodiment. [Figure 5]This is a block diagram of the search support server according to the same embodiment. [Figure 6] This flowchart shows the procedure for generating the search extension according to the same embodiment. [Figure 7] This is a conceptual diagram illustrating the procedure for generating search extensions according to the embodiment. [Figure 8] This flowchart shows the procedure for narrowing down the recommended word set in the same embodiment. [Figure 9] This figure shows an example of the search results screen for the same embodiment. [Figure 10] This figure shows an example of the search results screen for the same embodiment. [Figure 11] This figure shows the chat screen as the search results screen in the second embodiment. [Figure 12] This figure shows the training of the natural language processing model in the third embodiment. [Modes for carrying out the invention]

[0011] Examples of search extension generation systems, search extension generation methods, and search extension generation programs described herein are provided. This disclosure is not limited to these examples, and all modifications within the meaning and scope of the claims are intended to be included.

[0012] [Overview of the Information Processing System] FIG. 1 shows an example of a search expansion generation system 10. The search expansion generation system 10 supports search-linked advertisements. The search expansion generation system 10 includes at least a search support server 11. In the present embodiment, the search expansion generation system 10 is included in an information processing system 1 that provides a search service and search-linked advertisements. The information processing system 1 includes, in addition to the search support server 11, a search server 13, an advertisement server 14, a user management server 15, a large language model (LLM) server 16, and a user terminal 20. In the present embodiment, a large language model is used as a natural language processing model. Hereinafter, the large language model is referred to as "LLM". The user terminal 20, the search server 13, the search support server 11, the advertisement server 14, the user management server 15, and the LLM server 16 are connected via a network 17.

[0013] The search support server 11 is a server that obtains a set of recommended words. The set of recommended words is a set of words for search proposed to the user. The set of recommended words is a set of words highly relevant to the search query input by the user in the search box (search input field).

[0014] The search query may not necessarily accurately reflect the user's intention. For example, the user may not come up with keywords that accurately represent the information they are seeking. In particular, when the user's knowledge about the information they are looking for is insufficient, it becomes difficult to select appropriate keywords. Or, even though the user is seeking specific information, they may only come up with keywords for a broader concept than the information they are seeking, or vice versa. In the present embodiment, the set of recommended words is a set of words for which high advertisement performance is expected among the sets of words related to the search query. Advertisement performance refers to the results obtained by the advertisement. The advertisement performance index value is a quantification of the results. Hereinafter, the advertisement performance index value is abbreviated as the advertisement index value.

[0015] In addition, the search support server 11 also functions as a web server. In the present embodiment, the search support server 11 transmits a web page including a search box and an advertisement frame of search-linked advertisements to the user terminal 20.

[0016] The search support server 11 includes a processor 11A, a memory 11B, and a communication interface 11C. The processor 11A is a processor that executes one or more of the control processes disclosed in this specification. The processor 11A is at least one processing circuit. For example, the processor 11A is a CPU, GPU, NPU, etc. The processor 11A is not limited to performing software processing for all processes it executes. For example, the processor 11A may include a dedicated hardware circuit (e.g., an application-specific integrated circuit: ASIC) that performs hardware processing for at least a part of the processes it executes. That is, the processor 11A is a circuitry that includes one or more processors operating according to a computer program (software), one or more dedicated hardware circuits that execute at least a part of various processes, or a combination thereof.

[0017] The memory 11B stores the program 110. The program 110 includes at least one instruction. The memory 11B is composed of, for example, a main memory device and an auxiliary storage device. Examples of the memory 11B include a ROM, a RAM, a hard disk, etc. The memory 11B includes any available recording medium that can be accessed by a general-purpose or dedicated computer. The memory 11B is connected to the processor 11A via a bus. The program 110 corresponds to a search expansion generation program.

[0018] The communication interface 11C is an interface that establishes a communication path with other devices via the network 17 and executes data transmission and reception. The communication interface 11C is composed of hardware, software, or a combination thereof.

[0019] Network 17 may include the Internet or a LAN (Local Area Network). Network 17 may also include a core network and multiple base stations. Network 17 is a third-generation communication system such as W-CDMA or CDMA2000, LTE, a fourth-generation communication system, or a fifth-generation communication system.

[0020] The search server 13 has a processor 13A, memory 13B, and a communication interface 13C. These configurations are the same as those of the search support server 11. Memory 13B stores the crawler 130 and the index 131. Memory 13B also stores a search program (not shown). The crawler 130 automatically searches for web pages on the internet. The crawler 130 stores newly discovered web pages in the search engine index 131.

[0021] Index 131 stores data about web pages collected by crawler 130 in reverse index format. Index 131 has a data structure that includes keywords for referring to documents in which a word (token) appears, a list of identifiers for documents in which the word appears, and a posting list that stores the location of the word's appearance within the documents.

[0022] The search server 13 analyzes the search query by executing a search program and generates an input word set. The input word set contains at least one word (also called a token). The search server 13 also uses the index 131 to search for web pages that contain keywords related to the input word set entered by the user. The search server 13 ranks the detected web pages and narrows down the search to web pages closely related to the input word set. The search engine then sends the search results page, which includes snippets of the ranked web pages, to the user terminal 20 either via the search support server 11 or directly.

[0023] The ad server 14 acquires information about advertisements from advertiser terminals (not shown). Advertiser terminals are terminals used by advertisers. The ad server 14 also delivers advertisements related to the input word set received from the search server 13 to the user terminal 20. Furthermore, the ad server 14 tracks the display of advertisements and the user's actions regarding those advertisements.

[0024] The advertising server 14, and the search server 13, comprise a processor 14A, memory 14B, and a communication interface 14C. These configurations are the same as those of the search support server 11. Memory 14B stores the advertising DB 30 and the advertising performance DB 31. Memory 14B also stores a program (not shown) for providing advertisements.

[0025] Advertisers submit their ads to ad server 14. At the same time, advertisers create ad campaigns. An ad campaign includes target keywords for displaying search ads, bid amounts, ad duration, and the budget to be spent during the ad duration. An example of a bid amount is CPC (Cost Per Click).

[0026] The ad server 14 delivers ads, for example, through RTB (Real-Time Bidding). For example, when the ad server 14 receives an ad request input word set from the search server 13, it identifies the ad campaign corresponding to the search query. The ad server 14 then determines whether the bid amount of the identified ad campaign exceeds the floor price (base price) set for the ad medium. Furthermore, for ad campaigns that exceed the floor price, the ad server 14 selects the ad with the highest bid amount from among the submitted ads. At this time, the ad server 14 may select the ad based on the quality of the ad, such as the click-through rate. The ad server 14 sends the selected ad to the user terminal 20. The ad server 14 may also receive the search query from the user terminal 20 without going through the search support server 11.

[0027] The ad server 14 constitutes a platform that provides advertisements related to the input word set. The ad server 14 stores advertising performance data for the advertisements it has provided in the advertising performance DB 31.

[0028] The user management server 15 collects user activity history data. The user management server 15 includes a processor 15A, memory 15B, and a communication interface 15C. These configurations are the same as those of the search support server 11. Memory 15B stores the user management database 32. Memory 15B also stores a program that collects activity history data (not shown). The user management database 32 stores user activity history across various services. Examples of services include online shopping mall sales, financial or insurance transactions, online payment services, credit card payment services, video and other content distribution, mobile phone and other communication services, online accommodation booking services, online restaurant booking services, and online transportation booking services.

[0029] The LLM server 16 stores LLM16A, a language model constructed using a large amount of data and deep learning techniques. LLM16A is a learning model for various types of generative artificial intelligence. LLM16A learns from acquired documents using generative artificial intelligence. The LLM server 16 also extracts common features between the word sets contained in the documents and newly input word sets, based on the learned documents. The LLM server 16 then generates an abstract word set with the extracted features and a more specific word set with the extracted features.

[0030] The user terminal 20 is a device used by the user. An example of the user terminal 20 is a smartphone (multifunction telephone) or a personal computer. Other examples of the user terminal 20 include a tablet device, a wearable computer, or any other user terminal. The user terminal 20 may be a standalone device or a combination of multiple devices connected to each other in a manner that enables them to send and receive various types of data. The user terminal 20 comprises a processor 20A, memory 20B, a communication interface 20C, an input device 20D, and a display device 20E. The hardware configuration of the processor 20A, memory 20B, and communication interface 20C is the same as that of the search support server 11.

[0031] The processor 20A is connected to the input device 20D via an input interface. The input device 20D includes a touch panel, keyboard, mouse, operation buttons provided on the casing of the user terminal 20, etc. The user terminal 20 may include peripheral devices not shown. Peripheral devices may include a microphone and a speaker. The processor 20A is connected to the display device 20E via a display controller. The display device 20E is a device capable of displaying a screen, such as a liquid crystal display or an organic EL display.

[0032] Memory 20B stores various programs not shown in the diagram. For example, memory 20B stores programs for a web browser or a hybrid application including an in-app browser. For example, processor 20A retrieves a web page written in a markup language from the search support server 11 and displays the web page using a web browser. Display device 20E displays a search results screen based on the web page. Alternatively, processor 20A may retrieve a web page from the search server 13.

[0033] [Data structure] Next, we will describe an example of the data structure stored by each of the servers mentioned above. Figure 2 shows an example of the advertising performance table 40. The advertising performance table 40 is stored in the advertising performance DB 31. The advertising performance table 40 contains multiple advertising performance data 40A. The advertising performance table 40 is a table that stores the history based on search queries entered by multiple users.

[0034] The advertising performance data 40A is data that records keywords and advertising metrics set for an advertisement, linked together. Here, the keywords are referred to as the search word set. Advertisers register advertising content and keywords in the advertising server 14. When the advertising server 14 obtains the input word set, it uses the advertising performance data 40A to select advertising content 50D and delivers it to the media where the search was performed. When a user clicks on the advertising content, the user terminal 20 is redirected to a conversion site or the like.

[0035] The search word set is generated from past search queries. In this embodiment, the search query, when divided into words, is called a word set. Furthermore, in order to distinguish between the word set generated at the time of search execution and the word set accumulated in the advertising performance table 40, the former is called the input word set and the latter is called the search word set.

[0036] A search word set must contain at least one word. Ad metrics are performance metrics for an ad. The ad in question is the ad that appeared on the search results page when the search word set was used.

[0037] Examples of advertising performance metrics include CPC, CPM (Cost Per Mille), CTR (Click-Through Rate), CPV (Cost Per View), and CPA (Cost Per Action / Acquisition). CPC is the revenue or cost per click when an ad is clicked. Revenue is the revenue earned by the advertising medium, and cost is the cost paid by the advertiser. For example, if an ad is displayed multiple times in conjunction with a single set of search terms, the advertising metric value is updated using the newly acquired advertising metric value. In this case, the average or median of the advertising metric value may be stored.

[0038] For example, CPC is registered along with keywords when each advertiser submits their ad. If there are multiple different ads with the same keyword, the CPC stored in the ad performance data 40A may be the statistical value of the CPC associated with all of those ads. The statistical value may be the mean, median, etc.

[0039] For example, CPC may be the actual cost per click, calculated by dividing the advertising cost by the actual number of clicks during a specified period. The advertising cost is registered in advance along with the content of the ad delivery. If there are multiple ads included in the same advertising cost, the CPC statistics associated with the multiple ads may also be used. The statistics may be the mean, median, etc.

[0040] For example, CPC could be the cost per click determined by the distribution medium (e.g., a medium where items can be featured in an online shopping mall). The CPC included in advertising performance data 40A must fall under at least one of the three examples above. Alternatively, it may be a statistical value obtained by combining at least two of the above examples.

[0041] CPM is the revenue or cost per 1,000 impressions. CTR is the ratio of the number of clicks on an ad to the number of impressions. CPV is the revenue or cost when an ad (video) is viewed. CPA is the cost incurred when a user performs a specific consumer action in relation to the ad. For example, consumer actions include purchasing goods or services, registering user data, and downloading materials.

[0042] As users repeatedly perform searches, diverse advertising performance data 40A is accumulated in the advertising performance database 31. Furthermore, the reliability of the data is enhanced when advertising metrics are repeatedly updated for a single set of search terms.

[0043] Figure 3 shows examples of tables 41-43 in which user activity history data 44 is stored. The user management DB 32 stores the search history table 41, the browsing history table 42, and the purchase history table 43. In this embodiment, the search history data 41A, the browsing history data 42A, and the purchase history data 43A are collectively referred to as activity history data 44.

[0044] The search history table 41 contains at least one search history data 41A. The search history data 41A includes a user identifier, search word set, number of searches, search date and time, and search frequency. The search word set is a word set obtained by parsing the search query entered by the user. This word set contains at least one word. The number of searches is the number of times the same search word set has been used. The search date and time includes at least the most recent search date and time. The search frequency is the number of searches divided by a predetermined period. The predetermined period may be the number of days elapsed since the search word set was first entered.

[0045] The browsing history table 42 contains at least one browsing history data 42A. This includes a user identifier, the URL of the viewed page, a search word set, the number of views, and the date and time of viewing. The search word set consists of keywords contained in the viewed web page. For example, the viewed page is a page accessible from a link in the search results page.

[0046] The purchase history table 43 contains at least one purchase history data 43A. The purchase history data 43A includes a user identifier, keywords (search word set) for the products purchased by the user, the number of purchases, and the date and time of purchase.

[0047] The user management DB32 may also contain user management data for each user. This user management data may include at least one of the following: gender, age, date of birth, place of origin, place of residence, name, family structure, annual income, and occupation. This data can also be referred to as demographic attributes or demographic data.

[0048] The user management server 15 may collect behavioral history data 44 from the search server 13 or an external server. In this way, the user management server 15 accumulates diverse behavioral history data 44 as the user's actions are repeated. As the behavioral history data 44 accumulates, the accuracy of the user characteristics determined based on this data also improves.

[0049] Figure 4 shows an example of an advertising management table 45. The advertising management table 45 is stored in the advertising DB 30. The advertising management table 45 contains multiple advertising campaign data 45A. The advertising campaign data 45A includes a campaign identifier, advertiser identifier, keywords, bid amount, advertising period, and budget. The campaign identifier is an identifier assigned to a campaign set up by the advertiser. The advertiser identifier is the identifier of the advertiser. The keywords are target keywords used when selecting an ad. Keywords are set by the advertiser. The bid amount, advertising period, and budget are also set by the advertiser, etc. In addition, the URL of the advertising content is associated with the advertising campaign data 45A.

[0050] [Functional block diagram of the search support server] Figure 5 shows a functional block diagram of the search support server 11. The processor 11A of the search support server 11 functions as a prompt generation unit 111 and an information collection module 112 by executing program 110. The information collection module 112 acquires data related to the input word set from the search server 13. The information collection module 112 also acquires advertising performance data 40A. The information collection module 112 also acquires the recommended word set generated by LLM 16A.

[0051] The prompt generation unit 111 generates prompts to be input to the LLM 16A. The prompt includes an input word set, a search word set, advertising metric values ​​associated with the search word set, and instructions. The instructions include instructions to generate recommended word sets with high advertising metric values ​​from among the word sets associated with the input word set. The advertising metric values ​​are predicted from the relationship between the search word set and the advertising performance metric values.

[0052] [Steps for generating search extensibility, including suggested word sets] This disclosure describes a search extension generation method. The search extension generation method is executable by at least one processor 11A. The search extension generation method includes a method for obtaining a recommended word set.

[0053] The user terminal 20 logs in to a service provided by the search support server 11 using a user identifier. This service relates to an online shopping mall. For example, the search support server 11 sends a web page to display a screen that includes a search box. The user terminal 20 opens the web page sent from the search support server 11 using a web browser and displays the search screen, including the search box, on the display device 20E. Alternatively, if the user terminal 20 launches an application, the application screen is displayed. The application screen includes a search box. Alternatively, the user terminal 20 may log in to a service provided by the search server 13 and display the search screen, including the search box, on the display device 20E.

[0054] The user enters a search query into the search box and performs the search. The user terminal 20 sends the search request, along with the search query and user identifier, to the search server 13. The search server 13 analyzes the search query and extracts the input word set that will serve as keywords. For example, the search server 13 performs morphological analysis as part of the analysis of the search query. The search server 13 also searches index 131 according to the search algorithm. Then, the search server 13 extracts web pages in which at least one word from the input word set appears. At least one web page is extracted.

[0055] The search server 13 ranks the extracted web pages. The search server 13 sends the search results, including the web page title, snippet, and web page URL, to the user terminal 20. The snippet includes at least one of the page title and a summary of the page. The user terminal 20 expands the web page containing the search results to display the search results screen. In this case, pages with higher rankings may be displayed at the top of the screen.

[0056] Furthermore, the search server 13 transmits the input word set and user identifier to the search support server 11 and the advertising server 14. The ad server 14 identifies ad campaigns that use the input word set as target keywords and selects ads as described above. The ad server 14 sends data to the user terminal 20 that includes the URL from which the selected ad content is retrieved. At this time, the ad server 14 may also send ad content data to the user terminal 20. The ad content data includes image data or video data and the URL of the landing page.

[0057] Figure 6 shows the procedure by which the search support server 11 obtains a recommended word set. The search support server 11 obtains the input word set from the search server 13 (step S1). The search support server 11 also sends the input word set to the advertising server 14 and requests advertising performance data 40A. Here, the search support server 11 only needs to obtain the advertising performance data 40A corresponding to the input word set, and the sending and receiving of the input word set is performed between servers as appropriate. For example, when the advertising server 14 receives the input word set and user identifier from the search server 13, it may automatically send the advertising performance data 40A to the search support server 11. Alternatively, the search server 13 may send the input word set and user identifier only to the search support server 11.

[0058] The advertising server 14 searches the advertising performance database 31 and extracts advertising performance data 40A related to the input word set. The advertising server 14 sends the extracted advertising performance data 40A to the search support server 11. The search support server 11 retrieves the advertising performance data 40A (step S2).

[0059] Here, we will describe an example of how the ad server 14 extracts ad performance data 40A related to the input word set. For example, the ad server 14 may obtain ad performance data 40A that contains at least one word included in the input word set. Alternatively, the ad server 14 may identify words that have common characteristics with at least one word included in the input word set. A word set containing words with common characteristics is called a similar word set.

[0060] The ad server 14 then determines whether at least one word included in the input word set or in the similar word set is included as a search word set in the ad performance DB 31. If the ad server 14 determines that the word set is included as a search word set in the ad performance DB 31, it extracts the corresponding ad performance data 40A.

[0061] The following describes an example of identifying similar word sets. The ad server 14 may identify words belonging to the same or similar classification as the classification to which the words in the input word set belong, from the classifications registered in the ad performance DB 31 or ad DB 30. A classification is the classification to which a product belongs, and can also be rephrased as a category or type. For example, if the input word set is "orange", the ad server 14 may identify the word "fruit", which is a larger classification than the classification to which "orange" belongs, and extract ad performance data 40A that includes "fruit" in the search word set. Alternatively, if the input word set is "orange", the ad server 14 may identify the word "apple", which is another classification included in the larger classification of "fruit", and extract ad performance data 40A that includes "apple" in the search word set.

[0062] Alternatively, the ad server 14 may identify words with a high degree of similarity to the words included in the input word set and obtain ad performance data 40A containing those words. The similarity between words may be determined by converting the words into numerical vectors and using the distance or angle between the vectors. In this case, the ad server 14 may send the input word set to the LLM server 16 to obtain a similar word set that is similar to the input word set. The ad server 14 may then obtain ad performance data 40A containing at least one word included in the similar word set. Alternatively, if the ad server 14 determines that there is no ad performance data 40A containing words included in the input word set in the ad performance table 40, it may extract ad performance data 40A containing the similar word set.

[0063] Alternatively, the ad server 14 may identify other users with the same attributes as the user who entered the input word set, and identify keywords set for the ad items displayed to those other users or for the ad items on which those other users took action. User attributes are attribute of the behavioral history based on the behavioral history data 44 or demographic attributes, etc. Actions on ad items can be determined by performance indicators such as the number of clicks or click-through rate, the number of conversions or the conversion rate. Actions can also be determined by the cost per conversion or the amount of money earned from the conversion. The ad server 14 sorts the ad items displayed to other users or the ad items on which other users took action in order of the best ad metrics or the highest amounts, and obtains keywords associated with a predetermined number of top ad items. At this time, the keywords and ad metrics may be used instead of the ad performance data 40A, or the ad performance data 40A containing those keywords may be extracted. Alternatively, the above-mentioned similar word set may be obtained instead of keywords.

[0064] Furthermore, when the ad server 14 retrieves ad performance data 40A, it may prioritize retrieving data that contains new ad performance data. This is because the word sets with high ad performance change depending on the time and season. In other words, ad performance data 40A that contains new ad performance data reflects the latest ad trends.

[0065] The search support server 11 obtains the activity history data 44 from the user management server 15 (step S3). The search support server 11 transmits the user identifier and the input word set. The user management server 15 transmits the activity history data 44 containing the words from the input word set from tables 41 to 43, which contain the user identifier, to the search support server 11. The activity history data 44 obtained at this time includes the data with the most recent date.

[0066] The search support server 11 generates a prompt to obtain a recommended word set (step S4). The search support server 11 also obtains a recommended word set from the LLM server 16 by sending the generated prompt to the LLM server 16 (step S5).

[0067] The generation of a prompt (step S4) and the acquisition of a recommended word set (step S5) will be explained using Figure 7. The prompt 49 input to LLM16A includes the input word set 31, behavioral history data 44, instruction sentence 47, and advertising performance data 40A obtained from the user terminal 20. As described above, the advertising performance data 40A includes the search word set and the advertising metric values ​​associated with the search word set. The instruction sentence 47 includes instructions for generating a recommended word set with high advertising metric values ​​based on the relationship between the search word set and the advertising metric values. The instruction sentence 47 also includes instructions for generating a recommended word set that matches the user's behavioral tendencies predicted from the behavioral history data 44. The instruction sentence 47 may also include a command to specify the number of recommended word sets. Alternatively, the instruction sentence 47 may include a command to specify the minimum number of recommended word sets.

[0068] For example, suppose input word set 46 includes "orange" and behavioral history data 44 includes "gift". Also, suppose one advertising performance data 40A includes the search word sets "orange" and "luxury" and their CPC, and another advertising performance data 40A includes the search word sets "fruit" and "gift set" and their CPC. The search support server 11 includes these in the prompt and sends them to the LLM server 16. Behavioral history data 44 and advertising performance data 40A also include data other than the example above.

[0069] LLM16A learns the characteristics of search word sets that are predicted to have a high CPC from advertising performance data 40A. LLM16A also learns user behavior trends from behavioral history data 44. Based on these learning results, LLM16A generates recommended word sets 48, for example, one containing "oranges" and "gift set," and another containing "oranges" and "assortment."

[0070] The search support server 11 may select a predetermined number of recommended word sets from the multiple recommended word sets it has acquired. The process for selecting a predetermined number of recommended word sets will be described later. The ad server 14 also selects ad campaign data 45A from the ad database 30 in which at least one word included in the input word set 46 is included as a target keyword. In this process, the ad server 14 may also consider the bid amount, ad period, budget, etc. The ad server 14 also sends data related to the selected ad to the user terminal 20. The data related to the ad includes a URL for obtaining the ad content 70.

[0071] Returning to Figure 6, the search support server 11 sends the acquired recommended word set 48 to the user terminal 20 (step S6). The user terminal 20 generates a search results screen using the search results received from the search server 13, the advertising data received from the advertising server 14, and the recommended word set received from the search support server 11. Alternatively, the search support server 11 may generate data for displaying the search results screen using the search results, advertisements, and recommended word set, and send it to the user terminal 20. Alternatively, the search support server 11 may send the acquired recommended word set 48 to the search server 13. The search server generates data including the search results and the recommended word set 48, and sends it to the user terminal 20. The generation of the search results screen may be done using client-side rendering, server-side rendering, or any other format.

[0072] [Selection of recommended word set] Next, referring to Figure 8, we will explain how the search support server 11 selects a predetermined number of recommended word sets in step S5 described above. This process is based on the premise that the number of recommended word sets obtained from the LLM server 16 is greater than the number of recommended word sets sent to the user terminal 20.

[0073] For example, if the training data is insufficient or the content of the training data is ambiguous, LLM16A may output a recommended word set that deviates from the user's requirements. Therefore, the search support server 11 selects a word set that is estimated to have a high degree of confidence from among the recommended word sets obtained from the LLM server 16.

[0074] The search support server 11 determines whether the recommended word set obtained from the LLM server 16 is included in the advertising performance table 40 (step S5-1). At this time, the search support server 11 queries the advertising server 14 to see if there is a search word set that exactly matches the recommended word set.

[0075] If the search support server 11 determines that the recommended word set is not included in the advertising performance table 40 (step S5: NO), it proceeds to step S5-3. If the search support server 11 determines that the recommended word set is included in the advertising performance table 40 (step S5-1: YES), it obtains the advertising metric values ​​associated with the recommended word set from the advertising server 14. The search support server 11 selects the recommended word sets obtained from the LLM server 16 in descending order of their advertising metric values ​​(step S5-2).

[0076] The search support server 11 determines whether the number of selected recommended word sets is less than a predetermined number (step S5-3). The predetermined number is the number of recommended word sets to send to the user terminal 20, and is predetermined.

[0077] If the search support server 11 determines that the selected recommended word set is greater than or equal to a predetermined number (step S5-3: NO), it terminates the process. If the search support server 11 determines that the selected recommended word set is less than a predetermined number (step S5-3: YES), it determines whether or not the recommended word set is included in the search history table 41 corresponding to the user who performed the search (step S5-4).

[0078] If the search support server 11 determines that the recommended word set is not included in the search history table 41 (step S5-4: NO), it proceeds to step S5-5. If the search support server 11 determines that the recommended word set is included in the search history table 41 (step S5-4: YES), it extracts the corresponding search history data 41A.

[0079] The search support server 11 then selects a recommended word set according to the search status contained in the extracted search history data 41A (step S5-5). For example, the search support server 11 reads at least one of the search count, search date and time, and search frequency from the search history data that contains the same search word set as the recommended word set, and selects a recommended word set based on a predetermined algorithm. As an example, the search support server 11 selects the recommended word sets in order of the most recent search date and time. Alternatively, the search support server 11 selects the recommended word sets in order of the most frequent search count. Alternatively, the search support server 11 may rank the recommended word sets using a calculation formula with the search count, search date and time, and search frequency as parameters, and select the recommended word sets from the highest-ranked ones.

[0080] The search support server 11 determines whether the number of recommended word sets is less than a predetermined number (step S5-6). If the search support server 11 determines that the selected recommended word sets are equal to or greater than the predetermined number (step S5-6: NO), it terminates the process. If the search support server 11 determines that the selected recommended word sets are less than a predetermined number (step S5-6: YES), it selects a recommended word set using the advertising metric values ​​or search status of word sets similar to the recommended word set (step S5-7).

[0081] Specifically, the search support server 11 identifies word sets similar to the recommended word set. Identifying similar word sets is the same process as extracting advertising performance data 40A related to the input word set. If the search support server 11 determines that a similar word set is included in the advertising performance table 40, it performs the same process as in step S5-2 to select a recommended word set. The search support server 11 also determines whether a similar word set is included in the search history table 41. If the search support server 11 determines that a similar word set is included in the search history table 41, it performs the same process as in step S5-5 to select a recommended word set.

[0082] [Search Results Screen] Figure 9 shows the main parts of an example of a search results screen 50 displayed on the display device 20E by the user terminal 20. The search results screen 50 includes display areas 50A and 50C for search results and a display area 50B for recommended words. The search results screen 50 may also include a search box 50E.

[0083] The search query is entered into the search box 50E. The search results display areas 50A and 50C display the search results received from the search server 13 and the advertising content 50D received from the advertising server 14. For example, display area 50A displays the advertising content 50D, which is the result of a search using the advertising algorithm. In other words, search-linked advertisements are displayed in display area 50A. Display area 50C displays advertising content 50D and snippets found using organic search algorithms other than the advertising algorithm. Furthermore, the display mode of the search results display area 50A on the search results screen 50 is determined according to the medium.

[0084] The recommended word display area 50B contains multiple recommended word sets. When the user terminal 20 clicks on a recommended word set, it sends a search request to the search server 13 using the clicked word set as the input word set as the search query. The search server 13 uses the received input word set to perform a search in the manner described above. The search server 13 also sends the input word set to the advertising server 14 and the search support server 11. The search support server 11 uses the received input word set to obtain a new recommended word set and sends it to the user terminal 20 (steps S1 to S6). The user terminal 20 then generates the search results screen 50 again using the recommended word set and the advertising data received from the advertising server 14.

[0085] Figure 10 shows an example of the search results screen 51. The search results screen 51 is displayed after a recommended word set is selected in the display area 50B of the search results screen 50. In the search results screen 51, the recommended word set selected by the user is displayed in the search box 51E. The search results display areas 51A and 51C display the search results for the new recommended word set. Display area 51A displays advertising content 51D related to the selected recommended word set. In Figure 10, this includes advertising content 51D that is related to the recommended word set and leads to a web page where the advertised item is posted. For example, display area 51A displays advertising content 51D found using an advertising algorithm that uses the recommended word set. Display area 51C displays advertising content 51D and snippets for items found using a natural search algorithm that uses the recommended word set. The advertising content 51D associated with an item can also be called an item description image or thumbnail image. This advertising content 51D has high advertising metrics. Therefore, a high appeal effect can be expected from the user who performed the search.

[0086] The recommended word sets displayed in display area 51B may be more specific than the recommended word sets displayed in display area 50B, and the number of words in each word set may increase. As a result, the recommended word sets are gradually narrowed down to those that do not contradict the user's intent and have high advertising metrics. This allows users to reach web pages that match their potential needs, even if their own requests are not clearly defined when the search begins.

[0087] [Effects of this disclosure] According to this disclosure, the following effects can be achieved. (1-1) The search support server 11 inputs a prompt to the LLM16A and obtains a recommended word set. The prompt includes the input word set, the search word set and advertising metric values, and an instruction statement. The instruction statement indicates a recommended word set with a high predicted value for the advertising metric. By proposing a recommended word set that is expected to have high advertising performance in this way, it is possible to provide users with a new search experience that takes into account recent browsing and advertising trends. Furthermore, the suggested word set also reflects the browsing and advertising trends of advertisements placed on other services. Therefore, it is possible to propose appropriate keywords that meet the user's needs. In addition, if the recommended word set is accompanied by a link to transition to a landing page, it is possible to improve the advertising performance metric value.

[0088] (1-2) The search support server 11 inputs the behavioral history data 44 into the LLM 16A, and is able to obtain a set of recommended words that match the user's behavioral tendencies. (1-3) The search support server 11 uses the recommended word set selected by the user on the search results screen 50 as the input word set and further acquires recommended word sets. As a result, the search support server 11 can update the recommended word set to better match the user's requests. In addition, the advertising server 14 displays advertisements corresponding to the recommended word set on the user terminal 20. As a result, it is possible to improve the advertising performance indicator values.

[0089] (1-4) The search support server 11 selects a recommended word set that is included as a search word set in the advertising performance DB 31 and is associated with a high advertising metric value, and sends it to the user terminal 20. As a result, the advertising metric value can be improved through searches using the recommended word set.

[0090] (1-5) The search support server 11 sends the recommended word set, which is included as a search word set in the user management DB 32, to the user terminal 20. As a result, by repeatedly performing searches, advertisements that match the user's behavioral tendencies are displayed. Therefore, the advertising metrics can be improved.

[0091] (1-6) If the recommended word set is not included as a search word set in the advertising performance DB 31, the search support server 11 sends a similar word set to the input word set that has a high associated advertising metric value to the user terminal 20. Therefore, the advertising metric value can be improved through searches using the recommended word set.

[0092] (Second Embodiment) A second embodiment of the search extension generation system, search extension generation method, and search extension generation program will be described. In the second embodiment, a method for outputting the search results screen as a chat screen will be described.

[0093] The user terminal 20 logs in to the service provided by the search support server 11 using a user identifier. As shown in Figure 11, the search support server 11 sends web data to the user terminal 20 to display the chat screen 60 as a search results page. The chat screen 60 includes an input box 61, a send button 62 for sending a message to the LLM 16A, a chat area 63 for displaying messages, and an advertisement display area 64. The chat area 63 displays a message object 65, which is the first message, and a message object 66, which is the second message. An advertisement is displayed in the advertisement display area 64.

[0094] When the user terminal 20 detects that the send button 62 has been pressed, it sends the message entered in the input box 61 to the search support server 11. At this time, the user terminal 20 may send the message to the search support server 11 using WebSocket or an HTTP POST request. The user terminal 20 also converts the entered message into a message object. The message object 65 is displayed in the chat area 63.

[0095] The search support server 11 analyzes the received message and converts it into an input word set. The search support server 11 sends the input word set to the search server 13 and the advertising server 14. Then, the search support server 11 executes steps S1 to S6 described above. The LLM 16A generates a response sentence in addition to the recommended word set. In step S6, the LLM 16A sends the response sentence generated by the LLM 16A in addition to the recommended word set.

[0096] The user terminal 20 converts the acquired recommended word set and response data into a message object. Then, it displays the message object 66 in the chat area 63. The user terminal 20 also displays the advertisement selected by the advertisement server 14 based on the input word set in the advertisement display area 64.

[0097] In the second embodiment, in addition to the effects described in (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (2-1) The chat screen 60 includes a message object 65 which is the first message entered by the user, and a message object 66 which is the second message output by LLM16A. This allows the search support server 11 to narrow down the recommended word set in an interactive manner.

[0098] (Third embodiment) A third embodiment of a search extension generation system, a search extension generation method, a search extension generation program, and a method for generating a natural language processing model will be described. In the third embodiment, a natural language processing model is generated, and a recommended word set is generated using the natural language processing model.

[0099] As shown in Figure 12, in this embodiment, the search support server 11 stores the Natural Language Model 115 in memory 11B. The Natural Language Model 115 may also be stored in another server capable of sending and receiving data with the search support server 11. Hereinafter, the Natural Language Model 115 will be referred to as NLM115.

[0100] This section describes the procedure for training NLM115. NLM115 uses advertising performance data 40A and behavioral history data 44 as training data. The search extension generation system 10 prepares advertising performance data 40A and behavioral history data 44. The behavioral history data 44 includes the history of multiple searches performed using the user's account. The search support server 11 also pre-stores a natural language processing pre-training model 113, a tokenizer 114, and libraries (not shown). The pre-training model 113 is a machine learning model that has been trained in advance using a large dataset. The pre-training model 113 is also called a pre-trained model.

[0101] The tokenizer 114 converts the advertising performance data 40A and the behavioral history data 44 into a format that the pre-training model 113 can learn from. For example, the tokenizer 114 tokenizes the advertising performance data 40A and the behavioral history data 44.

[0102] The search support server 11 inputs tokenized advertising performance data 40A into the pre-training model 113 to learn the relationship between search word sets and advertising performance values. The search support server 11 also inputs tokenized behavioral history data 44 to learn user behavioral trends. At this time, the behavioral trends of one user may be pre-trained, or the behavioral trends of multiple users may be pre-trained. The search support server 11 stores the pre-training model 113, once training is complete, as NLM115. The search support server 11 uses the stored NLM115 in the same way as LLM16A in the first embodiment.

[0103] In the third embodiment, in addition to the effects described in (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (3-1) According to the third embodiment, an NLM115 can be obtained that has learned the relationship between the search word set and advertising performance values, as well as the user's behavioral tendencies.

[0104] [Example of changes] This embodiment can be implemented with the following modifications. This embodiment and the following modifications can be combined with each other to the extent that they do not contradict each other technically.

[0105] (User activity history) [Example of change 1] In the first embodiment, the search support server 11 input advertising performance data 40A and behavioral history data 44 into the LLM 16A, but the search support server 11 only needs to input at least the advertising performance data 40A. In this embodiment as well, the LLM server 16 can learn the relationship between the search word set and the advertising metric value, so it can generate a recommended word set that is expected to have a high advertising metric value.

[0106] [Example of change 2] In the first embodiment, the behavioral history data 44 included search history data 41A, browsing history data 42A, and purchase history data 43A. The behavioral history data 44 only needs to include at least one of the search history data 41A, browsing history data 42A, and purchase history data 43A.

[0107] (Generating a prompt) [Example of change 3] In each of the above embodiments, the search support server 11 generates the prompt, but the search server 13 may generate the prompt. Alternatively, the user terminal 20 may generate the prompt by executing a program and send it to the LLM server 16.

[0108] (Natural language processing model) [Example of change 4] In the first embodiment, a large-scale language model 16A was used as the natural language processing model. Alternatively, a small-scale language model (SLM) may be used in addition to this. A small-scale language model has fewer parameters than LLM16A.

[0109] (Retrieve recommended word set) [Example of change 5] In each of the above embodiments, the advertising server 14 extracts advertising performance data 40A related to the input word set. Alternatively, the search support server 11 may perform at least part of this process. For example, the search support server 11 may identify similar word sets and send them to the advertising server 14. The advertising server 14 determines whether the received similar word sets are stored in the advertising performance DB 31.

[0110] [Example of change 6] In each of the above embodiments, the search support server 11 is configured to obtain the recommended word set. Alternatively, the user terminal 20 may obtain the recommended word set from the LLM server 16. In this case, for example, the LLM server 16 uses a plugin or API (Application Programming Interface) to utilize the functions of at least one of the advertising server 14 and the user management server 15 to send and receive data with those servers 14 and 15. For example, the LLM server 16 may have plugins for the advertising server 14 and the user management server 15 installed. Alternatively, the program stored in the memory of the LLM server 16 may have an API embedded to communicate with the advertising server 14 and the user management server 15, or the LLM server 16 may use a Web API. When the user terminal 20 obtains a search query, it sends a request for the recommended word set to the LLM server 16 along with the search query. Upon receiving the request for the recommended word set, the LLM server 16 obtains advertising performance data 40A corresponding to the input word set from the advertising server 14. Furthermore, the LLM server 16 obtains user-specific behavioral history data 44 from the user management server 15. The LLM server 16 generates a recommended word set and sends it to the user terminal 20. The user terminal 20 renders the recommended word set and advertisements obtained from the ad server 14 to display the search results screens 50 and 51.

[0111] [Example of change 7] In each of the above embodiments, the search support server 11 is configured to obtain the recommended word set. Alternatively, the search server 13 may obtain the recommended word set from the LLM server 16. In this case, the search server 13 sends a request for the recommended word set to the LLM server 16 along with the input word set. Upon receiving the request for the recommended word set, the LLM server 16 obtains advertising performance data 40A corresponding to the input word set from the advertising server 14. The LLM server 16 also obtains user behavior history data 44 corresponding to the user from the user management server 15. The LLM server 16 generates the recommended word set and sends it to the search server 13.

[0112] [Example of change 8] In each of the above embodiments, the search support server 11 is configured to obtain the recommended word set. Alternatively, the advertising server 14 may obtain the recommended word set from the LLM server 16. In this case, the advertising server 14 sends a request for the recommended word set along with the input word set to the LLM server 16. Upon receiving the request for the recommended word set, the LLM server 16 obtains the advertising performance data 40A corresponding to the input word set from the advertising server 14. The LLM server 16 also obtains the user's behavior history data 44 from the user management server 15. The LLM server 16 generates the recommended word set and sends it to the advertising server 14.

[0113] [Example of change 9] In each of the above embodiments, the search support server 11 may input a prompt 49 to the LLM 16A that includes behavioral history data 44 but does not include advertising performance data 40A. Alternatively, the search support server 11 may input a prompt 49 to the LLM 16A that includes advertising performance data 40A but does not include behavioral history data 44. In other words, the search support server 11 obtains both a recommended word set based on behavioral history data 44 and a recommended word set based on advertising performance data 40A. These may be different or partially overlapping. The user terminal 20 may display both of these recommended word sets. Alternatively, the user terminal 20 may display these recommended word sets in separate areas.

[0114] (Refine the recommended word set) [Example of change 10] In the first embodiment, the search support server 11 determined whether the recommended word set obtained from LLM16A was included in the user management DB32 if it was not included as a search word set in the advertising performance DB31 (step S5-1: NO) (step S5-4). Alternatively, the search support server 11 may determine whether the recommended word set is included in the user management DB32 without determining whether the recommended word set is included as a search word set in the advertising performance DB31 (step S5-4).

[0115] (Display of search-linked ads) [Example of change 11] In each of the above embodiments, the search results screen 50 includes advertising content 50D related to the input word set and a recommended word set. In other words, the advertising content 50D is not selected using the recommended word set. Alternatively, or in addition to this, the search results screen 50 may include a recommended word set and advertising content 50D related to said recommended word set. In this embodiment, the display area 50A may display advertising content 50D that aligns with the user's potential needs, thereby enhancing the appeal of the advertising content 50D.

[0116] (search) [Example of change 12] In each of the above embodiments, the user terminal 20 sends a search request to the search server 13 along with the search query and user identifier. Alternatively, or in addition to this, the user terminal 20 may send a search request to the search server 13 in a non-logged-in state that does not recognize a specific user account. The search server 13 accepts non-logged-in searches. In this case, the search support server 11 inputs a prompt 49 into the LLM 16A that includes advertising performance data 40A but does not include behavioral history data 44. Alternatively, the search support server 11 may use search history data 41A that does not specify a user. In this embodiment, the user management server 15 creates a search history table 41 for each user, as well as a search history table 41 that does not specify a user. The latter search history table 41 records the most recently searched set of search words, the number of searches for each set of search words, etc. The search support server 11 inputs the most recently searched set of search words, or the set of search words with a high number of searches and their search counts, into the LLM 16A as search history data 41A. LLM16A learns search trends from search history data 41A.

[0117] [Example of change 13] In each of the embodiments described above, the user terminal 20 was described as logging in to the services provided by the search support server 11 using a user identifier. Alternatively, or in addition to this, the search support server 11 may identify the user identifier by obtaining visit history information from the user terminal 20. Visit history information is information about when the user has previously accessed the service, such as a "cookie".

[0118] [Example of change 14] In the search results screens 50 and 51, the recommended word sets may include links to transition to landing pages. In this case, it is possible to improve the advertising performance metrics.

[0119] <Information Processing System> [Example of change 15] In each of the above embodiments, the search support server 11, search server 13, advertising server 14, and user management server 15 are each separate servers. However, at least two of the search support server 11, search server 13, advertising server 14, and user management server 15 may be the same server. "Same server" means that they are the same as server machines or the same as server functions. The search server 13 and advertising server 14 may be the same server. Also, the search support server 11 and advertising server 14 may be the same server. The search support server 11 and search server 13 may be the same server. Furthermore, the above four servers may be a single server.

[0120] Furthermore, in each of the above embodiments, the search extension generation system 10 is provided with at least a search support server 11, but the search extension generation system 10 may also provide at least one of a search server 13, an advertising server 14, a user management server 15, and an LLM server 16. For example, the search extension generation system 10 may be a system that provides a search support server 11 and a user management server 15. Alternatively, the search extension generation system 10 may be a system that provides a search support server 11, an advertising server 14, and a user management server 15. The search extension generation system 10 may also include a user terminal 20.

[0121] [Example of change 16] In each of the above embodiments, the search support server 11 functioned as a web server and sent the recommended word set to the user terminal 20. Alternatively, the search support server 11 and the web server may be different servers. Different servers mean different server machines or different server functions.

[0122] [Example of change 17] The advertising server 14 may also place advertisements on the website of the online shopping mall. As a result, advertising performance data 40A based on keywords used in the operation of advertising on the online shopping mall, and the advertising cost for each keyword, is accumulated in the advertising performance DB 31.

[0123] In the above embodiments, the advertising server 14 is configured to have the advertising performance DB 31, but other servers may also have the advertising performance DB 31. Furthermore, the server that collects advertising performance and stores it in the advertising performance DB may be managed by a different administrator than the advertising server 14, or the services provided by those servers may be different. For example, the search support server 11 may have the advertising performance DB 31. Alternatively, the search server 13 or the user management server 15 may have the advertising performance DB 31. In addition, the various functions of the advertising server 14 may be distributed among multiple servers. The functions of the advertising server 14 include accepting bids for advertisements, selecting advertisements to display, and aggregating the results of advertisement placement. This allows for an expansion of the scope of data utilization for search-linked advertisements. It also allows for increased efficiency in advertising operations.

[0124] For example, the information processing system 1 may have an advertising performance server (not shown) having an advertising performance DB 31, separate from the advertising server 14. This advertising performance DB 31 stores advertising performance data 40A based on keywords and advertising costs (advertising metrics) for each keyword for advertisements mainly operated in the online shopping mall. For example, the advertising performance server collects the performance of advertisements placed on the website of the online shopping mall. The advertising performance server accepts advertisement submissions from stores in the online shopping mall. Advertisements are submitted with settings such as keywords related to product sales (search word sets in the above embodiment). The advertising server 14 or the advertising performance server places advertisements on the website or other media where the online shopping mall operates, using the set keywords. Users take actions such as clicking on the placed advertisements or purchasing products. The advertising performance server calculates advertising metrics in response to user actions. The advertising performance server also updates the advertising performance data 40A, which includes advertising metrics. Furthermore, the advertising performance server manages advertising costs such as advertising budgets and advertising expenses for advertisements in response to user actions.

[0125] The advertising performance data 40A accumulated in the advertising performance DB31 reflects user trends in online shopping malls, and therefore can be effectively utilized when placing advertisements on the online shopping mall's website.

[0126] Furthermore, the user management server 15 may also manage information about users who use the online shopping mall. [Example of change 18] In the first and second embodiments, the search support server 11 and the LLM server 16 are separate servers. Alternatively, the search support server 11 may be equipped with the LLM 16A. Or, the search server 13, advertising server 14, or user management server 15 may be equipped with the LLM 16A.

[0127] The embodiments and modifications described above are listed below. [A1] Memory that stores multiple instructions, Equipped with at least one processor, The at least one processor executes the plurality of instructions, The method involves obtaining the input word set entered for a search, wherein the input word set includes one or more words entered in a search associated with the user's user account. The acquisition of advertising performance data, wherein the advertising performance data includes a search word set and advertising performance metrics, the search word set is a word set containing one or more words entered in past searches, and the advertising performance metrics are metrics that indicate the performance of ads displayed in conjunction with searches using the search word set. Obtaining a recommended word set generated by inputting a prompt to a large-scale language model, wherein the prompt input to the large-scale language model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction statement, wherein the instruction statement is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value. A search extension generation system configured to transmit data to a user terminal for displaying a search results page, wherein the search results page includes a suggested word set for the user and advertisements related to the input word set.

[0128] [A2] The at least one processor, The system is configured to acquire the user's activity history data, wherein the activity history data includes a history of searches performed using the user account. The search extension generation system according to [A1] or [A2], wherein the prompt further includes the behavioral history data, and the instruction further includes an instruction to generate the recommended word set that matches the user's behavioral tendencies predicted from the behavioral history data.

[0129] [A3] The aforementioned behavioral history data includes keywords from pages previously viewed by the user, as described in [A2], and is part of the search extension generation system.

[0130] [A4] The aforementioned behavioral history data includes keywords for products previously purchased by the user, as described in [A2] or [A3], and is part of the search extension generation system.

[0131] [A5] The search results page includes a chat area containing a first message entered by the user into the user terminal and a second message containing a response from the large-scale language model to the first message and the recommended word set, as described in any one of items [A1] to [A4].

[0132] [A6] The aforementioned at least one processor, The recommended word set included in the search results page is obtained as a new input word set, Obtaining a new recommended word set generated by the large-scale language model, wherein the prompt input to the large-scale language model includes the new input word set, the search word set, the advertising performance metric values ​​associated with the search word set, and the instruction statement. A search extension generation system according to any one of items [A1] to [A5], further configured to send data to the user terminal for displaying the search results page containing the new recommended word set.

[0133] [A7] The search extension generation system according to any one of items [A1] to [A6], wherein obtaining the aforementioned advertising performance data means obtaining the aforementioned advertising performance data that includes at least one word included in the input word set.

[0134] [A8] The search extension generation system according to any one of [A1] to [A7], wherein obtaining the advertising performance data involves searching the database where the advertising performance data is stored for the search word set related to the input word set, before inputting the prompt into the large-scale language model, and obtaining the corresponding advertising performance data to complete the prompt.

[0135] [A9] The search results page includes the recommended word set, and the search extension generation system described in any one of items [A1] to [A8] includes advertising content that transitions to a web page containing the advertised item related to the recommended word set.

[0136] [A10] Obtaining the aforementioned advertising performance data means Identifying a similar word set that includes a word having a common characteristic with at least one word included in the aforementioned input word set, A search extension generation system according to any one of [A1] to [A9], comprising obtaining advertising performance data that includes at least one word from the similar word set in the search word set.

[0137] [A11] The aforementioned at least one processor, The system is further configured to select a predetermined number of recommended word sets to propose to the user from a plurality of recommended word sets obtained from the large-scale language model, Selecting a predetermined number of the aforementioned recommended word sets is, This involves determining whether the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, When it is determined that the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, the advertising performance metric values ​​associated with the aforementioned search word set are obtained, The search extension generation system described in any one of items [A1] to [A10], which involves selecting from among the multiple recommended word sets obtained from the large-scale language model the word set that is included as the search word set in the advertising performance database and has a high advertising performance metric value.

[0138] [A12] The aforementioned at least one processor, The system is configured to acquire the user's activity history data, further including the activity history data a history of searches performed using the user account. Sending data for displaying the search results page to the user terminal means selecting and sending a predetermined number of the recommended word sets from a plurality of recommended word sets obtained from the large-scale language model according to the search history, the search extension generation system according to any one of items [A1] to [A11].

[0139] [A13] The aforementioned at least one processor, The system is further configured to select a predetermined number of recommended word sets to propose to the user from a plurality of recommended word sets obtained from the large-scale language model, Selecting a predetermined number of the aforementioned recommended word sets is, This involves determining whether the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, If it is determined that the aforementioned recommended word set is not included in the advertising performance database as a search word set, then a similar word set that is similar to the aforementioned recommended word set is identified. Obtaining the advertising performance metrics associated with the aforementioned similar word set from the advertising performance database, The search extension generation system described in any one of items [A1] to [A12], which involves selecting the recommended word set with the highest acquired advertising performance metric value from among the multiple recommended word sets obtained from the large-scale language model.

[0140] [A14] The aforementioned advertising performance metric value is a value relating to the cost according to the performance of the advertisement, as described in any one of items [A1] to [A13] of the search extension generation system.

[0141] [A15] A platform that provides advertisements related to the aforementioned input word set includes an advertising performance database that stores the aforementioned advertising performance data, A search extension generation system according to any one of [A1] to [A14], wherein at least one processor constituting the platform stores the advertising performance data for the provided advertisements in the advertising performance database. [Explanation of Symbols]

[0142] 11A, 13A, 14A, 15A, 20A… Processors; 11B, 13B, 14B, 15B, 20B… Memory; 11C, 13C, 14C, 15C, 20C… Communication Interfaces; 40A… Advertising Performance Data; 16A… Large-Scale Language Model (LLM).

Claims

1. Memory that stores multiple instructions, Equipped with at least one processor, The at least one processor executes the plurality of instructions, The method involves obtaining the input word set entered for a search, wherein the input word set includes one or more words entered in a search associated with the user's user account. The acquisition of advertising performance data, wherein the advertising performance data includes a search word set and advertising performance metrics, the search word set is a word set containing one or more words entered in past searches, and the advertising performance metrics are metrics that indicate the performance of advertisements displayed in conjunction with searches using the search word set. Obtaining a recommended word set generated by inputting a prompt to a natural language processing model, wherein the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction sentence, and the instruction sentence is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value. A search extension generation system configured to transmit data to a user terminal for displaying a search results page, wherein the search results page includes a suggested word set for the user and advertisements related to the input word set.

2. The at least one processor, The system is configured to acquire the user's activity history data, wherein the activity history data includes a history of searches performed using the user account. The search extension generation system according to claim 1, wherein the prompt further includes the behavioral history data, and the instruction further includes an instruction to generate the recommended word set that matches the user's behavioral tendencies predicted from the behavioral history data.

3. The search extension generation system according to claim 2, wherein the behavioral history data includes keywords from pages previously viewed by the user.

4. The search extension generation system according to claim 2, wherein the behavioral history data includes keywords for products previously purchased by the user.

5. The search extension generation system according to claim 1, wherein the search results page includes a chat area that includes a first message entered by the user into the user terminal and a second message that includes a response sentence provided by the natural language processing model to the first message and the recommended word set.

6. The aforementioned at least one processor, The recommended word set included in the search results page is obtained as a new input word set, Obtaining a new recommended word set generated by the natural language processing model, wherein the prompt input to the natural language processing model includes the new input word set, the search word set, the advertising performance metric values ​​associated with the search word set, and the instruction sentence. The search extension generation system according to claim 1, further configured to transmit data to the user terminal for displaying the search results page containing the new recommended word set.

7. The search extension generation system according to claim 1, wherein obtaining the advertising performance data means obtaining the advertising performance data that includes at least one word included in the input word set.

8. The search extension generation system according to claim 1, wherein obtaining the advertising performance data involves, before inputting the prompt into the natural language processing model, searching the database storing the advertising performance data for the search word set related to the input word set, obtaining the corresponding advertising performance data, and supplementing the prompt with that data.

9. The search enhancement generation system according to claim 1, wherein the search results page includes advertising content that transitions to a web page containing advertised items related to the recommended word set.

10. Obtaining the aforementioned advertising performance data means Identifying a similar word set that includes a word having a common characteristic with at least one word included in the aforementioned input word set, The search extension generation system according to claim 1, comprising obtaining advertising performance data in which at least one word included in the similar word set is included in the search word set.

11. The aforementioned at least one processor, The system is further configured to select a predetermined number of suggested word sets for the user from a plurality of suggested word sets obtained from the natural language processing model, Selecting a predetermined number of the aforementioned recommended word sets is, This involves determining whether the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, When it is determined that the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, the advertising performance metric values ​​associated with the aforementioned search word set are obtained, The search extension generation system according to claim 1, wherein the system selects from among a plurality of recommended word sets obtained from the natural language processing model a word set that is included in the advertising performance database as a search word set and has a high advertising performance metric value.

12. The aforementioned at least one processor, The system is configured to acquire the user's activity history data, further including the activity history data a history of searches performed using the user account. The search extension generation system according to claim 1, wherein transmitting data for displaying the search results page to the user terminal means selecting and transmitting a predetermined number of recommended word sets from a plurality of recommended word sets obtained from the natural language processing model according to the search history.

13. The aforementioned at least one processor, The system is further configured to select a predetermined number of suggested word sets for the user from a plurality of suggested word sets obtained from the natural language processing model, Selecting a predetermined number of the aforementioned recommended word sets is, This involves determining whether the aforementioned recommended word set is included in the advertising performance database as the aforementioned search word set, If it is determined that the aforementioned recommended word set is not included in the advertising performance database as a search word set, then a similar word set that is similar to the aforementioned recommended word set is identified. Obtaining the advertising performance metrics associated with the aforementioned similar word set from the advertising performance database, The search extension generation system according to claim 1, wherein the system selects from among a plurality of recommended word sets obtained from the natural language processing model that have a high obtained advertising performance metric value.

14. The search extension generation system according to claim 1, wherein the advertising performance metric value is a value relating to the cost according to the performance of the advertisement.

15. A platform that provides advertisements related to the aforementioned input word set includes an advertising performance database that stores the aforementioned advertising performance data, The search extension generation system according to claim 1, wherein at least one processor constituting the platform stores the advertising performance data for the provided advertisements in the advertising performance database.

16. A method that can be executed by at least one processor, The method involves obtaining the input word set entered for a search, wherein the input word set includes one or more words entered in a search associated with the user's user account. The acquisition of advertising performance data, wherein the advertising performance data includes a search word set and advertising performance metrics, the search word set is a word set containing one or more words entered in past searches, and the advertising performance metrics are metrics that indicate the performance of advertisements displayed in conjunction with searches using the search word set. Obtaining a recommended word set generated by inputting a prompt to a natural language processing model, wherein the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction sentence, and the instruction sentence is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value. A search extension generation method comprising sending data to a user terminal for displaying a search results page, wherein the search results page includes a suggested word set for the user and advertisements related to the input word set.

17. At least one processor, The method involves obtaining the input word set entered for a search, wherein the input word set includes one or more words entered in a search associated with the user's user account. The acquisition of advertising performance data, wherein the advertising performance data includes a search word set and advertising performance metrics, the search word set is a word set containing one or more words entered in past searches, and the advertising performance metrics are metrics that indicate the performance of advertisements displayed in conjunction with searches using the search word set. Obtaining a recommended word set generated by inputting a prompt to a natural language processing model, wherein the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance metric value associated with the search word set, and an instruction sentence, and the instruction sentence is a word set related to the input word set and instructs the generation of a recommended word set with a high predicted value of the advertising performance metric value predicted from the relationship between the search word set and the advertising performance metric value. A search extension generating program that causes a user terminal to send data for displaying a search results page, wherein the search results page includes a suggested word set for the user and advertisements related to the input word set.

18. A method for generating a natural language processing model, wherein the natural language processing model is configured to generate one or more recommended word sets when one input word set is input. The above generation method involves at least one processor, The method involves obtaining advertising performance data, wherein the advertising performance data includes a plurality of search word sets and advertising performance metric values ​​related to each of the search word sets, each of which includes at least one word, and each advertising performance metric value is a metric value indicating the performance of an advertisement displayed in conjunction with a search using the corresponding search word set. The acquisition of user activity history data, wherein the activity history data includes the search history of multiple searches performed under the user account. Obtaining a pre-trained model for natural language processing, By inputting the advertising performance data into the pre-training model, the pre-training model learns the relationship between the search word set and the advertising performance indicator values. A method for generating a natural language processing model, comprising inputting the behavioral history data into the pre-training model to allow the pre-training model to learn the user's behavioral tendencies.

Images