Search expansion generation system, search expansion generation method, search expansion generation program, and natural language processing model generation method

Abstract

By suggesting appropriate search keywords that meet the user's needs, we can prevent a decline in advertising performance indicators. [Solution] The search support server 11 accepts an input word set including one or more words entered through a search request and a search linked to the user's user account, obtains advertising performance data including the search word set and an advertising performance index value, inputs a prompt including the input word set, search word set, advertising performance index value associated with the search word set, and instruction text to a natural language processing model to obtain a recommended word set that is predicted to have a high advertising performance index value, and sends data to the user terminal for displaying a search result page including the recommended word set.

Description

[Technical Field]

[0001] The present disclosure relates to a search expansion generation system, a search expansion generation method, a search expansion generation program, and a method for generating a natural language processing model. [Background technology]

[0002] Behavioral targeting using website search history is used as an advertising targeting technique. For example, search-related advertising is a method of displaying advertisements related to a search query entered by a user in a search box on a user's terminal. Patent Document 1 describes the accumulation of conversion queries, which are tokens obtained by performing morphological analysis on a search query. A search server searches for advertisements based on the conversion queries and an advertising database. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2024-004703 Summary of the Invention [Problem to be solved by the invention]

[0004] However, users do not always enter search queries that match their potential needs. If users do not enter appropriate keywords, the search results may not match their needs. For this reason, a new search experience that matches user needs was desired. [Means for solving the problem]

[0005] The present disclosure provides a search expansion generation system that solves the above-mentioned problems, the search expansion generation system including: a memory that stores a plurality of instructions; and at least one processor, the at least one processor executing the plurality of instructions to acquire an input word set input for a search, the input word set including one or more words input in a search associated with a user's user account; acquire advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; and generate a search expansion generation system by inputting a prompt to a natural language processing model. The system is configured to: obtain a recommended word set, wherein a prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction sentence, the instruction sentence instructing the generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; and send data to a user terminal for displaying a search result page, wherein the search result page includes the recommended word set to be proposed to the user, search results for the input word set, and advertisements related to the input word set.

[0006] The present disclosure provides a search expansion generation method for solving the above-mentioned problem, the search expansion generation method being executable by at least one processor, and includes: acquiring a set of input words entered for a search, the input word set including one or more words entered in a search associated with a user's user account; acquiring advertising performance data, the advertising performance data including a set of search words and an advertising performance index value, the search word set being a word set including one or more words entered in a past search, the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; and acquiring a set of recommended words generated by inputting a prompt to a natural language processing model. the prompt input to the natural language processing model includes the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction sentence, the instruction sentence instructing the generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; and sending data to a user terminal for displaying a search result page, the search result page including the recommended word set to be suggested to the user, search results for the input word set, and advertisements related to the input word set.

[0007] The present disclosure provides a search expansion generation program that solves the above-mentioned problems. The search expansion generation program causes at least one processor to execute the plurality of instructions to acquire an input word set input for a search, the input word set including one or more words input in a search associated with a user's user account; acquire advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; and acquire a recommended word set generated by inputting a prompt to a natural language processing model. the natural language processing model acquires a prompt including the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction sentence, the instruction sentence instructing the generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; and transmitting data to the user terminal for displaying a search result page, the search result page including the recommended word set to be suggested to the user, search results for the input word set, and advertisements related to the input word set.

[0008] The present disclosure provides a method for generating a natural language processing model, the natural language processing model being configured to generate one or more recommended word sets when one input word set is input, the method including: acquiring, by at least one processor, advertising performance data including a plurality of search word sets and advertising performance indicator values ​​associated with each of the search word sets, each of the search word sets including at least one word, each of the advertising performance indicator values ​​being an indicator value indicating the performance of an advertisement displayed in conjunction with a search using the corresponding search word set; acquiring user behavior history data including a search history of a plurality of searches performed on a user account; acquiring a natural language processing pre-training model; inputting the advertising performance data into the pre-training model to train the pre-training model on a relationship between the search word sets and the advertising performance indicator values; and inputting the behavior history data into the pre-training model to train the pre-training model on the user's behavioral tendencies. [Effects of the Invention]

[0009] According to the present disclosure, by suggesting appropriate keywords that meet the user's needs, a new search experience can be provided to the user. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram illustrating a schematic configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of an advertising performance table according to the embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a behavior history table according to the embodiment. [Figure 4] FIG. 10 is a diagram showing an example of an advertisement table according to the embodiment. [Figure 5]FIG. 2 is a block diagram of a search support server according to the embodiment. [Figure 6] 10 is a flowchart illustrating a procedure for generating a search extension according to the embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing a procedure for generating a search extension according to the embodiment. [Figure 8] 10 is a flowchart showing a procedure for narrowing down a recommended word set in the embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a search result screen according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a search result screen according to the embodiment. [Figure 11] FIG. 11 is a diagram showing a chat screen as a search result screen in the second embodiment. [Figure 12] FIG. 10 is a diagram illustrating training of a natural language processing model according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Examples of the search expansion generation system, search expansion generation method, and search expansion generation program of the present disclosure will be described below. The present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope of the claims.

[0012] [Outline of information processing system] FIG. 1 shows an example of a search expansion generation system 10. The search expansion generation system 10 supports search-related advertising. The search expansion generation system 10 includes at least a search support server 11. In this embodiment, the search expansion generation system 10 is included in an information processing system 1 that provides search services and search-related advertising. In addition to the search support server 11, the information processing system 1 also includes a search server 13, an advertisement server 14, a user management server 15, a large language model (LLM) server 16, and a user terminal 20. In this embodiment, a large language model is used as the natural language processing model. Hereinafter, the large language model is referred to as an "LLM." The user terminal 20, the search server 13, the search support server 11, the advertisement server 14, the user management server 15, and the LLM server 16 are connected via a network 17.

[0013] The search support server 11 is a server that acquires a recommended word set. The recommended word set is a search word set that is suggested to the user. The recommended word set is a word set that is highly relevant to the search query entered by the user in the search box (search input field).

[0014] A search query may not always accurately reflect a user's intention. For example, a user may not be able to come up with keywords that accurately express the information they are looking for. In particular, if the user does not have sufficient knowledge about the information they are looking for, it may be difficult to select appropriate keywords. Alternatively, even if the user is looking for specific information, they may only be able to come up with keywords that are broader in concept than the information they are looking for, or vice versa. In this embodiment, the recommended word set is a word set that is expected to have high advertising performance among word sets related to the search query. Advertising performance refers to the results obtained by advertising. An advertising performance index value is a numerical representation of the results. Hereinafter, the advertising performance index value will be abbreviated as advertising index value.

[0015] The search support server 11 also functions as a web server. In this embodiment, the search support server 11 transmits to the user terminal 20 a web page including a search box and an advertisement frame for a search-linked advertisement.

[0016] The search support server 11 includes a processor 11A, a memory 11B, and a communication interface 11C. The processor 11A is a processor that executes one or more of the control processes disclosed in this specification. The processor 11A is at least one processing circuit. For example, the processor 11A is a CPU, a GPU, an NPU, or the like. The processor 11A is not limited to a processor that performs software processing for all of the processes it executes. For example, the processor 11A may be equipped with a dedicated hardware circuit (e.g., an application-specific integrated circuit: ASIC) that performs hardware processing for at least some of the processes it executes. In other words, the processor 11A is a circuit that includes one or more processors that operate according to a computer program (software), one or more dedicated hardware circuits that execute at least some of the various processes, or a combination thereof.

[0017] The memory 11B stores the program 110. The program 110 includes at least one instruction. The memory 11B is composed of, for example, a main memory and an auxiliary memory. Examples of the memory 11B include a ROM, a RAM, and a hard disk. The memory 11B includes any available recording medium accessible by a general-purpose or dedicated computer. The memory 11B is connected to the processor 11A via a bus. The program 110 corresponds to a search expansion generation program.

[0018] The communication interface 11C is an interface that establishes a communication path with other devices and executes data transmission and reception via the network 17. The communication interface 11C is configured by hardware, software, or a combination thereof.

[0019] The network 17 may include the Internet or a local area network (LAN). The network 17 may also include a core network (backbone communication network) and multiple base stations. The network 17 may be a third-generation communication system such as W-CDMA or CDMA2000, or a fourth-generation communication system such as LTE, a fourth-generation communication system, or a fifth-generation communication system.

[0020] The search server 13 has a processor 13A, a memory 13B, and a communication interface 13C. These components are similar to those of the search support server 11. Memory 13B stores a crawler 130 and an index 131. Memory 13B also stores a search program (not shown). Crawler 130 automatically searches for web pages on the Internet. Crawler 130 stores detected new web pages in index 131 for the search engine.

[0021] The index 131 stores data about web pages collected by the crawler 130 in an inverted index format. The index 131 has a data structure that includes keywords for referencing documents in which words (tokens) appear, a list of identifiers of documents in which the words appear, and a posting list that stores the positions of the words in the documents.

[0022] The search server 13 executes a search program to analyze the search query and generate an input word set. The input word set includes at least one word (also called a token). The search server 13 also uses the index 131 to search for web pages that include keywords related to the input word set entered by the user. The search server 13 ranks the found web pages and narrows down the search to web pages that are closely related to the input word set. The search engine then transmits a search result page including snippets of the ranked web pages to the user terminal 20 via the search support server 11 or directly.

[0023] The advertisement server 14 acquires information about advertisements from advertiser terminals (not shown). The advertiser terminals are terminals used by advertisers. The advertisement server 14 also delivers advertisements related to the input word set received from the search server 13 to the user terminal 20. The advertisement server 14 also displays advertisements and tracks user behavior regarding the advertisements.

[0024] The advertisement server 14, the search server 13, includes a processor 14A, a memory 14B, and a communication interface 14C. These components are similar to those of the search support server 11. The memory 14B stores an advertisement DB 30 and an advertisement performance DB 31. The memory 14B also stores a program (not shown) for providing advertisements.

[0025] The advertiser submits an advertisement to the advertisement server 14. At this time, the advertiser also creates an advertisement campaign. The advertisement campaign includes target keywords for displaying search-related advertisements, a bid amount, an advertisement period, and a budget to be consumed within the advertisement period. An example of a bid amount is CPC (Cost Per Click).

[0026] The advertisement server 14 distributes advertisements, for example, through RTB (Real-Time Bidding). For example, when the advertisement server 14 receives an advertisement request input word set from the search server 13, it identifies an advertisement campaign corresponding to the search query. Then, the advertisement server 14 determines whether the bid amount of the identified advertisement campaign exceeds a floor price (base price) set for the advertisement medium. Furthermore, for advertisement campaigns that exceed the floor price, the advertisement server 14 selects the advertisement with the highest bid amount from among the submitted advertisements. At this time, the advertisement server 14 may select the advertisement based on the advertisement quality, such as click-through rate. The advertisement server 14 transmits the selected advertisement to the user terminal 20. Note that the advertisement server 14 may receive the search query from the user terminal 20 without going through the search support server 11.

[0027] The advertisement server 14 constitutes a platform for providing advertisements related to the input word set. The advertisement server 14 stores advertisement performance data on the provided advertisements in the advertisement performance DB 31.

[0028] The user management server 15 collects user behavior history data. The user management server 15 includes a processor 15A, a memory 15B, and a communication interface 15C. These components are configured in the same manner as the search support server 11. The memory 15B stores a user management DB 32. The memory 15B also stores a program (not shown) for collecting behavior history data. The user management DB 32 stores user behavior history data for various services. Examples of services include product sales at online shopping malls, financial product or insurance product transactions, online payment services, credit card payment services, distribution of content such as videos, communication services such as mobile phones, online accommodation reservation services, online restaurant reservation services, and online transportation reservation services.

[0029] The LLM server 16 stores the LLM 16A. The LLM 16A is a language model constructed using large amounts of data and deep learning technology. The LLM 16A is a variety of learning models for generative artificial intelligence. The LLM 16A learns acquired documents using generative artificial intelligence. Based on the learned documents, the LLM server 16 also extracts common features between a word set contained in the document and a newly input word set. The LLM server 16 then generates an abstract word set that includes the extracted features, and a more specific word set that also includes the extracted features.

[0030] The user terminal 20 is a device used by a user. An example of the user terminal 20 is a smartphone (multifunction phone) or a personal computer. Other examples of the user terminal 20 are a tablet device, a wearable computer, or any user terminal other than these devices. The user terminal 20 may be a standalone device or a combination of multiple devices connected so that they can send and receive various data to and from each other. The user terminal 20 includes a processor 20A, a memory 20B, a communication interface 20C, an input device 20D, and a display device 20E. The hardware configurations of the processor 20A, the memory 20B, and the communication interface 20C are the same as those of the search support server 11.

[0031] The processor 20A is connected to the input device 20D via an input interface. The input device 20D is a touch panel, a keyboard, a mouse, operation buttons provided on the housing of the user terminal 20, etc. The user terminal 20 may also be equipped with peripheral devices (not shown). The peripheral devices may include a microphone and a speaker. The processor 20A is connected to the display device 20E via a display controller. The display device 20E is a device capable of displaying a screen, such as a liquid crystal display or an organic EL display.

[0032] The memory 20B stores various programs (not shown). For example, the memory 20B stores a program for a hybrid application including a web browser or an in-app browser. For example, the processor 20A acquires a web page written in a markup language from the search support server 11 and displays the web page using the web browser. The display device 20E displays a search result screen based on the web page. Alternatively, the processor 20A may acquire the web page from the search server 13.

[0033] [Data Structure] Next, an example of the structure of the data stored in each of the above-mentioned servers will be described. 2 shows an example of the advertising performance table 40. The advertising performance table 40 is stored in the advertising performance DB 31. The advertising performance table 40 includes a plurality of pieces of advertising performance data 40A. The advertising performance table 40 is a table that accumulates histories based on search queries entered by a plurality of users.

[0034] The advertising performance data 40A is data in which keywords set for an advertisement are recorded in association with advertising index values. Here, the keywords are referred to as search word sets. Advertisers register advertising content and keywords in the advertising server 14. When the advertising server 14 acquires the input word set, it selects advertising content 50D using the advertising performance data 40A and distributes it to the medium where the search or the like was performed. When the user clicks on the advertising content, the user terminal 20 transitions to a conversion site or the like.

[0035] The search word set is generated from past search queries. In this embodiment, the word set refers to the search query divided into words. In addition, to distinguish between the word set used when a search is performed and the word set stored in the advertising performance table 40, the former is referred to as the input word set and the latter is referred to as the search word set.

[0036] The search word set includes at least one word. The ad metric value is a value of a performance metric for an advertisement when the advertisement is displayed on a search result screen in response to a search using the search word set.

[0037] Examples of advertising performance indicators include CPC, CPM (Cost Per Mille), CTR (Click-Through Rate), CPV (Cost Per View), and CPA (Cost Per Action / Acquisition). CPC is the revenue or cost per click when a displayed advertisement is clicked. Revenue is the revenue earned by the advertising medium, and cost is the cost paid by the advertiser. For example, if an advertisement is displayed multiple times in conjunction with one search word set, the advertising indicator value is updated using the newly acquired advertising indicator value. At this time, the average or median of the advertising indicator values ​​may be stored.

[0038] For example, the CPC is registered along with the keyword when each of multiple advertisers submits their ad. If there are multiple different ads with the same keyword, the CPC stored in the advertising performance data 40A may be a statistical value of the CPCs associated with the multiple ads. The statistical value may be an average value, a median value, or the like.

[0039] For example, the CPC may be a cost-per-click calculated by dividing the advertising cost by the actual number of clicks in a given period. The advertising cost is registered in advance along with the content of the ad. If there are multiple ads with the same advertising cost, the CPC may be a statistical value of the CPCs associated with the multiple ads. The statistical value may be an average value, a median value, or the like.

[0040] For example, the CPC may be a cost per click determined in a distribution medium (e.g., a medium capable of introducing items in an online shopping mall). The CPC included in the advertising performance data 40A corresponds to at least one of the above three examples, or may be a statistical value obtained by collecting statistics on at least two of the above examples.

[0041] CPM is the revenue or cost per 1,000 impressions. CTR indicates the ratio of the number of clicks on advertising content divided by the number of times it is displayed. CPV is the revenue or cost when a video that is advertising content is viewed. CPA is the cost incurred when a user takes a specific consumer action in response to advertising content. For example, consumer action includes purchasing a product or service, registering user data, or downloading materials.

[0042] As users repeatedly perform searches, a variety of advertisement performance data 40A is accumulated in the advertisement performance DB 31. Furthermore, when the advertisement index value for one search word set is repeatedly updated, the reliability of the data is increased.

[0043] 3 shows examples of tables 41 to 43 in which user behavior history data 44 is stored. The user management DB 32 stores a search history table 41, a browsing history table 42, and a purchase history table 43. In this embodiment, the search history data 41A, the browsing history data 42A, and the purchase history data 43A are collectively referred to as behavior history data 44.

[0044] The search history table 41 includes at least one search history data 41A. The search history data 41A includes a user identifier, a search word set, the number of searches, the search date and time, and the search frequency. The search word set is a word set obtained by analyzing the search query entered by the user. This word set includes at least one word. The number of searches is the number of searches using the same search word set. The search date and time includes at least the most recent search date and time. The search frequency is the number of searches divided by a specified period. The specified period may be the number of days that have elapsed since the search word set was first entered.

[0045] The browsing history table 42 includes at least one browsing history data 42A. The browsing history data 42A includes a user identifier, a URL of the viewed page, a search word set, the number of times the page was viewed, and the date and time of the view. The search word set is a keyword included in the viewed web page. For example, the viewed page is a page that can be accessed from a link included in the search result page.

[0046] The purchase history table 43 includes at least one purchase history data 43A. The purchase history data 43A includes a user identifier, keywords (search word sets) of products purchased by the user, the number of purchases, and purchase dates and times.

[0047] The user management DB 32 may further include user management data for each user. The user management data may include at least one of the following: gender, age, date of birth, birthplace, place of residence, name, family structure, annual income, and occupation. These data may also be referred to as demographic attributes or demographic data.

[0048] The user management server 15 may collect the behavior history data 44 from the search server 13 or an external server. In this way, various behavior history data 44 is accumulated in the user management server 15 as the user's behavior is repeated. As the behavior history data 44 is accumulated, the accuracy of the user's characteristics determined based on this data also improves.

[0049] FIG. 4 shows an example of the advertisement management table 45. The advertisement management table 45 is stored in the advertisement DB 30. The advertisement management table 45 includes a plurality of advertisement campaign data 45A. The advertisement campaign data 45A includes a campaign identifier, an advertiser identifier, keywords, a bid amount, an advertisement period, and a budget. The campaign identifier is an identifier assigned to a campaign set by an advertiser. The advertiser identifier is an identifier of the advertiser. The keyword is a target keyword used when selecting an advertisement. The keyword is set by the advertiser. The bid amount, advertisement period, and budget are also set by the advertiser, etc. The advertisement campaign data 45A is also associated with a URL of the advertisement content.

[0050] [Function block diagram of search support server] 5 shows a functional block diagram of the search support server 11. The processor 11A of the search support server 11 executes a program 110 to function as a prompt generation unit 111 and an information collection module 112. The information collection module 112 acquires data related to the input word set from the search server 13. The information collection module 112 also acquires advertising performance data 40A. The information collection module 112 also acquires the recommended word set generated by the LLM 16A.

[0051] The prompt generation unit 111 generates a prompt to be input to the LLM 16A. The prompt includes an input word set, a search word set, an advertising index value associated with the search word set, and an instruction statement. The instruction statement includes an instruction for generating a recommended word set having a high advertising index value from among word sets related to the input word set. The advertising index value is predicted from the relationship between the search word set and the advertising performance index value.

[0052] [Steps for generating search extensibility including suggested word sets] A search expansion generation method according to the present disclosure will be described. The search expansion generation method is a method executable by at least one processor 11A. The search expansion generation method includes a method for obtaining a set of suggested words.

[0053] The user terminal 20 uses a user identifier to log in to a service provided by the search support server 11. This service is related to an online shopping mall. For example, the search support server 11 transmits a web page for displaying a screen including a search box. The user terminal 20 uses a web browser to open the web page transmitted from the search support server 11 and displays a search screen including a search box on the display device 20E. Alternatively, if the user terminal 20 launches an application, the user terminal 20 displays an application screen. The application screen includes a search box. Alternatively, the user terminal 20 may log in to a service provided by the search server 13 and display a search screen including a search box on the display device 20E.

[0054] The user enters an arbitrary search query in the search box to execute a search. The user terminal 20 transmits the search request together with the search query and the user identifier to the search server 13. The search server 13 analyzes the search query and extracts a set of input words that serve as keywords. For example, the search server 13 performs morphological analysis to analyze the search query. The search server 13 also searches the index 131 according to a search algorithm. The search server 13 then extracts web pages in which at least one word from the input word set appears. At least one web page is extracted.

[0055] The search server 13 ranks the extracted web pages. The search server 13 transmits search results including the titles, snippets, and URLs of the web pages to the user terminal 20. The snippets include at least one of the titles and abstracts of the pages. The user terminal 20 unfolds the web pages including the search results and displays a search result screen. At this time, the web pages with the highest rankings may be displayed at the top of the screen.

[0056] In addition, the search server 13 transmits the input word set and the user identifier to the search support server 11 and the advertisement server 14. The advertisement server 14 identifies an advertisement campaign that uses the input word set as a target keyword and selects an advertisement as described above. The advertisement server 14 transmits data including a URL from which the selected advertisement content can be obtained to the user terminal 20. At this time, the advertisement server 14 may also transmit advertisement content data to the user terminal 20. The advertisement content data includes image data or video data and a landing page URL.

[0057] FIG. 6 shows the procedure by which the search support server 11 acquires a recommended word set. The search support server 11 acquires an input word set from the search server 13 (step S1). The search support server 11 also transmits the input word set to the advertisement server 14 and requests advertising performance data 40A. Here, the search support server 11 is only required to acquire advertising performance data 40A corresponding to the input word set, and the input word set is transmitted and received between servers as appropriate. For example, when the advertisement server 14 receives the input word set and the user identifier from the search server 13, it may automatically transmit the advertising performance data 40A to the search support server 11. Alternatively, the search server 13 may transmit the input word set and the user identifier only to the search support server 11.

[0058] The advertisement server 14 searches the advertisement performance DB 31 and extracts the advertisement performance data 40A related to the input word set. The advertisement server 14 transmits the extracted advertisement performance data 40A to the search support server 11. The search support server 11 acquires the advertisement performance data 40A (step S2).

[0059] Here, an example will be described in which the advertisement server 14 extracts advertisement performance data 40A related to the input word set. For example, the advertisement server 14 may acquire advertisement performance data 40A that includes at least one word included in the input word set. The advertisement server 14 may also identify words that share common characteristics with at least one word included in the input word set. A word set that includes words that share common characteristics is called a similar word set.

[0060] Then, the advertisement server 14 determines whether or not a word included in the input word set or at least one word included in the similar word set is included as a search word set in the advertisement performance DB 31. If the advertisement server 14 determines that the corresponding word set is included as a search word set in the advertisement performance DB 31, it extracts the advertisement performance data 40A.

[0061] An example of identifying a similar word set will be described. The advertisement server 14 may identify words that belong to the same or similar classification as the classification to which the words included in the input word set belong, from the classifications registered in the advertisement performance DB 31 or the advertisement DB 30. A classification is a classification to which a product belongs, and can also be referred to as a division or type. For example, if the input word set is "mandarin oranges," the advertisement server 14 may identify the word "fruits," which is a larger classification than the classification to which "mandarin oranges" belong, and extract advertisement performance data 40A that includes "fruits" in the search word set. Alternatively, if the input word set is "mandarin oranges," the advertisement server 14 may identify the word "apples," which is another classification included in the large classification of "fruits," and extract advertisement performance data 40A that includes "apples" in the search word set.

[0062] Alternatively, the advertisement server 14 may identify words that are highly similar to words included in the input word set and acquire advertising performance data 40A that includes those words. The similarity between words may be calculated by converting the words into numerical vectors and then calculating the distance or angle between the vectors. In this case, the advertisement server 14 may transmit the input word set to the LLM server 16 to acquire a set of similar words that are similar to the input word set. The advertisement server 14 may then acquire advertising performance data 40A that includes at least one word included in the similar word set. Alternatively, the advertisement server 14 may extract advertising performance data 40A that includes a similar word set when it determines that advertising performance data 40A that includes words included in the input word set is not present in the advertising performance table 40.

[0063] Alternatively, the advertising server 14 may identify other users with the same attributes as the user who input the input word set, and identify keywords set for advertised items displayed to the other users or advertised items on which the other users have taken action. User attributes may be behavioral history attributes or demographic attributes based on the behavioral history data 44. Actions on advertised items may be determined by performance indicators such as the number of clicks or click-through rate, the number of conversions or conversion rate, etc. Furthermore, actions may be determined by the cost per conversion or the amount leading to a conversion. The advertising server 14 sorts the advertised items displayed to other users or the advertised items on which the other users have taken action in order of best advertising index value or highest amount, and acquires keywords associated with a predetermined number of the top advertised items. Furthermore, the keywords and advertising index values ​​may be used instead of the advertising performance data 40A, or advertising performance data 40A including the keywords may be extracted. Furthermore, the similar word sets may be acquired instead of keywords.

[0064] Furthermore, when acquiring the advertising performance data 40A, the advertising server 14 may preferentially acquire data in which newer advertising performance data is stored. This is because word sets with high advertising performance change depending on the time and season. In other words, the advertising performance data 40A in which newer advertising performance data is stored is advertising performance data 40A that reflects the latest advertising trends.

[0065] The search support server 11 acquires the behavior history data 44 from the user management server 15 (step S3). The search support server 11 transmits the user identifier and the input word set. The user management server 15 transmits the behavior history data 44 containing the words in the input word set from among the tables 41 to 43 containing the user identifier to the search support server 11. The behavior history data 44 acquired at this time includes data containing the most recent date.

[0066] The search support server 11 generates a prompt for acquiring a recommended word set (step S4). The search support server 11 also acquires a recommended word set from the LLM server 16 by transmitting the generated prompt to the LLM server 16 (step S5).

[0067] The generation of a prompt (step S4) and the acquisition of a recommended word set (step S5) will be described using FIG. 7. The prompt 49 input to the LLM 16A includes the input word set 31, behavior history data 44, instruction 47, and advertising performance data 40A acquired from the user terminal 20. As described above, the advertising performance data 40A includes a search word set and an advertising index value associated with the search word set. The instruction 47 includes an instruction for generating a recommended word set with a high advertising index value based on the relationship between the search word set and the advertising index value. The instruction 47 also includes an instruction for generating a recommended word set that matches the user's behavioral tendency predicted from the behavior history data 44. The instruction 47 may also include an instruction specifying the number of recommended word sets. Alternatively, the instruction 47 may also include an instruction specifying the minimum number of recommended word sets.

[0068] For example, suppose the input word set 46 includes "mandarin oranges" and the behavioral history data 44 includes "gifts." Furthermore, suppose one piece of advertising performance data 40A includes a search word set of "mandarin oranges" and "luxury" and its CPC, and another piece of advertising performance data 40A includes a search word set of "fruits" and "gift sets" and its CPC. The search support server 11 includes these in a prompt and transmits them to the LLM server 16. The behavioral history data 44 and advertising performance data 40A also include data other than the above examples.

[0069] The LLM 16A learns the characteristics of search word sets that are predicted to have high CPCs from the advertising performance data 40A. The LLM 16A also learns user behavior trends from the behavior history data 44. Based on the learning results, the LLM 16A then generates a recommended word set 48 including, for example, "mandarin oranges" and "gift set" and a recommended word set 48 including "mandarin oranges" and "assortment."

[0070] The search support server 11 may select a predetermined number of recommended word sets from the plurality of acquired recommended word sets. The process of selecting a predetermined number of recommended word sets will be described later. The advertisement server 14 also selects from the advertisement DB 30 advertisement campaign data 45A in which at least one word included in the input word set 46 is included in the target keyword. At this time, the advertisement server 14 may take into consideration the bid amount, advertisement period, budget, etc. The advertisement server 14 also transmits data related to the selected advertisement to the user terminal 20. The data related to the advertisement includes a URL for acquiring the advertisement content 70.

[0071] Returning to FIG. 6 , the search support server 11 transmits the acquired recommended word set 48 to the user terminal 20 (step S6). The user terminal 20 generates a search result screen using the search results received from the search server 13, data related to advertisements received from the advertisement server 14, and the recommended word set received from the search support server 11. Alternatively, the search support server 11 may generate data for displaying the search result screen using the search results, advertisements, and recommended word set, and transmit the data to the user terminal 20. Alternatively, the search support server 11 may transmit the acquired recommended word set 48 to the search server 13. The search server generates data including the search results and recommended word set 48, and transmits the data to the user terminal 20. The search result screen may be generated by client-side rendering, server-side rendering, or any other format.

[0072] [Select recommended word set] Next, the process of selecting a predetermined number of recommended word sets by the search support server 11 in step S5 will be described with reference to Fig. 8. This process is based on the premise that the number of recommended word sets obtained from the LLM server 16 is greater than the number of recommended word sets to be sent to the user terminal 20.

[0073] For example, if there is little training data or the content of the training data is vague, the LLM 16A may output a recommended word set that does not meet the user's needs. For this reason, the search support server 11 selects a word set that is estimated to have a high reliability from the recommended word sets obtained from the LLM server 16.

[0074] The search support server 11 determines whether the recommended word set acquired from the LLM server 16 is included in the advertising performance table 40 (step S5-1). At this time, the search support server 11 inquires of the advertising server 14 whether there is a search word set that exactly matches the recommended word set.

[0075] If the search support server 11 determines that the recommended word set is not included in the advertising performance table 40 (step S5: NO), the process proceeds to step S5-3. If the search support server 11 determines that the recommended word set is included in the advertising performance table 40 (step S5-1: YES), the search support server 11 acquires the advertising index value associated with the recommended word set from the advertising server 14. The search support server 11 selects the recommended word sets acquired from the LLM server 16 in descending order of their advertising index value (step S5-2).

[0076] The search support server 11 determines whether the number of selected recommended word sets is less than a predetermined number (step S5-3). The predetermined number is the number of recommended word sets to be transmitted to the user terminal 20, and is determined in advance.

[0077] If the search support server 11 determines that the number of selected recommended word sets is equal to or greater than a predetermined number (step S5-3: NO), it terminates the process. If the search support server 11 determines that the number of selected recommended word sets is less than a predetermined number (step S5-3: YES), it determines whether the recommended word sets are included in the search history table 41 corresponding to the user who performed the search (step S5-4).

[0078] If the search support server 11 determines that the recommended word set is not included in the search history table 41 (step S5-4: NO), it proceeds to step S5-5. If the search support server 11 determines that the recommended word set is included in the search history table 41 (step S5-4: YES), it extracts the corresponding search history data 41A.

[0079] Then, the search support server 11 selects a recommended word set according to the search situation included in the extracted search history data 41A (step S5-5). For example, the search support server 11 reads out at least one of the number of searches, search date and time, and search frequency of search history data that includes the same search word set as the recommended word set, and selects a recommended word set based on a predetermined algorithm. As an example, the search support server 11 selects a recommended word set in descending order of the most recent search date and time. Alternatively, the search support server 11 may select a recommended word set in descending order of the most frequent searches. Alternatively, the search support server 11 may rank the recommended word sets using an arithmetic expression with the number of searches, search date and time, and search frequency as parameters, and select a recommended word set in descending order of the ranking.

[0080] The search support server 11 determines whether the number of recommended word sets is less than a predetermined number (step S5-6). If the search support server 11 determines that the number of selected recommended word sets is equal to or greater than the predetermined number (step S5-6: NO), it terminates the process. If the search support server 11 determines that the number of selected recommended word sets is less than the predetermined number (step S5-6: YES), it selects a recommended word set using the advertising index value or search situation of a word set similar to the recommended word set (step S5-7).

[0081] Specifically, the search support server 11 identifies a word set similar to the recommended word set. Identifying a similar word set is the same as the above process of extracting advertising performance data 40A related to the input word set. If the search support server 11 determines that a similar word set is included in the advertising performance table 40, it performs a process similar to step S5-2 to select a recommended word set. In addition, the search support server 11 determines whether a similar word set is included in the search history table 41. If the search support server 11 determines that a similar word set is included in the search history table 41, it performs a process similar to step S5-5 to select a recommended word set.

[0082] [Search results screen] 9 shows a main portion of an example of a search result screen 50 displayed on the display device 20E by the user terminal 20. The search result screen 50 includes search result display areas 50A and 50C and a recommended word display area 50B. The search result screen 50 may also include a search box 50E.

[0083] A search query is entered in the search box 50E. Search result display areas 50A and 50C display search results received from the search server 13 and advertising content 50D received from the advertising server 14. For example, display area 50A displays advertising content 50D that is a result of a search using an advertising algorithm. That is, search-related advertisements are displayed in display area 50A. Display area 50C displays advertising content 50D and snippets that are searched using an organic search algorithm other than the advertising algorithm. Furthermore, the display mode of search result display area 50A on the search result screen 50 is determined depending on the medium.

[0084] The recommended word display area 50B includes multiple recommended word sets. When a user clicks on a recommended word set, the user terminal 20 sends a search request to the search server 13 using the clicked word set as an input word set as a search query. The search server 13 performs a search using the received input word set in the manner described above. The search server 13 also sends the input word set to the advertisement server 14 and the search support server 11. The search support server 11 obtains a new recommended word set using the received input word set and sends it to the user terminal 20 (steps S1 to S6). The user terminal 20 regenerates the search result screen 50 using the recommended word set and data related to advertisements received from the advertisement server 14.

[0085] FIG. 10 shows an example of a search result screen 51. The search result screen 51 is displayed after a recommended word set is selected in the display area 50B of the search result screen 50. The search result screen 51 displays the recommended word set selected by the user in the search box 51E. Search results for the new recommended word set are displayed in the search result display areas 51A and 51C. Display area 51A displays advertising content 51D related to the selected recommended word set. FIG. 10 includes advertising content 51D related to the recommended word set and leading to a web page on which an advertised item is posted. For example, display area 51A displays advertising content 51D searched for using an advertising algorithm that uses the recommended word set. Furthermore, display area 51C displays advertising content 51D and snippets of items searched for using an organic search algorithm that uses the recommended word set. The advertising content 51D associated with an item can also be referred to as a description image or thumbnail image of the item. This advertising content 51D has a high advertising index value. Therefore, it is expected to have a high appeal to users who performed searches.

[0086] The recommended word sets displayed in display area 51B may be more specific than the recommended word sets displayed in display area 50B, or may contain a larger number of words. As a result, the recommended word sets are gradually narrowed down to those that do not contradict the user's intentions and have a high advertising index value. This allows the user to arrive at a web page that matches their potential needs, even if they start a search without clearly defining their own needs.

[0087] [Effects of this disclosure] According to the present disclosure, the following effects can be achieved. (1-1) The search support server 11 inputs a prompt to the LLM 16A and acquires a recommended word set. The prompt includes an input word set, a search word set, an advertising index value, and an instruction. The instruction indicates a recommended word set with a high predicted advertising index value. By suggesting a recommended word set that is expected to have high advertising performance, it is possible to provide the user with a new search experience that takes into account recent browsing and placement trends. In addition, the suggested recommended word set also reflects the browsing and placement trends of advertisements posted on other services. Therefore, it is possible to suggest appropriate keywords that meet the user's needs. Furthermore, if the recommended word set is provided with a link to transition to a landing page, it is possible to improve the advertising performance index value.

[0088] (1-2) The search support server 11 inputs the behavior history data 44 into the LLM 16A, so that it is possible to acquire a recommended word set that matches the user's behavioral tendencies. (1-3) The search support server 11 uses the recommended word set selected by the user on the search result screen 50 as the input word set and further acquires a recommended word set. This allows the search support server 11 to update the recommended word set to one that better matches the user's needs. Furthermore, the advertising server 14 displays an advertisement corresponding to the recommended word set on the user terminal 20. This allows the advertising performance index value to be improved.

[0089] (1-4) The search support server 11 selects a recommended word set that is included as a search word set in the advertising performance DB 31 and that is associated with a high advertising index value, and transmits the selected recommended word set to the user terminal 20. This allows the advertising index value to be improved through a search using the recommended word set.

[0090] (1-5) The search support server 11 transmits the recommended word set included in the user management DB 32 as a search word set to the user terminal 20. As a result, by repeating searches, advertisements that match the user's behavioral trends are displayed, thereby improving the advertising index value.

[0091] (1-6) When the recommended word set is not included as a search word set in the advertising performance DB 31, the search support server 11 transmits a recommended word set that is similar to the input word set and has a high associated advertising index value to the user terminal 20. This makes it possible to improve the advertising index value through a search using the recommended word set.

[0092] (Second embodiment) A second embodiment of a search expansion generation system, a search expansion generation method, and a search expansion generation program will be described below. In the second embodiment, a search result screen is output as a chat screen.

[0093] The user terminal 20 logs in to the service provided by the search support server 11 using the user identifier. 11, the search support server 11 transmits web data for displaying a chat screen 60 as a search result page to the user terminal 20. The chat screen 60 includes an input box 61, a send button 62 for sending a message to the LLM 16A, a chat area 63 for displaying messages, and an advertisement display area 64. A message object 65 which is a first message and a message object 66 which is a second message are displayed in the chat area 63. An advertisement is displayed in the advertisement display area 64.

[0094] When the user terminal 20 detects an event in which the send button 62 is operated, it transmits the message entered in the input box 61 to the search support server 11. At this time, the user terminal 20 may transmit the message to the search support server 11 using WebSocket or an HTTP POST request. The user terminal 20 also converts the entered message into a message object. The message object 65 is displayed in the chat area 63.

[0095] The search support server 11 analyzes the received message and sets an input word set. The search support server 11 transmits the input word set to the search server 13 and the advertisement server 14. The search support server 11 then executes steps S1 to S6 described above. The LLM 16A generates an answer sentence in addition to the recommended word set. In step S6, the answer sentence generated by the LLM 16A is transmitted in addition to the recommended word set.

[0096] The user terminal 20 converts the acquired data of the recommended word set and the answer sentence into a message object, and then displays the message object 66 in the chat area 63. The user terminal 20 also displays in the advertisement display area 64 an advertisement selected by the advertisement server 14 based on the input word set.

[0097] In the second embodiment, in addition to the effects (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (2-1) The chat screen 60 includes a message object 65 which is a first message input by the user and a message object 66 which is a second message output by the LLM 16A. This allows the search support server 11 to narrow down the recommended word set in an interactive format.

[0098] (Third embodiment) A third embodiment of a search expansion generation system, a search expansion generation method, a search expansion generation program, and a method for generating a natural language processing model will be described below. In the third embodiment, a natural language processing model is generated, and a recommended word set is generated using the natural language processing model.

[0099] 12, in this embodiment, the search support server 11 stores a natural language processing model 115 in memory 11B. The natural language processing model 115 may be stored in a separate server that can transmit and receive data to and from the search support server 11. Hereinafter, the natural language processing model 115 will be referred to as an NLM 115.

[0100] The following describes the procedure for training the NLM 115. The NLM 115 uses the advertising performance data 40A and the behavior history data 44 as learning data. The search expansion generation system 10 prepares advertising performance data 40A and behavior history data 44. The behavior history data 44 includes a history of multiple searches performed using a user's account. The search support server 11 also stores in advance a natural language processing pre-training model 113, a tokenizer 114, and a library (not shown). The pre-training model 113 is a machine learning model that has been trained in advance using a large-scale dataset. The pre-training model 113 is also called a pre-trained model.

[0101] The tokenizer 114 converts the advertisement performance data 40A and the behavior history data 44 into a format that can be learned by the pre-training model 113. For example, the tokenizer 114 tokenizes the advertisement performance data 40A and the behavior history data 44.

[0102] The search support server 11 inputs tokenized data of the advertising performance data 40A into the pre-training model 113, causing it to learn the relationship between search word sets and advertising performance values. The search support server 11 also inputs tokenized data of the behavior history data 44, causing it to learn user behavioral tendencies. At this time, the behavioral tendencies of one user may be learned in advance, or the behavioral tendencies of multiple users may be learned in advance. The search support server 11 stores the pre-training model 113 for which training has been completed as the NLM 115. The search support server 11 uses the stored NLM 115 in the same way as the LLM 16A of the first embodiment.

[0103] In the third embodiment, in addition to the effects (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (3-1) According to the third embodiment, it is possible to obtain the NLM 115 that has learned the relationship between search word sets and advertising performance values, and the behavioral trends of users.

[0104] [Example of change] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility.

[0105] (User behavior history) [Change Example 1] In the first embodiment, the search support server 11 inputs the advertising performance data 40A and the behavior history data 44 to the LLM 16A, but it is sufficient for the search support server 11 to input at least the advertising performance data 40A. In this embodiment, the LLM server 16 can also learn the relationship between the search word set and the advertising index value, and can therefore generate a recommended word set that is expected to have a high advertising index value.

[0106] [Change Example 2] In the first embodiment, the behavior history data 44 includes search history data 41A, browsing history data 42A, and purchase history data 43A. The behavior history data 44 may include at least one of the search history data 41A, browsing history data 42A, and purchase history data 43A.

[0107] (Prompt generation) [Change Example 3] In each of the above embodiments, the search support server 11 generates the prompt, but the search server 13 may generate the prompt. Alternatively, the user terminal 20 may execute a program to generate a prompt and transmit it to the LLM server 16.

[0108] (Natural Language Processing Model) [Change Example 4] In the first embodiment, the large-scale language model 16A is used as the natural language processing model. Instead of or in addition to this, a small language model (SLM) may be used. The small language model is a model with fewer parameters than the LLM 16A.

[0109] (Get suggested word sets) [Change Example 5] In each of the above embodiments, the advertisement server 14 extracts the advertisement performance data 40A related to the input word set. Alternatively, the search support server 11 may perform at least a part of this process. For example, the search support server 11 may identify a similar word set and transmit the similar word set to the advertisement server 14. The advertisement server 14 determines whether the received similar word set is stored in the advertisement performance DB 31.

[0110] [Change Example 6] In each of the above embodiments, the search support server 11 acquires the recommended word set. Alternatively, the user terminal 20 may acquire the recommended word set from the LLM server 16. In this case, for example, the LLM server 16 transmits and receives data between the advertisement server 14 and the user management server 15 using a plug-in or an API (Application Programming Interface) for utilizing the functions of at least one of the advertisement server 14 and the user management server 15. For example, a plug-in for the advertisement server 14 and the user management server 15 may be installed in the LLM server 16. Alternatively, a program stored in the memory of the LLM server 16 may be embedded with an API for linking with the advertisement server 14 and the user management server 15, or the LLM server 16 may use a WebAPI. When the user terminal 20 acquires a search query, it sends a request for a recommended word set along with the search query to the LLM server 16. Upon receiving the request for the recommended word set, the LLM server 16 acquires advertising performance data 40A corresponding to the input word set from the advertisement server 14. The LLM server 16 also acquires behavioral history data 44 corresponding to the user from the user management server 15. The LLM server 16 generates a recommended word set and transmits it to the user terminal 20. The user terminal 20 renders the recommended word set and the advertisement acquired from the advertisement server 14, and displays search result screens 50 and 51.

[0111] [Change Example 7] In each of the above embodiments, the search support server 11 is configured to acquire the recommended word set. Alternatively, the search server 13 may acquire the recommended word set from the LLM server 16. In this case, the search server 13 sends a request for the recommended word set to the LLM server 16 together with the input word set. Upon receiving the request for the recommended word set, the LLM server 16 acquires advertising performance data 40A corresponding to the input word set from the advertising server 14. The LLM server 16 also acquires behavior history data 44 corresponding to the user from the user management server 15. The LLM server 16 generates a recommended word set and sends it to the search server 13.

[0112] [Change Example 8] In each of the above embodiments, the search support server 11 is configured to acquire the recommended word set. Alternatively, the advertising server 14 may acquire the recommended word set from the LLM server 16. In this case, the advertising server 14 sends a request for the recommended word set to the LLM server 16 together with the input word set. Upon receiving the request for the recommended word set, the LLM server 16 acquires advertising performance data 40A corresponding to the input word set from the advertising server 14. The LLM server 16 also acquires behavior history data 44 corresponding to the user from the user management server 15. The LLM server 16 generates a recommended word set and transmits it to the advertising server 14.

[0113] [Change Example 9] In each of the above embodiments, the search support server 11 may input a prompt 49 that includes the behavior history data 44 but does not include the advertising performance data 40A to the LLM 16A. Alternatively, the search support server 11 may input a prompt 49 that includes the advertising performance data 40A but does not include the behavior history data 44 to the LLM 16A. In other words, the search support server 11 acquires both a recommended word set based on the behavior history data 44 and a recommended word set based on the advertising performance data 40A. These may be different or may partially overlap. The user terminal 20 may display both of these recommended word sets. Alternatively, the user terminal 20 may display these recommended word sets in separate areas.

[0114] (Narrowing down recommended word sets) [Change Example 10] In the first embodiment, when the recommended word set acquired from the LLM 16A is not included as a search word set in the advertising performance DB 31 (step S5-1: NO), the search support server 11 determines whether the recommended word set is included in the user management DB 32 (step S5-4). Alternatively, the search support server 11 may determine whether the recommended word set is included in the user management DB 32 (step S5-4) without determining whether the recommended word set is included as a search word set in the advertising performance DB 31.

[0115] (Display of search-related ads) [Change Example 11] In the above embodiments, the search result screen 50 includes advertising content 50D related to the input word set and a recommended word set. In other words, the advertising content 50D is not selected using the recommended word set. Alternatively or in addition, the search result screen 50 may include the recommended word set and advertising content 50D related to the recommended word set. In this embodiment, the display area 50A may display advertising content 50D that is in line with the user's potential needs, thereby increasing the appeal of the advertising content 50D.

[0116] (search) [Change Example 12] In each of the above embodiments, the user terminal 20 transmits a search request to the search server 13 along with a search query and a user identifier. Alternatively or additionally, the user terminal 20 may transmit a search request to the search server 13 in a non-logged-in state where a specific user account is not recognized. The search server 13 accepts searches in a non-logged-in state. In this case, the search support server 11 inputs a prompt 49 including advertising performance data 40A but not behavioral history data 44 to the LLM 16A. Alternatively, the search support server 11 may use search history data 41A that does not specify a user. In this embodiment, the user management server 15 creates a search history table 41 that does not specify a user in addition to a search history table 41 for each user. The latter search history table 41 records the most recently searched search word set, the number of searches for each search word set, etc. The search support server 11 inputs the most recently searched search word set or frequently searched search word sets and the number of searches for each search word set to the LLM 16A as search history data 41A. The LLM 16A learns search trends from the search history data 41A.

[0117] [Change Example 13] In the above embodiments, the user terminal 20 uses a user identifier to log in to a service provided by the search support server 11. Alternatively or additionally, the search support server 11 may identify the user identifier by acquiring visit history information from the user terminal 20. The visit history information is information about when a user has accessed a service in the past, such as a "cookie."

[0118] [Change Example 14] A link for transitioning to a landing page may be added to the recommended word set on the search result screens 50 and 51. In this case, it is possible to improve the advertising performance index value.

[0119] <Information Processing System> [Change Example 15] In each of the above embodiments, the search support server 11, search server 13, advertisement server 14, and user management server 15 are each separate servers. However, at least two of the search support server 11, search server 13, advertisement server 14, and user management server 15 may be the same server. The same server means that the server machine is the same, or the server functions are the same. The search server 13 and advertisement server 14 may be the same server. Furthermore, the search support server 11 and advertisement server 14 may be the same server. The search support server 11 and search server 13 may be the same server. Furthermore, the above four servers may be a single server.

[0120] Furthermore, in each of the above embodiments, the search expansion generation system 10 includes at least the search support server 11. However, the search expansion generation system 10 may also include at least one of the search server 13, the advertisement server 14, the user management server 15, and the LLM server 16. For example, the search expansion generation system 10 may be a system including the search support server 11 and the user management server 15. Alternatively, the search expansion generation system 10 may be a system including the search support server 11, the advertisement server 14, and the user management server 15. Furthermore, the search expansion generation system 10 may also include a user terminal 20.

[0121] [Change Example 16] In each of the above embodiments, the search support server 11 functions as a web server and transmits the recommended word set to the user terminal 20. Alternatively, the search support server 11 and the web server may be different servers. Different servers refer to different server machines or different server functions.

[0122] [Change Example 17] The advertisement server 14 may be a server that posts advertisements on the website of an online shopping mall. As a result, keywords used in the advertisement management of the online shopping mall and advertisement performance data 40A based on the advertisement costs for each keyword are accumulated in the advertisement performance DB 31.

[0123] In each of the above embodiments, the advertisement server 14 is configured to include the advertisement performance DB 31, but another server may also include the advertisement performance DB 31. Furthermore, the server that collects advertisement performance and stores it in the advertisement performance DB may be managed by a different administrator than the advertisement server 14, or the services provided by these servers may be different. For example, the search support server 11 may include the advertisement performance DB 31. Alternatively, the search server 13 or the user management server 15 may include the advertisement performance DB 31. Furthermore, the various functions of the advertisement server 14 may be distributed among multiple servers. The functions of the advertisement server 14 include accepting advertisement bids, selecting advertisements to be displayed, and aggregating the results of the advertisement display. This can expand the scope of data usage for search-based advertisements. Furthermore, it can also improve the efficiency of advertising operations.

[0124] For example, the information processing system 1 may include an advertisement result server (not shown) having an advertisement result DB 31, separate from the advertisement server 14. The advertisement result DB 31 stores advertisement result data 40A based on keywords and the advertising costs (advertising index values) for each keyword in advertisements mainly run in the online shopping mall. For example, the advertisement result server collects the results of advertisements posted on the website of the online shopping mall. The advertisement result server accepts advertisement submissions from stores in the online shopping mall. The advertisements are submitted along with keywords related to product sales (search word sets in the above embodiment). The advertisement server 14 or the advertisement result server uses the set keywords to post advertisements on websites, etc., which are media on which the online shopping mall is operated. Users take actions, such as clicking on posted advertisements or purchasing products, in response to the posted advertisements. The advertisement result server calculates advertisement index values ​​according to user actions. The advertisement result server also updates the advertisement result data 40A, including the advertisement index values. The advertisement result server also manages advertising costs, such as advertising budgets and advertising expenses, according to user actions.

[0125] The advertising performance data 40A stored in the advertising performance DB 31 reflects user trends in the online shopping mall, and therefore the advertising performance data 40A can be effectively utilized for placing advertisements on the online shopping mall website.

[0126] The user management server 15 may also manage information about users who use the online shopping mall. [Change Example 18] In the first and second embodiments, the search support server 11 and the LLM server 16 are separate servers. Alternatively, the search support server 11 may include the LLM 16A. Alternatively, the search server 13, the advertisement server 14, or the user management server 15 may include the LLM 16A.

[0127] The following are some aspects that can be understood based on the above-described embodiment and modifications. [A1] a memory for storing a plurality of instructions; at least one processor; the at least one processor executing the instructions, obtaining a set of input words entered for a search, the set of input words including one or more words entered through a search associated with the user's user account; Acquiring advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; Obtaining a recommended word set generated by inputting a prompt into a large-scale language model, wherein the prompt input into the large-scale language model includes the input word set, the search word set, the advertising performance indicator value associated with the search word set, and an instruction statement, and the instruction statement instructs generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance indicator value predicted from the relationship between the search word set and the advertising performance indicator value; A search expansion generation system configured to: send data to a user terminal for displaying a search result page, the search result page including the recommended word set for suggestion to the user and advertisements related to the input word set.

[0128] [A2] the at least one processor: further configured to obtain behavioral history data of the user, the behavioral history data including a history of searches performed on the user account; The search expansion generation system described in [A1] or [A2], wherein the prompt further includes the behavioral history data, and the instruction sentence further includes an instruction to generate the recommended word set that matches the user's behavioral tendencies predicted from the behavioral history data.

[0129] [A3] The search expansion generation system according to [A2], wherein the behavioral history data includes keywords of pages previously viewed by the user.

[0130] [A4] The search expansion generation system according to [A2] or [A3], wherein the behavioral history data includes keywords for products purchased in the past by the user.

[0131] [A5] The search result page includes a chat area including a first message input by the user to the user terminal and a second message including a response sentence to the first message from the large-scale language model and the recommended word set.

[0132] [A6] the at least one processor: acquiring the recommended word set included in the search result page as a new input word set; obtaining the new set of recommended words generated by the large-scale language model, wherein the prompts input to the large-scale language model include the new set of input words, the search word set, the advertising performance metric value associated with the search word set, and the instruction sentence; A search expansion generation system described in any one of [A1] to [A5], further configured to send data to the user terminal for displaying the search result page including the new set of recommended words.

[0133] [A7] The search expansion generation system described in any one of [A1] to [A6], wherein acquiring the advertising performance data is acquiring the advertising performance data that includes at least one word included in the input word set.

[0134] [A8] The search expansion generation system according to any one of [A1] to [A7], wherein obtaining the advertising performance data comprises searching a database that stores the advertising performance data for advertising performance data that includes the search word set related to the input word set before inputting the prompt into the large-scale language model, and obtaining the corresponding advertising performance data to complement the prompt.

[0135] [A9] The search result page includes the recommended word set and advertising content related to the recommended word set that transitions to a web page containing an advertised item.

[0136] [A10] The acquiring of the advertising performance data includes: identifying a similar word set that includes words that share a common characteristic with at least one word in the input word set; The search expansion generation system according to any one of [A1] to [A9], further comprising obtaining the advertising performance data in which the search word set includes at least one word included in the similar word set.

[0137] [A11] the at least one processor: further configured to select a predetermined number of the recommendation word sets for suggesting to the user from a plurality of the recommendation word sets obtained from the large-scale language model; Selecting a predetermined number of the suggested word sets includes: determining whether the recommended word set is included as the search word set in an advertisement performance database; When it is determined that the recommended word set is included as the search word set in the advertising performance database, acquiring the advertising performance indicator value associated with the search word set; The search expansion generation system according to any one of [A1] to [A10], wherein a word set that is included as the search word set in the advertising performance database and has a high advertising performance index value is selected from the plurality of recommended word sets obtained from the large-scale language model.

[0138] [A12] the at least one processor: further configured to obtain behavioral history data of the user, the behavioral history data including a history of searches performed on the user account; The search expansion generation system described in any one of [A1] to [A11], wherein sending data to the user terminal for displaying the search result page comprises selecting and sending a predetermined number of the recommended word sets from the multiple recommended word sets obtained from the large-scale language model according to the search history.

[0139] [A13] the at least one processor: further configured to select a predetermined number of the recommendation word sets for suggesting to the user from a plurality of the recommendation word sets obtained from the large-scale language model; Selecting a predetermined number of the suggested word sets includes: determining whether the recommended word set is included as the search word set in an advertisement performance database; When it is determined that the recommended word set is not included in the advertisement performance database as the search word set, identifying a similar word set that is similar to the recommended word set; obtaining the advertising performance indicator values ​​associated with the similar word sets from the advertising performance database; The search expansion generation system described in any one of [A1] to [A12], wherein the recommended word set having the highest advertising performance index value is selected from the plurality of recommended word sets obtained from the large-scale language model.

[0140] [A14] The search expansion generation system according to any one of [A1] to [A13], wherein the advertisement performance index value is a value relating to a cost depending on advertisement performance.

[0141] [A15] a platform that provides advertisements related to the input word set, the platform comprising an advertisement performance database that stores the advertisement performance data; The search expansion generation system according to any one of [A1] to [A14], wherein the at least one processor constituting the platform stores the advertising performance data for the provided advertisement in the advertising performance database. [Explanation of symbols]

[0142] 11A, 13A, 14A, 15A, 20A...processor, 11B, 13B, 14B, 15B, 20B...memory, 11C, 13C, 14C, 15C, 20C...communication interface, 40A...advertising performance data, 16A...LLM, a large-scale language model.

Claims

1. a memory for storing a plurality of instructions; at least one processor; the at least one processor executing the instructions, obtaining a set of input words entered for a search, the set of input words including one or more words entered through a search associated with the user's user account; Acquiring advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; Obtaining a recommended word set generated by inputting a prompt into a natural language processing model, wherein the prompt input into the natural language processing model includes the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction statement, and the instruction statement instructs generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; A search expansion generation system configured to: send data to a user terminal for displaying a search result page, the search result page including the recommended word set for suggestion to the user and advertisements related to the input word set.

2. the at least one processor: further configured to obtain behavioral history data of the user, the behavioral history data including a history of searches performed on the user account; The search expansion generation system of claim 1 , wherein the prompt further includes the behavioral history data, and the instruction sentence further includes an instruction to generate the recommended word set that matches the user's behavioral tendencies predicted from the behavioral history data.

3. The search expansion generation system of claim 2 , wherein the behavior history data includes keywords of pages previously viewed by the user.

4. The search expansion and generation system of claim 2 , wherein the behavioral history data includes keywords for products previously purchased by the user.

5. 2. The search expansion generation system of claim 1, wherein the search result page includes a chat area including a first message input by the user to the user terminal and a second message including a response sentence to the first message from the natural language processing model and the recommended word set.

6. the at least one processor: acquiring the recommended word set included in the search result page as a new input word set; obtaining the new set of recommended words generated by the natural language processing model, wherein the prompt input to the natural language processing model includes the new set of input words, the search word set, the advertising performance metric value associated with the search word set, and the instruction sentence; The search expansion generating system of claim 1 , further configured to: send data to the user terminal for displaying the search result page including the new set of recommended words.

7. The search expansion generation system of claim 1 , wherein obtaining the advertising performance data comprises obtaining the advertising performance data that includes at least one word included in the input word set.

8. 2. The search expansion generation system of claim 1, wherein the acquiring of the advertising performance data comprises searching a database that stores the advertising performance data for the advertising performance data including the search word set related to the input word set before inputting the prompt into the natural language processing model, and acquiring the corresponding advertising performance data to complement the prompt.

9. The search expansion generation system of claim 1 , wherein the search result page includes advertising content that is related to the set of recommended words and that transitions to a web page containing an advertised item.

10. The acquiring of the advertising performance data includes: identifying a similar word set including words that share a common characteristic with at least one word in the input word set before inputting the prompt into the natural language processing model; The search expansion generation system of claim 1 , further comprising: obtaining the advertising performance data in which the search word set includes at least one word included in the similar word set.

11. the at least one processor: further configured to select a predetermined number of the recommendation word sets for suggesting to the user from the plurality of recommendation word sets obtained from the natural language processing model; Selecting a predetermined number of the suggested word sets includes: determining whether the recommended word set is included as the search word set in an advertisement performance database; When it is determined that the recommended word set is included as the search word set in the advertising performance database, acquiring the advertising performance indicator value associated with the search word set; 2. The search expansion generation system according to claim 1, wherein a word set that is included as the search word set in the advertising performance database and has a high advertising performance index value is selected from the plurality of recommended word sets obtained from the natural language processing model.

12. the at least one processor: further configured to obtain behavioral history data of the user, the behavioral history data including a history of searches performed on the user account; 2. The search expansion generation system of claim 1, wherein transmitting data to the user terminal for displaying the search result page comprises selecting and transmitting a predetermined number of the recommended word sets from the plurality of recommended word sets obtained from the natural language processing model according to the search history.

13. the at least one processor: further configured to select a predetermined number of the recommendation word sets for suggesting to the user from the plurality of recommendation word sets obtained from the natural language processing model; Selecting a predetermined number of the suggested word sets includes: determining whether the recommended word set is included as the search word set in an advertisement performance database; When it is determined that the recommended word set is not included in the advertisement performance database as the search word set, identifying a similar word set that is similar to the recommended word set; obtaining the advertising performance indicator values ​​associated with the similar word sets from the advertising performance database; The search expansion generation system according to claim 1 , wherein the system is configured to select, from among the plurality of recommendation word sets obtained from the natural language processing model, the recommendation word set having a high obtained advertising performance index value.

14. The search expansion generation system according to claim 1 , wherein the advertisement performance index value is a value related to a cost depending on the performance of the advertisement.

15. a platform that provides advertisements related to the input word set, the platform comprising an advertisement performance database that stores the advertisement performance data; The search expansion generation system of claim 1 , wherein the at least one processor comprising the platform stores the advertising performance data for the served advertisements in the advertising performance database.

16. 2. The system of claim 1, wherein the advertising performance indicator value is one of cost per click (CPC), cost per mille (CPM), cost per view (CPV), and cost per action / acquisition (CPA).

17. The search expansion generation system described in claim 1, wherein the advertising performance index value is set by the advertiser at the time of submitting the advertisement.

18. 1. A method executable by at least one processor, comprising: obtaining a set of input words entered for a search, the set of input words including one or more words entered through a search associated with the user's user account; Acquiring advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; Obtaining a recommended word set generated by inputting a prompt into a natural language processing model, wherein the prompt input into the natural language processing model includes the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction statement, and the instruction statement instructs generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; A search expansion generation method including: sending data to a user terminal for displaying a search result page, the search result page including the recommended word set for suggesting to the user and advertisements related to the input word set.

19. At least one processor obtaining a set of input words entered for a search, the set of input words including one or more words entered through a search associated with the user's user account; Acquiring advertising performance data, the advertising performance data including a search word set and an advertising performance index value, the search word set being a word set including one or more words input in a past search, and the advertising performance index value being an index value indicating the performance of an advertisement displayed in conjunction with a search using the search word set; Obtaining a recommended word set generated by inputting a prompt into a natural language processing model, wherein the prompt input into the natural language processing model includes the input word set, the search word set, the advertising performance index value associated with the search word set, and an instruction statement, and the instruction statement instructs generation of the recommended word set, which is a word set related to the input word set and has a high predicted value of the advertising performance index value predicted from the relationship between the search word set and the advertising performance index value; A search expansion generation program that executes the following: sending data to a user terminal for displaying a search result page, the search result page including the recommended word set to suggest to the user and advertisements related to the input word set.

20. A method for generating a natural language processing model, the natural language processing model being configured to generate one or more recommendation word sets when an input word set is input, The generating method includes at least one processor: Acquiring advertising performance data, the advertising performance data including a plurality of search word sets and a CPC (Cost Per Click) which is an advertising performance index value related to each of the search word sets, each of the search word sets including at least one word, and each of the CPCs being an index value indicating the cost when an advertisement displayed in conjunction with a search using the corresponding search word set is selected; Obtaining behavioral history data of a user, the behavioral history data including a search history of multiple searches performed on the user account; Obtaining a pre-trained natural language processing model; inputting the advertising performance data into the pre-training model to allow the pre-training model to learn the relationship between the search word set and the CPC; inputting the behavioral history data into the pre-training model to allow the pre-training model to learn the behavioral tendencies of the user.

Images