Conditional content selection procedure

Abstract

A system, method, and / or algorithm facilitates efficiently delivering content to client devices to fill content slots of an information resource having multiple content slots. A server receives a request from a client device for content to be included in the information resource and, in response, performs one or more content selection procedures to select one or more content items for one or more content slots. Based on the results of the content selection procedure(s), the server may conditionally perform or not perform one or more additional content selection procedures for one or more other content slots of the information resource. Alternatively, or in addition, the server may apply one or more constraints based on the format or content of one or more content items selected during the initial content selection procedure(s) to limit which content items may be candidates in one or more additional content selection procedures.

Description

Technical Field

[0001] The present disclosure relates to a content selection procedure, and more specifically, to a procedure for selecting content provided to a client device for presentation to a user within a content slot of an information resource.

Background Art

[0002] For the purpose of generally presenting the context of the present disclosure, the background art provided herein is described. The achievements of the inventor(s) listed herein are not explicitly or implicitly recognized as prior art to the present disclosure, either within the scope described in this background art section or in aspects of this document that may not be eligible as prior art at the time of filing and in other cases.

[0003] In some scenarios, such as when a user is visiting a web page or viewing a screen provided by a dedicated application, the user is presented with additional content that is dynamically added within a specific reserved area of the page or screen. For example, a web page may reserve one or more "slots" for third-party content, and the client device may request that the server supply a content item, such as a digital advertisement or other third-party content item, to fill these slots. Typically, the client device sends a request for a given content slot to the server and causes the server to execute a procedure for selecting a specific content item. Next, the server supplies the selected content item to the client device, and the client device fills / fills the slot with the content item. For example, in the case of a digital advertisement, the content selection procedure usually includes an auction. After some pre-filtering of the advertisements (e.g., based on various targeting parameters), bid scores and ranks are determined for the remaining candidates. The winner of the auction is determined based on the rank, and the winner's advertisement is used to fill the content slot.

[0004] When multiple content slots are available to fill a webpage, user interface, or other information resource, various complexities can arise. For example, different slots and different content item formats may be associated with different priorities and / or have dependencies (e.g., conflicts) with each other. As an example in the context of digital advertising, if a sponsored content item (ad) is selected to fill a prominent ad slot (e.g., at the top of the page) in order to enhance or maintain its value, it may be desirable to avoid placing another ad in a nearby slot that could distract and / or annoy the user. In other examples, it may be desirable to avoid filling a single page with too many ads and / or to avoid placing competing ads (e.g., for similar products) in adjacent slots on a page. Therefore, various decisions need to be made regarding whether to select a particular ad, which ad slots should be filled, and / or whether to apply certain constraints to which ads may be candidates in the ad selection process.

[0005] In some existing systems, this decision-making process is carried out by conducting an auction for all available ad slots and then deciding whether to drop a particular ad from among the auction winners. For example, in response to a request from a client device, the server may conduct a first auction for a first slot on a page and a second auction for a second slot on the same page. If the first auction has a certain outcome, such as the first slot being filled (or the first slot being filled with an ad for a particular product or brand), the winning ad for the second slot may be discarded. Otherwise, the winning ad for the second slot is used to fill the second slot. However, this approach has certain technical drawbacks. Firstly, the auction is conducted for all requested slots (e.g., all slots on the page the user is visiting), but only a subset of the winning ads may be delivered to the user's device, which usually results in a waste of processing resources. Specifically, the server's processing resources (e.g., processing time, processing power) are wasted by conducting auctions even for slots that ultimately do not display an ad. Secondly, since ads in some slots may be dropped based on the results / winners of other slots, the "fill rate" of ad slots may decrease, resulting in inefficient delivery of ad content by the server (and therefore fewer impressions and less ad revenue).

[0006] In other systems, the decision-making process is carried out by the client device making multiple remote procedure calls (RPCs) in sequence to the server to fetch advertisements for different slots on the page. Under this approach, the client device waits for and considers the results / responses received from the server, after making the first RPC for the first slot, before making the second RPC for the second slot. This approach also has significant drawbacks. Firstly, because the client device must sequentially wait for results from the server for each RPC / slot, a delay occurs between each request and the response from the server, increasing latency (e.g., web page load time), which can sometimes become frustrating for the user. Secondly, because the client device needs to perform a considerable amount of front-end processing (e.g., deciding whether to make a subsequent RPC based on the results of the previous RPC), the processing load can cause the client device to crash (e.g., the browser or other application crashes), resulting in a poor user experience. This can be particularly problematic when the client device has relatively limited memory, as can happen with smart TV devices, for example. Thirdly, because decision-making takes place on the client device, there is generally less information available on the server, which can hinder the server's ability to make future decisions (e.g., for future content selection) or provide useful analysis or problem-solving feedback (e.g., information showing ad performance or information showing why ads were not shown). [Overview of the project]

[0007] In some embodiments described herein, the server determines whether to perform one or more content selection procedures to fill a content slot(s) of the information resource, and / or the manner in which the content selection procedures(s) are performed, based on the results of one or more previous content selection procedures used to fill other content slots(s) of the information resource.

[0008] In one exemplary embodiment, a method for efficiently supplying content to a client device includes receiving requests from a server and a client device for content to be included in an information resource having multiple content slots, and, in response to the requests, the server selecting one or more content items. Selecting one or more content items includes (1) performing a first content selection procedure using selection parameters associated with at least candidate content items to determine a first content selection result for a first subset of the multiple content slots, and (2) conditionally performing or not performing a second content selection procedure for a second subset of the multiple content slots based on the first content selection result. The method also includes the server causing the client device to fill one or more of the multiple content slots with the selected one or more content items.

[0009] In another exemplary embodiment, a method for efficiently supplying content to a client device includes receiving requests from a server and a client device for content to be included in an information resource having multiple content slots, and, in response to the requests, the server selecting multiple content items. Selecting multiple content items includes (1) performing one or more first content selection procedures using selection parameters associated with at least candidate content items to select a first set of one or more content items for the multiple content slots, and (2) applying one or more constraints in one or more second content selection procedures for a second subset of the multiple content slots, based on the format or content of the first set of content items, to limit which content items may be candidates. The method also includes the server causing the client device to fill at least some of the multiple content slots with the selected multiple content items.

[0010] In some embodiments, the server includes one or more processors and one or more memories, the one or more memories storing instructions that, when executed by one or more processors, cause one or more processors to execute any of the embodiments described above. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram of an exemplary system that can implement technologies for efficiently delivering content to client devices. [Figure 2] The first exemplary information resource is shown, having content slots configured to display content items selected by a content selection procedure. [Figure 3]A second exemplary information resource is shown, having content slots configured to display content items selected by a content selection procedure. [Figure 4] Figure 1 is a diagram of a sequence of messages illustrating an exemplary embodiment in which the content server selects content for the client device shown in Figure 1 and supplies the selected content to the client device. [Figure 5] This is a flowchart illustrating an exemplary method for efficiently delivering content to client devices. [Figure 6] This is a flowchart illustrating another exemplary method for efficiently delivering content to client devices. [Modes for carrying out the invention]

[0012] In general, in the embodiments of the disclosed invention, the server performs a conditional content selection procedure. More specifically, in response to receiving a request for content from a client device, the server performs one or more content selection procedures to fill one or more content slots of an information resource (e.g., a web page or application screen / user interface displayed on the client device), and for one or more additional content slots of the information resource, the server conditions the execution of a content selection procedure(s) on the result of a previous content selection procedure(s).

[0013] In the specific exemplary embodiments and scenarios described herein, the content items used to fill content slots are advertisements, and / or the corresponding content selection procedure is an auction. However, the content items could instead be any other type of content that can be dynamically selected for presentation within the information resource, and the content selection procedure could instead be any other type of procedure suitable for selecting content (e.g., a procedure for selecting content based on the relevance score of content items).

[0014] In one aspect of the present invention, the server conditionally executes or refrains from executing one or more content selection procedures based on the results of one or more previous content selection procedures for the same information resource slot. In some embodiments, for example, the server executes a second content selection procedure only if a first content selection procedure does not result in the selection of any content items in a particular content slot. In other embodiments, the server conditionally executes or refrains from executing a second content selection procedure based on one or more characteristics of the content items selected during the first content selection procedure. For example, the server may use a machine learning model to determine the relevance score of the content items selected during the first content selection procedure, and then execute a second content selection procedure only if the relevance score exceeds some predetermined threshold.

[0015] In another aspect of the present invention, the server conditions how one or more content selection procedures are performed on the results of one or more previous content selection procedures for the same information resource slot. In particular, the server may apply one or more constraints based on the format and / or content of the content items selected during the first content selection procedure, and the constraint(s) may function to limit which content items may be candidates for the second content selection procedure. In various embodiments, for example, the server may exclude content items from the second content selection procedure based on the category or format of those content items (e.g., based on whether the category or format of the content item is the same as the content item resulting from the first content selection procedure), based on user cost, based on the level of similarity between the content item and the content item selected during the first content selection procedure, and / or other factors.

[0016] In either of the two embodiments described above, in some embodiments, the server may sort the content slots of the information resource according to their priority levels before the server performs the first content selection procedure to determine the order in which the server should attempt to fill the content slots. In some embodiments, at least some of the content slots of the information resource may be grouped together, in which case the server may perform the content selection procedures for the content items in a given group at least partially in parallel with one another.

[0017] These and other embodiments can improve the efficiency of content selection procedures (e.g., by using fewer processing resources), reduce latency (e.g., by shortening web page load times), prevent "crashes" on client devices (e.g., when client devices are particularly resource-constrained devices), and / or provide other technical advantages.

[0018] For example, in the first embodiment (where the server conditionally performs or does not perform one or more additional content selection procedures for an information resource), (1) the server uses fewer processing resources than if the server automatically performed content selection procedures for all content slots of a multi-slot information resource (e.g., based on a single request from a client device), (2) latency or delay on the client device is reduced compared to when the client device has to send a series of separate requests to the server for various slots (e.g., when loading a web page containing multiple content slots), and / or (3) the client device is less likely to crash compared to when the client device has to process responses to each of a series of separate requests sent to the server to decide whether to send the next request.

[0019] In the second aspect (a configuration that conditions a server to perform one or more additional content selection procedures based on the results of one or more previous content selection procedures for the same information resource), (1) the content item slot fill rate is typically improved compared to when the server automatically performs content selection procedures for all content slots of a multi-slot information resource (e.g., based on a single request from a client device), (2) latency or delay on the client device is reduced compared to when the client device has to send a series of separate requests to the server for various slots (e.g., when loading a web page containing multiple content slots), and / or (3) the likelihood of crashes on the client device is reduced compared to when the client device has to process responses to each of a series of separate requests sent to the server to determine whether to send the next request.

[0020] Furthermore, in both of these embodiments, more information is generally available to the server compared to the case where a client device makes multiple content requests and analyzes each request result / response against a given slot of information resources. Such information may help the server make more informed decisions (e.g., for future content selection), and / or enable the server to provide useful analytical or problem-solving feedback (e.g., information indicating ad performance, or information indicating why an ad was not delivered).

[0021] FIG. 1 shows an exemplary system 100 in which one or more techniques for efficiently supplying content to a client device 102 may be implemented. The exemplary system 100 includes a client device 102, a content server 104, several publishers 106, several content providers 108, and a network 110. The content server 104 is remote from the client device 102 and communicatively coupled to the client device 102 via the network 110. The communication connections / network connections shown in FIG. 1 for the publishers 106 and the content providers 108 represent communication / network connections with computing devices or systems associated with the publishers 106 and the content providers 108, respectively.

[0022] The network 110 may be a single communication network (e.g., the Internet) and, in some embodiments, also includes one or more additional networks. As just one specific example, the network 110 may include a cellular network, the Internet, and a server-side local area network (LAN). Although FIG. 1 shows only a single client device 102, it is understood that the content server 104 may also communicate with several other client devices that are generally similar to the client device 102.

[0023] Generally, a client device 102 can access information resources supplied or published by a publisher 106, and a content server 104 selects content items supplied by a content provider 108 to fill content slots within those information resources. For example, an information resource could be a webpage of a website hosted by a publisher 106, and a content provider 108 could supply advertisements or other content items that the content server 104 can then select to fill content slots within those webpages. As another example, an information resource could be a screen or user interface of an application (e.g., a mobile app) supplied by a publisher 106, and a content provider 108 could supply advertisements or other content items that the content server 104 can use to fill content slots within those user interfaces. As yet another example, an information resource could be a video, and content slots could be distributed over time throughout the video. However, in some embodiments, multiple (some or all) content slots of an information resource may exist simultaneously in the information resource without temporary distribution. The content item itself can be text, an image, or a video, or any combination thereof, and may or may not include an audio component.

[0024] The client device 102 can be, or can include, any stationary, mobile, or portable computing device (such as a smartphone, tablet computer, laptop computer, desktop computer, smart wearable device such as smart glasses or a smartwatch, a computer of a vehicle's head unit, etc.) with wired and / or wireless communication capabilities. In the exemplary embodiment of FIG. 1, the client device 102 includes a network interface 120, a processor 122, a memory module 124, and a display 126. The processor 122 can be a single processor (e.g., a central processing unit (CPU)), or can include a set of processors (e.g., multiple CPUs, or one or more CPUs and one or more graphics processing units (GPUs)).

[0025] The memory module 124 includes one or more computer-readable non-transitory storage units or devices that can include persistent (e.g., hard disk) and / or non-persistent memory components. The memory module 124 stores instructions executable by the processing unit 122 for performing various operations, including instructions for various software applications and data generated and / or used by such applications. In the exemplary embodiment of FIG. 1, the memory module 124 stores at least the application 130, which can be, for example, a web browser application (e.g., Chrome), or a mobile application downloaded from an application store.

[0026] Generally, application 130 is executed by processor 122 and presents information resources to the user of client device 102 via display 126, at least one of these information resources includes multiple content slots for dynamically presenting content items. In an embodiment where application 130 is a web browsing application, for example, the information resource could be a web page hosted by one of the publishers 106, and the browser causes client device 102 to download HTML, scripts, and / or other code of the web page for presentation to the user via display 126. In another example, application 130 could be a video sharing application such as YouTube, and the information resource could be a user interface generated by the video sharing application and presented via display 126. In yet another example, application 130 could be a video playback application, and the information resource could be a video played by the video playback application.

[0027] The display 126 includes hardware, firmware, and / or software configured to allow the user to view the visual output of the client device 102, and may use any suitable display technology (e.g., LED, OLED, LCD, etc.). In some embodiments, the display 126 is incorporated into a touchscreen having both display and manual input functions. Furthermore, in some embodiments where the client device 102 is a wearable device, the display 126 is a transparent visibility component (e.g., the lens of smart glasses) that shows the state of integrated electronic components. For example, the display 126 may include a micro-LED or OLED electronic device embedded in the lens of smart glasses.

[0028] The network interface 120 includes hardware, firmware, and / or software configured to enable the client device 102 to exchange electronic data with the content server 104 via the network 110. For example, the network interface 120 may include a cellular communication transceiver, a WiFi transceiver, and / or transceivers for one or more other wired and / or wireless communication technologies.

[0029] Figure 1 shows the client device 102 as a single component communicating directly (i.e., via the network 110) with the content server 104; however, in some embodiments, the subcomponents of the client device 102 shown in Figure 1 are instead divided into two or more user-side devices. As just one example, smart glasses may include a processor 122, memory 124, and a display 126, while a smartphone may include a separate processing unit, separate memory, separate display, and a network interface 120. The smart glasses (or smart helmet, etc.) may then communicate with the smartphone (e.g., via Bluetooth) as needed to enable the operations described herein.

[0030] The content server 104 includes a network interface 140, a processor 142, and memory 144. The network interface 140 includes hardware, firmware, and / or software configured to enable the content server 104 to exchange electronic data with client device 102 and other similar client devices via the network 110. For example, the network interface 140 may include a wired or wireless router and a modem. The processor 142 may be a single processor or may include multiple processors. As used herein, the term “server” may refer to a single computing device located in a single location, or to multiple coordinated computing devices that are either located in the same location or distributed remotely.

[0031] Memory 144 is a computer-readable, non-temporary storage unit or device, or a collection of units / devices, which may include persistent and / or non-persistent memory components. Memory 144 stores instructions for the content selector 150, which can be executed by the processor 142. The content selector 150 includes a conditional content selection procedure (CSP) module 152, which includes a slot priority module 154, a content filter 156, and an association module 158. One, some, or all of modules 154, 156, and 158 may be optionally omitted from the system 100.

[0032] Generally, the content selector 150 selects content items that the content server 104 will supply / transmit to the client device 102 (or other client devices) via the network 110. The content selector 150 selects specific content items using one or more content selection procedures performed by the conditional CSP module 152. The content filter 156 can apply one or more constraints to limit which content items may serve as candidates for selection by the conditional CSP module 152 based on the results of one or more previous content selection procedures (described further below) and / or one or more other factors (e.g., user preferences).

[0033] The content selection procedure performed by the conditional CSP module 152 could be, for example, an auction, in which the conditional CSP module 152 selects a winner based on the bids that various content providers 108 communicate to the content server 104, or bids that the content server 104 generates for the various content providers 108 if it does not otherwise (for example, based on input from the content providers 108, such as the preferred highest bid). In a more specific example, each content provider 108 could submit the highest bid for the placement of its content item, and the conditional CSP module 152 could select the content item associated with the content provider 108 with the highest bid. The content server 104 could then charge the winning content provider 108 the highest bid (or the bid by the second-highest bidder, for example) and select the content item associated with that content provider 108 to fill the content slot under consideration.

[0034] The content selection procedure may additionally or alternatively take into account selection parameters other than the bid amount, such as the relevance of a particular content item (or relevance to an information resource, etc.) to the user of the client device 102, which is determined by the relevance module 158 and / or user preferences, etc. The content selection procedure may be entirely rule-based or may have one or more random components (e.g., a random selection from a candidate pool after filtering by the content filter 156).

[0035] At least some of the content selection procedures performed by the conditional CSP module 152 have a conditional aspect. In some embodiments, for example, the execution of a content selection procedure for a given content slot(s) is conditional on the results of a content selection procedure for one or more other content slots of the same information resource. Furthermore or alternatively, in some embodiments, the content filter 156 may conditionally apply or not apply one or more constraints that limit which content items can be candidates for selection in one or more content slots, based on the results of a content selection procedure for one or more other content slots of the same information resource. In some embodiments, the slot priority module 154 determines the priority / ordering of individual content slots or groups / subsets of content slots, and the conditional CSP module 152 executes a conditional content selection procedure for one, some, or all of the content slots or content slot subsets in the determined order.

[0036] The operation of the conditional CSP module 152 and its constituent modules will be described in further detail below in relation to various exemplary embodiments.

[0037] Content items supplied by the content provider 108 (e.g., created and / or sourced) and available for selection by the content selector 150 may, in some embodiments, be stored in a content database 160. The content database 160 may be stored in memory 144, or in one or more remote memories, for example, from the content server 104.

[0038] In some embodiments, the publisher 106 and / or content provider 108 maintain accounts associated with services supplied by the content server 104. For example, the publisher 106 may create such accounts to monetize information resources that the publisher publishes or otherwise makes available (e.g., by selling advertisements in content slots on web pages hosted by the publisher), and / or the content provider 108 may create such accounts to find and purchase content slots where it would be particularly advantageous to present that content (e.g., advertisements). In these embodiments, information associated with the publisher and / or content provider accounts may be stored in an account database 162. The account database 162 may be stored in memory 144, or in one or more memory locations that are remote from the content server 104, for example. Account information may include information such as entity name, application level, and entity preferences (e.g., brand control preferences). In some embodiments, the account information includes selection parameters (e.g., bid amount or highest bid amount) related to different content providers 108, which are used by the conditional CSP module 152 when selecting content as described elsewhere in this specification.

[0039] In some embodiments, the system 100 is configured differently from that shown in Figure 1. For example, the content server 104 may be associated with the same entity that supplies the information resources, in which case the system 100 may omit the publisher 106 (for example, the activities described herein for the publisher 106 may instead be performed by the content server 104). Furthermore, or instead, the content server 104 may be associated with the same entity that sources content items to fill content slots in the information resources, in which case the system 100 may omit the content provider 108 (for example, the activities described herein for the content provider 108 may instead be performed by the content server 104).

[0040] Figures 2 and 3 show exemplary information resources having content slots configured such that the content selector 150 of Figure 1 selects content items. The information resources in Figure 2 or Figure 3 may be sourced (e.g., hosted) by one of the publishers 106, and after a client device 102 downloads the information resources (e.g., from publisher 106) via the network 110 and network interface 120, etc., they may be presented on the display 126 by the application 130.

[0041] Figure 2 shows an information resource 200, which is the user interface of a video sharing application (e.g., YouTube), and the user interface can be supplied as a web page (e.g., if application 130 is a web browser) or as the screen of a dedicated application (e.g., if application 130 is a dedicated video sharing application). The information resource 200 includes a primary or "masthead" content slot 202 and secondary content slots 204A-D, with the primary content slot 202 being more prominent than each of the secondary content slots 204A-D due to its higher position within the information resource 200 and having larger dimensions (height and / or width).

[0042] In some embodiments, where application 130 is a web browser application and information resource 200 is a web page hosted by one of the publishers 106, application 130 downloads and executes the code of the web page (such as HTML code) and causes it to present the web page via display 126. The web page code may also include a script (e.g., JavaScript) that, when interpreted by application 130, causes the client device 102 to request the content from content server 104 to satisfy, or conditionally satisfy, at least some (i.e., two or more) of content slots 202 and 204A-D. In some embodiments, where application 130 is a dedicated application that instead generates information resource 200, application 130 causes the client device 102 to request the content from content server 104 when information resource 200 is first generated (e.g., when application 130 is launched). In either case (web page or dedicated application screen), the request sent to content server 104 may be a single remote procedure call (RPC), as described below in relation to Figure 4.

[0043] Figure 3 shows an information resource 300, which is the user interface of a search engine, and the user interface may be supplied as a web page (for example, if application 130 is a web browser) or as the screen of a dedicated application (for example, if application 130 is a search engine mobile application). The information resource 300 includes several sponsored content slots 320A-C, each consecutive slot being further down in the list presented to the information resource 300, followed by a number of content slots 330A-F representing search results. The search may be based on a text string query entered by the user in the query field 302. In various embodiments, the search results used to fill the content slots 330A-F (e.g., Uniform Resource Locator (URL) links) may be selected by the content server 104 or a server other than the content server 104 (e.g., a dedicated search engine server) using any appropriate search technique or algorithm.

[0044] In some embodiments where application 130 is a web browser application and information resource 300 is a web page, application 130 downloads and executes the code of the web page (such as HTML code) and presents the web page via display 126. The web page code may also include a script (e.g., JavaScript) that, when interpreted by application 130, causes the client device 102 to request content from content server 104 to satisfy, or conditionally satisfy, at least content slots 320A-C. In some embodiments where application 130 is a dedicated application that instead generates information resource 300, application 130 causes the client device 102 to request content from content server 104 when information resource 300 is first generated (e.g., when application 130 is launched). In either case (web page or dedicated application screen), the request sent to content server 104 may be a single RPC, as described below in relation to Figure 4.

[0045] In some embodiments, the information resource 300 presents results in a manner different from that shown in Figure 3. For example, the search engine may utilize generative artificial intelligence (e.g., based on a large language model (LLM)), and the information resource 300 may provide a paragraph-style text response to a single user query instead of supplying an ordered list of search results. In one such embodiment, sponsored content slots 320A-C are scattered throughout the generated text response at fixed or dynamically changing positions.

[0046] In some embodiments, content slots 202 and 204A-D in Figure 2, or content slots 320A-C in Figure 3, are all configured to display the same common type of content (e.g., video only, images only, text only, or text and images). In other embodiments, different content slots of the same information resource may display different types of content depending on which content is selected for the slot by the content server 104. For example, the content selector 150 may be constrained to select only video content items for content slot 202 (by the content filter 156), while the remaining content slots 204A-D may be able to select content items such as images (or images and text, etc.) or videos (or video and text, etc.).

[0047] In some embodiments, if one of the content slots 202, 204A-D, or 320A-C is filled with a content item, the user of client device 102 can select that content item using the input devices of client device 102 (e.g., by tapping a touchscreen or clicking a mouse button) and be redirected to the page or other information resource associated with the selected content item. For example, if the content item is an advertisement filling content slot 202 or 204A in Figure 2, or content slot 320A in Figure 3, when the user selects that content item, application 130 can redirect the user to a landing page hosted by the content provider's server. The landing page may allow the user to navigate to a page where they can purchase, for example, the item advertised by the content item. In embodiments where the initial information resource (200 or 300) is a web page, this may include redirecting the user from the URL of the initial web page to the URL / web page of the content provider. In embodiments where the initial information resource (200 or 300) is a dedicated application screen, this may include automatically opening a web browser application and opening the URL of the content provider's web page.

[0048] Figures 2 and 3 are for illustrative purposes only, and it should be understood that the techniques described herein can instead be applied to information resources having different numbers of content slots, different sizes of content slots, different relative positions of content slots, etc. Furthermore, in some embodiments and / or scenarios, the user may need to navigate the entire information resource (e.g., by scrolling or panning) to view a particular content slot. In some embodiments, multiple content slots are delivered over time (e.g., if the information resource is a video, multiple slots are interrupted for video advertisement breaks).

[0049] Several embodiments of how the content selector 150 selects content items to fill / fill content slots 202, 204A-D in Figure 2, or content slots 320A-C in Figure 3, are described with reference to Figure 4. Figure 4 shows an exemplary scenario 400 in which the content server 104 in Figure 1 selects and delivers content to the client device 102 in Figure 1. In scenario 400, the client device 102 accesses an information resource (402) having multiple content slots (e.g., information resources 200 or 300). In an embodiment where the information resource is a web page hosted by one of the publishers 106, event 402 may include the client device 102 accessing the web page (perhaps after the user has navigated to the publisher's website) by receiving the web page code (e.g., HTML, JavaScript, etc.) from the publisher 106 via the network 110 using the web page's URL, at least in part. In embodiments where the information resource is the user interface / screen of a (dedicated) application 130, event 402 may include the client device 102 launching the application 130 and, optionally, navigating to a specific user interface provided by the application 130.

[0050] After accessing (and in response to) an information resource (e.g., automatically) (402), the client device 102 (e.g., application 130) requests content from the content server 104 to fill the content slots of the information resource (404). The request may consist of a single RPC. In some embodiments where the information resource is a web page, event 404 may be automatically triggered by application 130 in response to application 130 interpreting a script embedded in the web page code received from the server hosting the web page. In some embodiments where the information resource is a user interface / screen of a dedicated application (application 130), event 404 may be automatically triggered by application 130 in response to a user opening or launching application 130, or in response to a user navigating to a particular user interface / screen of application 130. Other triggers for event 404 are also possible.

[0051] In response to receiving the request, the content server 104 (e.g., the content selector 150) processes the request (410). Event 410 may include, for example, reformatting the request message and / or determining the function of the client device 102 based on the request message. In some embodiments, event 410 is omitted. The slot priority module 154 sorts the content slots of the information resource (e.g., slots 202 and 204A-D in Figure 2, or slots 320A-C in Figure 3) according to the slot priority (412). Sorting (412) serves to establish an order in which the conditional CSP module 152 will perform, or consider performing, a content selection procedure for a particular content slot or a particular subset of content slots. The slot priority module 154 may sort the slots or subsets of slots of the information resource based on priority information (e.g., that of the request message in event 404) sent to the content server 104 by the client device 102 (412). The client device 102 may obtain priority information from, for example, an information resource, from information associated with each content slot or slot subset (e.g., flags or field values), and / or from application 130. In some embodiments, the order / priority is fixed and event 412 is omitted.

[0052] Priority information may reflect the desirability of content slots (e.g., prominence such as location or size). For information resource 200, for example, content slot 202 may have the highest priority, content slot 204A the second highest, content slot 204B the third highest, content slot 204C the fourth highest, and content slot 204D the fifth highest. Alternatively, content slot 202 may have the highest priority, while content slots 204A-D are grouped together, all having the same priority and the second highest priority. For information resource 300, content slot 320A may have the highest priority, content slot 320B the second highest, and content slot 320C the third highest. Other priorities and / or groupings are also possible.

[0053] In event 414, the conditional CSP module 152 executes a content selection procedure (e.g., an auction or a relevance-based selection procedure) on the highest-priority content slot (e.g., content slot 202 or 320A), or executes a first set of content selection procedures on each of the slots in a subset of the highest-priority content slot (e.g., content slots 302A and 302B). Next, the conditional CSP module 152 processes (416) the results of the content selection procedure(s) executed in event 414 to determine how, or whether, a conditional selection procedure should be executed on the next highest-priority content slot or slot subset in event 420.

[0054] In some embodiments, in event 416, the conditional CSP module 152 decides, based on the result, whether or not to perform event 420 (and possibly any other subsequent similar operations such as event 424). For example, the conditional CSP module 152 may decide not to perform a content selection procedure (e.g., an auction) for content slot 204A if (1) the content selection procedure in event 414 results in any kind of content item being selected for content slot 202, but (2) the content selection procedure should be performed if the content selection procedure in event 414 results in no kind of content item being selected for content slot 202. As another example, the conditional CSP module 152 can determine that (1) if the content selection procedure in event 414 results in a content item being selected for content slot 202, then the content selection procedure will not be executed for any of the content slots 204A to D, but (2) if the content selection procedure in event 414 does not result in a content item being selected for content slot 202, then the content selection procedure should be executed for each of the content slots 204A to D. As yet another example, the conditional CSP module 152 can determine that (1) if the content selection procedure in event 414 results in a content item being selected for content slot 320A, then the content selection procedure will not be executed for content slot 320B, but (2) if the content selection procedure in event 414 does not result in a content item being selected for content slot 320A, then the content selection procedure should be executed.

[0055] The content selection procedure performed by the conditional CSP module 152 may not result in a content item being selected for a particular content slot for various reasons, depending on the embodiment and scenario. For example, the content selection procedure may result in the content server 104 rejecting the content item due to security concerns associated with the content item (such as suspicious scripts within the content item) or incompatibility with the preferences of the user of the client device 102 or the publisher 106 (for example, if the content filter 156 applies certain constraints after the content selection procedure is completed).

[0056] In other embodiments, the processing in event 416 analyzes other aspects of the result of the content selection procedure(s) in event 414 when deciding whether or not to execute event 420 (and / or subsequent events in Figure 4). For example, conditional CSP module 152 may decide that if the content selection procedure for content slot 202 in event 414 results in a content item being selected with a bid above a certain minimum amount, then the content selection procedure will not be executed for any of content slots 204A-D. In another embodiment, conditional CSP module 152 may decide whether or not to execute the content selection procedure(s) in event 414 based on one or more characteristics of the one or more content items selected in event 414. For example, conditional CSP module 152 may decide that if the content selection procedure in event 414 results in a content item of a certain type (e.g., video) being selected for content slot 202, then the content selection procedure will not be executed for any of content slots 204A-D in event 420.

[0057] In another exemplary embodiment, the conditional CSP module 152 conditionally executes or does not execute a content selection procedure(s) in event 420 based on the relevance score of at least one content item selected during the content selection procedure(s) in event 414. For example, the relevance module 158 (or another module / software, or possibly another server) could use a machine learning model (e.g., a neural network) to determine a relevance score for each of the one or more content items selected in event 414, and the relevance module 158 could execute a content selection procedure(s) in event 420 only if the determined relevance score(s) (or each of a certain number of relevance scores) exceeds some predetermined threshold. The relevance score could represent, for example, the relevance to a user's search query (e.g., a text string) or the relevance to an information resource.

[0058] As a more specific example, in a scenario where information resource 200 is presented to a user on client device 102 in response to a user's video search query, the relevance module 158 can use a machine learning model to calculate relevance or search scores for content items selected by content selector 150 (in event 414) to fill content slot 202. If the relevance / search score is greater than some threshold (meaning the content item is particularly relevant to the search), the conditional CSP module 152 performs another content selection procedure (e.g., an auction) in event 420 to fill content slot 204A (or performs such a procedure for each of slots 204A-D). However, if the search score of the content item is not greater than the threshold, the conditional CSP module 152 reduces the likelihood of confusing the user with irrelevant or useless information by not performing a content selection procedure(s) in event 420.

[0059] In some embodiments, in event 416, the conditional CSP module 152 instead (or additionally) determines how event 420 is performed based on the results of the content selection procedure(s) in event 414. In particular, the content filter 156 can apply one or more constraints to limit which content items (from the content database 160) can serve as candidates in the content selection procedure(s) in event 420, based on the format or content of the content items(s) selected in event 414.

[0060] For example, the conditional CSP module 152 may exclude content items from consideration in event 420 based on one or more categories of the content items selected in event 414, the user cost associated with one or more of the selected content items, one or more formats of the selected content items (e.g., excluding content items of the same format as the selected content items), and / or the level of similarity with one or more of the selected content items. The level of similarity may be indicated by a similarity score. The content server 104 (e.g., content selector 150) may determine the similarity score of a pair of content items using machine learning techniques, such as a neural network or other machine learning model trained on the content item pair using supervised or unsupervised learning techniques. The machine learning model may manipulate (accept as input) specific parameters that describe the characteristics of the content items (e.g., format, category, length of the content items), or it may directly manipulate, for example, image, text, or video frame information. In some embodiments, the content server 104 uses generative artificial intelligence (e.g., a large-scale language model (LLM)) to determine the level of similarity between two content items that both contain text.

[0061] Applying constraints based on similarity to previously selected content items, and / or based on the category or format of previously selected content items, can avoid scenarios where certain types / formats of content items are over-selected, offering various advantages. For example, if a content slot corresponds to an in-stream video ad interruption, it may be desirable to prevent client device users from receiving multiple long video ads (e.g., spanning a certain threshold length, such as 60 seconds). Therefore, if a long video ad was selected in the initial interruption, content filter 156 can exclude any long video ad (e.g., those exceeding the threshold length) in one or more subsequent ad interruptions (as candidates for the content selection procedure in event 420). As another example, content filter 156 can be useful in enforcing brand safety constraints related to ads or content providers. For example, content filter 156 can exclude any ad related to a competitor of a content provider selected in event 414 (as candidates for the content selection procedure in event 420). The list of competitors may be associated with the account of each content provider 108 in the account database 162, or the content server 104 may identify potential competitors by accessing other information (for example, the categories of goods or services sold as shown in databases 160 or 162).

[0062] In some embodiments, the content filter 156 uses a machine learning model to apply its constraints to the content items. For example, event 420 may include applying one or more parameters indicating the format or content of the content item(s) selected in event 414 as input to a machine learning model (such as a neural network), and deriving at least one constraint from the output of the machine learning model.

[0063] As shown in Figure 4, if there are subsequent priority levels for a content slot (or slot subset) determined by the slot priority module 154, additional events similar to events 416 and / or 420 may occur, or be conditionally occurring. For example, the conditional CSP module 152 may process the results of the content selection procedure(s) of event 420 in event 422 (e.g., similar to event 416), or conditionally execute or not execute (or conditionally restrict execution) one or more further content selection procedures in event 424 (e.g., similar to event 420).

[0064] In some embodiments where specific content slots (e.g., slots 204A-D) are grouped together into subsets for prioritization / sorting purposes, the conditional CSP module 152 performs content selection procedures in parallel (i.e., simultaneously) or at least partially in parallel (i.e., at least partially overlapping in time) on all content slots within the group / subset to reduce the latency associated with filling the content slots of the information resource.

[0065] After events 410-424 are completed (or, as described above, may not be performed at all in the case of events 420-424), the content server 104 responds to the client device 102 (for example, by returning an RPC response) (430). The response in event 430 may include the content item(s) selected during one or more content selection procedures in events 414, 420, and 424, or may include links to the selected content item(s). In event 430, the client device 102 stores the selected content item(s) in the appropriate content slot(s) of the information resource. In some embodiments, the client device 102 does not need to create any RPC other than the single RPC in event 404 to obtain the content item(s) to fill the slots in the information resource.

[0066] Next, exemplary methods for efficiently selecting content on a client device will be described with reference to Figures 5 and 6. Either or both of the methods in Figures 5 and 6 may be implemented as instructions stored in one or more computer-readable media and executed on one or more processors of one or more computing devices. For example, the method(s) may be implemented by the processor 142 of the content server 104 in Figure 1 when executing the instructions of the content selector 150.

[0067] First, referring to method 500 in Figure 5, a request for content to be included in an information resource having multiple content slots is received in block 502 from a client device (e.g., client device 102) (e.g., event 404). In block 504, one or more content items are selected in response to the request. Block 504 includes determining a first content selection result for a first subset of the multiple content slots (e.g., event 414) by performing a first content selection procedure of one or more first content selection procedures using selection parameters (e.g., bid amount, relevance score, etc.) associated with at least the candidate content items. Block 504 also includes conditionally performing or not performing a second content selection procedure for a second subset of the multiple content slots based on the first content selection result. In some embodiments, the first subset of the multiple content slots consists of only a single content slot, and / or the second subset of the multiple content slots consists of only a single content slot. In other embodiments, one or both subsets include multiple content slots.

[0068] In block 506, the client device is configured to fill one or more content slots from among several content slots with one or more selected content items. Block 506 may, for example, respond to event 430.

[0069] In some embodiments, block 504 refrains from performing a second content selection procedure(s) when selecting a first set of one or more content items to fill a first subset of multiple content slots, and instead includes performing a second content selection procedure(s) when no content items are selected to fill a first subset of multiple content slots.

[0070] In some embodiments, determining a first content selection result in block 504 includes selecting a first set of one or more content items to satisfy a first subset of multiple content slots, and selecting one or more content items in block 504 includes performing or not performing a second content selection procedure(s) conditionally based on one or more characteristics of the first set of content items. One or more characteristics of the first set of content items may include one or more relevance scores associated with the first set of content items (e.g., determined by the relevance module 158 using a machine learning model).

[0071] In some embodiments, method 500 also includes determining that a first subset has a higher priority than a second subset before selecting one or more content items, and performing a first content selection procedure(s) in block 504 in response to this determination.

[0072] In some embodiments, a first subset of a plurality of content slots includes at least a first content slot and a second content slot, a first content selection procedure(s) includes a third content selection procedure for the first content slot and a fourth content selection procedure for the second content slot, and executing the first content selection procedure(s) includes executing the third content selection procedure at least partially in parallel with executing the fourth content selection procedure.

[0073] Next, referring to method 600 in Figure 6, a request for content to be included in an information resource having multiple content slots is received in block 602 from a client device (e.g., client device 102) (e.g., event 404). In block 604, multiple content items are selected in response to the request. Block 604 includes selecting a first set of one or more content items in a first subset of multiple content slots (e.g., slot 202, slot 320A, or slots 320A and 320B, etc.) by performing a first content selection procedure (e.g., event 414) using at least selection parameters (e.g., bid, relevance score, etc.) associated with the candidate content items. Block 604 also includes applying one or more constraints that limit which content items may be candidates in a second content selection procedure for a second subset of the multiple content slots, based on the format or content of the first set of content items. In some embodiments, a first subset of multiple content slots consists of only a single content slot, and / or a second subset of multiple content slots consists of only a single content slot. In other embodiments, one or both subsets include multiple content slots.

[0074] In block 606, the client device is configured to fill at least some of the multiple content slots with selected content items. Block 606 may, for example, correspond to event 430.

[0075] In some embodiments, applying one or more constraints includes excluding content items from a second content selection procedure(s) based on the category of content items (e.g., excluding content items associated with the category of at least one content item in a first set of content items), excluding content items from a second content selection procedure(s) based on the user cost associated with at least one content item in the first set of content items, excluding content items having the same format as at least one content item in the first set of content items, and / or excluding content items based on a similarity score indicating the level of similarity with at least one content item in the first set of content items. In the latter case, method 600 may further include using a machine learning model to determine the similarity score.

[0076] In some embodiments, applying constraints in block 604 includes applying one or more parameters that indicate the format or content of a first set of content items as input to a machine learning model, and deriving at least one of the constraints from the output of the machine learning model.

[0077] In some embodiments, method 600 also includes determining that a first subset has a higher priority than a second subset before selecting content items in block 604, and performing a first content selection procedure(s) in block 604 is in response to determining that the first subset has a higher priority.

[0078] In some embodiments, a first subset of content slots includes at least a first content slot and a second content slot, a first content selection procedure(s) includes a third content selection procedure for the first content slot and a fourth content selection procedure for the second content slot, and executing the first content selection procedure(s) in block 604 includes executing the third content selection procedure at least partially in parallel with executing the fourth content selection procedure.

[0079] In some embodiments, the technologies disclosed herein use artificial intelligence to facilitate the efficient selection of content items to fill content slots in an information resource. Artificial intelligence (AI) is a segment of computer science that focuses on creating models that can perform tasks with little or no human intervention. Systems of artificial intelligence can utilize, for example, machine learning, natural language processing, and computer vision. Machine learning and its subsets, such as deep learning, focus on developing models that can infer outputs from data. Outputs may include, for example, prediction and / or classification. Natural language processing focuses on analyzing and generating human language. Computer vision focuses on analyzing and interpreting images and videos. Systems of artificial intelligence can include generative models that generate new content, such as images, videos, text, audio, and / or other content, in response to input prompts and / or based on other information.

[0080] Exemplary machine learning models include neural networks or other multi-layered nonlinear models. Exemplary neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some exemplary machine learning models can leverage attention mechanisms such as self-attention. For example, some machine learning models can include multi-head self-attention models (e.g., transformer models).

[0081] Models can be trained using a variety of training or learning techniques. Training can include supervised learning, unsupervised learning, and reinforcement learning. Techniques such as backpropagation can be used during training. For example, a loss function can be backpropagated through the model(s) to update one or more of the model(s) parameters (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent can be used to iteratively update parameters over several training iterations. Several generalization techniques (e.g., weight decay, dropout) can be used to improve the generalization ability of the trained model.

[0082] Prior to domain-specific alignment, the model(s) can be pre-trained. For example, a model can be pre-trained on a general corpus of training data and then fine-tuned to a more targeted corpus of training data. The model can be aligned using prompts designed to elicit domain-specific outputs. The prompts can be designed to include learned prompt values ​​(e.g., soft prompts). The trained model(s) may be validated before use using input data other than the training data and may be further updated or improved based on additional feedback / input during use.

[0083] In some embodiments, the content server 104 (e.g., the conditional CSP module 152) can use one or more of the machine learning models described above to perform any one or more of the operations discussed herein in relation to machine learning. For example, the content server 104 can use one or more such machine learning models, as described above, to determine the relevance score of content items, the similarity score between content items, and / or the constraints applied by the content filter 156.

[0084] While the preceding text describes in detail numerous different aspects and embodiments of the present invention, it should be understood that the scope of the patent is defined by the claims language set out at the end of this patent. The embodiments for carrying out the invention should be interpreted as illustrative only and do not describe all possible embodiments, for describing all possible embodiments would be impractical, if not impossible. Numerous alternative embodiments can be implemented using the current art or art developed after the filing date of this patent, and these remain within the scope of the claims. The disclosure herein assumes at least the following examples:

[0085] Example 1. A method for efficiently supplying content to a client device, the method comprising: receiving requests from a server and the client device for content to be included in an information resource having a plurality of content slots; in response to the requests, the server selects one or more content items, including (i) determining a first content selection result for a first subset of the plurality of content slots by performing a first content selection procedure of one or more first content selection procedures using selection parameters associated with at least candidate content items, and (ii) conditionally performing or not performing a second content selection procedure of one or more second content selection procedures for a second subset of the plurality of content slots based on the first content selection result; and causing the server to cause the client device to fill one or more content slots of the plurality of content slots with the selected one or more content items.

[0086] Example 2. The method according to Example 1, wherein selecting one or more content items includes not performing the second one or more content selection procedure when a first set of one or more content items is selected to fill the first subset of the plurality of content slots, and performing the second one or more content selection procedure when no content or content items are selected to fill the first subset of the plurality of content slots.

[0087] Example 3. The method according to Example 1, wherein determining the first content selection result includes selecting a first set of one or more content items to satisfy the first subset of the plurality of content slots, and selecting the one or more content items includes conditionally performing or not performing the second one or more content selection procedure based on one or more characteristics of the first set of content items.

[0088] Example 4. The method according to Example 3, wherein the one or more characteristics of the first set of content items include one or more relevance scores associated with the first set of content items.

[0089] Example 5. The method according to Example 4, further comprising using a machine learning model to determine the one or more relevance scores by the server.

[0090] Example 6. The method according to any one of Examples 1 to 5, further comprising the server determining that the first subset has a higher priority than the second subset before selecting the one or more content items, wherein the execution of the first one or more content selection procedure is in response to the determination that the first subset has a higher priority than the second subset.

[0091] Example 7. The method according to Example 6, wherein the first subset of the plurality of content slots includes at least a first content slot and a second content slot, the first one or more content selection procedures include a third content selection procedure for the first content slot and a fourth content selection procedure for the second content slot, and executing the first one or more content selection procedures includes executing the third content selection procedure at least partially in parallel with executing the fourth content selection procedure.

[0092] Example 8. The method according to any one of Examples 1 to 6, wherein the first subset of the plurality of content slots consists of a single content slot, and the second subset of the plurality of content slots consists of a single content slot, or both.

[0093] Example 9. The method according to any one of Examples 1 to 8, wherein the request is a Remote Procedure Call (RPC).

[0094] Example 10. A method for efficiently supplying content to a client device, comprising: receiving requests from a server and the client device for content to be included in an information resource having a plurality of content slots; selecting a plurality of content items, which includes, in response to the requests, the server performing a first content selection procedure of one or more first content selection procedures for a first subset of the plurality of content slots by using selection parameters associated with at least candidate content items; and applying one or more constraints to limit which content items may be candidates in a second content selection procedure of a second subset of the plurality of content slots based on the format or content of the first set of content items; and causing the server to cause the client device to fill at least some of the plurality of content slots with the selected plurality of content items.

[0095] Example 11. The method according to Example 10, wherein applying one or more constraints includes excluding content items from the second one or more content selection procedure based on content item categories.

[0096] Example 12. The method according to Example 11, wherein excluding content items from the second one or more content selection procedures based on content item categories includes excluding content items associated with the category of at least one content item from the first set of content items.

[0097] Example 13. The method according to any one of Examples 10 to 12, wherein applying the one or more constraints includes excluding content items from the second one or more content selection procedure based on the user cost associated with at least one content item from the first set of content items.

[0098] Example 14. The method according to any one of Examples 10 to 13, wherein applying one or more of the restrictions includes excluding content items that have the same format as at least one content item in the first set of content items.

[0099] Example 15. The method according to any one of Examples 10 to 14, wherein applying one or more of the constraints includes excluding content items based on a similarity score indicating a level of similarity with at least one content item from a first set of content items.

[0100] Example 16. The method according to any one of Examples 10 to 14, further comprising using a machine learning model to determine the similarity score by the server.

[0101] Example 17. The method according to any one of Examples 10 to 16, wherein applying the one or more constraints includes applying one or more parameters indicating the format or content of the first set of content items as input to a machine learning model, and deriving at least one of the one or more constraints from the output of the machine learning model.

[0102] Example 18. The method according to any one of Examples 10 to 17, wherein the request is a Remote Procedure Call (RPC).

[0103] Example 19. The method according to any one of Examples 10 to 18, further comprising the server determining that the first subset has a higher priority than the second subset before selecting the plurality of content items, wherein the execution of the first one or more content selection procedures is in response to the determination that the first subset has a higher priority than the second subset.

[0104] Example 20. The method according to Example 19, wherein the first subset of the plurality of content slots includes at least a first content slot and a second content slot, the first one or more content selection procedures include a third content selection procedure for the first content slot and a fourth content selection procedure for the second content slot, and executing the first one or more content selection procedures includes executing the third content selection procedure at least partially in parallel with executing the fourth content selection procedure.

[0105] Example 21. The method according to any one of Examples 10 to 20, wherein the first subset of the plurality of content slots consists of a single content slot, and the second subset of the plurality of content slots consists of a single content slot, or both.

[0106] Example 22. A server comprising one or more processors and one or more memories, wherein the one or more memories store instructions, and when an instruction is executed by the one or more processors, the one or more processors cause the one or more processors to perform the method described in any one of Examples 1 to 21.

[0107] The following additional considerations apply to the preceding discussion. Throughout this specification, multiple examples may implement a component, operation, or structure described as a single example. While individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed simultaneously, and nothing requires the operations to be performed in the order illustrated. Structures and functions represented as separate components in the examples may be implemented as a combined structure or component. Similarly, structures and functions represented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter of this disclosure.

[0108] Unless otherwise expressed, any discussion in this disclosure using terms such as “process,” “calculate,” “determine,” “represent,” “display,” and similar terms may refer to the actions or processing of a machine (e.g., a computer) that operates or transforms data that is represented as a physical (e.g., electronic, magnetic, or optical) quantity within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0109] Where used in this disclosure, any reference to “one implementation” or “an implementation” means that the specific features, structures, or characteristics described in relation to the implementation are included in at least one implementation. The phrase “in one implementation” appearing in various places in this specification does not necessarily refer to the same implementation in all instances.

[0110] Where used in this disclosure, “comprises,” “comprising,” “includes,” “including,” “has,” “having,” or any other variation thereof are intended to encompass non-exclusive inclusion. For example, a process, method, item, or apparatus that includes a list of elements is not necessarily limited to these elements alone and may include other elements not expressly enumerated or inherent in such process, method, item, or apparatus. Furthermore, unless expressly stated otherwise, “or” refers to an inclusive or not an exclusive or. For example, for example, condition A or B is satisfied by any one of the following: A is true (or exists) and B is false (or does not exist), A is false (does not exist) and B is true (or exists), and both A and B are true (or exist).

[0111] A person skilled in the art will understand, through the principles described herein, further alternative structural and functional designs for efficiently selecting and supplying content to a client device to fill content slots of an information resource. Therefore, while specific embodiments and applications have been described and outlined, it should be understood that the disclosed embodiments are not limited to the exact structures and components disclosed herein. Various modifications, changes, and variations obvious to a person skilled in the art may be made in the arrangement, operation, and details of the methods and apparatus disclosed herein without departing from the spirit and scope of the appended claims.

Claims

1. A method for efficiently delivering content to client devices, The server receives requests from the client device for content to be included in an information resource having multiple content slots. In response to the aforementioned request, the server performs the following actions: A first content selection result for a first subset of the plurality of content slots is determined by performing a first content selection auction of one or more content selection auctions using selection parameters associated with at least candidate content items, and Based on the first content selection result, conditionally perform or refrain from performing one or more second content selection auctions for a second subset of the plurality of content slots. Selecting one or more content items that include, The server causes the client device to fill one or more of the multiple content slots with the selected one or more content items. The method, including the method described above.

2. Selecting one or more of the aforementioned content items means When a first set of one or more content items is selected to fill the first subset of the plurality of content slots, the second auction for one or more content items is not performed, and When no content items are selected to satisfy the first subset of the plurality of content slots, the second one or more content selection auctions are performed. The method according to claim 1, including the method described in claim 1.

3. Determining the first content selection result includes selecting a first set of one or more content items to satisfy the first subset of the plurality of content slots, Selecting one or more content items means conditionally performing or not performing one or more second content selection auctions based on one or more characteristics of the first set of content items. The method according to claim 1, including the method described in claim 1.

4. The method according to claim 3, wherein the one or more characteristics of the first set of content items include one or more relevance scores associated with the first set of content items.

5. The method according to claim 4, further comprising using a machine learning model to determine the one or more relevance scores by the server.

6. The server further includes determining, before selecting one or more content items, that the first subset has a higher priority than the second subset. The method according to claim 1, wherein performing the first one or more content selection auctions is in response to determining that the first subset has a higher priority than the second subset.

7. The first subset of the plurality of content slots includes at least a first content slot and a second content slot, The first one or more content selection auctions include a third content selection auction for the first content slot and a fourth content selection auction for the second content slot. The method according to claim 6, wherein performing the first one or more content selection auctions includes performing the third content selection auction at least partially in parallel with performing the fourth content selection auction.

8. The first subset of the plurality of content slots consists of a single content slot, and The second subset of the plurality of content slots consists of a single content slot. The method according to claim 1, wherein one or both of the above.

9. The method according to claim 1, wherein the request is a remote procedure call (RPC).

10. A method for efficiently delivering content to client devices, The server receives requests from the client device for content to be included in an information resource having multiple content slots. In response to the aforementioned request, the server performs the following actions: Selecting a first set of one or more content items for a first subset of the plurality of content slots by performing a first content selection auction of one or more content items using selection parameters associated with at least candidate content items, and Applying one or more constraints to restrict which content items may be candidates in a second content selection auction for a second subset of the plurality of content slots, based on the format or content of the first set of content items. Selecting multiple content items, including, The server causes the client device to fill at least some of the multiple content slots with the selected multiple content items. The method, including the method described above.

11. Applying the aforementioned one or more constraints means The method according to claim 10, comprising excluding content items from one or more second content selection auctions based on content item categories.

12. Excluding content items from one or more of the aforementioned second content selection auctions based on content item categories is: To exclude content items associated with the category of at least one content item from the first set of content items, The method according to claim 11, including the method described in claim 11.

13. Applying the aforementioned one or more constraints means Excluding content items from the second one or more content selection auctions based on the user cost associated with at least one content item from the first set of content items, The method according to claim 10, including the method described in claim 10.

14. Applying the aforementioned one or more constraints means To exclude content items that have the same format as at least one content item from the first set of content items, The method according to claim 10, including the method described in claim 10.

15. Applying the aforementioned one or more constraints means This includes excluding content items based on a similarity score indicating the level of similarity with at least one content item from the first set of content items, The method according to claim 10.

16. The method according to claim 15, further comprising the server determining the similarity score using a machine learning model.

17. Applying the aforementioned one or more constraints means Applying one or more parameters indicating the format or content of the first set of content items as input to a machine learning model, Deriving at least one of the one or more constraints from the output of the machine learning model, The method according to claim 10, including the method described in claim 10.

18. The method according to claim 10, wherein the request is a remote procedure call (RPC).

19. The process further includes the server determining, before selecting the plurality of content items, that the first subset has a higher priority than the second subset. The method according to claim 10, wherein performing the first one or more content selection auctions is in response to determining that the first subset has a higher priority than the second subset.

20. The first subset of the plurality of content slots includes at least a first content slot and a second content slot, The first one or more content selection auctions include a third content selection auction for the first content slot and a fourth content selection auction for the second content slot. The method according to claim 19, wherein performing the first one or more content selection auctions includes performing the third content selection auction at least partially in parallel with performing the fourth content selection auction.

21. The first subset of the plurality of content slots consists of a single content slot, and The second subset of the plurality of content slots consists of a single content slot. The method according to claim 10, wherein one or both of the above.

22. A server comprising one or more processors and one or more memories, wherein the one or more memories store instructions, and when an instruction is executed by the one or more processors, the server causes the one or more processors to perform the method according to any one of claims 1 to 21.

Images