Conditional content selection process

Abstract

Systems, methods, and / or algorithms facilitate efficient provision of content to a client device to fill a content slot of an information resource having a plurality of content slots. The server receives a request from the client device for content to be included in the information resource and in response performs one or more content selection processes to select one or more content items for one or more content slots. The server may conditionally perform or not perform one or more additional content selection processes on one or more other content slots of the information resource based on a result of the content selection process. Alternatively or additionally, the server may apply one or more constraints to limit which content items may be candidates in one or more additional content selection processes based on the format or content of one or more content items selected during the initial content selection process.

Description

Technical Field

[0001] The present disclosure relates to content selection processes, and more particularly to processes for selecting content to provide to a client device for presentation to a user within a content slot of an information resource. Background Art

[0002] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. The work of the presently named inventors (to the extent that it is described in this background section) and aspects of the specification that might not have been considered prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present disclosure.

[0003] In some scenarios, such as when a user visits a web page or views a screen provided by a dedicated application, additional, dynamically added content is presented to the user in certain reserved areas of the page or screen. For example, a web page may reserve one or more "slots" for third-party content, and a client device may request a server to provide content items (such as digital advertisements or other third-party content items) to fill those slots. Typically, a client device sends a request to a server for a given content slot, causing the server to perform a process of selecting a specific content item. The server then provides the selected content item to the client device, which fills / fills the slot with the content item. For example, in the case of digital advertising, the content selection process typically involves an auction. After a certain pre-screening of the advertisement (e.g., based on various targeted parameters), the bid scores and rankings of the remaining candidates are determined. The auction winner is determined based on the ranking, and the winner's advertisement is used to fill the content slot.

[0004] In situations where there are multiple content slots available to be filled on a web page, a user interface, or another information resource, various complications may arise. For example, different slots and different content item formats may be associated with different priorities, and / or may be dependent on each other (e.g., conflicting). As an example in a digital advertising context: in order to enhance or maintain the value of a prominent (e.g., top of the page) ad slot, when selecting a sponsored content item (ad) to fill the slot, it may be preferred to avoid placing another ad in a nearby slot, which may otherwise distract the user and / or annoy the user. In other examples, it may be preferred to avoid filling a single page with too many ads and / or avoid placing competing ads (e.g., for similar products) in adjacent slots of a page. Therefore, various decisions need to be made regarding whether to select a particular ad, which ad slots should be filled, and / or whether to apply certain constraints to which ads may be candidates in the ad selection process.

[0005] In some existing systems, this decision-making process is performed by performing an auction for all available ad slots and then determining whether to exclude certain ads from the auction winners. For example, in response to a request from a client device, a server may perform a first auction for a first slot on a page and a second auction for a second slot on the same page. If the first auction has a certain result, such as the first slot has been filled (or the first slot is filled with an ad for a particular product or brand, etc.), the winning ad for the second slot may be discarded; otherwise, the second slot is filled with the winning ad for the second slot. However, this approach has certain technical disadvantages. First, because the auction is run for all requested slots (e.g., all slots on the page that the user is visiting), but only a subset of the winning ads can be delivered to the user's device, there is typically a waste of processing resources. Specifically, running an auction wastes processing resources (e.g., processing time, processing power) at the server even for slots in which no ads are ultimately presented. Second, because ads for some slots may be eliminated based on the results / winners for other slots, the ad slot "fill rate" may be reduced, resulting in the server being unable to efficiently serve ad content (and therefore resulting in fewer impressions, lower ad revenue, etc.).

[0006] In other systems, the decision-making process is performed by the client device issuing multiple remote procedure calls (RPCs) to the server in sequence to obtain advertisements for different locations on the page. According to this approach, the client device issues a first RPC to the server for a first slot, and then waits for and considers the result / response obtained from the server, then issues a second RPC to the server for a second slot, and so on. This approach also has significant disadvantages. First, because the client device must wait for the result from the server for each of the RPCs / slots in turn, and there is a delay between each request and the response from the server, the latency (e.g., web page load time) increases, which may increase to the point of annoying the user. Second, because the client device must perform a large amount of front-end processing (e.g., to determine whether to issue a subsequent RPC based on the result of the previous RPC), the processing load may cause the client device to crash (e.g., the browser or other application crashes), and thus result in a poor user experience. This may be particularly problematic when the client device has relatively limited memory, such as may occur with smart TV devices, for example. Third, because decision making occurs at the client device, less information is typically available to the server, which can hinder the server's ability to make future decisions (e.g., for future content selections) or provide useful analysis or troubleshooting feedback (e.g., information indicating ad performance or information indicating why an ad was not delivered, etc.). Summary of the invention

[0007] In some implementations described herein, a server determines whether to perform one or more content selection processes to fill content slots of an information resource based on results from one or more earlier content selection processes used to fill other content slots of the information resource and / or determines a manner in which to perform the content selection processes.

[0008] In one example implementation, a method for efficiently providing content to a client device includes: receiving, by a server from a client device, a request for content to be included in an information resource having a plurality of content slots; and selecting, by the server, one or more content items in response to the request. Selecting the one or more content items includes: (1) performing a first content selection process at least by using selection parameters associated with candidate content items to determine a first content selection result for a first subset of the plurality of content slots; and (2) conditionally performing or not performing a second content selection process for a second subset of the plurality of content slots based on the first content selection result. The method also includes causing, by the server, the client device to fill one or more content slots of the plurality of content slots with the selected one or more content items.

[0009] In another example implementation, a method for efficiently providing content to a client device includes: receiving, by a server from a client device, a request for content to be included in an information resource having a plurality of content slots; and selecting, by the server, a plurality of content items in response to the request. Selecting the plurality of content items includes: (1) selecting a first set of one or more content items for a first subset of the plurality of content slots by performing a first one or more content selection process at least using selection parameters associated with candidate content items; and (2) applying one or more constraints to limit which content items may be candidates in a second one or more content selection process for a second subset of the plurality of content slots based on a format or content of the content items of the first set. The method also includes causing, by the server, the client device to fill at least some of the plurality of content slots with the selected plurality of content items.

[0010] In some implementations, a server includes: one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform any of the above implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram of an example system in which techniques for efficiently providing content to client devices may be implemented.

[0012] Figure 2 A first example information resource having a content slot configured to display a content item selected by a content selection process is depicted.

[0013] Figure 3 A second example information resource having a content slot configured to display a content item selected by a content selection process is depicted.

[0014] Figure 4 It depicts Figure 1 The content server is Figure 1 A message sequence chart of an example implementation of selecting content for a client device and providing the selected content to the client device.

[0015] Figure 5 is a flow chart of an example method for efficiently providing content to a client device.

[0016] Figure 6 is a flow chart of another example method for efficiently providing content to a client device. DETAILED DESCRIPTION

[0017] In general, in various aspects of the disclosed invention, a server performs a conditional content selection process. Specifically, in response to receiving a request for content from a client device, the server performs one or more content selection processes to fill one or more content slots of an information resource (e.g., a web page or application screen / user interface displayed at the client device), and for one or more additional content slots of the information resource, the server determines the execution of a content selection process based on the results of an earlier content selection process.

[0018] In certain exemplary implementations and scenarios described herein, the content items used to fill the content slots are advertisements, and / or the corresponding content selection process is an auction. However, the content items may alternatively be any other type of content that can be dynamically selected for presentation within an information resource, and the content selection process may alternatively be any other type of process suitable for selecting content (e.g., a process for selecting content based on a relevance score of a content item).

[0019] In one aspect of the invention, a server conditionally performs or does not perform one or more content selection processes based on the results of one or more earlier content selection processes for a slot of the same information resource. In some implementations, for example, the server performs a second content selection process only if the first content selection process does not result in any content item being selected for a particular content slot. In other implementations, the server conditionally performs or does not perform the second content selection process based on one or more characteristics of the content item selected during the first content selection process. For example, the server may use a machine learning model to determine a relevance score for the content item selected during the first content selection process, and then perform the second content selection process only if the relevance score exceeds a predetermined threshold.

[0020] In another aspect of the present invention, the server decides the manner of executing one or more content selection processes, alternatively or also based on the results of one or more earlier content selection processes for the slot of the same information resource. Specifically, the server may apply one or more constraints based on the format and / or content of the content items selected during the first content selection process, wherein these constraints are used to limit which content items can be candidates in the second content selection process. In various implementations, for example, the server may exclude those content items from the second content selection process based on the category or format of the content items (e.g., based on whether the category or format of the content items is the same as the content items generated by the first content selection process), based on user cost, based on the similarity level between the content items and the content items selected during the first content selection process, and / or based on other factors.

[0021] For either of the above two aspects, in some implementations, the server may sort the content slots of the information resource according to their priority levels before performing the first content selection process to determine the order in which the server should attempt to fill the content slots. In some implementations, at least some of the content slots of the information resource may be grouped together, in which case the server may perform content selection processes for content items within a given group at least partially in parallel with each other.

[0022] These aspects and other aspects may improve the efficiency of the content selection process (e.g., using fewer processing resources), reduce latency (e.g., reducing the load time of a web page), prevent "crashes" at the client device (e.g., if the client device is a highly resource-constrained device), and / or provide other technical advantages.

[0023] For example, in the first aspect (wherein the server conditionally performs or does not perform one or more additional content selection processes for an information resource), (1) the server uses fewer processing resources than if the server automatically performs the content selection process for all content slots in a multi-slot information resource (e.g., based on a single request from a client device), (2) latency or delay at the client device is reduced (e.g., when loading a web page with multiple content slots) compared to a situation where the client device must send a series of separate requests to the server for different slots, and / or (3) crashes are less likely to occur at the client device than if the client device must process a response to each of a series of separate requests sent to the server in order to determine whether to send the next request.

[0024] In the second aspect (a manner in which a server determines to perform one or more additional content selection processes based on the results of one or more earlier content selection processes for the same information resource), (1) the fill rate of content item slots is generally increased compared to a situation in which the server automatically performs a content selection process for all content slots in a multi-slot information resource (e.g., based on a single request from a client device), (2) latency or delay at the client device is reduced (e.g., when loading a web page with multiple content slots) compared to a situation in which the client device must send a series of separate requests to the server for different slots, and / or (3) crashes are less likely to occur at the client device than in a situation in which the client device must process a response to each of a series of separate requests sent to the server in order to determine whether to send the next request.

[0025] Furthermore, in both aspects, there is typically more information available to the server than in the case where, for a given information resource's slot, the client device issues multiple content requests and analyzes each request result / response. This information can help the server make more informed decisions (e.g., for future content selection) and / or allow the server to provide useful analysis or troubleshooting feedback (e.g., information indicating ad performance, or information indicating why an ad was not delivered, etc.).

[0026] Figure 1 An example system 100 is shown in which one or more techniques for efficiently providing content to a client device 102 may be implemented. The example system 100 includes a client device 102, a content server 104, a plurality of publishers 106, a plurality of content providers 108, and a network 110. The content server 104 is remote from the client device 102 and is communicatively coupled to the client device 102 via the network 110. Figure 1 The communication / network connections shown for the publisher 106 and the content provider 108 in represent communication / network connections with computing devices or systems associated with the publisher 106 and the content provider 108, respectively.

[0027] The network 110 may be a single communication network (e.g., the Internet), and in some implementations may also include one or more additional networks. As just one specific example, the network 110 may include a cellular network, the Internet, and a server-side local area network (LAN). Figure 1 Only a single client device 102 is shown, but it should be understood that the content server 104 may also communicate with a plurality of other client devices that are generally similar to the client device 102 .

[0028] Typically, the client device 102 can access information resources supplied or published by the publisher 106, and the content server 104 selects content items provided by the content provider 108 to fill the content slots within those information resources. For example, the information resource can be a web page of a website hosted by the publisher 106, and the content provider 108 can provide advertisements or other content items that the content server 104 can then select to fill the content slots within those web pages. As another example, the information resource can be a screen or user interface of an application (e.g., a mobile app) provided by the publisher 106, and the content provider 108 can provide advertisements or other content items that the content server 104 can use to fill the content slots within those user interfaces. As another example, the information resource can be a video, and the content slots can be distributed throughout the video in time. However, in some implementations, multiple (some or all) content slots of the information resource exist in the information resource at the same time without time distribution. The content item itself can be text, an image, or a video, or any combination thereof, and may or may not include an audio component.

[0029] The client device 102 may be or include any fixed, mobile, or portable computing device with wired and / or wireless communication capabilities (e.g., a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart wearable device (such as smart glasses or a smart watch), a vehicle head unit computer, etc.). Figure 1 In an example implementation of , the client device 102 includes a network interface 120, a processor 122, a memory 124, and a display 126. The processor 122 may be a single processor (e.g., a central processing unit (CPU)), or may include a set of processors (e.g., multiple CPUs, or one or more CPUs and one or more graphics processing units (GPUs)).

[0030] Memory 124 includes one or more computer-readable, non-transitory storage units or devices, which may include persistent (e.g., a hard disk) and / or non-persistent memory components. Memory 124 stores instructions that are executable on processing unit 122 to perform various operations, including instructions for various software applications and data generated and / or used by such applications. Figure 1 In an example implementation of , the memory 124 stores at least an application 130, which may be, for example, a web browser application (eg, Chrome) or a mobile application downloaded from an application store.

[0031] Generally speaking, the application 130 is executed by the processor 122 to present information resources to a user of the client device 102 via the display 126, wherein at least one of those information resources includes a plurality of content slots for dynamically presenting content items. In an implementation where the application 130 is a web browsing application, for example, the information resource may be a web page hosted by one of the publishers 106, wherein the browser causes the client device 102 to download the HTML, scripts, and / or other code of the web page for presentation to the user via the display 126. As another example, the application 130 may be a video sharing application, such as YouTube, and the information resource may be a user interface generated by the video sharing application and presented via the display 126. As yet another example, the application 130 may be a video playing application, and the information resource may be a video played by the video playing application.

[0032] The display 126 includes hardware, firmware, and / or software configured to enable a user to view visual output of the client device 102, and may use any suitable display technology (e.g., LED, OLED, LCD, etc.). In some implementations, the display 126 is incorporated into a touch screen having both display and manual input capabilities. Additionally, in some implementations where the client device 102 is a wearable device, the display 126 is a transparent viewing component with integrated electronic components (e.g., the lens of smart glasses). For example, the display 126 may include micro-LED or OLED electronics embedded in the lens of smart glasses.

[0033] The network interface 120 includes hardware, firmware, and / or software configured to enable the client device 102 to exchange electronic data with the content server 104 via the network 110. For example, the network interface 120 may include a cellular communication transceiver, a WiFi transceiver, and / or a transceiver for one or more other wired and / or wireless communication technologies.

[0034] Although Figure 1 Client device 102 is shown as a single component that communicates directly with content server 104 (ie, communicates via network 110), but in some implementations, Figure 1 The subcomponents of the client device 102 shown in are instead divided among two or more user-side devices. As just one example, a pair of smart glasses may include a processor 122, a memory 124, and a display 126, while a smartphone may include another processing unit, another memory, another display, and a network interface 120. The smart glasses (or smart helmet, etc.) can then communicate with the smartphone (e.g., via Bluetooth) as needed to implement the operations described herein.

[0035] Content server 104 includes a network interface 140, a processor 142, and a memory 144. Network interface 140 includes hardware, firmware, and / or software configured to enable client device 104 to exchange electronic data with client device 102 and other similar client devices via network 110. For example, network interface 140 may include a wired or wireless router and modem. Processor 142 may be a single processor or may include two or more processors. As used herein, the term "server" may refer to a single computing device located at a single location, or to multiple coordinated computing devices located at the same location or remotely distributed.

[0036] The memory 144 is a computer-readable non-transitory storage unit or device, or a collection of units / devices, which may include persistent and / or non-persistent memory components. The memory 144 stores instructions for the content selector 150, which may be executed by the processor 142. The content selector 150 includes a conditional content selection process (CSP) module 152, which includes a slot priority module 154, a content filter 156, and a relevance module 158. One, some, or all of the modules 154, 156, and 158 may be optionally omitted from the system 100.

[0037] In general, the content selector 150 selects content items that the content server 104 then provides / sends to the client device 102 (or other client devices) via the network 110. The content selector 150 selects particular content items using one or more content selection processes implemented by the conditional CSP module 152. The content filter 156 may apply one or more constraints to limit which content items are available as candidates for selection by the conditional CSP module 152 based on the results of one or more earlier content selection processes (as discussed further below) and / or based on one or more other factors (e.g., user preferences).

[0038] The content selection process performed by the conditional CSP module 152 can be an auction, for example, where the conditional CSP module 152 selects a winner based on bid amounts provided by individual content providers 108 to the content server 104 or bid amounts otherwise generated by the content server 104 for individual content providers 108 (e.g., based on input from the content providers 108, such as a preferred maximum bid amount). As a more specific example, each content provider 108 can provide a maximum bid amount for the delivery of its content item, and the conditional CSP module 152 can select the content item associated with the content provider 108 with the highest bid. The content server 104 can then charge the winning content provider 108 the highest bid amount (or the bid amount of the second highest bidder, etc.) and select the content item associated with that content provider 108 to fill the content slot under consideration.

[0039] The content selection process may also or alternatively consider selection parameters other than bid amount, such as relevance of a particular content item to the user of the client device 102 (or to an information resource, etc.), as determined by relevance module 158, and / or user preferences, etc. The content selection process may be entirely rule-based, or may have one or more random components (e.g., random selection from a pool of candidates after screening by content filter 156), etc.

[0040] At least some of the content selection processes performed by the conditional CSP module 152 have conditional aspects. In some implementations, for example, the performance of the content selection process for one or more given content slots is determined based on the results of the content selection process for one or more other content slots in the same information resource. Additionally or alternatively, in some implementations, the content filter 156 may conditionally apply or not apply one or more constraints that restrict which content items may be candidates for selection for one or more content slots based on the results of the content selection process for one or more other content slots in the same information resource. In some implementations, the slot priority module 154 determines the priority / ordering of individual content slots or groups / subsets of content slots, and the conditional CSP module 152 performs the conditional content selection process on one, some, or all of the content slots or subsets of content slots based on the determined ordering.

[0041] The operation of the conditional CSP module 152 and its component modules are discussed in further detail below in conjunction with various example implementations.

[0042] In some implementations, content items provided (e.g., created and / or generated) by content provider 108 and available for selection by content selector 150 may be stored in content database 160. Content database 160 may be stored in memory 144, or may be stored in one or more memories remote from content server 104, for example.

[0043] In some implementations, publishers 106 and / or content providers 108 hold accounts associated with services provided by content servers 104. For example, publishers 106 may create such accounts in order to monetize information resources that they publish or otherwise provide (e.g., by selling advertisements in content slots on web pages hosted by the publisher), and / or content providers 108 may create such accounts in order to locate and purchase content slots that are particularly conducive to presenting their content (e.g., advertisements). In these implementations, information associated with publisher and / or content provider accounts may be stored in an account database 162. Account database 162 may be stored in memory 144, or may be stored in one or more memories remote from content servers 104, for example. Account information may include information such as entity name, subscription level, entity preferences (e.g., brand control preferences), etc. In some implementations, account information includes selection parameters (e.g., bid amounts or maximum bid amounts) associated with different content providers 108 for use by conditional CSP module 152 to select content as discussed elsewhere herein.

[0044] In some implementations, the system 100 is arranged in a manner similar to Figure 1 For example, content server 104 may be associated with the same entity that generates the information resource, in which case system 100 may omit publisher 106 (e.g., the activities described herein with respect to publisher 106 may be performed by content server 104 instead). Additionally or alternatively, content server 104 may be associated with the same entity that generates content items for filling content slots in the information resource, in which case system 100 may omit content provider 108 (e.g., the activities described herein with respect to content provider 108 may be performed by content server 104 instead).

[0045] Figure 2 and Figure 3 depicts an example information resource with content slots, Figure 1 The content selector 150 is configured to select a content item for the content slot. Figure 2 or Figure 3 The information resource may be generated (eg, hosted) by one of publishers 106 and presented on display 126 by application 130 after client device 102 downloads the information resource (eg, downloads the information resource from publisher 106), for example, via network 110 and network interface 120.

[0046] Figure 2An information resource 200 is depicted that is a user interface for a video sharing application (e.g., YouTube), where the user interface may be provided as a web page (e.g., if application 130 is a web browser) or as a screen of a dedicated application (e.g., if application 130 is a dedicated video sharing application). Information resource 200 includes a primary or "header" content slot 202 and secondary content slots 204A-204D, where primary content slot 202 is more prominent because it is located higher within information resource 200 and has a larger size (height and / or width) relative to each of secondary content slots 204A-204D.

[0047] In some implementations where the application 130 is a web browser application and the information resource 200 is a web page hosted by one of the publishers 106, the application 130 downloads and executes the code (e.g., HTML code) of the web page to present the web page via the display 126. The web page code may also include a script (e.g., JavaScript) that, when interpreted by the application 130, causes the client device 102 to request content from the content server 104 to fill or conditionally fill at least some (i.e., two or more) of the content slots 202 and 204A to 204D. In some implementations where the application 130 is instead a dedicated application that generates the information resource 200, the application 130 causes the client device 102 to request the content from the content server 104 when the information resource 200 is first generated (e.g., when the application 130 is launched). In either case (web page or dedicated application screen), the request sent to the content server 104 may be as follows in conjunction with Figure 4 The single remote procedure call (RPC) in question.

[0048] Figure 3 An information resource 300 is depicted that is a user interface for a search engine, where the user interface may be provided as a web page (e.g., if the application 130 is a web browser) or as a screen of a dedicated application (e.g., if the application 130 is a search engine mobile application). The information resource 300 includes a plurality of sponsored content slots 320A-320C, where each successive slot further down within a list presented on the information resource 300 is followed by a plurality of content slots 330A-330F representing search results. The search may be based on a text string query entered by a user in a query field 302. In various implementations, the search results (e.g., uniform resource locator (URL) links) used to populate the content slots 330A-330F may be selected by the content server 104 or a server other than the content server 104 (e.g., a dedicated search engine server) using any suitable search technique or algorithm.

[0049] In some implementations where the application 130 is a web browser application and the information resource 300 is a web page, the application 130 downloads and executes the code (e.g., HTML code) of the web page to present the web page via the display 126. The web page code may also include a script (e.g., JavaScript) that, when interpreted by the application 130, causes the client device 102 to request content from the content server 104 to fill or conditionally fill at least the content slots 320A to 320C. In some implementations where the application 130 is instead a dedicated application that generates the information resource 300, the application 130 causes the client device 102 to request the content from the content server 104 when the information resource 300 is first generated (e.g., when the application 130 is launched). In either case (web page or dedicated application screen), the request sent to the content server 104 may be as follows in conjunction with Figure 4 The single RPC in question.

[0050] In some implementations, information resource 300 is provided in a format similar to Figure 3 The results may be presented in a different manner than shown. For example, the search engine may utilize generative artificial intelligence (e.g., based on a large language model (LLM)), and the information resource 300 may provide a single paragraph-style text response to the user's query rather than providing an ordered list of search results. In one such implementation, the sponsored content slots 320A-320C are interspersed in the generated text response at fixed or dynamically changing locations.

[0051] In some implementations, Figure 2 The contents of slots 202 and 204A to 204D or Figure 3 The content slots 320A to 320C of the information resource may all be configured to display content of the same general type (e.g., video only, image only, text only, or text and image, etc.). In other implementations, different content slots of the same information resource may display different types of content, depending on which content is selected for the slot by the content server 104. For example, the content selector 150 may be constrained (by the content filter 156) to select only video content items for the content slot 202, but may be able to select image (or image and text, etc.) or video (or video and text, etc.) content items for the remaining content slots 204A to 204D.

[0052] In some implementations, when one of the content slots 202, 204A-204D, 320A-320C is already filled with a content item, a user of the client device 102 may select the content item using an input device of the client device 102 (e.g., by tapping a touch screen, clicking a mouse button, etc.) to move to a page or other information resource associated with the selected content item. For example, if the content item is filled with Figure 2 The content in slot 202 or 204A or Figure 3 320A in the content slot 320A, the user's selection of the content item may cause the application 130 to direct the user to a login page hosted by the content provider's server. For example, the login page may allow the user to navigate to a page that enables the user to purchase the item advertised by the content item. In an implementation where the initial information resource (200 or 300) is a web page, this may include transferring the user from the initial web page URL to the URL / web page of the content provider. In an implementation where the initial information resource (200 or 300) is a dedicated application screen, this may include automatically opening a web browser application and opening the URL of the content provider's web page.

[0053] It should be understood that Figure 2 and Figure 3 For illustration purposes only, and the techniques described herein may alternatively be applied to information resources having different numbers of content slots, different sizes of content slots, different relative positions of content slots, etc. In addition, in some implementations and / or scenarios, a user may need to navigate across the information resource (e.g., by scrolling or panning) to view certain content slots. In some implementations, multiple content slots are distributed over time (e.g., if the information resource is a video, there are multiple slots for video ad insertion).

[0054] Now refer to Figure 4 Description wherein the content selector 150 selects content items to fill / populate according to some implementations Figure 2 The content slots 202, 204A to 204D or Figure 3 The contents of slots 320A to 320C are arranged in a manner. Figure 4 An example scenario 400 is depicted, wherein Figure 1 The content server 104 is Figure 1 4. The client device 102 of the embodiment of the present invention selects content and provides content to the client device 102. In the scenario 400, the client device 102 accesses 402 an information resource (e.g., the information resource 200 or 300) having multiple content slots. In an implementation in which the information resource is a web page hosted by one of the publishers 106, the event 402 may include the client device 102 using the URL of the web page to access the web page at least in part by receiving the code (e.g., HTML, JavaScript, etc.) of the web page from the publisher 106 via the network 110 (possibly after the user navigates to the website of the publisher 106). In an implementation in which the information resource is a user interface / screen of a (dedicated) application 130, the event 402 may include the client device 102 launching the application 130 and possibly navigating to a specific user interface provided by the application 130.

[0055] After (e.g., automatically and in response to) accessing 402 the information resource, the client device 102 (e.g., application 130) requests 404 content from the content server 104 to fill the content slot of the information resource. The request may consist of a single RPC. In some implementations where the information resource is a web page, event 404 may be automatically triggered by application 130 in response to application 130 interpreting a script embedded in the web page code received from the server hosting the web page. In some implementations where the information resource is a user interface / screen of a dedicated application (application 130), event 404 may be automatically triggered by application 130 in response to a user opening or launching application 130 or in response to a user navigating to a specific user interface / screen of application 130. Other triggers of event 404 are also possible.

[0056] In response to receiving the request, the content server 104 (e.g., the content selector 150) processes 410 the request. Event 410 may include, for example, reformatting the request message, and / or determining the capabilities of the client device 102 based on the request message, etc. In some implementations, event 410 is omitted. The slot priority module 154 prioritizes the content slots (e.g., Figure 2 The slots 202 and 204A to 204D or Figure 3 The slots 320A to 320C of the information resource are sorted 412. The sorting 412 is used to establish the order in which the conditional CSP module 152 will perform or consider performing the content selection process for a particular content slot or a particular subset of content slots. The slot priority module 154 can sort 412 the slots or subsets of slots of the information resource based on priority information sent by the client device 102 (e.g., in the request message at event 404) to the content server 104. For example, the client device 102 may have obtained the priority information from the information resource, from information associated with each content slot or subset of slots (e.g., a flag or field value), and / or from the application 130. In some implementations, the order / priority is fixed and event 412 is omitted.

[0057] Priority information may reflect the desirability of the content slot (e.g., prominence of location, size, etc.). For information resource 200, for example, content slot 202 may have the highest priority, content slot 204A may have the second highest priority, content slot 204B may have the third highest priority, content slot 204C may have the fourth highest priority, and content slot 204D may have the fifth highest priority. Alternatively, content slot 202 may have the highest priority, while content slots 204A to 204D may be grouped together and all have the same second highest priority. For information resource 300, content slot 320A may have the highest priority, content slot 320B may have the second highest priority, and content slot 320C may have the third highest priority. Other priorities and / or groupings are also possible.

[0058] At event 414, the conditional CSP module 152 performs a content selection process (e.g., an auction or relevance-based selection process, etc.) on the content slot with the highest priority (e.g., content slot 202 or 320A), or performs a first set of content selection processes on corresponding slots in the subset of content slots with the highest priority (e.g., content slots 302A and 302B). The conditional CSP module 152 then processes 416 the results of the content selection process performed at event 414 to determine how or whether the conditional selection process should be performed on the next highest priority content slot or subset of slots at event 420.

[0059] In some implementations, at event 416, conditional CSP module 152 decides whether to perform or not perform event 420 (and possibly any other subsequent similar operations, such as event 424) based on the result. For example, (1) if the content selection process at event 414 results in any type of content item being selected for content slot 202, conditional CSP module 152 may determine not to perform a content selection process (e.g., auction) for content slot 204A, but (2) if the content selection process at event 414 results in no type of content item being selected for content slot 202, conditional CSP module 152 may determine that the content selection process should be performed. As another example, conditional CSP module 152 may determine that (1) the content selection process is not to be performed on any of content slots 204A-204D if the content selection process at event 414 results in a content item being selected for content slot 202, but (2) the content selection process is not to be performed on each of content slots 204A-204D if the content selection process at event 414 results in no content item being selected for content slot 202. As yet another example, conditional CSP module 152 may determine that (1) the content selection process is not to be performed on content slot 320B if the content selection process at event 414 results in a content item being selected for content slot 320A, but (2) the content selection process is to be performed if the content selection process at event 414 results in no content item being selected for content slot 320A.

[0060] The content selection process performed by the conditional CSP module 152 may not result in the selection of a content item for a particular content slot for various reasons, depending on the implementation and scenario. For example, the content selection process may result in the content server 104 rejecting a content item due to security issues associated with the content item (e.g., suspicious scripts within the content item) or due to incompatibility with the preferences of the user of the client device 102 or the publisher 106 (e.g., if the content filter 156 applies certain constraints after the content selection process is completed), etc.

[0061] In other implementations, when determining whether to execute or not execute event 420 (and / or Figure 4414), processing at event 416 analyzes other aspects of the results of the content selection process at event 414. For example, if the content selection process for content slot 202 at event 414 results in a content item being selected with a bid exceeding a certain minimum dollar amount, conditional CSP module 152 may determine not to perform the content selection process for any of content slots 204A to 204D. In another implementation, conditional CSP module 152 may determine whether to perform the content selection process at event 414 based on one or more characteristics of the one or more content items selected at event 414. For example, if the content selection process at event 414 results in a content item of a particular type (e.g., a video) being selected for content slot 202, conditional CSP module 152 may determine not to perform the content selection process for any of content slots 204A to 204D at event 420.

[0062] In another example implementation, the conditional CSP module 152 conditionally performs or does not perform the content selection process at event 420 based on the relevance score of at least one content item selected during the content selection process (or processes) at event 414. For example, the relevance module 158 (or another module / software, or perhaps another server) may use a machine learning model (e.g., a neural network) to determine the relevance score of each of the one or more content items selected at event 414, and only when the determined relevance score (or each of a certain number of relevance scores) exceeds a certain predetermined threshold, the relevance module 158 may perform the content selection process at event 420. The relevance score may indicate, for example, relevance to a user's search query (e.g., a text string), or relevance to an information resource, etc.

[0063] As a more specific example, in a scenario where information resources 200 are presented to a user of client device 102 in response to a video search query by the user, relevance module 158 may use a machine learning model to calculate a relevance or search score for the content item selected by content selector 150 (at event 414) to fill content slot 202. If the relevance / search score is greater than a certain threshold (meaning that the content item is highly relevant to the search), conditional CSP module 152 runs another content selection process (e.g., an auction) to fill content slot 204A (or runs such a process for each of slots 204A-204D) at event 420. However, if the search score for the content item is not greater than the threshold, conditional CSP module 152 does not run the content selection process at event 420 to reduce the likelihood of annoying the user with irrelevant or useless information.

[0064] In some implementations, at event 416, conditional CSP module 152 determines, alternatively (or additionally) based on the results of the content selection process at event 414, the manner in which event 420 is to be performed. Specifically, content filter 156 may apply one or more constraints to limit which content items (from content database 160) are available as candidates in the content selection process at event 420 based on the format or content of the content items selected at event 414.

[0065] For example, the conditional CSP module 152 may exclude a content item from consideration at event 420 based on the category of the one or more content items selected at event 414, the user cost associated with one or more of the selected content items, the format of one or more of the selected content items (e.g., excluding content items having the same format as the selected content items), and / or the similarity level with one or more of the selected content items. The similarity level may be indicated by a similarity score. The content server 104 (e.g., the content selector 150) may use machine learning techniques (such as a neural network or other machine learning model trained on the content item pairs using supervised or unsupervised learning techniques) to determine the similarity score of the content item pairs. The machine learning model may operate on certain parameters (e.g., content item format, category, length, etc.) indicating the characteristics of the content items (accepted as input), or may operate directly on, for example, image, text, or video frame information. In some implementations, the content server 104 uses generative artificial intelligence (e.g., a large language model (LLM)) to determine the similarity level between two content items that both include text.

[0066] By applying constraints based on the category or format of an earlier selected content item and / or based on similarity to an earlier selected content item, scenarios in which too many content items of a particular type / format / etc. are selected can be avoided, thereby providing various benefits. For example, if the content slot corresponds to an in-stream video ad spot, it may be desirable to avoid having the client device user experience multiple long video ads (e.g., exceeding a certain threshold length, such as 60 seconds). Thus, if a long video ad is selected for the first spot, the content filter 156 can exclude any long video ads (e.g., exceeding a threshold length) (as a candidate for the content selection process at event 420) for one or more subsequent ad spots. As another example, the content filter 156 can help enforce brand safety constraints associated with an ad or content provider. For example, the content filter 156 can exclude any ads associated with a competitor of the content provider whose ad was selected at event 414 (as a candidate for the content selection process at event 420). A competitor list may be associated with each content provider's 108 account in account database 162, or content server 104 may access other information (e.g., the categories of goods or services sold indicated in databases 160 or 162) to identify possible competitors, etc.

[0067] In some implementations, content filter 156 applies its constraints to content items using a machine learning model. For example, event 420 may include applying one or more parameters indicating the format or content of the content item selected at event 414 as input to a machine learning model (e.g., a neural network), and deriving at least one of the constraints from an output of the machine learning model.

[0068] like Figure 4 As can be seen, if the slot priority module 154 determines that there are content slots (or slot subsets) with subsequent priority levels, additional events similar to events 416 and / or 420 may occur or conditionally occur. For example, the conditional CSP module 152 may process the result of the content selection process of event 420 at event 422 (e.g., similar to event 416), and conditionally perform or not perform one or more further content selection processes (or conditionally apply constraints to the performance of one or more further content selection processes) at event 424 (e.g., similar to event 420), and so on.

[0069] In some implementations where certain content slots (e.g., slots 204A through 204D) are grouped together into subsets for prioritization / ranking purposes, the conditional CSP module 152 performs the content selection process on all content slots within the group / subset in parallel (i.e., simultaneously) or at least partially in parallel (i.e., at least partially overlapping in time) in order to reduce latency associated with populating the content slots of the information resource.

[0070] After events 410 to 424 are completed (or may not be performed at all in the case of events 420 to 424, as discussed above), the content server 104 responds 430 to the client device 102 (e.g., returns an RPC response). The response for event 430 may include the content item selected during one or more of the content selection processes of events 414, 420, 424, or may include a link to the selected content item. At event 430, the client device 102 fills the appropriate content slot of the information resource with the selected content item. In some implementations, the client device 102 need not issue any RPCs other than the single RPC of event 404 in order to obtain the content item for filling the slot of the information resource.

[0071] Now refer to Figure 5 and Figure 6 Example methods for efficiently selecting content for client devices are discussed. Figure 5 and Figure 6 Either or both of the methods may be implemented as instructions stored on one or more computer-readable media and executed on one or more processors in one or more computing devices. For example, the method may be implemented by Figure 1 The processor 142 of the content server 104 in the implementation when executing the instructions of the content selector 150.

[0072] First reference Figure 5 In a method 500, a request is received at block 502 from a client device (e.g., client device 102) for content to be included in an information resource having a plurality of content slots (e.g., event 404). At block 504, one or more content items are selected in response to the request. Block 504 includes performing a first one or more content selection process, at least by using selection parameters associated with candidate content items (e.g., bid amounts, relevance scores, etc.), to determine a first content selection result for a first subset of the plurality of content slots (e.g., event 414). Block 504 also includes conditionally performing or not performing a second one or more content selection process on a second subset of the plurality of content slots based on the first content selection result. In some implementations, the first subset of the plurality of content slots consists of only a single content slot, and / or the second subset of the plurality of content slots consists of only a single content slot. In other implementations, one or both subsets include a plurality of content slots.

[0073] At block 506 , the client device is caused to fill one or more content slots of the plurality of content slots with the selected one or more content items. For example, block 506 may correspond to event 430 .

[0074] In some implementations, block 504 includes prohibiting (not performing) a second content selection process when a first set of one or more content items is selected to fill a first subset of the plurality of content slots, and performing the second content selection process instead when no content items are selected to fill the first subset of the plurality of content slots.

[0075] In some implementations, determining the first content selection result at block 504 includes selecting a first set of one or more content items to fill a first subset of the plurality of content slots, and selecting the one or more content items at block 504 includes conditionally performing or not performing a second content selection process based on one or more characteristics of the content items of the first set. The one or more characteristics of the content items of the first set may include one or more relevance scores associated with the content items of the first set (e.g., determined by the relevance module 158 using a machine learning model).

[0076] In some implementations, method 500 also includes, prior to selecting the one or more content items, determining that the first subset has a higher priority than the second subset, and performing the first content selection process at block 504 in response to the determination.

[0077] In some implementations, a first subset of multiple content slots includes at least a first content slot and a second content slot, the first content selection process includes a third content selection process for the first content slot and a fourth content selection process for the second content slot, and executing the first content selection process includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

[0078] Next reference Figure 6 In a method 600 of an information resource having a plurality of content slots, a request is received from a client device (e.g., client device 102) at block 602 for content to be included in an information resource having a plurality of content slots (e.g., event 404). At block 604, a plurality of content items are selected in response to the request. Block 604 includes performing a first one or more content selection process (e.g., event 414) to select a first set of one or more content items for a first subset of the plurality of content slots (e.g., slot 202, slot 320A, or slots 320A and 320B, etc.) by at least using selection parameters associated with the candidate content items (e.g., bids, relevance scores, etc.). Block 604 also includes applying one or more constraints based on the format or content of the content items of the first set to limit which content items may be candidates in a second one or more content selection process for a second subset of the plurality of content slots. In some implementations, the first subset of the plurality of content slots consists of only a single content slot, and / or the second subset of the plurality of content slots consists of only a single content slot. In other implementations, one or both subsets include a plurality of content slots.

[0079] At block 606 , the client device is caused to fill at least some of the plurality of content slots with the selected content items. For example, block 606 may correspond to event 430 .

[0080] In some implementations, applying the one or more constraints includes excluding content items from the second content selection process based on content item categories (e.g., excluding content items associated with a category of at least one of the content items of the first episode), excluding content items from the second content selection process based on user costs associated with at least one of the content items of the first episode, excluding content items having the same format as at least one of the content items of the first episode, and / or excluding content items based on a similarity score indicating a level of similarity to at least one of the content items of the first episode. In the latter case, method 600 may further include determining the similarity score using a machine learning model.

[0081] In some implementations, applying the constraints at box 604 includes applying one or more parameters indicating the format or content of the content items of the first episode as input to the machine learning model, and deriving at least one of the constraints from an output of the machine learning model.

[0082] In some implementations, method 600 further includes determining, prior to selecting the content item at block 604, that the first subset has a higher priority than the second subset, and performing the first content selection process at block 604 is responsive to determining that the first subset has a higher priority.

[0083] In some implementations, the first subset of content slots includes at least a first content slot and a second content slot, the first content selection process includes a third content selection process for the first content slot and a fourth content selection process for the second content slot, and executing the first content selection process at box 604 includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

[0084] In some implementations, the technology disclosed herein uses artificial intelligence to facilitate efficient selection of content items to fill content slots in information resources. Artificial intelligence (AI) is a branch of computer science that focuses on creating models that can perform tasks with little human intervention. For example, artificial intelligence systems can utilize machine learning, natural language processing, and computer vision. Machine learning and its subsets (such as deep learning) focus on developing models that can infer outputs from data. Outputs may include, for example, predictions and / or classifications. Natural language processing focuses on analyzing and generating human language. Computer vision focuses on analyzing and interpreting images and videos. Artificial intelligence systems may include generative models that generate new content, such as images, videos, text, audio, and / or other content, in response to input prompts and / or based on other information.

[0085] Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some machine learning models may include multi-head self-attention models (e.g., transformer models).

[0086] A variety of training or learning techniques can be used to train the model. Training can implement supervised learning, unsupervised learning, reinforcement learning, etc. Training can use techniques such as, for example, error back propagation. For example, a loss function can be back propagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques can be used to iteratively update parameters in multiple training iterations. A variety of generalization techniques (e.g., weight decay, dropout) can be used to improve the generalization ability of the model being trained.

[0087] The model may be pre-trained prior to domain-specific alignment. For example, the model may be pre-trained on a general training data corpus and fine-tuned on a more targeted training data corpus. The model may be aligned using prompts designed to elicit domain-specific outputs. The prompts may be designed to include learned prompt values ​​(e.g., soft prompts). The trained model may be validated prior to its use using input data other than the training data, and may be further updated or refined during its use based on additional feedback / input.

[0088] In some implementations, the content server 104 (e.g., the conditional CSP module 152) can use any one or more of the above-described machine learning models in conjunction with machine learning to perform any one or more of the operations discussed herein. For example, the content server 104 can use one or more such machine learning models to determine relevance scores for content items, determine similarity scores between content items, and / or determine constraints to be applied by the content filter 156, as discussed above.

[0089] Although the foregoing text describes many different aspects and implementations of the present invention in detail, it should be understood that the scope of this patent is limited by the text of the claims set forth at the end of this patent. The detailed description should be understood as exemplary only and does not describe every possible implementation, because it is impractical or even impossible to describe every possible implementation. A variety of alternative implementations may be implemented using prior art or technology developed after the filing date of this patent, which still falls within the scope of the claims. The disclosure herein contemplates at least the following examples:

[0090] Example 1. A method for efficiently providing content to a client device, the method comprising: receiving, by a server, a request from the client device for content to be included in an information resource having multiple content slots; selecting, by the server in response to the request, one or more content items, comprising: (i) performing a first one or more content selection processes at least by using selection parameters associated with candidate content items to determine a first content selection result for a first subset of the multiple content slots; and (ii) conditionally performing or not performing a second one or more content selection processes for a second subset of the multiple content slots based on the first content selection result; and causing the server to fill one or more content slots of the multiple content slots with the selected one or more content items.

[0091] Example 2. A method as described in Example 1, wherein selecting the one or more content items includes: when a first set of one or more content items is selected to fill the first subset of the multiple content slots, not performing the second one or more content selection processes; and when no content any content items are selected to fill the first subset of the multiple content slots, performing the second one or more content selection processes.

[0092] Example 3. A method as described in Example 1, wherein: determining the first content selection result includes selecting a first set of one or more content items to fill the first subset of the multiple content slots; and selecting the one or more content items includes conditionally executing or not executing the second one or more content selection processes based on one or more characteristics of the content items of the first set.

[0093] Example 4. The method of Example 3, wherein the one or more characteristics of the content items of the first set include one or more relevance scores associated with the content items of the first set.

[0094] Example 5. The method of Example 4 further includes: the server using a machine learning model to determine the one or more relevance scores.

[0095] Example 6. The method as described in any one of Examples 1 to 5 further includes: before selecting the one or more content items, the server determines that the first subset has a higher priority than the second subset, wherein the first one or more content selection processes are performed in response to determining that the first subset has a higher priority than the second subset.

[0096] Example 7. A method as described in Example 6, wherein: the first subset of the multiple content slots includes at least a first content slot and a second content slot; the first one or more content selection processes include a third content selection process for the first content slot and a fourth content selection process for the second content slot; and executing the first one or more content selection processes includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

[0097] Example 8. A method as described in any of Examples 1 to 6, wherein one or both of the following: the first subset of the plurality of content slots consists of a single content slot; and the second subset of the plurality of content slots consists of a single content slot.

[0098] Example 9. A method as described in any of Examples 1 to 8, wherein the request is a remote procedure call (RPC).

[0099] Example 10. A method for efficiently providing content to a client device, the method comprising: a method for efficiently providing content to a client device, the method comprising: receiving, by a server from the client device, a request for content to be included in an information resource having multiple content slots; in response to the request, selecting, by the server, multiple content items, comprising: selecting a first set of one or more content items for a first subset of the multiple content slots by performing a first one or more content selection processes at least by using selection parameters associated with candidate content items; and applying one or more constraints to limit which content items can serve as candidates in a second one or more content selection processes for a second subset of the multiple content slots based on the format or content of the content items in the first set; and causing the client device, by the server, to fill at least some of the multiple content slots with the selected multiple content items.

[0100] Example 11. The method of Example 10, wherein applying the one or more constraints comprises excluding content items from the second one or more content selection processes based on content item categories.

[0101] Example 12. The method of Example 11, wherein excluding content items from the second one or more content selection processes based on the content item category comprises: excluding content items associated with a category of at least one content item in the first set of content items.

[0102] Example 13. The method of any one of Examples 10 to 12, wherein applying the one or more constraints comprises excluding a content item from the second one or more content selection processes based on a user cost associated with at least one content item of the first set of content items.

[0103] Example 14. The method of any one of Examples 10 to 13, wherein applying the one or more constraints comprises excluding content items having the same format as at least one content item in the first set of content items.

[0104] Example 15. The method of any one of Examples 10 to 14, wherein applying the one or more constraints comprises excluding content items based on a similarity score indicating a level of similarity to at least one content item of the first set of content items.

[0105] Example 16. The method of any one of Examples 10 to 14, further comprising: the server using a machine learning model to determine the similarity score.

[0106] Example 17. A method as described in any of Examples 10 to 16, wherein applying the one or more constraints includes: applying one or more parameters indicating the format or content of the content items of the first set as input to a machine learning model; and deriving at least one of the one or more constraints from an output of the machine learning model.

[0107] Example 18. A method as described in any of Examples 10 to 17, wherein the request is a remote procedure call (RPC).

[0108] Example 19. The method as described in any one of Examples 10 to 18 further includes: before selecting the multiple content items, the server determines that the first subset has a higher priority than the second subset, wherein performing the first one or more content selection processes is in response to determining that the first subset has a higher priority than the second subset.

[0109] Example 20. A method as described in Example 19, wherein: the first subset of the multiple content slots includes at least a first content slot and a second content slot; the first one or more content selection processes include a third content selection process for the first content slot and a fourth content selection process for the second content slot; and executing the first one or more content selection processes includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

[0110] Example 21. A method as described in any of Examples 10 to 20, wherein one or both of the following: the first subset of the plurality of content slots consists of a single content slot; and the second subset of the plurality of content slots consists of a single content slot.

[0111] Example 22. A server comprising: one or more processors; and one or more memories, wherein the one or more memories store instructions that, when executed by the one or more processors, cause the one or more processors to perform a method as described in any one of Examples 1 to 21.

[0112] The following additional considerations apply to the foregoing discussion. Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although the individual operations of one or more methods are shown and described as separate operations, one or more of the individual operations may be performed simultaneously, and it is not required to perform the operations in the order shown. The structures and functions presented as separate components in the example configuration may be implemented as combined structures or components. Similarly, the structures and functions presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements all fall within the scope of the subject matter of the present disclosure.

[0113] Unless expressly stated otherwise, discussions in this disclosure using words such as "process," "compute," "calculate," "determine," "present," "display," etc. may refer to the action or process of a machine (e.g., a computer) manipulating or transforming data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0114] As used in this disclosure, any reference to "one implementation" or "implementation" means that a particular element, feature, structure, or characteristic described in conjunction with the implementation is included in at least one implementation or implementation. The appearance of the phrase "in one implementation" in various places in this specification does not necessarily refer to the same implementation.

[0115] As used in this disclosure, the terms "comprises," "comprising," "includes," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that includes a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, condition A or condition B is satisfied by any of the following: A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exists).

[0116] After reading this disclosure, those skilled in the art will understand additional alternative structures and functional designs for efficiently selecting content and providing content to client devices to fill the content slots of information resources through the principles described herein. Therefore, although specific implementations and applications have been illustrated and described, it should be understood that the disclosed implementations are not limited to the precise structures and components disclosed in this disclosure. Various modifications, changes and variations may be made to the arrangement, operation and details of the methods and devices disclosed in this disclosure without departing from the spirit and scope defined in the appended claims, and these modifications, changes and variations will be apparent to those skilled in the art.

Claims

1. A method for efficiently providing content to a client device, the method comprising: receiving, by a server from the client device, a request for content to be included in an information resource having a plurality of content slots; In response to the request, the server selects one or more content items, including determining a first content selection result for a first subset of the plurality of content slots by performing a first one or more content selection processes using at least selection parameters associated with the candidate content items, and conditionally performing or not performing a second one or more content selection processes on a second subset of the plurality of content slots based on the first content selection result; as well as The client device is caused, by the server, to fill one or more content slots of the plurality of content slots with the selected one or more content items.

2. The method of claim 1 , wherein selecting the one or more content items comprises: when selecting a first set of one or more content items to fill said first subset of said plurality of content slots, not performing said second one or more content selection processes; as well as The second one or more content selection processes are performed when no content any content items are selected to fill the first subset of the plurality of content slots.

3. The method of claim 1, wherein: Determining the first content selection result includes selecting a first set of one or more content items to fill the first subset of the plurality of content slots; as well as Selecting the one or more content items includes conditionally performing or not performing the second one or more content selection processes based on one or more characteristics of the content items of the first set. 4 . The method of claim 3 , wherein the one or more characteristics of the content items of the first collection include one or more relevance scores associated with the content items of the first collection.

5. The method of claim 4, further comprising: The one or more relevance scores are determined by the server using a machine learning model.

6. The method according to any one of claims 1 to 5, further comprising: prior to selecting the one or more content items, determining, by the server, that the first subset has a higher priority than the second subset, Wherein performing the first one or more content selection processes is in response to determining that the first subset has a higher priority than the second subset.

7. The method of claim 6, wherein: said first subset of said plurality of content slots comprises at least a first content slot and a second content slot; the first one or more content selection processes include a third content selection process for the first content slot and a fourth content selection process for the second content slot; and Executing the first one or more content selection processes includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

8. The method of any one of claims 1 to 6, wherein one or both of the following: said first subset of said plurality of content slots consists of a single content slot; and The second subset of the plurality of content slots consists of a single content slot.

9. The method of any one of claims 1 to 8, wherein the request is a remote procedure call (RPC).

10. A method for efficiently providing content to a client device, the method comprising: receiving, by a server from the client device, a request for content to be included in an information resource having a plurality of content slots; In response to the request, the server selects a plurality of content items, including selecting a first set of one or more content items for a first subset of the plurality of content slots by performing a first one or more content selection processes using at least selection parameters associated with candidate content items; as well as applying one or more constraints to limit which content items are candidates for a second one or more content selection process for a second subset of the plurality of content slots based on the format or content of the content items of the first set; as well as The client device is caused, by the server, to fill at least some of the plurality of content slots with the selected plurality of content items.

11. The method of claim 10, wherein applying the one or more constraints comprises: Content items are excluded from the second one or more content selection processes based on content item categories.

12. The method of claim 11, wherein excluding content items from the second one or more content selection processes based on content item categories comprises: Content items associated with a category of at least one of the content items of the first set of content items are excluded.

13. The method of any one of claims 10 to 12, wherein applying the one or more constraints comprises: Content items are excluded from the second one or more content selection processes based on a user cost associated with at least one content item in the first set of content items.

14. The method of any one of claims 10 to 13, wherein applying the one or more constraints comprises: Content items having the same format as at least one content item of the content items of the first set are excluded.

15. The method of any one of claims 10 to 14, wherein applying the one or more constraints comprises: Content items are excluded based on a similarity score indicating a level of similarity to at least one content item of the first set of content items.

16. The method of any one of claims 10 to 14, further comprising: The similarity score is determined by the server using a machine learning model.

17. The method of any one of claims 10 to 16, wherein applying the one or more constraints comprises: applying as input to a machine learning model one or more parameters indicative of the format or content of the content items of the first set; as well as At least one of the one or more constraints is derived from an output of the machine learning model.

18. The method of any one of claims 10 to 17, wherein the request is a remote procedure call (RPC).

19. The method of any one of claims 10 to 18, further comprising: prior to selecting the plurality of content items, determining, by the server, that the first subset has a higher priority than the second subset, Wherein performing the first one or more content selection processes is in response to determining that the first subset has a higher priority than the second subset.

20. The method of claim 19, wherein: said first subset of said plurality of content slots comprises at least a first content slot and a second content slot; the first one or more content selection processes include a third content selection process for the first content slot and a fourth content selection process for the second content slot; and Executing the first one or more content selection processes includes executing the third content selection process at least partially in parallel with executing the fourth content selection process.

21. The method of any one of claims 10 to 20, wherein one or both of the following: said first subset of said plurality of content slots consists of a single content slot; and The second subset of the plurality of content slots consists of a single content slot.

22. A server, comprising: one or more processors; and one or more memories storing instructions which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 21.

Images