Recommendation method and system and storage medium

Abstract

The invention provides a recommendation method and system and a storage medium. The recommendation method comprises the steps that the recommendation system can receive a recommendation request and sequentially execute screening of a plurality of links on an initial set containing a plurality of objects until at least one object to be recommended is obtained through screening, and the recommendation request is responded based on the at least one object. The screening process of the recommendation system in a target link in the links comprises the steps that the recommendation system determines an available queue length range of the target link based on an input set of the target link by taking the recommendation system meeting a preset response duration constraint as a target, and selects an available queue length range of the target link by taking the recommendation system meeting a preset computing power constraint as a target; and determining a target queue length in the queue length range, and based on the target queue length, screening out a part of objects from the input set as an output set of the target link. When recommendation is carried out based on the recommendation method, the recommendation system can meet the response duration constraint and the computing power constraint at the same time, and the recommendation effect is improved.

Description

Technical Field

[0001] This specification relates to the field of Internet technology, and in particular to a recommendation method, system, and storage medium. Background Art

[0002] When recommending content on the internet, recommendation systems typically respond to recommendation requests through multiple screening stages, including recall, rough sorting, and refined sorting. During this screening stage, the recommendation system predicts and scores the recommendation evaluation indicators of candidate content to select content with better recommendation evaluation indicators.

[0003] The recommendation system consumes a certain amount of computing power when predicting and scoring the recommendation evaluation indicators of candidate recommendation content during the screening process. The recommendation system can allocate different computing power resources to each recommendation request to optimize the overall recommendation evaluation indicators of the entire recommendation system.

[0004] Currently, recommendation systems can use the overall computing power of the recommendation system as a constraint to allocate computing power for each recommendation request. However, this method of allocating computing power is not accurate enough and may lead to poor recommendation results.

[0005] The content of the background technology section is merely information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the Invention

[0006] This specification provides a recommendation method, system, and storage medium that enable a recommendation system to respond to recommendation requests while meeting response time constraints and computing power constraints, thereby improving the recommendation effect of the recommendation system.

[0007] In the first aspect, the present specification provides a recommendation method, which is applied to a recommendation system, and the method includes: receiving a recommendation request; and performing multiple screening steps on an initial set containing multiple objects in sequence until at least one object to be recommended is screened out, and responding to the recommendation request based on the at least one object, wherein the multiple steps include a target step, and the screening process of the target step includes: obtaining an input set of the target step, wherein the input set is the initial set or the output set of the previous step, determining the queue length range available for the target step with the goal of the recommendation system meeting a preset response time constraint, determining a target queue length within the queue length range with the goal of the recommendation system meeting a preset computing power constraint, wherein the predicted recommendation index corresponding to the target queue length is greater than the predicted recommendation index corresponding to other queue lengths within the queue length range, and based on the target queue length, screening out some objects in the input set as the output set of the target step.

[0008] In some embodiments, the response time constraint includes: the response time of the recommendation system does not exceed the preset response time, and with the goal of the recommendation system meeting the preset response time constraint, the queue length range available for the target link is determined, including: determining the first queue length that is maximally supported by the target link based on the preset response time; obtaining the second queue length that is minimally supported by the target link; and determining the queue length range based on the first queue length and the second queue length.

[0009] In some embodiments, the queue length range is determined based on the first queue length and the second queue length, including: obtaining a target correction parameter corresponding to the recommendation system, the target correction parameter is obtained based on the first operating data of the recommendation system in a historical period, and is used to correct the queue length range to adjust the response timeout of the recommendation system in the current period; correcting the first queue length based on the target correction parameter to obtain a third queue length, the third queue length being smaller than the first queue length; and using the third queue length as the upper limit of the queue length range and the second queue length as the lower limit of the queue length range to obtain the queue length range.

[0010] In some embodiments, the first operating data includes the actual response timeout rate of the recommendation system during the historical period, and the target correction parameter is obtained in the following manner: obtaining the target response timeout rate of the recommendation system; and determining the target correction parameter based on the difference between the actual response timeout rate and the target timeout rate.

[0011] In some embodiments, the target correction parameter includes a queue length increment or a queue length multiple, and the first queue length is corrected based on the target correction parameter to obtain a third queue length, including: obtaining the third queue length based on the sum of the first queue length and the queue length increment, or obtaining the third queue length based on the product of the first queue length and the queue length multiple.

[0012] In some embodiments, the maximum supported first queue length of the target link is determined based on the preset response time, including: determining the remaining processing time based on the preset response time and the processed time of the recommendation request; determining the maximum supported first queue length of the target link based on the remaining processing time and the fourth queue length corresponding to the target link in the current time period, wherein the fourth queue length in each time period is pre-set based on the number of historical recommendation requests of the recommendation system in the time period.

[0013] In some embodiments, with the goal of the recommendation system satisfying a preset computing power constraint, a target queue length is determined within the queue length range, including: obtaining a first coefficient corresponding to the recommendation system, the first coefficient being determined with the goal of the recommendation system satisfying the preset computing power constraint, and characterizing the extent of change in the recommendation indicator when each unit of computing power is increased in the recommendation system; determining the predicted recommendation evaluation indicator corresponding to each queue length within the queue length range based on the first coefficient; and determining the target queue length within the queue length range based on the predicted recommendation evaluation indicator corresponding to each queue length within the queue length range.

[0014] In some embodiments, the first coefficient is obtained in the following manner, including: obtaining actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests within a historical period, and determining the second coefficient based on the actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests; obtaining second operating data of the recommendation system within the historical period, the second operating data representing the use of computing power by the recommendation system; and with the goal of the recommendation system meeting a preset computing power constraint, correcting the second coefficient based on the second operating data to obtain the first coefficient.

[0015] In some embodiments, the computing power constraint includes: the resource utilization rate of the recommendation system does not exceed the target resource utilization rate, and the second coefficient is determined based on the actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests, including: determining the second coefficient based on the target resource utilization rate, and the actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests; the second operating data includes the actual resource utilization rate of the recommendation system in the historical period, with the goal of the recommendation system meeting the preset computing power constraint, and correcting the second coefficient based on the second operating data to obtain the first coefficient, including: updating the second coefficient based on the actual resource utilization rate and the target resource utilization rate to obtain the first coefficient.

[0016] In some embodiments, determining the predicted recommended evaluation index corresponding to each queue length within the queue length range based on the first coefficient includes: for each candidate queue length within the queue length range: obtaining a first evaluation index that can be generated by the candidate queue length; based on the first coefficient and the resource consumption required for the candidate queue length, obtaining a second evaluation index that needs to be consumed by the candidate queue length; and determining the predicted recommended evaluation index corresponding to the candidate queue length based on the difference between the first evaluation index and the second evaluation index.

[0017] In some embodiments, the recommendation request is triggered by a target user, and obtaining the first evaluation indicator that can be generated by the candidate queue length includes: predicting the value score of the target user; and determining the first evaluation indicator that can be generated by the candidate queue length based at least on the value score of the target user.

[0018] In some embodiments, determining a first evaluation indicator that can be generated by the candidate queue length is performed at least based on the value score of the target user, including: predicting the exposure probability corresponding to the recommendation request; obtaining the target weight corresponding to the candidate queue length; and determining the first evaluation indicator that can be generated by the candidate queue length based on the value score of the target user, the exposure probability and the target weight.

[0019] In some embodiments, the target queue length is determined within the queue length range based on the predicted recommendation evaluation indicators corresponding to each queue length within the queue length range, including: determining the largest predicted recommendation evaluation indicator among the predicted recommendation evaluation indicators corresponding to each queue length within the queue length range; and using the queue length corresponding to the largest predicted recommendation evaluation indicator as the target queue length.

[0020] In some embodiments, the multiple links include: a recall link, a rough sorting link, a fine sorting link and a re-sorting link; the screening process of the multiple links satisfies the following conditions: the target queue length determined in the screening process of the i+1th link is less than the target queue length determined in the screening process of the i-th link.

[0021] In a second aspect, this specification provides a recommendation system, comprising: at least one storage medium storing at least one instruction set for performing data processing related to content recommendation; and at least one processor communicatively connected to the at least one storage medium, wherein when the recommendation system is running, the at least one processor reads the at least one instruction set and implements the method described in any one of the first aspects according to the instructions of the at least one instruction set.

[0022] In a third aspect, this specification also provides a computer-readable non-volatile storage medium, wherein the computer-readable non-volatile storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, it implements the recommended method provided in any one of the first aspects.

[0023] Other functions of the recommended method provided by this specification will be partially listed in the following description. The creative aspects of the recommended method provided by this specification can be fully explained by practicing or using the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 A schematic diagram showing a recommendation scenario provided according to an embodiment of this specification is shown;

[0026] Figure 2 shows a hardware structure diagram of a computing system provided according to an embodiment of this specification;

[0027] Figure 3 A flowchart of a recommended method provided according to an embodiment of this specification is shown;

[0028] Figure 4 A schematic diagram showing the flow of the first and second stages of the target link provided according to an embodiment of this specification; and

[0029] Figure 5The flowchart of the recommended method provided by the embodiment of this specification is shown. DETAILED DESCRIPTION

[0030] The following description provides specific application scenarios and requirements for this specification, with the goal of enabling those skilled in the art to make and use the contents of this specification. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but is intended to be accorded the broadest scope consistent with the claims.

[0031] The terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. For example, as used herein, the singular forms "a," "an," and "the" may also include the plural forms unless the context clearly indicates otherwise. When used in this specification, the terms "comprise," "include," and / or "contain" are intended to refer to the presence of the associated integers, steps, operations, elements, and / or components, but do not preclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups or the addition of other features, integers, steps, operations, elements, components, and / or groups in the system / method.

[0032] These and other features of this specification, as well as the operation and function of the associated elements of the structure, and the economical assembly and manufacture of the components, can be significantly improved with consideration of the following description. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be expressly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0033] The flowcharts used in this specification illustrate operations implemented by systems according to some embodiments of the present specification. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. Rather, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0034] The following is an introduction to the application scenarios of this manual.

[0035] The technical solution provided in this specification is applicable to scenarios where a recommendation system is used to recommend information to users. In this scenario, when a user performs a preset operation through a client, the client can send a recommendation request to the recommendation system. After receiving the recommendation request, the recommendation system will filter through multiple links to obtain the recommended content corresponding to the recommendation request. For the target link among the multiple links, the recommendation method provided in this specification can be used to determine the output set of the target link based on the input set of the target link. The input set of the target link includes multiple objects, each of which can represent a candidate recommendation content. The candidate recommendation content can be any form of information, including but not limited to: goods, services, information, etc.

[0036] This specification provides a recommendation method that can be performed by a recommendation system. The recommendation system can receive a recommendation request, and then the recommendation system can sequentially perform multiple steps of screening on an initial set containing multiple objects until at least one object to be recommended is screened out, and respond to the recommendation request based on the at least one object, wherein the multiple steps include a target step, and the screening process of the target step includes: the recommendation system obtains an input set of the target step, the input set is the initial set or the output set of the previous step, with the goal of the recommendation system meeting a preset response time constraint, determining the available queue length range of the target step, with the goal of the recommendation system meeting a preset computing power constraint, determining a target queue length within the queue length range, wherein the predicted recommendation index corresponding to the target queue length is greater than the predicted recommendation index corresponding to other queue lengths within the queue length range, and based on the target queue length, screening out some objects from the input set as the output set of the target step.

[0037] In this specification, recommendation evaluation metrics are related to the expected recommendation effect and are used to evaluate whether the recommendation achieved the expected effect. The expected recommendation effect is related to the needs of the actual application scenario and is not limited in this specification. For example, the expected recommendation effect may include, but is not limited to, one or more of the following: high conversion rate, high revenue, high click-through rate, etc.

[0038] In the recommendation scheme provided in this specification, the recommendation system achieves the target link through two stages. Among them, in the first stage, the range of queue lengths available for the target link is determined with the goal of the recommendation system meeting the preset response time (RT) constraint; in the second stage, the target queue length is determined within the queue length range with the goal of the recommendation system meeting the preset computing power constraint. By binding the queue length range with the response time constraint of the recommendation system, it is ensured that the recommendation request can be processed within the preset response time, avoiding the timeout of the recommendation request. Then, when the recommendation request meets the preset response time constraint, the target queue length with the optimal predicted recommendation indicator is determined within the queue length range. Through two stages, the recommendation system coordinates the response time constraint and the computing power constraint, so that the recommendation system improves the recommendation effect while meeting the response time constraint and the computing power constraint.

[0039] Figure 1 FIG1 shows a schematic diagram of a recommendation scenario provided according to an embodiment of this specification. Figure 1 As shown, the scenario 100 may include a recommendation system 11 and a client 12 . Figure 1 The scenario 100 shown may be a recommendation scenario for a certain Internet product. For example, an Internet product may include a client and a server, wherein: Figure 1 The client 12 in the example may correspond to a client of an Internet product. Figure 1 The recommendation system 11 in the example may correspond to the service end of an Internet product.

[0040] See also Figure 1 After receiving the recommendation request, the recommendation system 11 will perform multiple steps of screening on the initial set in order to obtain the recommended content corresponding to the recommendation request. For example, Figure 1 The screening of multiple links can include "Link 1", "Link 2", "Link 3" and "Link 4". The screening of multiple links is a funnel-type screening. The recommendation system 11 can further screen the output results of the previous link in the subsequent links, that is, each link can take the output set of the previous link as input (the input of link 1 can be a preset initial set). The recommendation system can perform the screening of multiple links in sequence until the last link is executed, and the recommended content corresponding to the recommendation request is obtained by screening. After obtaining the recommended content, the recommendation system 11 can send the recommended content to the client 12 to respond to the recommendation request. After receiving the recommended content, the client 12 can render and display the recommended content in the corresponding position.

[0041] As an example, any link in the multiple recommendation links can be used as the target link. For the target link in the multiple links, the recommendation system can determine the available queue length range for the target link with the goal of meeting the preset response time constraint. With the goal of meeting the preset computing power constraint, the target queue length is determined within the queue length range. Then, the target link output set is obtained based on the input set of the target link.

[0042] In some embodiments, the recommendation method provided herein can be executed on a recommendation system 11. In this case, the recommendation system 11 can store data or instructions for executing the recommendation method described herein and can execute or be used to execute such data or instructions. In some embodiments, the recommendation system 11 can include hardware devices capable of data information processing and the necessary programs to operate the hardware devices.

[0043] The recommendation system 11 may correspond to a single computing device or a computing cluster composed of multiple computing devices. The recommendation system 11 may also be referred to as the server corresponding to the client 12, or as a subsystem of the server.

[0044] The client 12 may send a recommendation request to the recommendation system 11 in response to a preset operation of the user.

[0045] In some embodiments, the client 12 may be installed with one or more application programs (APPs). The APPs can provide the ability to receive preset operations and an interface. The APPs include, but are not limited to, financial APPs, web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social platform software, etc.

[0046] In some embodiments, a target app may be installed on the client 12. The client 12 may receive a preset operation through the target app. In some embodiments, the target app may, in response to receiving the preset operation, send a recommendation request to the recommendation system. The target app may also receive recommendation information from the recommendation system 11 and display the recommendation information on the client 12. The recommendation information may be considered the result of the recommendation system executing the recommendation action corresponding to the recommendation request.

[0047] It should be understood that Figure 1 The number of clients 12 in FIG. 1 is merely illustrative. Any number of clients 12 may be provided depending on implementation requirements.

[0048] Figure 2 FIG2 shows a hardware structure diagram of a computing system provided according to an embodiment of this specification. The computing system 200 can be used as Figure 1 The recommendation system 11 in the embodiment executes the recommendation method described in this specification.

[0049] like Figure 2 As shown, computing system 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, computing system 200 may further include communication port 250 and internal communication bus 210. Computing system 200 may further include I / O component 260.

[0050] The internal communication bus 210 can connect various system components, such as the storage medium 230 , the processor 220 , the communication port 250 , and the I / O component 260 .

[0051] I / O components 260 support input / output between computing system 200 and other components.

[0052] Communication port 250 is used for data communication between computing system 200 and the outside world. For example, communication port 250 can be used for data communication between computing system 200 and a network. Communication port 250 can be a wired communication port or a wireless communication port.

[0053] Storage medium 230 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 235. Storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, which may include a program, routine, object, component, data structure, procedure, module, etc.

[0054] At least one processor 220 may be communicatively connected to at least one storage medium 230. When the computing system 200 is running, the at least one processor 220 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the recommended method provided in this specification. The processor 220 may execute the steps included in the recommended method. The processor 220 may be in the form of one or more processors. In some embodiments, the processor 220 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0055] For illustrative purposes only, the computing system 200 shown in the accompanying drawings only shows one processor 220. However, it should be noted that the computing system 200 described herein may also include multiple processors. Therefore, the operations and / or method steps disclosed herein may be performed by a single processor or jointly by multiple processors. For example, if the computing system 200 is described herein as performing steps A and B by the processor 220, it should be understood that steps A and B may also be performed jointly or separately by two different processors 220 (e.g., the first processor performs step A and the second processor performs step B, or the first and second processors jointly perform steps A and B).

[0056] Figure 3 FIG2 shows a flow chart of a recommendation method provided according to an embodiment of the present specification. As mentioned above, the recommendation system 11 can execute the recommendation method of the present specification.

[0057] like Figure 3 As shown, recommended methods may include:

[0058] S310: Receive a recommendation request.

[0059] In some embodiments, a recommendation request can be sent by a client to a recommendation system. This recommendation request can be triggered in a variety of situations. For example, when a target user performs a preset operation through a client, the recommendation system can receive a corresponding recommendation request from the client. Examples of preset operations include: accessing an item, making a payment, adding an item to a shopping cart, submitting an order, and confirming receipt.

[0060] When the client detects that the target user has performed a preset action through the client, it can generate a recommendation request corresponding to the preset action. The recommendation request contains at least the preset action that triggered the recommendation and the corresponding user account information of the client. The client can then send the generated recommendation request to the recommendation system, receive the recommended content from the recommendation system, and render and display the page related to the preset action based on the received recommended content.

[0061] S320: Execute multiple steps of screening on the initial set containing multiple objects in sequence until at least one object to be recommended is obtained through screening, and respond to the recommendation request based on the at least one object, wherein the multiple steps include a target step, and the screening process of the target step includes S321-S324.

[0062] In some embodiments, when filtering multiple links, the recommendation system may select any link as the target link and use the recommendation method provided in this specification to obtain the output set for that link. For example, the filtering of multiple links may include a recall link, a rough sorting link, a fine sorting link, and a re-ranking link, and the target link may be any of the above links.

[0063] Among them, the recall link is the screening of the first link, and the input of the recall link can be an initial set, which includes all available recommended content corresponding to the recommendation request. For example, the recommendation system can pre-configure the initial set corresponding to each user, and the initial set includes multiple objects, each of which can represent a recommended content. When the recommendation system receives a recommendation request, it can obtain the corresponding initial set based on the user corresponding to the recommendation request. Then, the recommendation system can perform multiple screening links in sequence until at least one object to be recommended (that is, at least one recommended content corresponding to the recommendation request) is screened, and respond to the recommendation request based on the at least one object to be recommended.

[0064] Figure 4 A schematic diagram of the process flow of the first and second stages of the target link provided according to an embodiment of this specification is shown; Figure 5 The following is a timing diagram of the process of the recommended method provided in accordance with the embodiment of this specification. Figure 4 and Figure 5 , and detailed description is given of steps S321-S324 involved in the screening process of the target link.

[0065] S321: Obtain the input set of the target link, where the input set is the initial set or the output set of the previous link.

[0066] In some embodiments, referring to the above example, the multiple steps, in order of priority, may include: a recall step, a rough sorting step, a fine sorting step, and a re-sorting step. When the target step is the recall step, the input set is the initial set; when the target step is the fine sorting step, the input set is the output set of the rough sorting step. As an example, the screening process of the multiple steps satisfies the following condition: the target queue length determined during the screening process of the i+1th step is less than the target queue length determined during the screening process of the i-th step. That is, the screening process of the multiple steps is a funnel-shaped screening process, where the target queue length corresponding to the subsequent step is less than the target queue length corresponding to the previous step. After the recommendation system screens the initial set through multiple steps, the objects in the output set output by the last step are the objects to be recommended and can be used to respond to recommendation requests. For example, assuming that the i-th step is the rough sorting step and the i+1th step is the fine sorting step, the target queue length corresponding to the fine sorting step is less than the target queue length corresponding to the rough sorting step.

[0067] In some embodiments, the recommendation request may come from a client. After determining at least one recommended object corresponding to the recommendation request, the recommendation system may respond to the recommendation request based on the recommended object and send recommended content information corresponding to the recommended object to the corresponding client, thereby displaying the recommendation information on the corresponding client.

[0068] For example, assuming that the recommendation request is generated based on a user's visit to a target page, the object to be recommended may be recommended content displayed in at least one booth on the target page. When the recommendation system responds to the recommendation request based on the object to be recommended, it may send the recommended content corresponding to each booth on the target page (determined based on the object to be recommended) to the corresponding client and instruct the client to render and display the target page based on the recommended content.

[0069] refer to Figure 4 In the target link, the recommendation system can obtain the output set of the target link through two stages. The first stage uses the response time as a constraint to ensure that the recommendation request meets the response time constraint (the queue length does not exceed j′); the second stage is to coordinate the response time constraint with the computing power constraint (determine the target queue length j * ), and improve the recommendation effect as much as possible under limited resources. In the following steps, S322 corresponds to the first stage, and S323 corresponds to the second stage.

[0070] S322: With the goal of ensuring that the recommendation system meets a preset response time constraint, determine the available queue length range for the target link.

[0071] In a recommendation system, response time refers to the time interval from when the recommendation system receives a recommendation request to when it determines the recommended content and returns it to the client. A recommendation system meeting a preset response time constraint means that the recommendation request received by the recommendation system can be processed within the preset response time. In the target phase, the recommendation system evaluates each object in the input set to determine the object to be recommended in the output set. The evaluation time for each object can be considered a fixed value. This means that there is a corresponding relationship between the queue length of the target phase and the response time required by the target phase. Therefore, constraining the response time can be considered as constraining the queue length of the target phase.

[0072] In some embodiments, the response time constraint includes ensuring that the recommendation system's response time does not exceed a preset response time. The recommendation system may determine a first queue length that is maximum supported by the target link based on the preset response time. The recommendation system may then determine a second queue length that is minimum supported by the target link and determine a queue length range based on the first and second queue lengths.

[0073] In some embodiments, the recommendation system may determine a remaining processing time based on a preset response time and a processing time of the recommendation request, and then determine a maximum first queue length supported by the target link based on the remaining processing time.

[0074] In some embodiments, the target link may be any one of multiple links. When the target link does not have a preceding link, the preset response time is the remaining processing time. When the target link does have a preceding link, the remaining processing time is the preset response time minus the corresponding processing time of the preceding link. Here, the method for determining the first queue length is described using the target link being the fine sorting link as an example.

[0075] As an example, when the target link is the fine sorting link, the processed time of the recommendation request is the time taken for the recall link and the rough sorting link, and the remaining processing time is the preset response time minus the processed time of the recommendation request. In the fine sorting link, the recommendation system will evaluate each object in the queue to obtain the corresponding predicted recommendation evaluation index for each object in the queue. The length of the first queue is the remaining processing time divided by the time required to evaluate an object. For example, assuming that the remaining processing time is 200 milliseconds and the time required to evaluate an object is 2 milliseconds, the length of the first queue is 200 milliseconds / 2 millimeters = 100, that is, the maximum supported first queue length of the target link is 100.

[0076] In this embodiment, the recommendation system determines the length of the first queue based on the remaining processing time corresponding to the target link. When the computing power is sufficient, the recommendation system will not time out when making precise recommendations based on the length of the first queue, thereby constraining the response time. As a result, the recommendation system can achieve a significant improvement in the recommendation effect while meeting the constraints of response time and computing power.

[0077] In this embodiment, the recommendation system can determine the maximum first queue length supported by the target link in the current period based on the maximum queue length supported by the target link in the historical period corresponding to the current period, ensuring that the queue upper limit of the current period always meets the preset response time constraint, thereby enabling the recommendation system to achieve a significant improvement in the recommendation effect while meeting the response time and computing power constraints.

[0078] In some embodiments, the minimum supported second queue length (j0) of the target link can be pre-configured, the first queue length is greater than or equal to the second queue length, and the second queue length can be determined based on the actual operating scenario of the recommendation system. This specification does not limit the setting method of the second queue length.

[0079] In some embodiments, the recommendation system determines a queue length range based on a first queue length and a second queue length, including: obtaining a target correction parameter corresponding to the recommendation system, the target correction parameter being obtained based on first operating data of the recommendation system in a historical period and used to correct the queue length range to adjust the response timeout of the recommendation system in the current period; correcting the first queue length based on the target correction parameter to obtain a third queue length (j′), the third queue length being less than the first queue length; and using the third queue length as the upper limit of the queue length range and the second queue length as the lower limit of the queue length range to obtain a queue length range ([j0, j′]). The response timeout of the recommendation system includes at least one of the following: a timeout rate of recommendation requests, and a recommendation failure rate (or success rate) of the recommendation model caused by a timeout of the recommendation request.

[0080] In some embodiments, when the computing power of the recommendation system is sufficient, timeouts will not occur when making recommendations based on the first queue length. However, in actual applications, the computing power resources of the recommendation system change at any time. When the computing power resources are insufficient, timeouts may occur. In this case, the recommendation system can determine a target correction parameter for the first queue length based on the first operating data of the recommendation system in the historical period, and correct the first queue length based on the target correction parameter (i.e., compress the first queue length) to obtain a third queue length. When the recommendation system executes the second stage based on the third queue length, the response timeout situation of the recommendation system in the current period can be better than the response timeout situation in the historical period, further reducing the occurrence of response timeouts in the recommendation system. The historical period and the current period can be time periods at the minute level. For example, the historical period can be the last 1 minute. The length of the historical period and the current period can be longer or shorter than 1 minute. For example, the length of the historical period and the current period can be half a minute, 2 minutes, 3 minutes, etc.

[0081] In some embodiments, taking the time point when the recommendation system starts to determine the length of the third queue as an example, the historical period can be 1 minute before the time point when the recommendation system starts to determine the length of the third queue; the current period can be 1 minute after the recommendation system determines the length of the third queue.

[0082] In some embodiments, the target correction parameter may include a queue length increment or a queue length multiple. The recommendation system may derive the third queue length based on the sum of the first queue length and the queue length increment. Alternatively, the recommendation system may derive the third queue length based on the product of the first queue length and the queue length multiple.

[0083] As an example, since the third queue length is smaller than the first queue length, when the target correction parameter is the queue length increment, the queue length increment is a negative number (set to Δ), then the third queue length is equal to the first queue length + Δ.

[0084] Alternatively, when the target correction parameter is a queue length multiple, and the queue length multiple is a positive number less than 1, the third queue length is equal to the first queue length multiplied by the queue length multiple.

[0085] In some embodiments, the target correction parameter can be calculated by a feedback control algorithm and updated based on the feedback control algorithm. As an example, the first operating data may include the actual response timeout rate of the recommendation system in a historical period, and the target operating data may include the target response timeout rate of the recommendation system in the current period. The recommendation system may determine the target correction parameter based on the difference between the actual response timeout rate and the target timeout rate, as well as the fourth queue length corresponding to the target link in the current period. The difference between the actual response timeout rate and the target timeout rate may include the difference between the actual response timeout rate and the target timeout rate, or the ratio of the actual response timeout rate to the target timeout rate.

[0086] In some embodiments, the fourth queue length in each time period is pre-set based on the number of historical recommendation requests received by the recommendation system in the time period. As an example, the recommendation system can divide a day into multiple time periods. Since the load of the recommendation system in different time periods (i.e., the number of recommendation requests received) is different, the queue length (i.e., the fourth queue length) that the recommendation system is expected to support for each recommendation request in the target link in each time period will also change over time. Figure 5 ,The recommendation system records and collects the number of historical recommendation requests in each time period in the previous day, and then calculates the fourth queue length corresponding to each time period offline based on the collected number of historical recommendation requests in each time period.

[0087] In some embodiments, reference Figure 5 The recommendation system can, near-line, use a proportional-integral-derivative (PID) control algorithm, using the difference between the actual response timeout rate and the target timeout rate as a feedback parameter. Based on the feedback parameter and the fourth queue length corresponding to the target link in the current time period, the recommendation system can calculate a target correction parameter. Furthermore, after obtaining the target correction parameter, the recommendation system can perform the above steps every other time period to cyclically update the target correction parameter, thereby updating the third queue length based on the actual response timeout rate in the historical time period and the fourth queue length of the corresponding time period, and then executing the second stage based on the updated third queue length.

[0088] In this embodiment, the recommendation system ensures that recommendation requests can be processed within the preset response time by binding the queue length range to the recommendation system's response time constraint, avoiding timeouts and improving recommendation effectiveness. The recommendation system calculates the fourth queue length offline based on the number of historical recommendation requests in each time period, and calculates the target correction parameter based on the feedback parameter and the fourth queue length corresponding to the target link in the current time period. The recommendation system uses the target correction parameter obtained in this way to correct the first queue length to obtain a more accurate third queue length. When executing the second stage based on the third queue length, the probability of recommendation request timeouts is lower, thereby improving the recommendation effectiveness and operating more stably.

[0089] S323: With the goal of ensuring that the recommendation system meets a preset computing power constraint, a target queue length is determined within the queue length range, wherein a predicted recommendation indicator corresponding to the target queue length is greater than predicted recommendation indicators corresponding to other queue lengths within the queue length range.

[0090] In some embodiments, the target queue length corresponding to the target link is the queue length with the largest corresponding prediction recommendation index within the queue length range. As an example, the recommendation system can use x ij Indicates whether to make a recommendation based on the queue with a queue length of j for the recommendation request i in the second stage of the target link, where x ij The value of is [0, 1]. ij When the value of is 0, it means that for the recommendation request i, the recommendation system does not make recommendations based on the queue with a queue length of j; when x ij When the value of is 1, it means that for recommendation request i, the recommendation system makes recommendations based on the queue with a queue length of j. In this case, the second stage of the target link of the recommendation system can be implemented based on the following objective function:

[0091]

[0092] Among them, reward ij represents the first evaluation metric that the recommendation system can generate for recommendation request i based on a queue with queue length j. This formula indicates that when responding to a recommendation request, the recommendation system needs to determine the queue length with the maximum predicted recommendation metric (i.e., the target queue length) based on the queue length and the first evaluation metric corresponding to the queue length, and respond to the recommendation request based on the target queue length. Referring to the example in S322, in the objective function of the second stage of the target phase, the upper limit of the queue length range is the third queue length, and the lower limit is the second queue length.

[0093] For the above objective function, the following constraints are also included:

[0094]

[0095] in, It is used to represent the kth computing power consumption when the recommendation system makes a recommendation based on the target queue with a queue length of j for the recommendation request i. The constraint represents the sum of k types of computing power consumption (i.e., the upper limit of computing power allocated by the recommendation system to recommendation requests). As an example, when the recommendation system evaluates the objects in the target queue in the target phase, it may be based on multiple models or algorithms, and each model or algorithm can correspond to a computing power consumption. The sum of the computing power consumed by these multiple models or algorithms is not greater than rt ij It represents the response time of the recommendation system for recommendation request i based on the queue with a queue length of j. Indicates the preset response time constraint corresponding to the recommendation request i. Indicates the total number of recommendation requests i received in the current period.

[0096] In some embodiments, the recommendation system can transform the above formula and constraints into a dual problem, that is, in the second stage of the target phase, it can be implemented based on the following objective function:

[0097]

[0098] Among them, λ k 、μ i And v is the Lagrange multiplier, which is used to transform the constraints into part of the objective function. k Used to represent the change in the recommendation index when each unit of computing power is increased in the recommendation system; μ i It is used to represent the response time constraint for responding to recommendation request i in the recommendation system without timeout; v is used to represent the constraint on the total number of recommendation requests i in each time period.

[0099] The constraints corresponding to the above objective function are:

[0100]

[0101] Since the total amount of recommendation requests i that the recommendation system responds to in each period can be constrained by setting a threshold for the number of recommendation requests received, it can be determined that the total amount of recommendation requests i that the recommendation system responds to in each period meets the constraint and can be omitted from the objective function, that is, the target queue length j * It can be expressed by the following formula:

[0102]

[0103] Among them, the objective function indicates that under the condition of complying with the response time constraint and the computing power constraint, the maximum prediction recommendation evaluation index is determined among the prediction recommendation evaluation indexes corresponding to the queue lengths within the queue length range; and the queue length corresponding to the maximum prediction recommendation evaluation index is used as the target queue length j * And, in the second stage of the target link, for the recommendation request i, based on the queue length j * The queue is recommended.

[0104] For the above objective function, since the response time of the recommendation request in the target link is constrained in the first stage of the target link, that is, in the second stage, when the recommendation system processes based on any length of the queue length j, the recommendation request has satisfied the response time constraint, so the response time-related constraints can also be omitted from the objective function, that is, the target queue length j * It can be expressed by the following objective function:

[0105]

[0106] The range of queue length j is [j0, j′].

[0107] In some embodiments, the recommendation system may first obtain a first coefficient (ie, λ k ), where the first coefficient is determined with the goal of ensuring the recommendation system meets a preset computing power constraint and represents the magnitude of change in the recommendation index for each unit of computing power increase in the recommendation system. Then, based on the first coefficient, a predicted recommendation evaluation index corresponding to each queue length within a queue length range is determined; and based on the predicted recommendation evaluation index corresponding to each queue length within the queue length range, a target queue length within the queue length range is determined.

[0108] As an example, for a recommendation request i, the recommendation system may include k types of computing power consumption when making recommendations based on a target queue with a queue length of j. Each computing power consumption corresponds to a first coefficient λ. k For example, assuming that the recommendation system uses a click-through rate (ctr) prediction model and three conversion rate (cvr) prediction models when evaluating objects in the target queue, the ctr prediction model and each cvr prediction model each correspond to a computing power consumption, that is, there are four types of computing power consumption, the first of which is the computing power consumption of the ctr prediction model. The corresponding first coefficient is λ1, and the second to fourth coefficients are the computing power consumption corresponding to the three CVR prediction models. The corresponding first coefficients are λ2, λ3, and λ4 respectively.

[0109] In some embodiments, reference Figure 5 The recommendation system can obtain the actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests within a historical period, and determine the second coefficient based on the actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests (i.e., the data set to be solved). Then, the second operating data of the recommendation system within the historical period is obtained, and the second operating data represents the usage of computing power by the recommendation system; and with the goal of ensuring that the recommendation system meets the preset computing power constraint, the second coefficient is corrected based on the second operating data to obtain the first coefficient.

[0110] In some embodiments, reference Figure 5 , the second coefficient can be solved near-line, that is, the length of the historical period is also at the minute level, for example, it can be 1 minute, 5 minutes or 10 minutes. The second operating data can be obtained through the interface provided by the monitoring platform pre-configured in the recommendation system. Figure 5 The dataset to be solved shown in the figure can include the actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests in at least one time period. For example, assuming that a time period is 1 minute, the dataset to be solved can include the actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests in 10 historical time periods (i.e., within the previous 10 minutes). The number of historical time periods included in the dataset to be solved can be determined based on the actual application situation and is not limited in this specification.

[0111] In some embodiments, the computing power constraint includes ensuring that the resource utilization of the recommendation system does not exceed a target resource utilization. The near-line solution for the second coefficient may include determining the second coefficient based on the target resource utilization, the actual recommendation evaluation metrics corresponding to multiple historical recommendation requests, and the actual computing power consumption. The second coefficient also represents the magnitude of the change in the recommendation metric for each additional unit of computing power in the recommendation system.

[0112] As an example, the recommendation system can determine the second coefficient based on the actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests through a mixed integer programming (MIP) solver, on the premise of satisfying computing power constraints.

[0113] In some embodiments, the recommendation system may determine multiple candidate second coefficients corresponding to each of the multiple historical recommendation requests based on the actual recommendation evaluation indicators and actual computing power consumption. In this case, the recommendation system may use the average of the multiple candidate second coefficients as the determined second coefficient, or use the median of the multiple candidate second coefficients as the determined second coefficient.

[0114] In some embodiments, for each historical recommendation request, the recommendation system may first obtain the actual recommendation evaluation index and actual computing power consumption corresponding to the historical recommendation request. Then, the recommendation system may increase computing power by one unit based on the actual computing power consumption, assuming that the resource utilization rate of the recommendation system does not exceed the target resource utilization rate. Using a MIP solver, the recommendation system may predict the predicted recommendation evaluation index after this one-unit increase in computing power. Finally, the recommendation system may determine a candidate second coefficient based on the ratio of the predicted recommendation evaluation index to the actual recommendation evaluation index.

[0115] In some embodiments, referring to the above example, the recommendation system includes multiple models or algorithms, each model or algorithm corresponds to a second coefficient. In this case, the obtained second coefficient can be represented by a tensor or matrix, which includes the second coefficient corresponding to each model or algorithm.

[0116] In some embodiments, after the recommendation system obtains the second coefficient through near-line solution, it can also update the second coefficient through a near-line feedback algorithm to obtain the first coefficient. As an example, the second operating data includes the actual resource utilization rate of the recommendation system during a historical period. The recommendation system can update the second coefficient based on the actual resource utilization rate and the target resource utilization rate to obtain the first coefficient.

[0117] For example, taking the PID control algorithm as the feedback control algorithm, the recommendation system can obtain the feedback parameter (i.e., the actual resource usage rate of the recommendation system in the period) every time period, and correct the second coefficient based on the PID control algorithm according to the difference between the feedback parameter and the target parameter (target resource usage rate) to obtain the first coefficient. Figure 5 , the recommendation system can obtain the second operating data after the current period ends, and use the actual resource usage rate corresponding to the second operating data as a feedback parameter, and correct the second coefficient through the PID control algorithm to obtain the first coefficient.

[0118] In this embodiment, the recommendation system can obtain the second coefficient in the near-line by solving the actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests, thereby realizing minute-level updates of the second coefficient. Compared with solving the second coefficient offline, it can more accurately reflect the current status of the recommendation system, making the solved second coefficient closer to the optimal solution.

[0119] In addition, the recommendation system can also update the second coefficient through a feedback algorithm based on the actual resource utilization rate and the target resource utilization rate in the historical period to obtain the first coefficient, so that when the recommendation system executes the second stage of the target link based on the first coefficient, it can more accurately express the state changes of the recommendation system, reduce the gap between the solution result and the actual optimal solution, further improve the recommendation effect of the recommendation system, and ensure the robustness of the recommendation system.

[0120] In some embodiments, for each candidate queue length within the queue length range: the recommendation system may obtain a first evaluation indicator (reward) that the candidate queue length can generate. ij ), and based on the first coefficient (λ k ) and the resource consumption required for the candidate queue length (i.e., computing power consumption ), obtain the second evaluation index required to consume the candidate queue length And according to the difference between the first evaluation index and the second evaluation index, determining the prediction recommendation evaluation index corresponding to the candidate queue length.

[0121] In some embodiments, a recommendation request is triggered by a target user, and the recommendation system may predict the target user's value score; and determine a first evaluation metric that can be generated by the candidate queue length based at least on the target user's value score. For example, the recommendation system may also predict the exposure probability corresponding to the recommendation request, obtain a target weight corresponding to the candidate queue length, and determine the first evaluation metric that can be generated by the candidate queue length based on the target user's value score, exposure probability, and target weight.

[0122] As an example, the first evaluation indicator reward ij It can be obtained by the following formula:

[0123] reward ij =expo_rate*cpm i *α j

[0124] Among them, expo_rate is the exposure probability corresponding to the recommendation request, cpm i is the cost per thousand impressions of the target user, which is used to characterize the value score of the target user, α j is the target weight corresponding to the candidate queue length.

[0125] In some embodiments, the value score of the target user can be obtained by solving a multi-layer perceptron (MLP). The recommendation system can estimate the value score of the target user through MLP based on the following formula:

[0126] cpm i =E(cpm|expocpm )

[0127] Among them, expo cpm Refers to the historical exposure data of the target user. This formula indicates that based on the historical exposure data of the target user, the expected CPM value of the target user (i.e. the value score CPM of the target user) is predicted. i ).

[0128] For example, the target user's historical exposure data may include the target user's characteristics and the target user's actual CPM value. The target user's characteristics may include at least one of the target user's user characteristics, contextual characteristics, content characteristics, or exposure characteristics. User characteristics may include the target user's historical click-through rate, historical conversion rate, and historical search content; contextual characteristics may include environmental characteristics of the target user's client, such as time, geographic location, network information, and device characteristics; content characteristics may include the target user's preferred recommended content categories; and exposure characteristics may include exposure frequency and exposure booth characteristics. User characteristics may be recorded in the form of key-value pairs.

[0129] The recommendation system can pre-deploy a trained MLP model, which is trained based on the user's historical exposure data. The trained MLP model has the ability to predict the target user's CPM based on the target user's characteristics. When the recommendation system receives a recommendation request, it can input the target user's characteristics corresponding to the recommendation request into the MLP model to obtain the CPM corresponding to the target user. i .

[0130] In some embodiments, the exposure probability corresponding to the recommendation request can be obtained by the following formula:

[0131] expo_rate=P(expo expo_rate |The target link has an object returned)

[0132] Among them, P(expo expo_rate The target phase has an object returned) refers to the probability of the returned object being exposed if the condition "the target phase has an object returned" is met. Specifically, the expo_rate is the probability of the object in the output set determined in the target phase being exposed. The target phase has an object returned when the output set (including at least one object) is determined in the target phase.

[0133] In some embodiments, the target weight corresponding to the candidate queue length may be the relative value of the candidate queue length. For example, the relative value α of the candidate queue length is j It can be obtained by the following formula:

[0134]

[0135] Among them, ecpmj refers to the revenue per thousand impressions when the queue length is j. is the average RPM for all queue lengths. For example, if When the queue length is j = 200, the corresponding ecpmj = 120, when the queue length is j = 100, the corresponding ecpmj = 100; then α j200 =120 / 100=1.2, α j100 =100 / 100=1. Based on multiple historical recommendation requests, the recommendation system can obtain the queue length and corresponding revenue per thousand impressions determined when the recommendation system processes each historical recommendation request, and then obtain ecpmj and

[0136] In this embodiment, the recommendation system determines the first evaluation index that can be generated by the candidate queue length based on the value score, exposure probability, and target weight of the target user. The obtained first evaluation index can more accurately represent the value that can be generated by the candidate queue length, thereby improving the estimation accuracy, enabling the recommendation system to more accurately allocate computing power, and further improving the recommendation effect of the recommendation system. In this specification, in the second stage of the target phase, the recommendation system determines the predicted recommendation evaluation index corresponding to the candidate queue length based on at least the value score of the target user corresponding to the recommendation request, and determines the target queue length based on the predicted recommendation evaluation index corresponding to each candidate queue length. In this process, when the value score of the target user is low, the corresponding target queue length will also be shorter, that is, the load of the recommendation system will be reduced, and the timeout rate will also be reduced accordingly. In the next period, the actual response timeout rate (i.e., feedback parameter) of the recommendation system in the first stage will be much smaller than the target timeout rate. The recommendation system can adjust the queue length increment to make the updated third queue length longer (increase the third queue length while meeting the response time constraint and computing power constraint). That is, the upper limit of the queue length range of the second stage of the target phase will increase. When the value score of the target user is high, its target queue length will also increase relatively. After multiple cycles, the recommendation system will gradually shorten the target queue length for recommendation requests from target users with lower value scores and increase the target queue length for recommendation requests from target users with higher value scores. After executing these steps multiple times, the target queue length for recommendation requests with higher value scores will be longer than that for recommendation requests with lower value scores, while meeting response time constraints. This allows for personalized computing power allocation based on the target user's value score, while still meeting response time constraints.

[0137] In this embodiment, the recommendation system decouples the global computing power constraint from the response time constraint of a single recommendation request by constraining the response time in the first stage and constraining the computing power in the second stage in the target link. When the recommendation request meets the response time constraint, the response time of the recommendation request can be fully utilized to improve the recommendation effect. In addition, in the second stage, the recommendation system allocates computing power based on the actual computing power consumption of the previous period and the value score of the target user, so that the recommendation system can allocate a longer target queue length for recommendation requests with a higher value score for the target user and a shorter target queue length for recommendation requests with a lower value score for the target user in the event of a timeout or insufficient computing power. In addition, under the premise of meeting the response time constraint and computing power constraint, computing power resources can be allocated more specifically, thereby making the recommendation effect of the recommendation system better.

[0138] S324: Based on the target queue length, select some objects from the input set as the output set of the target link.

[0139] In some embodiments, the recommendation system may determine, based on the target queue length, a number of objects from the input set corresponding to the target queue length as the output set for the target link. For example, the recommendation system may sort multiple objects in the input set in descending order according to a preset evaluation metric, and determine the number of objects with the highest ranking corresponding to the target queue length as the output set for the target link. The preset evaluation metric may be obtained in a preceding link of the target link, or may be pre-set, and this specification does not impose any restrictions on this.

[0140] In summary, in the recommendation method and system provided in this specification, the recommendation system achieves the target link through two stages. Among them, in the first stage, the range of queue lengths available for the target link is determined with the goal of the recommendation system meeting the preset response time constraint; in the second stage, the target queue length is determined within the queue length range with the goal of the recommendation system meeting the preset computing power constraint. By binding the queue length range with the response time constraint of the recommendation system, it is ensured that the recommendation request can be processed within the preset response time, reducing the probability of timeout of the recommendation request. Then, when the recommendation request meets the preset response time constraint, the recommendation system can determine the target queue length with the best prediction recommendation index within the queue length range. Through two stages, the recommendation system coordinates the response time constraint and the computing power constraint, so that the recommendation system improves the recommendation effect under the premise of meeting the response time constraint and the computing power constraint.

[0141] Another aspect of this specification provides a computer-readable, non-transitory storage medium storing at least one instruction set for performing data processing related to content recommendation. When executed by a processor, the at least one instruction set directs the processor to implement the steps of the recommendation method described herein. In some possible implementations, various aspects of this specification may also be implemented as a program product comprising program code. When the program product is executed on a computing system 200, the program code is configured to cause the computing system 200 to perform the steps of the recommendation method described herein. The program product for implementing the aforementioned method may comprise a portable compact disc read-only memory (CD-ROM) comprising the program code and may be executed on the computing system 200. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system. The program product may utilize any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of computer-readable storage media include: an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing. Program code for performing the operations described herein may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may execute entirely on the computing system 200, partially on the computing system 200, as a stand-alone software package, partially on the computing system 200 and partially on a remote computing device, or entirely on the remote computing device.

[0142] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0143] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and may not be limiting. Although not expressly stated herein, those skilled in the art will understand that this specification encompasses various reasonable changes, improvements, and modifications to the embodiments. Such changes, improvements, and modifications are intended to be suggested by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0144] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, “one embodiment,” “an embodiment,” and / or “some embodiments” mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is emphasized and should be understood that two or more references to “an embodiment,” “one embodiment,” or “an alternative embodiment” in various parts of this specification do not necessarily refer to the same embodiment. Furthermore, particular features, structures, or characteristics may be appropriately combined in one or more embodiments of this specification.

[0145] It should be understood that in the foregoing descriptions of the embodiments of this specification, to facilitate understanding of a feature and to simplify this specification, various features are combined in a single embodiment, figure, or description thereof. However, this does not necessarily mean that these features are combined. When reading this specification, a person skilled in the art may label some of the devices as separate embodiments. In other words, the embodiments of this specification can also be understood as the integration of multiple sub-embodiments. The content of each sub-embodiment is also valid even when it includes fewer than all the features of a single previously disclosed embodiment.

[0146] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, articles, and the like, cited herein, except to the extent that it is inconsistent or conflicting with this document or that it has a limiting effect on the broadest scope of the claims, is hereby incorporated by reference for all purposes now or hereafter connected with this document. In addition, in the event of any inconsistency or conflict between the description, definition, and / or use of a term in any material and the description, definition, and / or use of a term in this document, the term in this document shall control.

[0147] Finally, it should be understood that the embodiments of the application disclosed herein are illustrative of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can adopt alternative configurations based on the embodiments in this specification to implement the application in this specification. Therefore, the embodiments of this specification are not limited to the embodiments precisely described in the application.

Claims

1. A recommendation method, applied to a recommendation system, comprising: receiving referral requests; as well as Performing a plurality of steps of screening on an initial set of multiple objects in sequence until at least one object to be recommended is obtained, and responding to the recommendation request based on the at least one object, wherein the plurality of steps includes a target step, and the screening process of the target step includes: Obtaining the input set of the target link, wherein the input set is the initial set or the output set of the previous link, With the goal of ensuring that the recommendation system meets the preset response time constraint, the available queue length range of the target link is determined. With the goal of the recommendation system meeting the preset computing power constraint, a target queue length is determined within the queue length range, wherein the predicted recommendation index corresponding to the target queue length is greater than the predicted recommendation indexes corresponding to other queue lengths within the queue length range, and Based on the target queue length, some objects are screened out from the input set as the output set of the target link.

2. The method according to claim 1, wherein The response time constraint includes: the response time of the recommendation system does not exceed a preset response time. With the goal of the recommendation system meeting the preset response time constraint, determining the available queue length range for the target link includes: Determine a first queue length that is maximally supported by the target link based on the preset response time; Obtaining a minimum second queue length supported by the target link; and The queue length range is determined based on the first queue length and the second queue length.

3. The method according to claim 2, wherein: Determining the queue length range based on the first queue length and the second queue length includes: Obtaining a target correction parameter corresponding to the recommendation system, the target correction parameter being obtained based on first operating data of the recommendation system in a historical period and being used to correct the queue length range to adjust a response timeout of the recommendation system in a current period; Correcting the first queue length based on the target correction parameter to obtain a third queue length, where the third queue length is smaller than the first queue length; and The queue length range is obtained by taking the third queue length as the upper limit of the queue length range and taking the second queue length as the lower limit of the queue length range.

4. The method according to claim 3, wherein: The first operating data includes the actual response timeout rate of the recommendation system in the historical period, and the target correction parameter is obtained in the following manner: Obtaining a target response timeout rate of the recommendation system and a fourth queue length corresponding to the target link in a current time period; as well as The target correction parameter is determined based on the difference between the actual response timeout rate and the target timeout rate, and the fourth queue length corresponding to the target link in the current time period, wherein the fourth queue length in each time period is pre-set based on the number of historical recommendation requests of the recommendation system in the time period.

5. The method according to claim 3, wherein: The target correction parameter includes a queue length increment or a queue length multiple, and correcting the first queue length based on the target correction parameter to obtain a third queue length includes: The third queue length is obtained based on the sum of the first queue length and the queue length increment, or The third queue length is obtained based on the product of the first queue length and the multiple of the queue length.

6. The method according to claim 2, wherein: Determining a first queue length that is maximally supported by the target link based on the preset response time includes: Determining a remaining processing time based on the preset response time and the processing time of the recommendation request; and A first queue length that is maximally supported by the target link is determined based on the remaining processing time.

7. The method according to claim 1, wherein With the recommendation system satisfying a preset computing power constraint as a goal, determining a target queue length within the queue length range includes: Obtaining a first coefficient corresponding to the recommendation system, where the first coefficient is determined with the recommendation system satisfying a preset computing power constraint and represents a change in the recommendation indicator when the computing power in the recommendation system increases by one unit; Determining, based on the first coefficient, a prediction recommendation evaluation index corresponding to each queue length within the queue length range; and The target queue length is determined within the queue length range based on the prediction recommendation evaluation index corresponding to each queue length within the queue length range.

8. The method according to claim 7, wherein: The first coefficient is obtained in the following manner, including: Obtaining actual recommendation evaluation indicators and actual computing power consumption corresponding to multiple historical recommendation requests within a historical period, and determining a second coefficient based on the actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests; Obtaining second operating data of the recommendation system during the historical period, the second operating data representing usage of computing power by the recommendation system; and With the goal of ensuring that the recommendation system meets a preset computing power constraint, the second coefficient is corrected based on the second operating data to obtain the first coefficient.

9. The method according to claim 8, wherein The computing power constraint includes: the resource utilization rate of the recommendation system does not exceed the target resource utilization rate; and determining the second coefficient based on the actual recommendation evaluation index and the actual computing power consumption corresponding to the multiple historical recommendation requests includes: Determining the second coefficient based on the target resource usage rate, and actual recommendation evaluation indicators and actual computing power consumption corresponding to the multiple historical recommendation requests; The second operating data includes the actual resource usage of the recommendation system during the historical period. With the goal of ensuring that the recommendation system meets a preset computing power constraint, the second coefficient is corrected based on the second operating data to obtain the first coefficient, including: Based on the actual resource usage rate and the target resource usage rate, the second coefficient is updated to obtain the first coefficient.

10. The method according to claim 7, wherein: Determining the prediction recommendation evaluation index corresponding to each queue length within the queue length range based on the first coefficient includes: For each candidate queue length within the queue length range: Obtaining a first evaluation indicator that can be generated by the candidate queue length; Obtaining a second evaluation index required for resource consumption of the candidate queue length based on the first coefficient and the resource consumption required for the candidate queue length; and Determine a prediction recommendation evaluation index corresponding to the candidate queue length according to a difference between the first evaluation index and the second evaluation index.

11. The method according to claim 10, wherein: The recommendation request is triggered by a target user, and obtaining the first evaluation indicator that can be generated by the candidate queue length includes: Predicting a value score for the target user; and A first evaluation indicator that can be generated by the candidate queue length is determined based at least on the value score of the target user.

12. The method according to claim 11, wherein Determining a first evaluation metric that can be generated by the candidate queue length based at least on the value score of the target user includes: Predicting the exposure probability corresponding to the recommendation request; Obtaining a target weight corresponding to the candidate queue length; and A first evaluation index that can be generated by the candidate queue length is determined based on the value score of the target user, the exposure probability, and the target weight.

13. The method according to claim 7, wherein: The determining the target queue length within the queue length range based on the prediction recommendation evaluation indicator corresponding to each queue length within the queue length range includes: Determining the maximum prediction recommendation evaluation index among the prediction recommendation evaluation indexes corresponding to the queue lengths within the queue length range; and The queue length corresponding to the maximum prediction recommendation evaluation index is used as the target queue length.

14. The method according to claim 1, wherein The multiple links include: recall link, rough sorting link, fine sorting link and re-sorting link; The screening process of the multiple links meets the following condition: the target queue length determined in the screening process of the (i+1)th link is smaller than the target queue length determined in the screening process of the (i)th link.

15. A recommendation system comprising: at least one storage medium storing at least one instruction set for performing data processing related to content recommendation; as well as At least one processor is communicatively connected to the at least one storage medium, wherein when the recommendation system is running, the at least one processor reads the at least one instruction set and implements the method according to any one of claims 1 to 14 according to the instructions of the at least one instruction set.

16. A computer-readable non-volatile storage medium, wherein: The computer-readable non-volatile storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, the method according to any one of claims 1 to 14 is implemented.

Images