Hybrid sorting model training method and tourism search hybrid sorting method and device

Abstract

The invention relates to the technical field of computers, and provides a hybrid sorting model training method and a tourism search hybrid sorting method and device. The hybrid sorting model training method comprises the steps that sample data is constructed and comprises state data, action data and reward data, the state data represents candidate objects of all tourism categories and attributes of users, the action data represents a hybrid sorting result of the candidate objects of all the tourism categories, and the reward data represents operation of the users on the hybrid sorting result; using the sample data to train a hybrid sorting model constructed based on reinforcement learning: calculating an action value of the current state according to the state data and the action data of the current state, and calculating an action value of the next state according to the state data and the action data of the next state, and calculating model loss and adjusting model parameters according to the reward data of the current state, the action value of the current state and the action value of the next state. Through reinforcement learning and personalized recommendation, tourism search efficiency and user experience are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a hybrid sorting model training method, a tourism search hybrid sorting method and equipment. Background Art

[0002] With the rapid development of the tourism industry and the popularization of the Internet, more and more users tend to use search engines or travel websites to plan and arrange travel. However, in travel search scenarios, users are often faced with a large number of search results, including objects of multiple categories such as tickets, day tours, and entertainment. Current search sorting algorithms often only consider the sorting of objects in a single category and cannot meet the comprehensive needs of users well. For example, a user may be interested in ticket prices, day tour itineraries, and entertainment activities for a certain destination at the same time, but traditional search sorting algorithms can often only sort according to the relevance of objects in a certain category and cannot comprehensively consider the information of objects in multiple categories. This causes users to switch back and forth between different search results, wasting time and energy.

[0003] Traditional hybrid sorting methods usually sort the search results of objects of different categories separately, and then mix them according to certain rules. For example, a weighted average can be taken according to the relevance of each category object, or the search results of objects of different categories can be arranged alternately according to a certain rule. However, this traditional hybrid sorting method has some disadvantages. First, most traditional methods are based on single-point scoring, that is, a score is calculated for each object separately, ignoring the impact of the arrangement of different objects on user behavior. Secondly, traditional hybrid sorting methods cannot comprehensively consider the characteristics of objects of different categories and user preferences. Users may attach different importance to factors such as the relevance, price, and evaluation of objects of different categories, and traditional methods often cannot flexibly sort according to users' personalized needs.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In view of this, the present invention provides a hybrid sorting model training method, a tourism search hybrid sorting method and a device, aiming to solve the hybrid sorting problem of multiple category objects in tourism search scenarios. The present invention improves the efficiency and user experience of tourism search by introducing reinforcement learning and personalized recommendation, bringing development value to the business.

[0006] According to one aspect of the present invention, a hybrid sorting model training method is provided, comprising: constructing sample data: including state data, action data and reward data, wherein the state data represents attributes of candidate objects of each tourism category and users, the action data represents a mixed sorting result of candidate objects of each tourism category, and the reward data represents the user's operation on the mixed sorting result; wherein the state data and the action data are generated by data embedding simulation, and the reward data is calculated by data embedding collection; using the sample data to train a hybrid sorting model constructed based on reinforcement learning: calculating the action value of the current state according to the state data and action data of the current state, calculating the action value of the next state according to the state data and action data of the next state, and calculating the model loss and adjusting the model parameters according to the reward data of the current state, the action value of the current state and the action value of the next state.

[0007] In some embodiments, the status data includes: candidate object queue characteristics of each tourism category, user portrait characteristics, user preference characteristics for objects and categories, context characteristics, and search characteristics.

[0008] In some embodiments, the travel category includes a ticket category, a day tour category, and an entertainment category, and the characteristics of the candidate objects include the relevance, popularity, order volume, and evaluation of the candidate objects.

[0009] In some embodiments, the influencing factors of the reward data include: click operation, order operation, hit rate and NDCG index; the reward data is calculated based on each influencing factor and its weight.

[0010] In some embodiments, the hybrid sorting model is constructed based on a competitive cross deep Q network, which adopts the architecture of a competitive deep Q network and cross-splices the state data and the action data.

[0011] In some embodiments, the action value of the current state is calculated based on the state data and action data of the current state, including: based on the state data of the current state, generating a candidate object queue of each tourism category of the current state represented by a vector through parameter sharing as the state information of the current state; based on the action data of the current state, generating the action information of the current state represented by a matrix; multiplying the state information of the current state to obtain the state value of the current state; cross-multiplying the state information of the current state with the action information of the current state and then adding them to obtain the advantage of the current state; adding the state value of the current state to the advantage of the current state to obtain the action value of the current state.

[0012] In some embodiments, the model loss is calculated using the following loss function: Among them, r is the reward data of the current state, γ is the discount factor, Q(s,a) is the action value of the current state, and Q(s`,a`) is the action value of the next state.

[0013] According to another aspect of the present invention, a hybrid ranking method for tourism search is provided, comprising: obtaining a tourism search request; based on the tourism search request, roughly ranking objects of each tourism category to obtain a rough ranking result for each tourism category; obtaining candidate objects of each tourism category from the rough ranking result, and finely ranking the candidate objects of each tourism category to obtain a fine ranking result for each tourism category; based on the fine ranking result of each tourism category, obtaining a hybrid ranking result for each tourism category based on a hybrid ranking model as described in any of the above embodiments; and generating tourism search results based on the hybrid ranking result.

[0014] In some embodiments, based on the hybrid ranking model, a hybrid ranking result of each tourism category is obtained, including: obtaining a plurality of top-ranked candidate objects of each tourism category from the refined ranking results of each tourism category; generating state information of the current state according to the plurality of top-ranked candidate objects of each tourism category and the attributes of the user corresponding to the tourism search request; inputting the state information of the current state and the candidate action information into the hybrid ranking model to obtain the action information of the next state with the maximum action value predicted by the hybrid ranking model; and obtaining the hybrid ranking result of each tourism category according to the action information of the next state with the maximum action value.

[0015] In some embodiments, generating travel search results based on the mixed sorting results includes: removing duplicates from the mixed sorting results of each travel category and the rough sorting results of each travel category to generate travel search results.

[0016] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor; a memory, wherein the memory stores executable instructions; wherein when the executable instructions are executed by the processor, the hybrid ranking model training method described in any of the above embodiments is implemented, or the tourism search hybrid ranking method described in any of the above embodiments is implemented.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided for storing a program, wherein when the program is executed by a processor, the hybrid ranking model training method described in any of the above embodiments is implemented, or the tourism search hybrid ranking method described in any of the above embodiments is implemented.

[0018] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the hybrid ranking model training method described in any of the above embodiments, or implements the tourism search hybrid ranking method described in any of the above embodiments.

[0019] The beneficial effects of the present invention compared with the prior art include at least:

[0020] The present invention comprehensively considers the object information and different combination information of each tourism category, as well as the user's preference information for different tourism categories and objects by characterizing the state data of the attributes of the candidate objects of each tourism category and the user; constructs a reinforcement learning model that adapts to the personalized needs of users by characterizing the reward data of the user's operation on the mixed arrangement results of the candidate objects of the tourism category, and can dynamically adjust the weights and sorting rules of objects of different categories according to the user's search behavior and feedback; simulates the state data and action data by data embedding, and collects the reward data, fully considering the iterativeness of the model.

[0021] The personalized hybrid sorting solution for tourism search scenarios based on reinforcement learning of the present invention can effectively solve the problem of mixed sorting of objects of various categories in tourism search scenarios. By introducing reinforcement learning and personalized recommendations, the efficiency of tourism search and user experience can be improved, bringing development value to the business.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0024] Figure 1 A schematic diagram showing the steps of a hybrid sorting model training method according to an embodiment of the present invention is shown;

[0025] Figure 2 A schematic diagram showing the architecture of a DC-DQN model in an embodiment of the present invention is shown;

[0026] Figure 3 A schematic diagram showing the flow of training and deployment of a hybrid sorting model in an embodiment of the present invention;

[0027] Figure 4 A schematic diagram showing the steps of the hybrid ranking method for tourism search in an embodiment of the present invention;

[0028] Figure 5 A schematic diagram showing a flow chart of a hybrid ranking method for tourism search in an embodiment of the present invention;

[0029] Figure 6 A schematic structural diagram of an electronic device in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to make the present invention more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art.

[0031] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] In addition, the processes shown in the drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, some steps may be combined or partially combined, and the actual execution order may be changed according to the actual situation.

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and features in different embodiments may be combined with each other.

[0034] Figure 1 The main steps of the hybrid sorting model training method in the embodiment of the present invention are shown in FIG. Figure 1 As shown, the hybrid sorting model training method provided by the embodiment of the present invention includes:

[0035] S110, constructing sample data: including state data, action data and reward data, the state data represents the attributes of the candidate objects of each tourism category and the user, the action data represents the mixed arrangement results of the candidate objects of each tourism category, and the reward data represents the user's operation on the mixed arrangement results; wherein the state data and action data are generated by data embedding simulation, and the reward data is calculated by data embedding collection;

[0036] S120, using sample data to train a hybrid sorting model built based on reinforcement learning: calculate the action value of the current state based on the state data and action data of the current state, calculate the action value of the next state based on the state data and action data of the next state, calculate the model loss and adjust the model parameters based on the reward data of the current state, the action value of the current state and the action value of the next state.

[0037] The present invention comprehensively considers the object information and different combination information of each tourism category, as well as the user's preference information for different tourism categories and objects by characterizing the state data of the attributes of the candidate objects of each tourism category and the user; constructs a reinforcement learning model that adapts to the personalized needs of users by characterizing the reward data of the user's operation on the mixed arrangement results of the candidate objects of the tourism category, and can dynamically adjust the weights and sorting rules of objects of different categories according to the user's search behavior and feedback; simulates the state data and action data by data embedding, and collects the reward data, fully considering the iterativeness of the model.

[0038] The personalized hybrid sorting solution for tourism search scenarios based on reinforcement learning of the present invention can effectively solve the problem of mixed sorting of objects of various categories in tourism search scenarios. By introducing reinforcement learning and personalized recommendations, the efficiency of tourism search and user experience can be improved, bringing development value to the business.

[0039] In some embodiments, the status data includes: candidate object queue characteristics of each tourism category, user portrait characteristics, user preference characteristics for objects and categories, context characteristics, and search characteristics.

[0040] In this way, the present invention not only considers the information of a single object when sorting objects of different categories, but also considers the information of different combinations of upper and lower objects; in addition, not only the user's preference for a single object is considered, but also the user's preference for different object types.

[0041] In some embodiments, the tourism category includes ticket category, day tour category and entertainment category, and the characteristics of the candidate object include the relevance, popularity, order volume and evaluation of the candidate object. The object refers to a specific product, such as a ticket to a certain attraction.

[0042] In this way, the present invention provides users with accurate, comprehensive and personalized search results by comprehensively considering factors such as the relevance, popularity, orders, evaluations, contextual relationships, etc. of various categories of objects such as tickets, day trips, and entertainment.

[0043] In some embodiments, the influencing factors of the reward data include: click operation, order operation, hit rate and NDCG index; the reward data is calculated based on each influencing factor and its weight.

[0044] The weight of each influencing factor can be set as needed, and the specific calculation method of each influencing factor can also be set as needed, for example, weighted summation can be performed.

[0045] In a specific example, the hybrid sorting model is constructed using a reinforcement learning method, in which the construction of the quadruple is defined as follows. State S: Contains the top 20 candidate product queue features of each category in the current round, the portrait features on the user side, the user's preference features for products and categories, the global context, and search-related features. Action A: Represents the pit position decision of each category in the current round, for example, [1,2,3,2,2] represents the pit position result of [tickets, entertainment, day tour, entertainment, entertainment]. Reward R: Reward represents the benefit brought by taking action A in state S. In the scenario of the present invention, the reward is defined as the user's click and conversion on the mixed ranking list, and the top hit rate and ndcg indicators are integrated into the calculation of the reward formula. Next state S: Due to the characteristics of the application scenario of the present invention, each action only includes 5 pit position results. Therefore, in the previous state, the candidate queues of each category will remove the products in the corresponding category according to the category in the currently executed action, and the remaining candidate products will form the next state.

[0046] In some embodiments, the hybrid sorting model is constructed based on a competitive cross deep Q network, which adopts the architecture of a competitive deep Q network and cross-splices state data and action data.

[0047] Figure 2 The architecture of the DC-DQN model in the embodiment of the present invention is shown in FIG. Figure 1 and Figure 2As shown, in some embodiments, when calculating the action value of the current state according to the state data and action data of the current state, the action value of the current state is calculated based on the architecture of the Q-net side, including: according to the state data of the current state (including object-related features of the ticket category tkt, object-related features of the day tour category odt, object-related features of the entertainment category act, user-related features, etc.), by sharing parameters (Sharing Parms), respectively generate a candidate object queue of each tourism category of the current state represented by a vector as the state information of the current state (including the state information of the ticket category tkt, the state information of the day tour category odt, and the state information of the entertainment category act); according to the action data (action) of the current state, generate the action information of the current state represented by a matrix; multiply the state information of the current state (MLP) to obtain the state value of the current state (V(S)); cross-multiply the state information of the current state and the action information of the current state (Multiplicaion) and then add them to obtain the advantage of the current state (A(S,a)); add the state value of the current state to the advantage of the current state to obtain the action value of the current state (Q(S,a)).

[0048] The architecture and process of calculating the action value of the next state based on the state data and action data of the next state are consistent with the architecture and process of calculating the action value of the current state mentioned above, so Figure 2 Not repeated in. Figure 2 The architecture on the Q-net side calculates the action value of the next state based on the state data and action data of the next state, and the action value of the next state is calculated based on the architecture on the Target-Net side, including: generating candidate object queues of each tourism category of the next state represented by vectors as the state information of the next state by parameter sharing according to the state data of the next state; generating action information of the next state represented by a matrix according to the action data of the next state; multiplying the state information of the next state to obtain the state value of the next state; cross-multiplying the state information of the next state with the action information of the next state and adding them to obtain the advantage of the next state; adding the state value of the next state to the advantage of the next state to obtain the action value of the next state (Q(S`,a`)).

[0049] The hybrid sorting model constructed based on reinforcement learning in the present invention is named Dueling Cross Deep Q-Network (DC-DQN, competitive cross deep Q network), which adopts the framework structure of Dueling-DQN (competitive deep Q network). The core idea of ​​Dueling-DQN is to decompose the value function into a state value function and an advantage function, so that the value of the state and action can be better modeled. The state value function represents the expected return under a given state, and the advantage function represents the additional return relative to the state value function under a given state and action. Considering that the action and the state are related, the present invention proposes to cross the action and the state, that is, different actions may cause the next state to change, so the intersection of the action and the state is added to the model for learning. At the same time, the present invention also adds a state representation generation module to the value network, which is used to uniformly learn the product information, user information, context information, etc. in the candidate queue to represent the state. Finally, the state representation information extracted by the value function part and the product aggregation information after the execution of the action extracted by the advantage function part are fused to output the Q value of the current candidate action.

[0050] Among them, the loss function used in model training is:

[0051]

[0052] r is the reward data of the current state, γ is the discount factor, Q(s,a) is the action value of the current state, is the action value of the next state. B represents a batch of data.

[0053] The model training of the present invention adopts an offline training method, and is deployed online after the training is completed. In the offline training stage, the state and action of the user interaction are simulated through the buried log data, and the reward is calculated by the user's real click order to construct the training data. In the online reasoning stage, the state is constructed by carefully sorting products in each category, and the state information and candidate action set are input into the model for prediction to obtain the action with the largest Q value as the pit result.

[0054] Figure 3 The process of training and deploying the hybrid sorting model in the embodiment of the present invention is shown in FIG. Figure 3 , and combined Figure 1 and Figure 2 As shown, in a specific example, the training and deployment of the hybrid sorting model includes:

[0055] S310: Construct state information according to the characteristics of the business scenario, and the state information includes characteristic information such as products so that the model can make correct decisions. The state information is related to the task goal of sorting category objects in the tourism search scenario, and can reflect the environment and task status of the model.

[0056] S320: Define the action space to ensure that each action in the action space is feasible, that is, it can be executed in the environment and the state transition can be observed after the action is executed.

[0057] S330: Define the reward function, and determine the reward calculation logic by considering the task objectives in combination with the business scenario characteristics.

[0058] S340: Data embedding, which is to obtain necessary feature information through embedding, especially the user behavior (click, order, etc.) embedding data, and fully consider the iterativeness of the model.

[0059] At this point, the process of building sample data is completed.

[0060] S350: Construct training samples. Construct training samples that meet the specifications according to the model training method and input.

[0061] S360: Offline model training parameter adjustment. During reinforcement learning model training, attention should be paid to the complexity of the network, the learning rate should be controlled, and the appropriate discount factor should be adjusted to ensure sufficient sample training.

[0062] At this point, the process of using sample data to train the hybrid sorting model built based on reinforcement learning is completed.

[0063] S370: Online model deployment. Performance issues need to be considered when deploying models online. The action space and state space should be adjusted according to the machine performance in the business scenario.

[0064] The embodiment of the present invention also provides a tourism search hybrid sorting method. Figure 4 The main steps of the hybrid ranking method for tourism search in the embodiment of the present invention are shown in FIG. Figure 4 , and combined Figures 1 to 3 As shown, the tourism search hybrid sorting method provided by the embodiment of the present invention includes:

[0065] S410, obtaining a travel search request.

[0066] S420: Roughly sort objects of each tourism category according to the tourism search request to obtain rough sorting results of each tourism category.

[0067] S430, obtaining candidate objects of each tourism category from the rough ranking results, and performing fine ranking on the candidate objects of each tourism category to obtain fine ranking results of each tourism category.

[0068] The coarse sorting and the fine sorting can be set with corresponding sorting algorithms as needed. In this embodiment, an existing sorting algorithm is used.

[0069] S440, obtaining a mixed ranking result of each tourism category based on the mixed ranking model according to the refined ranking result of each tourism category.

[0070] Among them, based on the hybrid ranking model, a hybrid ranking result of each tourism category is obtained, specifically including: obtaining multiple candidate objects with a high ranking for each tourism category from the refined ranking results of each tourism category; generating state information of the current state according to the multiple candidate objects with a high ranking for each tourism category and the attributes of the user corresponding to the tourism search request; inputting the state information of the current state and the candidate action information into the hybrid ranking model to obtain the action information of the next state with the maximum action value predicted by the hybrid ranking model; and obtaining the hybrid ranking result of each tourism category according to the action information of the next state with the maximum action value.

[0071] S450, generating travel search results according to the mixed sorting results.

[0072] The generating of the tourism search results according to the mixed sorting results specifically includes: performing de-duplication sorting on the mixed sorting results of each tourism category and the rough sorting results of each tourism category to generate the tourism search results.

[0073] Figure 5 The specific process of the hybrid ranking method for tourism search in the embodiment of the present invention is shown; Figure 5 , and combined Figures 1 to 4 As shown, in a specific example, the tourism search hybrid ranking method includes:

[0074] S501, obtaining a user's travel search request.

[0075] S502: The front end generates a trace ID for the request.

[0076] S503, search engine recalls and roughly ranks top 300. This refers to recalling all objects of tourism categories and obtaining the top 300 objects by sorting them according to the rough ranking algorithm. 300 is an example value and can be replaced by different numbers in different application scenarios. Here, tourism categories include tickets, day trips, and entertainment, and objects include products under each tourism category.

[0077] S504, unified sorting service entry. Refers to performing fine sorting of each tourism category, generating ticket fine sorting queue tkt_list (including 200 products, for example), day tour fine sorting queue odt_list (including 60 products, for example), and entertainment fine sorting queue act_list (including 40 products, for example).

[0078] S505, mixed sorting service. That is, the mixed sorting model is used to perform mixed sorting of objects of each tourism category. Specifically, it includes: obtaining the top 20 tickets [t1, t2, ..., t20] from the ticket sorting queue tkt_list: [t1, t2, ..., t200], obtaining the top 20 day tours [o1, o2, ..., o20] from the day tour sorting queue odt_list: [o1, o2, ..., o60], and obtaining the top 20 entertainment [a1, a2, ..., a20] from the entertainment sorting queue act_list: [a1, a2, ..., a40], which constitute the candidate object queue of each tourism category. Then, the candidate object queue of each tourism category, as well as user information (user portrait information, user preference information for objects and categories, etc.), context information, real-time features and other external data are sent to the feature service to generate feature data represented by vectors.

[0079] Among them, the candidate object queues of each tourism category obtained above, that is, a total of 60 products are divided into black box model features. In addition, the white box model features of all the recalled products in the top 300 are obtained. The feature acquisition of the two models can be carried out in parallel.

[0080] S506, input model feature splicing, that is, using the mixed sorting model to perform cross splicing of features, for details, please refer to the above description.

[0081] S507, black box model inference service. That is, the hybrid ranking model is used to infer the action information of the next state. Specifically, the inferred Action result is: [0,0,0,1,1,0,1,2,0,0,1,2,1,0,0,1,1,1,2,2]. Then, the top20 final ranking result bbox_rank: [t1,t2,t3,o1,o2,t4,o3,a1,t5,t6,o4,a2,o5,t7,t8,o6,o7,o8,a3,a4].

[0082] In addition, the white-box model version id is used to obtain the ranking result wbox_rank of 300 products in the white-box model. wbox_rank is the same as the rough ranking top300 mentioned above.

[0083] S508, bbox_rank and wbox_rank are deduplicated to obtain the final mixed ranking result of 300 products.

[0084] S509, return to the search engine, feed back the search result data to the front end through the card service, and present it to the user.

[0085] After the refined sorting stage in the search sorting scenario, the present invention is applied. Each category business line calls the refined sorting model of its own business line for sorting based on the recall results, and then transmits the sorting results to the mixed sorting model for mixed unified sorting, and finally outputs the pit results of each category, and then fills the refined sorting results of each category into the corresponding pit according to the refined sorting order. The method of mixed sorting of multiple types of products based on reinforcement learning of the present invention is highly versatile and can be quickly implemented.

[0086] An embodiment of the present invention also provides an electronic device, including a processor and a memory, wherein executable instructions are stored in the memory, and when the executable instructions are executed by the processor, the hybrid sorting model training method described in any of the above embodiments is implemented, or the tourism search hybrid sorting method described in any of the above embodiments is implemented.

[0087] Figure 6 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. It should be understood that: Figure 6 The modules are only schematically shown, and these modules may be virtual software modules or actual hardware modules. The combination and splitting of these modules and the addition of other modules are all within the protection scope of the present invention.

[0088] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 include but are not limited to: at least one processor 610, at least one memory 620, and a bus 630 connecting different platform components (including the memory 620 and the processor 610).

[0089] The memory 620 stores program codes, which can be executed by the processor 610, so that the processor 610 executes the steps of the hybrid ranking model training method / tourism search hybrid ranking method described in any of the above embodiments.

[0090] The memory 620 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) and / or a cache memory, and may further include a read-only memory unit (ROM).

[0091] Memory 620 may also include programs / utilities having one or more program modules, such as, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0092] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0093] The electronic device 600 can also communicate with one or more external devices, and the external device can be one or more of the devices such as keyboard, pointing device, Bluetooth device, etc. These external devices enable the user to communicate interactively with the electronic device 600. The electronic device 600 can also communicate with one or more other computing devices, and the computer device shown includes a router and a modem. This communication can be carried out through an input / output (I / O) interface. In addition, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter. The network adapter can communicate with other modules of the electronic device 600 through a bus 630.

[0094] An embodiment of the present invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the hybrid ranking model training method described in any of the above embodiments, or implements the tourism search hybrid ranking method described in any of the above embodiments.

[0095] The storage medium may be a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the storage medium of the present invention is not limited thereto, and may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0096] The storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media include, but are not limited to, an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0097] The readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. The data signal of this propagation may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable signal medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0098] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device, such as through the Internet using an Internet service provider.

[0099] An embodiment of the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the hybrid ranking model training method described in any of the above embodiments, or implements the tourism search hybrid ranking method described in any of the above embodiments.

[0100] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A hybrid sorting model training method, characterized in that: include: Constructing sample data: including state data, action data and reward data, wherein the state data represents the attributes of the candidate objects of each tourism category and the user, the action data represents the mixed arrangement results of the candidate objects of each tourism category, and the reward data represents the user's operation on the mixed arrangement results; The state data and the action data are generated by data embedding simulation, and the reward data is collected and calculated by data embedding; The sample data is used to train a hybrid sorting model built based on reinforcement learning: the action value of the current state is calculated according to the state data and action data of the current state, the action value of the next state is calculated according to the state data and action data of the next state, and the model loss is calculated and the model parameters are adjusted according to the reward data of the current state, the action value of the current state and the action value of the next state.

2. The hybrid sorting model training method according to claim 1, characterized in that: The state data includes: candidate object queue features of each tourism category, user portrait features, user preference features for objects and categories, context features, and search features.

3. The hybrid sorting model training method according to claim 2, characterized in that: Tourism categories include ticket categories, day tour categories, and entertainment categories, and the characteristics of candidate objects include the relevance, popularity, order volume, and evaluation of the candidate objects.

4. The hybrid sorting model training method according to claim 1, characterized in that: The influencing factors of the reward data include: click operation, order operation, hit rate and NDCG index; The reward data is calculated based on various influencing factors and their weights.

5. The hybrid sorting model training method according to claim 1, characterized in that: The hybrid sorting model is constructed based on a competitive cross-depth Q network, which adopts the architecture of a competitive deep Q network and cross-joins the state data and the action data.

6. The hybrid sorting model training method according to claim 5, characterized in that: Calculate the action value of the current state based on the state data and action data of the current state, including: According to the state data of the current state, a candidate object queue of each tourism category of the current state represented by a vector is generated respectively by parameter sharing as the state information of the current state; According to the action data of the current state, the action information of the current state represented by the matrix is ​​generated; Multiply the state information of the current state to obtain the state value of the current state; The state information of the current state and the action information of the current state are cross-multiplied and then added to obtain the advantage of the current state; Add the state value of the current state to the advantage of the current state to get the action value of the current state.

7. The hybrid sorting model training method according to claim 6, characterized in that: Calculate the model loss and use the following loss function: Among them, r is the reward data of the current state, γ is the discount factor, Q(s,a) is the action value of the current state, and Q(s`,a`) is the action value of the next state.

8. A hybrid ranking method for tourism search, characterized in that: include: Get travel search request; According to the tourism search request, roughly sort the objects of each tourism category to obtain a rough sorting result of each tourism category; Obtaining candidate objects for each tourism category from the rough ranking results, and performing fine ranking on the candidate objects for each tourism category to obtain fine ranking results for each tourism category; According to the refined ranking result of each tourism category, based on the mixed ranking model as described in any one of claims 1 to 7, a mixed ranking result of each tourism category is obtained; Generate travel search results according to the mixed sorting results.

9. The tourism search hybrid ranking method as claimed in claim 8, characterized in that: Based on the hybrid sorting model, the hybrid sorting results of each tourism category are obtained, including: Obtaining multiple top-ranked candidate objects for each tourism category from the refined ranking results of each tourism category; Generate status information of the current status according to the top-ranked candidate objects of each tourism category and the attributes of the user corresponding to the tourism search request; Inputting the state information of the current state and the candidate action information into the hybrid sorting model to obtain the action information of the next state with the maximum action value predicted by the hybrid sorting model; According to the action information of the next state with the largest action value, a mixed sorting result of each tourism category is obtained.

10. The tourism search hybrid ranking method according to claim 9, characterized in that: Generating travel search results according to the mixed sorting results includes: The mixed sorting results of each tourism category and the rough sorting results of each tourism category are de-duplicated and sorted to generate tourism search results.

11. An electronic device, characterized in that: include: processor; A memory, wherein executable instructions are stored in the memory; Wherein, when the executable instructions are executed by the processor, the hybrid sorting model training method described in any one of claims 1-7 is implemented, or the tourism search hybrid sorting method described in any one of claims 8-10 is implemented.

12. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, it implements the hybrid ranking model training method described in any one of claims 1-7, or implements the tourism search hybrid ranking method described in any one of claims 8-10.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the hybrid ranking model training method described in any one of claims 1 to 7, or implements the tourism search hybrid ranking method described in any one of claims 8 to 10.

Images