Multi-service information sorting system, method, storage medium and electronic device

Abstract

This specification discloses a multi-business information sorting system, method, storage medium and electronic device, wherein the system includes a feature calculation module, which responds to a business access request for a target business and determines the feature field corresponding to the target business according to the type of the target business; constructs a feature sample set based on information matched with the feature field determined from a historical information set; a training module, which uses the feature sample set to train a training model to obtain a sorting model corresponding to the target business; an online sorting module, which responds to a sorting request directed to the target business and uses the sorting model corresponding to the target business to perform information sorting and obtain a sorting result; wherein the sorting result is used to characterize the display order of information. The above scheme can connect to multiple businesses and can provide targeted sorting services for multiple businesses.

Description

Technical Field

[0001] The present application relates to the technical field of multi-service information sorting, and in particular to a multi-service information sorting system, method, storage medium and electronic device. Background Art

[0002] With the rapid development and widespread use of the internet, it has become one of the primary ways people obtain information. Internet-based search plays an indispensable role in efficiently acquiring information. Internet users often search for answers by entering keywords and phrases in search systems.

[0003] Typically, if the information retrieved through a search isn't unique, it needs to be sorted so that it can be presented to the user in a prioritized order. For users, the most direct evaluation is whether the top few items in the display order meet their needs. In other words, the ranking of the retrieved information significantly impacts the user experience. Summary of the Invention

[0004] The embodiments of this specification provide a multi-service information sorting system, method, storage medium, and electronic device to partially solve the above-mentioned problems existing in the prior art.

[0005] The embodiments of this specification adopt the following technical solutions:

[0006] In a first aspect, the present application provides a multi-service information sorting system, comprising:

[0007] a feature calculation module, in response to a service access request for a target service, determining a feature field corresponding to the target service according to the type of the target service; and constructing a feature sample set based on information matching the feature field determined from the historical information set;

[0008] A training module, configured to train a model to be trained using the feature sample set to obtain a ranking model corresponding to the target business;

[0009] The online sorting module is used to respond to the sorting request directed to the target business, adopt the sorting model corresponding to the target business, perform information sorting, and obtain a sorting result; wherein the sorting result is used to represent the display order of the information.

[0010] In an optional embodiment of the present specification, the feature calculation module is specifically used to: determine the type of the target service as the target type; determine, among the historical services that have been accessed in the past, a historical service whose type matches the target type as the first reference service; and determine the feature field of the target service based on the feature field of the first reference service.

[0011] In an optional embodiment of this specification, the system further includes: a model selection module;

[0012] The model selection module is configured to: determine the type of samples in the feature sample set; and based on a preset correspondence between sample types and candidate models, determine a candidate model corresponding to the type of the sample from the preset candidate models as the model to be trained;

[0013] The sample type is any one of the following: continuous, discrete, and sequential.

[0014] In a second aspect, this specification provides a multi-service information sorting method, the method comprising:

[0015] In response to a service access request for a target service, determining a feature field corresponding to the target service according to a type of the target service;

[0016] Constructing a feature sample set based on information matching the feature field determined from the historical information set;

[0017] Using the feature sample set to train the model to be trained, to obtain a ranking model corresponding to the target business;

[0018] In response to a sorting request directed to the target business, a sorting model corresponding to the target business is used to perform information sorting to obtain a sorting result; wherein the sorting result is used to represent the display order of the information.

[0019] In an optional embodiment of the present specification, the method further includes: for each target business, taking the correspondence between the business identifier of the target business and the model identifier of the sorting model corresponding to the target business as a designated relationship; and constructing a designated relationship set based on the designated relationships corresponding to each target business.

[0020] In an optional embodiment of the present specification, a sorting model corresponding to the target business is used to perform information sorting to obtain a sorting result, including: selecting a specified relationship whose business identifier matches the business identifier of the target business from a specified relationship set as a target relationship; and using the sorting model to which the model identifier corresponding to the target relationship belongs to sort the information to be sorted carried in the sorting request to obtain a sorting result.

[0021] In an optional embodiment of the present specification, the method further includes: determining the description information of the target service from the service access request; and based on the correspondence between the preset description information and the service type, determining the service type that matches the service description information as the type of the target service.

[0022] In an optional embodiment of the present specification, the description information of the target service is determined from the service access request; among the historical services that have completed access in the past, the historical service whose description information matches the description information of the target service is determined as the second reference service; and the type of the target service is determined based on the type of the second reference service.

[0023] In an optional embodiment of the present specification, determining, according to the type of the target service, a feature field corresponding to the target service includes:

[0024] Determining the type of the target business as the target type;

[0025] Determining, from among historical services that have been accessed in the past, a historical service whose type matches the target type as a first reference service;

[0026] The characteristic field of the target service is determined according to the characteristic field of the first reference service.

[0027] In an optional embodiment of the present specification, determining the characteristic field of the target service according to the characteristic field of the first reference service includes:

[0028] Displaying the characteristic field of the first reference service as a display field;

[0029] If a user adjustment operation on the display field is detected, adjusting the display field based on the adjustment operation;

[0030] If a confirmation operation of the user is detected, the currently displayed fields are used as characteristic fields of the target service.

[0031] In an optional embodiment of this specification, the method further includes:

[0032] Determine the type of samples in the feature sample set; wherein the type of the samples is any one of the following: continuous, discrete, and sequential;

[0033] Based on the correspondence between the preset feature types and the candidate models, the candidate model corresponding to the type of the feature is determined from the preset candidate models as the model to be trained.

[0034] In an optional embodiment of the present specification, determining the type of the sample in the feature sample set includes:

[0035] For each sample in the feature sample set, a first target feature is determined from the features of the sample; the type of the first target feature is used as the type of the sample; wherein the first target feature is the feature with the strongest characterization capability for the sample among the features.

[0036] In an optional embodiment of the present specification, determining the type of the sample in the feature sample set includes:

[0037] For each sample in the feature sample set, cluster the features of the sample according to the feature type; use the type of the feature in the target class as the type of the sample; wherein the target class is the class containing the largest number of features among the clustered classes.

[0038] In an optional embodiment of the present specification, a sorting model corresponding to the target service is used to perform information sorting to obtain a sorting result, including:

[0039] Reading a characteristic field corresponding to the target service from local data;

[0040] Constructing a second target feature according to information matching the feature field in the information to be sorted carried in the sorting request;

[0041] The second target feature is processed using a ranking model corresponding to the target business to obtain a ranking result.

[0042] In an optional embodiment of the present specification, there are at least two models to be trained; and

[0043] Using a sorting model corresponding to the target business among the sorting models to perform information sorting to obtain a sorting result, including: using at least two sorting models corresponding to the target business to sort the information to be sorted carried in the sorting request to obtain sorting results corresponding to each of the at least two sorting models;

[0044] The method further includes: determining a ranking effect score of each ranking result according to a user's operation on each ranking result; and determining a ranking model with a higher ranking effect score among the at least two ranking models as a final ranking model for the target business.

[0045] In an optional embodiment of the present specification, a sorting model corresponding to the target service is used to perform information sorting to obtain a sorting result, including:

[0046] Using a ranking model corresponding to the target business, performing information ranking to obtain an intermediate result;

[0047] According to the attributes of each piece of information to be sorted, the order of each piece of information to be sorted in the intermediate result is adjusted to obtain a sorting result.

[0048] In a third aspect, this specification provides a multi-service information recommendation system, including:

[0049] A recall module, used to recall information to be sorted according to keywords; and the multi-service sorting system in any one of the aforementioned first aspects, wherein the multi-service sorting system is used to sort the information to be sorted recalled by the recall module to obtain a sorting result.

[0050] In an optional embodiment of this specification, the multi-service information recommendation system further includes a display module;

[0051] The display module is used to display the information recalled by the recall module based on the sorting result output by the multi-service sorting system.

[0052] In a fourth aspect, this specification provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the multi-service information sorting method in the first aspect is implemented.

[0053] In a fifth aspect, this specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the multi-service information sorting method in the first aspect described above is implemented.

[0054] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0055] The multi-business information sorting system, method, storage medium and electronic device in the embodiments of this specification adopt a multi-business information sorting system to sort the recalled information. On the one hand, different sorting strategies are adopted based on different businesses, so that the sorting results obtained by sorting are more targeted to the business, and the sorting results can better reflect the characteristics of the business. On the other hand, the sorting strategy adopted in the sorting process is mainly reflected in: a feature sample set that matches the business is determined in a targeted manner, and the sorting model trained with the feature sample set can also output a more reasonable sorting result for the business. Thereafter, during the online use process, if a sorting request for the business is received, sorting is performed based on the sorting model trained for the business, and a sorting result that better meets the user's needs can also be obtained. In addition, the above scheme can connect to multiple businesses and provide targeted sorting services for multiple businesses. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation of this specification. In the drawings:

[0057] Figure 1a A schematic diagram of the architecture of a multi-service information sorting system provided in an embodiment of this specification;

[0058] Figure 1b A schematic diagram of the communication relationship between components of the multi-service information sorting system provided in an embodiment of this specification;

[0059] Figure 2 A flowchart of a multi-service information sorting method provided in an embodiment of this specification;

[0060] Figure 3 A schematic diagram of the process of selecting a model to be trained in the multi-service information sorting method provided in an embodiment of this specification;

[0061] Figure 4 A schematic diagram of the online sorting process in the multi-service information sorting method provided in the embodiments of this specification;

[0062] Figure 5 The embodiments of this specification provide corresponding Figure 2 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0064] The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0065] Figure 1a This is a schematic diagram of the architecture of the multi-service information sorting system in this manual. Figure 1a As shown, the multi-service information sorting system includes a feature calculation module, a training module and an online sorting module, wherein the feature calculation module is communicated with the training module and the online sorting module respectively; the training module is communicated with the online sorting module.

[0066] In an optional embodiment of the present specification, the feature calculation module is used to respond to a service access request for a target service and, based on the type of the target service, determine a feature field corresponding to the target service; and construct a feature sample set based on information matching the feature field determined from the historical information set. The training module is used to train a to-be-trained model using the feature sample set to obtain a ranking model corresponding to the target service. The online ranking module is used to respond to a ranking request directed to the target service and, using the ranking model corresponding to the target service, perform information ranking to obtain a ranking result; wherein the ranking result is used to represent the display order of the information.

[0067] In another optional embodiment of the present specification, the feature calculation module is specifically used to: determine the type of the target service as the target type; determine, among the historical services that have been accessed in the past, a historical service whose type matches the target type as the first reference service; and determine the feature field of the target service based on the feature field of the first reference service.

[0068] In another optional embodiment of the present specification, the multi-business information sorting system further includes: a model selection module, the model selection module being used to determine the type of samples in the feature sample set; based on the correspondence between the preset sample type and the alternative model, determining the alternative model corresponding to the type of the sample from the preset alternative models as the model to be trained; wherein the type of the sample is any one of the following: continuous, discrete, and sequential.

[0069] like Figure 1b As shown, the multi-service information ranking system consists of the following components: a computing resource server, a storage server, a service management backend, an online deployment server, and a database. The computing resource server sends the model file (which may include ranking model parameters) and the encoding file (which may include service feature fields) to the storage server for storage. The computing resource server stores the offline indicator model distribution path in the database.

[0070] The multi-business information sorting process in this specification involves two stages: an offline stage and an online stage. In the offline stage, the multi-business information sorting system connects with the sorting service demander and obtains historical data collected from the business demander. Thereafter, a feature sample set is extracted from the historical data. A model matching the business is trained based on the feature sample set. In the online stage, the multi-business information sorting system obtains a sorting request from the sorting service demander, and the sorting request carries the information to be sorted. Then, the information to be sorted is sorted to obtain a sorting result. Optionally, thereafter, the multi-business information sorting system returns the sorting result to the sorting service demander, and the sorting service demander displays the information based on the sorting result.

[0071] The feature calculation module participates in both the offline and online phases. The training module participates in the offline phase, and the online ranking module participates in the online phase.

[0072] like Figure 2 As shown, the multi-service information sorting process performed by the multi-service information sorting system in this specification may specifically include one or more of the following steps:

[0073] S200: In response to a service access request for a target service, determine a feature field corresponding to the target service according to the type of the target service.

[0074] The multi-service information ranking system described in this specification is intended to provide services for multiple services. A target service is one of the services provided by the multi-service information ranking system. Any service connected to the multi-service information ranking system can be considered a target service in this specification. Specifically, this step can be performed by the feature calculation module of the service information ranking system.

[0075] In an optional embodiment of the present specification, the service access request carries a historical information set of the sorting service requirements of the target service, which has been collected based on the target service. The feature field is used to determine, from the historical information set, a feature for sorting the to-be-sorted information corresponding to the target service.

[0076] For example, the type of business 1 is online shopping business, the historical information set of business 1 is a set of user online shopping information collected historically, and the characteristic fields of business 1 are user age, user gender, and user occupation.

[0077] The multi-service information sorting system in this specification can be maintained through the following access service status table.

[0078]

[0079] Data determines the upper limit of the model. Feature selection is designed because not all features have positive feedback on offline indicators. On the contrary, some features will cause significant noise in the model. In view of this, this manual uses the feature calculation module to determine the feature field to reduce the noise caused by other data.

[0080] S202: Constructing a feature sample set according to the information matched with the feature field determined from the historical information set.

[0081] In an optional embodiment of the present specification, the feature calculation module can use field matching to determine information matching the feature field from the historical information set as the feature sample set. Specifically, this step is performed by the feature calculation module of the business information ranking system.

[0082] In another optional embodiment of the present specification, the feature calculation module can use field matching to determine the information that matches the feature field from the historical information set. Feature construction is performed on the information determined from the historical information set to obtain features that characterize the historical data from different dimensions. The set of feature constructions is used as a feature sample set. In this embodiment, the information can be processed according to the feature processing function to obtain samples in the feature sample set. For example, the tf-idf method can be used to calculate the keywords of the material and the click rate of the material within a certain period of time, or the word2vec method can be used to pre-train the ID of the multi-valued class into a dense numerical vector as a sample in the feature sample set.

[0083] S204: Using the feature sample set, the to-be-trained model is trained to obtain a ranking model corresponding to the target business.

[0084] The multi-service information ranking system in this specification is intended to provide ranking services for multiple services, so the ranking model trained by the multi-service information ranking system may not be unique. In an optional embodiment of this specification, during the offline phase, for each target service, the corresponding relationship between the service identifier of the target service and the model identifier of the ranking model corresponding to the target service can be used as a designated relationship; based on the designated relationships corresponding to each target service, a designated relationship set is constructed. Specifically, this step is performed by the training module of the service information ranking system.

[0085] Optionally, the specified relationship combination can be represented by the following business model table.

[0086] ID name Business Information Request address Request Parameters 100 DeepFM An e-commerce business url rank_method = DeepFM 101 FM An e-commerce business url rank_method = FM ... … … … ...

[0087] In another optional embodiment of this specification, a service interface is created based on the service identifier of the first target service and the model information of the sorting model corresponding to the first target service; the sorting model corresponding to the second target service in each sorting model is used to perform information sorting to obtain a sorting result, including: querying from each service interface a service interface whose service identifier matches the service identifier of the second target service as the target interface; and sorting the information to be sorted carried in the sorting request using the sorting model to which the model information corresponding to the target interface belongs to obtain a sorting result. In this embodiment, the service interface can be shown in the following table:

[0088] Interface parameters illustrate app_id A unique identifier assigned to each business rank_method Ranking Model user_id Search user id item_ids Recalled document ID list query Search for key words or phrases context Contextual information, such as location, time, device information, etc.

[0089] The ranking model's model file contains a used feature JSON file. When the model file is imported and loaded onto the online server, the feature JSON file is read. When the ranking model requests real-time features from the feature calculation module, it filters all features to obtain the features specified in the feature JSON file. This ensures that the offline trained model data is consistent with the online prediction input data.

[0090] S206: In response to the sorting request directed to the target business, use the sorting model corresponding to the target business to perform information sorting to obtain a sorting result; wherein the sorting result is used to represent the display order of the information.

[0091] During the online phase, if a sorting request is received, the service identifier of the target service specified by the sorting request is determined from the sorting request. Then, from the specified relationship set, a designated relationship whose service identifier matches the service identifier of the target service is selected as the target relationship. The sorting model corresponding to the model identifier of the target relationship is used to sort the information to be sorted contained in the sorting request, obtaining a sorting result. Specifically, this step is performed by the online sorting module of the service information sorting system.

[0092] When a ranking service requester completes a search request, Elasticsearch (ES), a distributed search and analytics engine, first retrieves a subset of documents that closely match the search results from a vast database. This JSON-formatted retrieved information includes the user ID, query, and a list of item_ids (object IDs). Some contextual information about the search is also returned. This information is used as an interface parameter, along with the app_id (which can be used as a business ID) and rank_method parameters. A request is then sent to the online search ranking interface to sort the ES results.

[0093] The multi-business information sorting system and method in the embodiments of this specification adopt a multi-business information sorting system to sort the recalled information. On the one hand, different sorting strategies are adopted based on different businesses, so that the sorting results obtained by sorting are more targeted to the business, and the sorting results can better reflect the characteristics of the business. On the other hand, the sorting strategy adopted in the sorting process is mainly reflected in: a feature sample set that matches the business is determined in a targeted manner, and the sorting model trained with the feature sample set can also output a more reasonable sorting result for the business. Thereafter, during the online use process, if a sorting request for the business is received, sorting is performed based on the sorting model trained previously for the business, and a sorting result that better meets user needs can also be obtained. In addition, the above scheme can connect to multiple businesses and can provide targeted sorting services for multiple businesses.

[0094] 1. Determine the characteristic fields.

[0095] As can be seen from the foregoing, the multi-service information ranking system in this specification can determine a characteristic field for a target service. The process of determining the characteristic field will now be described. The process of determining the characteristic field can be performed by a characteristic calculation module of the multi-service information ranking system.

[0096] The characteristic fields in this specification are determined based on the type of the target service. In an optional embodiment of this specification, the process of determining the type of the target service may be: determining the description information of the target service from the service access request; and based on a preset correspondence between the description information and the service type, determining the service type that matches the service description information as the type of the target service.

[0097] In another optional embodiment of the present specification, the process of determining the type of the target service may be: determining the description information of the target service from the service access request; determining, among the historical services that have completed access in the past, a historical service whose description information matches the description information of the target service as a second reference service; and determining the type of the target service based on the type of the second reference service.

[0098] After determining the type of the target service, characteristic fields are determined based on the target service type. Specifically, the target service type is determined as the target type; among previously accessed historical services, a historical service whose type matches the target type is determined as a first reference service; characteristic fields of the first reference service are displayed as display fields; if a user adjustment operation on the display field is detected, the display field is adjusted based on the adjustment operation; if a user confirmation operation is detected, the currently displayed fields are used as the characteristic fields of the target service.

[0099] Optionally, the multi-service information sorting system in this specification provides an interactive interface. Users can modify the display fields through the interactive interface. The modification method can be any of the following: adding, deleting, and modifying.

[0100] After the characteristic fields are determined, a characteristic sample set may be determined from the historical information set based on the characteristic fields.

[0101] Due to the diverse nature of businesses, the historical information collected also varies. Therefore, for specific businesses, the platform will work with the business development personnel to determine which log fields need to be provided. For example, e-commerce businesses require information such as product price and sales volume. News content businesses require information such as word count and region. The information in the historical information collection in this specification may include at least one of the following: user historical search logs, user log tables, and material log tables.

[0102] A user's historical search log reflects search interactions and their subjective preferences over time. Each independent search behavior generates a search log entry. In search, user behavior typically includes search queries, impressions, and clicks. Each behavior log entry consists of several essential fields, such as query (the search keyword or phrase), behavior_type (behavior type), and the specific behavior.

[0103] The user log table records basic attributes and profile information filled in or noted by registered users during their active period, such as age, region, and entry tags. User logs can update user information in a timely manner and reflect some basic user status.

[0104] In search services, material logs primarily refer to recalled documents. They record information after a document is created and can be updated promptly based on document modifications. Searched material logs typically include the document's URL, title, and content.

[0105] To distinguish between user actions generated through search and recommendations, and to ensure that user search history can also serve as a recommendation engine without interference from recommendation data, user actions such as clicks on items are divided into two types in the behavior log: actions generated in search scenarios are recorded in the search behavior log; actions generated in all scenarios are recorded in the general behavior log, which is used to construct the behavior dataset for recommendation training.

[0106] Search behavior log fields need to be correlated, counted, converted, and aggregated through the feature platform to generate a series of trainable values. For example, for e-commerce businesses, user behavior logs can be used to calculate features such as the user's purchase list over the past 30 days, the list of products viewed in the last hour, and the average order amount. For community content businesses, keywords can be calculated, and NLP processing can be used to generate feature vectors for titles or summaries.

[0107] In an optional embodiment of the present specification, a historical information set is obtained; wherein the log information includes information collected for the user's search behavior and non-search behavior; a first information set is constructed based on the search behavior information in the log information; a second information set is constructed based on the non-search behavior information in the log information; and the historical information set is constructed based on the first information set and the second information set.

[0108] After building the historical information set, the feature calculation module calculates the training set file (CSV file) according to the scheduled task, and then uses the rsync tool to transfer it to the designated folder on the training machine. The rsync tool requires a configuration file, and the destination address is set by the specific module. Independent calculation and file management are performed based on the business ID and model ID.

[0109] As can be seen from the foregoing, the multi-service information ranking system of this specification is capable of determining matching feature fields for a target service. In a further optional embodiment of this specification, the multi-service information ranking system further includes a model selection module, which determines, through the model selection module, the model most suitable for ranking the corresponding information of the target service from among a number of candidate models.

[0110] 2. Select the model to be trained.

[0111] In an optional embodiment of the present specification, the multi-service information sorting system locally pre-stores several candidate models. Optionally, the model in the present specification is an artificial intelligence model, such as an LSTM (Long Short-Term Memory) model, K-Nearest Neighbors, etc.

[0112] The process of determining the corresponding model to be trained for the target business in each candidate model can be performed by the training module (such as Figure 1a The process can be:

[0113] S300: Determine the type of samples in the feature sample set; wherein the type of the samples is any one of the following: continuous, discrete, and sequential.

[0114] For some models, selecting all features may not yield optimal training results. For example, FM and LR are not adept at analyzing continuous numerical features, but perform better with categorical discrete features. We also provide model selection training to select models and feature combinations that have better offline metrics and demonstrate better generalization performance in real-world scenarios. This functionality will be integrated into the business model distribution management backend, facilitating flexible testing and timely adjustments.

[0115] In an optional embodiment of the present specification, the type of a sample in the feature sample set may be determined by: for each sample in the feature sample set, determining a first target feature from the features of the sample; and using the type of the first target feature as the type of the sample; wherein the first target feature is the feature with the strongest characterization capability for the sample among the features. The characterization capability in the present specification may be characterized by the positive contribution of a feature to the convergence speed of the model during training. The higher the degree of improvement in the convergence speed of the model, the stronger the characterization capability of the feature. The characterization capability may be an empirical value.

[0116] In another optional embodiment of the present specification, for each sample in the feature sample set, the features of the sample are clustered according to the type of the feature; the type of the feature in the target class is used as the type of the sample; wherein the target class is the class containing the largest number of features among the classes obtained by clustering.

[0117] Suitable models for continuous samples include tree models such as LightGBM and CatBoost, and DNNs. These models can identify the magnitude relationship between values. Suitable models for discrete samples include FM, LR, and SVM. Suitable models for sequential samples include RNN models such as LSTM, TextCNN models, or pooling models.

[0118] S302: Based on the correspondence between the preset feature types and the candidate models, determine the candidate model corresponding to the type of the feature from the preset candidate models as the model to be trained.

[0119] Feature types on the platform are categorized into three types: continuous, discrete, and sequence. The available models have varying performance capabilities for each of these three types. Continuous features are typically derived from floating-point numerical features, such as statistical features or height. Continuous features are often processed using bucketing or ensemble tree training. Discrete features, typically categorical, are the most common and widely distributed features, as much data in the industry is represented by IDs. Discrete features can reflect uniqueness, but can lead to excessively long input dimensions and sparse data. They can generally be fed into the model using one-hot or hashing, or pre-trained into dense vectors. Discrete features generally perform better in linear models such as FM and LR. Sequence features, also known as multi-valued features, commonly include multi-valued labels and time series. Sequence feature parsing is often complex. Common approaches include encoding using multi-onehot methods or initializing embeddings for each element and then pooling. Some scenarios, which consider temporal features, require pre-training using RNN models. Consequently, each feature type has different processing methods, resulting in significant differences in model performance. Therefore, the platform can obtain the corresponding type of each feature and the ranking results of the performance of each model under this type. In other words, the platform can make a judgment on the feature type based on the current feature and give the preferred ranking of the model for this type of feature.

[0120] 3. Online stage.

[0121] After obtaining the sorting model of the target business through the aforementioned steps, sorting of the information to be sorted can be performed based on the sorting request.

[0122] Specifically, the online sorting module includes a real-time collection submodule and a sorting submodule. The real-time collection submodule interfaces with the sorting service demander.

[0123] S400: receiving a sorting request from a sorting service demander, the sorting request carrying information to be sorted, and sending the information to be sorted to a feature calculation module.

[0124] This step can be performed by the real-time acquisition submodule.

[0125] S402: Read the characteristic field corresponding to the target service from local data.

[0126] This step can be performed by the feature calculation module.

[0127] S404: Construct a second target feature according to the information matching the feature field in the information to be sorted carried in the sorting request.

[0128] This step can be performed by the feature calculation module.

[0129] S406: Process the second target feature using a ranking model corresponding to the target business to obtain a ranking result.

[0130] This step can be performed by the sorting submodule.

[0131] In an optional embodiment of the present specification, the sorting result is also outputted in the online stage based on the attributes of the information to be sorted. Specifically, the online sorting module also includes an operation position management submodule. The position management submodule includes a business rule input interface, which is used to input business rules. The business rules include several sorting reference fields, and each sorting reference field corresponds to a field weight. The position management submodule receives the output of the sorting submodule as an intermediate result. For each information in the intermediate result, the attributes of the information are determined, and the attributes of the information are characterized by multiple attribute fields.

[0132] The campsite management submodule identifies each sorting reference field and, if matching attribute information exists in any of the attribute fields, designates that field as the designated field. The sum of the weights for each designated field is then used as the first weight for that information. The ranking of that information in the intermediate results is used as the second weight. The sum of the first and second weights is used as the overall weight for that information. The information is then sorted based on the overall weight to produce a ranking result.

[0133] In a further optional embodiment of the present specification, the multi-business information sorting system can also analyze the sorting effect of the sorting model obtained in the offline stage, and then adjust the offline training process based on the results of the analysis. Specifically, in the offline stage, at least two models to be trained are determined for the target business, and then each model to be trained is trained separately to obtain at least two sorting models corresponding to at least the target business. Thereafter, in the online stage, the AB testing method is used to allocate traffic to the at least two sorting models, and the sorting results of the at least two sorting models are obtained, and the sorting results are used to display each information. Afterwards, for each sorting result, the user's sorting effect score for the information displayed with the sorting result is determined. The sorting model with the higher sorting effect score among the at least two sorting models is determined as the final sorting model for the target business.

[0134] The ranking effect score is obtained based on at least one of the click-through rate and NDCG (Normalized Discounted Cumulative Gain). Specifically, the ranking effect score is positively correlated with at least one of the click-through rate and NDCG.

[0135] For example, when performing AB testing on the ranking model, the test data can be recorded using the following AB testing table.

[0136] Strategy ID Business ID Business Model ID Diversion ratio state 1 10000 100 50％ Enable 2 10000 101 50％ Enable ... … … …

[0137] To improve response speed during the online phase, the computing resources provided by the feature calculation module are divided into multiple feature processing processes (hereinafter referred to as "workers"), and output is performed according to different business interfaces. For example, there are workers responsible for calculating query features, workers for calculating document features, and workers for calculating user features. Furthermore, using different feature processing processes for feature calculation allows for independent and separate requests when business interfaces are requested, improving data transmission capabilities.

[0138] This specification further provides a multi-service information recommendation system, comprising a recall module and the aforementioned multi-service ranking system. The recall module is communicatively connected to the aforementioned multi-service ranking system. The recall module is configured to recall information to be ranked based on keywords. The multi-service ranking system is configured to rank the information to be ranked recalled by the recall module to obtain a ranking result.

[0139] Optionally, the recall module responds to the user's information acquisition request, parses keywords from the information acquisition request, and then uses the information recalled based on the keywords as the information to be sorted.

[0140] The multi-service information recommendation system in this specification may further include a display module. The display module is configured to display the information recalled by the recall module based on the ranking result output by the multi-service ranking system.

[0141] The embodiments of this specification also provide a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1a The process of sorting the multi-service information provided.

[0142] The embodiments of this specification also propose Figure 5 The schematic structure diagram of the electronic device shown in FIG. Figure 5 At the hardware level, the electronic device may include a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile storage into the memory and then runs it to implement any of the above-mentioned multi-service information sorting processes.

[0143] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as the combination of logic device XOR software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic device.

[0144] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD through their own programming, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0145] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0146] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0147] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0148] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0150] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0152] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0153] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0154] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0156] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Thus, this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0158] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0159] The foregoing is merely an example of the present invention and is not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A multi-service information sorting system, characterized in that: include: a feature calculation module, responding to a service access request for a target service and determining a feature field corresponding to the target service according to a type of the target service; The target service type is determined according to the description information of the target service in the service access request; Using a field matching method, information matching the characteristic field is determined from a historical information set, where the historical information set is collected historically by the ordering service demander of the target business based on the target business; Performing feature construction on the information matched with the feature field determined from the historical information set to construct a feature sample set; A training module, configured to train a model to be trained using the feature sample set to obtain a ranking model corresponding to the target business; An online sorting module is used to respond to a sorting request directed to the target business, adopt a sorting model corresponding to the target business, perform information sorting on each piece of information to be sorted carried in the sorting request, and obtain a sorting result; wherein the sorting result is used to represent the display order of each piece of information to be sorted carried in the sorting request.

2. The system according to claim 1, wherein The feature calculation module is specifically used to: determine the type of the target service as the target type; determine the historical service whose type matches the target type among the historical services that have been accessed in the past as the first reference service; and determine the feature field of the target service based on the feature field of the first reference service.

3. The system according to claim 1, wherein: The system further comprises: a model selection module; The model selection module is configured to: determine the type of samples in the feature sample set; and based on a preset correspondence between sample types and candidate models, determine a candidate model corresponding to the type of the sample from the preset candidate models as the model to be trained; The sample type is any one of the following: continuous, discrete, and sequential.

4. A multi-service information sorting method, characterized in that: include: In response to a service access request for a target service, determining a feature field corresponding to the target service according to a type of the target service; The target service type is determined according to the description information of the target service in the service access request; Using a field matching method, information matching the characteristic field is determined from a historical information set, where the historical information set is collected historically by the ordering service demander of the target business based on the target business; Performing feature construction on the information matched with the feature field determined from the historical information set to construct a feature sample set; Using the feature sample set to train the model to be trained, to obtain a ranking model corresponding to the target business; In response to a sorting request directed to the target business, a sorting model corresponding to the target business is used to perform information sorting on each piece of information to be sorted carried in the sorting request to obtain a sorting result; wherein the sorting result is used to characterize the display order of each piece of information to be sorted carried in the sorting request.

5. The method according to claim 4, wherein The method further includes: for each target business, taking the correspondence between the business identifier of the target business and the model identifier of the ranking model corresponding to the target business as a designated relationship; and constructing a designated relationship set based on the designated relationships corresponding to the respective target businesses; Using a sorting model corresponding to the target business, information sorting is performed to obtain a sorting result, including: selecting a specified relationship whose business identifier matches the business identifier of the target business from a specified relationship set as a target relationship; using a sorting model to which the model identifier corresponding to the target relationship belongs, sorting each piece of information to be sorted carried in the sorting request to obtain a sorting result.

6. The method according to claim 4, wherein The method further comprises: Determining the description information of the target service from the service access request; determining the service type that matches the description information of the target service as the type of the target service based on the correspondence between the preset description information and the service type; or, Determine the description information of the target service from the service access request; among the historical services that have completed access in the past, determine the historical service whose description information matches the description information of the target service as the second reference service; determine the type of the target service based on the type of the second reference service.

7. The method according to claim 4, wherein Determining, according to the type of the target service, a characteristic field corresponding to the target service, including: Determining the type of the target business as the target type; Determining, from among historical services that have been accessed in the past, a historical service whose type matches the target type as a first reference service; The characteristic field of the target service is determined according to the characteristic field of the first reference service.

8. The method according to claim 4, wherein The method further comprises: Determine the type of samples in the feature sample set; wherein the type of the samples is any one of the following: continuous, discrete, and sequential; Based on the correspondence between the preset sample type and the candidate models, the candidate model corresponding to the type of the sample is determined from the preset candidate models as the model to be trained.

9. The method according to claim 8, wherein Determining the type of the sample in the feature sample set includes: For each sample in the feature sample set, determine a first target feature from the features of the sample; use the type of the first target feature as the type of the sample; wherein the first target feature is the feature with the strongest characterization capability for the sample among the features; or, For each sample in the feature sample set, cluster the features of the sample according to the feature type; use the type of the feature in the target class as the type of the sample; wherein the target class is the class containing the largest number of features among the clustered classes.

10. The method according to claim 4, wherein Using a ranking model corresponding to the target business, performing information ranking to obtain a ranking result, including: Reading a characteristic field corresponding to the target service from local data; Constructing a second target feature according to information matching the feature field in the information to be sorted carried in the sorting request; The second target feature is processed using a ranking model corresponding to the target business to obtain a ranking result.

11. The method according to claim 4, wherein There are at least two models to be trained; and Using a sorting model corresponding to the target business among the sorting models to perform information sorting to obtain a sorting result, including: using at least two sorting models corresponding to the target business to sort the information to be sorted carried in the sorting request to obtain sorting results corresponding to each of the at least two sorting models; The method further includes: determining a ranking effect score of each ranking result according to a user's operation on each ranking result; and determining a ranking model with a higher ranking effect score among the at least two ranking models as a final ranking model for the target business.

12. The method according to claim 4, wherein Using a ranking model corresponding to the target business, performing information ranking to obtain a ranking result, including: Using a ranking model corresponding to the target business, performing information ranking to obtain an intermediate result; According to the attributes of each piece of information to be sorted, the order of each piece of information to be sorted in the intermediate result is adjusted to obtain a sorting result.

13. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the steps of the multi-service information sorting method described in any one of claims 4 to 12 when executing the program stored in the memory.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-service information sorting method according to any one of claims 4 to 12 are implemented.

Images