Text ranking method, text ranking device, electronic equipment and storage medium

Abstract

The application provides a text ranking method, a text ranking device, an electronic device and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: constructing a training sample set, the training sample set comprising at least one training sample, each training sample comprising a sample query text, a sample search text, an initial relevance score and an initial relevance category; inputting the sample query text and the sample search text into a pre-constructed initial text ranking model, and adjusting parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model; obtaining a query text pair of a target object, each query text pair comprising an initial query text and a candidate search text; inputting the query text pair into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a text ranking result. The application embodiment improves the accuracy of text ranking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a text ranking method, a text ranking device, an electronic device and a storage medium. BACKGROUND

[0002] At present, in a search scenario, a user can search and purchase an item by inputting a query word, and search results are ranked according to the relevance of the search word and the item, that is, the results with high relevance are ranked in the front, and the results with low relevance are ranked in the back. The training effect of an existing text ranking model is poor, thereby affecting the accuracy of text ranking. Therefore, how to improve the accuracy of text ranking has become a technical problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a text ranking method, a text ranking device, an electronic device and a storage medium, which aims to improve the accuracy of text ranking.

[0004] To achieve the above purpose, a first aspect of the embodiments of the present application provides a text ranking method, which comprises:

[0005] A training sample set is constructed, the training sample set comprises at least one training sample, and each training sample comprises a sample query text, a sample search text, an initial relevance score and an initial relevance category;

[0006] The sample query text and the sample search text are input into a pre-constructed initial text ranking model, and the initial text ranking model is adjusted in parameters according to the initial relevance score and the initial relevance category, to obtain a target text ranking model;

[0007] At least one query text pair of a target object is obtained, and each query text pair comprises an initial query text and a candidate search text of the initial query text;

[0008] The query text pair is input into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text, to obtain a text ranking result.

[0009] In some embodiments, the initial text ranking model comprises an encoding sub-model, a ranking sub-model and a classification sub-model, and the inputting of the sample query text and the sample search text into the pre-constructed initial text ranking model and the adjustment of the initial text ranking model in parameters according to the initial relevance score and the initial relevance category to obtain the target text ranking model comprises:

[0010] inputting the sample query text and the sample search text into the initial text ranking model, performing encoding processing on the sample query text and the sample search text respectively by the encoding sub-model, to obtain a first text feature corresponding to the sample query text and a second text feature corresponding to the sample search text;

[0011] performing feature processing on the first text feature and the second text feature by the ranking sub-model to obtain a sample relevance score;

[0012] performing feature processing on the first text feature and the second text feature by the classification sub-model to determine a sample relevance category;

[0013] determining a first loss value according to the initial relevance score and the sample relevance score;

[0014] determining a second loss value according to the initial relevance category and the sample relevance category;

[0015] adjusting model parameters of the initial text ranking model according to the first loss value and the second loss value, and continuing to train the adjusted initial text ranking model based on the training sample set until the first loss value and the second loss value meet a preset training end condition, to obtain a target text ranking model.

[0016] In some embodiments, the sample relevance category includes a plurality of preset categories, and the performing feature processing on the first text feature and the second text feature by the classification sub-model to determine a sample relevance category includes:

[0017] performing feature processing on the first text feature and the second text feature by the classification sub-model to obtain a category probability set, the category probability set including probability values of the sample relevance category belonging to each preset category;

[0018] determining the sample relevance category according to the category probability set.

[0019] In some embodiments, the classification sub-model includes a first classification layer and a second classification layer, and the performing feature processing on the first text feature and the second text feature by the classification sub-model to obtain a category probability set includes:

[0020] performing feature processing on the first text feature and the second text feature by the first classification layer to obtain a first classification probability, the first classification probability being used to represent a probability value of the sample relevance category belonging to a first preset category;

[0021] The first text feature and the second text feature are processed by the second classification layer to obtain a second classification probability, the second classification probability being used to represent a probability value of the sample relevance category belonging to a second preset category.

[0022] A category probability set is obtained according to the first classification probability and the second classification probability.

[0023] In some embodiments, the second loss value is determined according to the initial relevance category and the sample relevance category, including:

[0024] An initial category weight is determined according to the initial relevance category;

[0025] A sample category weight is determined according to the sample relevance category;

[0026] The second loss value is determined according to the first classification probability, the second classification probability, the initial category weight and the sample category weight.

[0027] In some embodiments, the at least one query text pair of the target object is obtained, including:

[0028] An initial query text of the target object is obtained;

[0029] The initial query text is matched with a pre-obtained search text library to obtain at least one candidate search text;

[0030] The initial query text and each candidate search text are combined to obtain at least one query text pair.

[0031] In some embodiments, the query text pair is input into the target text ranking model to enable the target text ranking model to process the initial query text and the candidate search text to obtain a text ranking result, including:

[0032] The query text pair is input into the target text ranking model to enable the target text ranking model to process the initial query text and the candidate search text to obtain a target relevance score;

[0033] The query text pair is ranked according to the target relevance score to obtain a text ranking result.

[0034] To achieve the above object, a second aspect of the embodiments of the present application provides a text ranking device, the device comprising:

[0035] a sample construction module, configured to construct a training sample set, the training sample set comprising at least one training sample, each of the training samples comprising a sample query text, a sample search text, an initial relevance score, and an initial relevance category;

[0036] a model training module, configured to input the sample query text and the sample search text into a pre-constructed initial text ranking model, and adjust parameters of the initial text ranking model according to the initial relevance score and the initial relevance category, to obtain a target text ranking model;

[0037] a text pair acquisition module, configured to acquire at least one query text pair of a target object, each of the query text pairs comprising an initial query text and a candidate search text of the initial query text;

[0038] a ranking module, configured to input the query text pair into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text, to obtain a text ranking result.

[0039] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0040] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0041] The text ranking method, the text ranking device, the electronic device and the storage medium provided by the present application can construct a training sample set, the training sample set comprising at least one training sample, each of the training samples comprising a sample query text, a sample search text, an initial relevance score, and an initial relevance category. The sample query text and the sample search text are input into a pre-constructed initial text ranking model, and parameters of the initial text ranking model are adjusted according to the initial relevance score and the initial relevance category, to obtain a target text ranking model. Then, at least one query text pair of a target object is acquired, each of the query text pairs comprising an initial query text and a candidate search text of the initial query text. The query text pair is input into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text, to obtain a text ranking result. The embodiments of the present application can combine the relevance score and the relevance category to perform ranking, and improve the accuracy of text ranking. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1is a method flowchart of step S120 in

[0043] Figure 2 is Figure 1 is a method flowchart of step S230 in

[0044] Figure 3 is Figure 2 is a method flowchart of step S310 in

[0045] Figure 4 is Figure 3 is a method flowchart of step S250 in

[0046] Figure 5 is Figure 2 is a method flowchart of step S130 in

[0047] Figure 6 is Figure 1 is a method flowchart of step S140 in

[0048] Figure 7 is Figure 1 is a structure schematic diagram of a text sorting device provided by an embodiment of the present application.

[0049] Figure 8 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application.

[0050] Figure 9 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0052] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0054] First, the terms involved in the present application are analyzed:

[0055] Artificial Intelligence (AI): is a new technical science of studying, developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; Artificial Intelligence is a branch of computer science, and Artificial Intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems, etc. Artificial Intelligence can simulate the information process of human consciousness and thinking. Artificial Intelligence is also the theory, method, technology and application system of using digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0056] BM25 algorithm: is an algorithm used to evaluate the relevance between search words and documents, and is an algorithm based on a probabilistic retrieval model. Among them, BM is the abbreviation of Best Match, and 25 refers to the 25th iteration of the algorithm.

[0057] Term Frequency-Inverse Document Frequency (TF-IDF) algorithm: is a commonly used weighting technique for information retrieval and data mining, that is, to evaluate the importance of a word to a file set or a document in a corpus.

[0058] Deep Structured Semantic Model (DSSM) based on deep network: is to map query words and text into a common dimensional semantic space, and to maximize the cosine similarity between query and doc semantic vectors to train the implicit semantic model and achieve the purpose of retrieval.

[0059] BERT (Bidirectional Encoder Representations from Transformers) model: used to further increase the generalization ability of word vector model, fully describe the character level, word level, sentence level and even inter-sentence relationship features, which is based on the construction of Transformer. There are three kinds of Embedding in BERT, namely Token Embedding, Segment Embedding and Position Embedding; among them, Token Embedding is word vector, the first word is CLS mark, which can be used for subsequent classification tasks; Segment Embeddings is used to distinguish two sentences, because pre-training not only does LM but also classification task with two sentences as input; Position Embeddings, the position word vector here is not the trigonometric function in transfor, but the one learned by BERT after training. But BERT directly trains a PositionEmbedding to retain position information, initializes a vector at each position randomly, adds it to the model training, and finally gets an Embedding containing position information. Finally, the combination of Position Embedding and word Embedding, BERT chooses to directly splice them.

[0060] Token is a computer term, which means token (temporary) in computer identity authentication and token in lexical analysis, representing the object of the right to perform certain operations.

[0061] Cross-entropy loss function: is the most commonly used loss function in classification, cross-entropy is used to measure the difference between two probability distributions, to measure the difference between the distribution learned by the model and the real distribution.

[0062] At present, in a search scenario, a user can search and purchase an item by inputting a query word, and the sorting of search results greatly affects the user experience. Among them, the search results are sorted according to the relevance of the search word and the commodity, that is, the results with high relevance are placed in front, and the results with low relevance are placed in back. There are mainly two kinds of current mainstream sorting methods: (1) a keyword-based sorting method, which mainly includes BM25, TF-IDF and the like. This method mainly considers the literal relevance degree of each word and commodity, and thus ignores the potential semantic information of the search word; (2) a deep semantic matching method, such as DSSM, a matching matrix and the like. This method constructs semantic vector representations of the search word and the commodity title, and sorts the results by vector similarity calculation. This method considers the semantic information of the word and the commodity title, but does not fully utilize sample annotation information. In addition, the training effect of the existing text sorting model is not good, thereby affecting the accuracy of text sorting. Therefore, how to improve the accuracy of text sorting has become a technical problem to be solved.

[0063] Based on this, the embodiment of the application provides a text sorting method, a text sorting device, an electronic equipment and a storage medium, aiming to improve the accuracy of text sorting.

[0064] The text sorting method, the text sorting device, the electronic equipment and the storage medium provided by the embodiment of the application are specifically explained by the following embodiment. First, the text sorting method in the embodiment of the application is described.

[0065] The embodiment of the application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence basic technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and the like. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology and machine learning / deep learning and the like.

[0066] The text sorting method provided by the embodiments of the present application can be applied to a terminal, can be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc.; the server end can be configured as a separate physical server, can be configured as a server cluster or a distributed system formed by multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform; and the software can be an application that implements the text sorting method, but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0068] It should be noted that in each specific embodiment of the present application, when relevant processing needs to be performed on data related to the identity or characteristics of the user, such as user information, user behavior data, user history data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards in relevant countries and regions. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate permission or separate consent of the user will be obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for enabling the embodiments of the present application to normally operate will be obtained.

[0069] Please refer to Figure 1 , Figure 1 is an optional flowchart of the text sorting method provided by the embodiments of the present application, Figure 1The method in the method can include but is not limited to including steps S110 to S140:

[0070] Step S110, constructing a training sample set, the training sample set including at least one training sample, each training sample including sample query text, sample search text, initial relevance score, initial relevance category;

[0071] Step S120, inputting the sample query text and the sample search text into the pre-constructed initial text ranking model, and adjusting the parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model;

[0072] Step S130, obtaining at least one query text pair of the target object, each query text pair including an initial query text and a candidate search text of the initial query text;

[0073] Step S140, inputting the query text pair into the target text ranking model to enable the target text ranking model to process the initial query text and the candidate search text to obtain a text ranking result.

[0074] The steps S110 to S140 shown in the embodiments of the present application, due to the existing text ranking method for query text and search text, mainly utilize keyword matching or adopt a deep interaction relevance model based on a multi-similarity matrix for text ranking, and the sample annotation information of the query text and the search text is not fully utilized. The present application constructs a training sample set, the training sample set including at least one training sample, each training sample including sample query text, sample search text, initial relevance score, initial relevance category. Then, input the sample query text and the sample search text into the pre-constructed initial text ranking model, and adjust the parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model. Then, at least one query text pair of the target object is obtained, each query text pair including an initial query text and a candidate search text of the initial query text. Finally, the query text pair is input into the target text ranking model to enable the target text ranking model to process the initial query text and the candidate search text to obtain a text ranking result. The embodiments of the present application can improve the accuracy of text ranking.

[0075] It should be noted that the application scenarios of the embodiments of the present application can include a user terminal device and a server device, the user terminal device being configured to send at least one query text pair of a target object to the server device, and the server device being configured to execute the text ranking method provided by the embodiments of the present application. After obtaining at least one query text pair of the target object input by the user terminal device, the text ranking method of the present application is executed.

[0076] In step S110 of some embodiments, first, a training sample set is constructed, the training sample set including at least one training sample, each training sample including a sample query text, a sample search text, an initial relevance score, and an initial relevance category. The sample query text is used to represent a word text that needs to be queried. The sample search text is used to represent product description information obtained by searching the sample query text. The initial relevance score is used to represent a relevance score between the sample query text and the sample search text marked in advance, which can be any value between 0 and 1. The higher the relevance score, the stronger the relevance between the sample query text and the sample search text. The initial relevance category is used to represent the relevance category between the sample query text and the sample search text marked in advance. The higher the relevance category level between the sample query text and the sample search text, the higher the ranking of the sample search text in the search result.

[0077] In step S120 of some embodiments, the sample query text and the sample search text of the training sample set are input into the pre-constructed initial text ranking model as expected input, and the initial text ranking model is adjusted in parameters according to the initial relevance score and the initial relevance category marked in advance to obtain a target text ranking model. The initial text ranking model of the present application uses multiple tasks, and trains the model using the relevance score and the relevance category marked for the sample query text and the sample search text, which can better obtain the feature representation of the sample query text and the sample search text, so as to improve the accuracy of the relevance score of the query text and the search text, and further improve the accuracy of the text ranking.

[0078] It should be noted that the present application can input each pair of sample query text and sample search text into the initial text ranking model for model training, so that the target text ranking model obtained can accurately predict the relevance score of the input query text and search text. In addition, in order to improve the training efficiency of the model, the present application can also input multiple pairs of sample query text and sample search text, and simultaneously input the sample ranking distribution that has been ranked into the initial text ranking model for model training. The relevance score of each pair of sample query text and sample search text is calculated, and a predicted ranking distribution is obtained according to the obtained relevance score. The model parameters are further adjusted according to the sample ranking distribution and the predicted ranking distribution, and a target text ranking model with better ranking effect is obtained.

[0079] Please refer to Figure 2 , Figure 2 is an optional flowchart of step S120 provided by the embodiments of the present application. In some embodiments, the initial text ranking model includes an encoding sub-model, a ranking sub-model, and a classification sub-model, and step S120 can specifically include but is not limited to steps S210 to S260:

[0080] In step S210, the sample query text and the sample search text are input into the initial text ranking model, and the sample query text and the sample search text are respectively encoded by an encoding submodel to obtain first text features corresponding to the sample query text and second text features corresponding to the sample search text.

[0081] In step S220, the first text features and the second text features are processed by a ranking submodel to obtain a sample relevance score.

[0082] In step S230, the first text features and the second text features are processed by a classification submodel to determine a sample relevance category.

[0083] In step S240, a first loss value is determined according to the initial relevance score and the sample relevance score.

[0084] In step S250, a second loss value is determined according to the initial relevance category and the sample relevance category.

[0085] In step S260, the model parameters of the initial text ranking model are adjusted according to the first loss value and the second loss value, and the adjusted initial text ranking model is continuously trained based on the training sample set until the first loss value and the second loss value meet a preset training end condition, so as to obtain a target text ranking model.

[0086] In step S210 of some embodiments, in order to capture the semantic information of the text more deeply, the original application can construct an initial text ranking model based on a BERT deep learning model. The initial text ranking model includes an encoding submodel, a ranking submodel and a classification submodel. The sample query text and the sample search text are input into the initial text ranking model, and the sample query text and the sample search text are respectively encoded by the encoding submodel to obtain first text features corresponding to the sample query text and second text features corresponding to the sample search text. In the training process of the model, the same encoding submodel is used to encode the sample query text and the sample search text, so that the obtained feature embedding information is more accurate, and the accuracy of the relevance score calculation is improved.

[0087] It should be noted that through the encoding processing of the sample query text and the sample search text by the encoding sub-model, for example, the [cls] and the [sep] can be used for text segmentation of the sample query text w1 and the sample search text w2, to obtain a segmented form of the sample query text w1 as [cls][w11, w12, ···, w1n][sep], wherein [w11, w12, ···, w1n] represent token words in the sample query text respectively; and a segmented form of the sample search text w2 as [cls][w21, w22, ···, w2n][sep], wherein [w21, w22, ···, w2n] represent token words in the sample search text respectively. Then, the token words in the segmented form are encoded to obtain feature embedding information, which can be a 768-dimensional feature vector representation.

[0088] In steps S220 to S230 of some embodiments, the ranking sub-model is used to predict the relevance score of the sample query text and the sample search text, and the classification sub-model is used to predict the relevance category of the sample query text and the sample search text. The first text feature and the second text feature are processed by the ranking sub-model to obtain a sample relevance score, which is used to represent the predicted relevance score of the sample query text and the sample search text. The first text feature and the second text feature are processed by the classification sub-model to determine a sample relevance category, which is used to represent the predicted relevance score of the sample query text and the sample search text.

[0089] It should be noted that the ranking sub-model and the classification sub-model can both use a model based on a full connection layer, and the model parameters of the ranking sub-model and the classification sub-model are not the same. Since the ranking sub-model is used to predict the relevance score, the model parameters of the corresponding full connection layer can be set to 768*1, where 1 represents the predicted relevance score. The classification sub-model is used to predict the relevance category, and the model parameters of the corresponding full connection layer can be set to 768*2, where 2 represents the predicted two categories of relevance categories. When the model parameters of the corresponding full connection layer are set to 768*3, 3 represents the predicted three categories of relevance categories.

[0090] Please refer to Figure 3 , Figure 3 is an optional flowchart of step S230 provided by the embodiments of the present application. In some embodiments, the sample relevance category includes a plurality of preset categories, and step S230 can specifically include but is not limited to steps S310 to S320:

[0091] In step S310 and step S320 of some embodiments, in order to better obtain the feature representation of the sample query text and the sample search text, the sample relevance category is divided into a plurality of preset categories, the first text feature and the second text feature are processed by the classification sub-model to obtain a category probability set, the category probability set includes probability values of the sample relevance category being each preset category, that is, probability values of the first text feature and the second text feature belonging to each preset category are obtained. All probability values obtained in the category probability set are compared in value to determine the preset category with the maximum probability value, and the preset category is the predicted sample relevance category. The embodiments of the present application predict the sample relevance category of the sample query text and the sample search text, fully consider the category level information between texts, and improve the accuracy of text ranking.

[0092] In step S320, the sample relevance category is determined according to the category probability set.

[0093] In step S310 and step S320 of some embodiments, in order to better obtain the feature representation of the sample query text and the sample search text, the sample relevance category is divided into a plurality of preset categories, the first text feature and the second text feature are processed by the classification sub-model to obtain a category probability set, the category probability set includes probability values of the sample relevance category being each preset category, that is, probability values of the first text feature and the second text feature belonging to each preset category are obtained. All probability values obtained in the category probability set are compared in value to determine the preset category with the maximum probability value, and the preset category is the predicted sample relevance category. The embodiments of the present application predict the sample relevance category of the sample query text and the sample search text, fully consider the category level information between texts, and improve the accuracy of text ranking.

[0094] It should be noted that the number and specific category division of the preset category can be flexibly adjusted according to actual needs, which is not limited here. For example, when the sample relevance category includes two preset categories, which are the relevant category and the irrelevant category, the category label level of the relevant category can be set to 1, and the category label level of the irrelevant category can be set to 2; or when the sample relevance category includes three preset categories, which are the completely relevant category, the partially relevant category and the completely irrelevant category, the category label level of the completely relevant category can be set to 1, the category label level of the partially relevant category can be set to 2, and the category label level of the completely irrelevant category can be set to 3.

[0095] It should be noted that, for example, when a text search is needed on the input about a certain product, the sample query text can be a water cup, sports shoes, etc. For example, when a text search is performed on the sample query text of sports shoes, and the sample relevance category is divided into three preset categories, the results of the text search can be marked with an initial relevance score and an initial relevance category, that is, as shown in Table 1.

[0096] Sample query text Sample search terms Initial relevance score Initial relevance category Sneakers A sneakers 0.9 1 Sneakers Sneakers 0.5 2 Teacup Figure 4 0 3

[0097] Table 1

[0098] In Table 1, each row constitutes a training sample, and each training sample includes a sample query text, a sample search text, an initial relevance score of the sample query text and the sample search text, and an initial relevance category of the sample query text and the sample search text.

[0099] Referring to Figure 4 , Figure 5 is an optional flowchart of step S310 provided by the embodiments of the present application. In some embodiments, the classification sub-model includes a first classification layer and a second classification layer, and step S310 can specifically include but is not limited to steps S410 to S430:

[0100] In step S410, the first text feature and the second text feature are processed by the first classification layer to obtain a first classification probability, which is used to represent a probability value of the sample relevance category belonging to a first preset category;

[0101] In step S420, the first text feature and the second text feature are processed by the second classification layer to obtain a second classification probability, which is used to represent a probability value of the sample relevance category belonging to a second preset category;

[0102] In step S430, a category probability set is obtained according to the first classification probability and the second classification probability.

[0103] In steps S410 to S430 of some embodiments, the classification sub-model is used to predict the relevance category, and when the classification sub-model includes the first classification layer and the second classification layer, the model parameters of the full connection layer corresponding to the classification sub-model can be set to 768*2, where 2 represents the number of predicted relevance categories, and the parameters of the classification layer corresponding to each relevance category are 768*1. Assuming that the classification sub-model includes two preset categories, i.e., the sample relevance category can be divided into a relevant category and an irrelevant category, and the relevant category label level can be set to 1 and the irrelevant category label level can be set to 2, the first classification layer corresponds to the category label level 1, and the second classification layer corresponds to the category label level 2. The first text feature and the second text feature are processed by the first classification layer to obtain a first classification probability, which is used to represent a probability value of the sample relevance category belonging to the first preset category, i.e., the relevant category. The first text feature and the second text feature are processed by the second classification layer to obtain a second classification probability, which is used to represent a probability value of the sample relevance category belonging to the second preset category, i.e., the irrelevant category. Then, the first classification probability and the second classification probability are combined to obtain a category probability set.

[0104] It should be noted that when the sample correlation category includes three preset categories, namely, the full correlation category, the partial correlation category and the full non-correlation category, and the category label level of the full correlation category is set to 1, the category label level of the partial correlation category is set to 2, and the category label level of the full non-correlation category is set to 3. Then a third classification layer can be added in the classification sub-model, and the first text feature and the second text feature are processed according to each classification layer to obtain the probability value of the sample correlation category in each preset category.

[0105] It should be noted that the probability value of each preset category is calculated by using the classification layer, the gradient of each layer is calculated layer by layer, and then the gradient obtained in the reverse process is used to update the weight of each layer.

[0106] In steps S240 to S250 of some embodiments, in order to improve the accuracy of the model on the correlation score and the classification, a first loss value is determined according to the initial correlation score and the sample correlation score, and a second loss value is determined according to the initial correlation category and the sample correlation category. Specifically, the relative entropy loss function is used to calculate the ranking loss of the initial correlation score and the sample correlation score, as shown in formula (1), to obtain the first loss value, denoted as rank_loss.

[0107] rank_loss=∑ x∈X p1(x)log(p1(x) / q1(x)) (1)

[0108] Wherein, p1(x) represents the initial correlation score marked in advance; q1(x) represents the sample correlation score predicted by the model under each preset category; X represents a sample set of a training sample set; x represents each training sample in the training sample set.

[0109] Please refer to Figure 5 , Figure 6 is an optional flowchart of step S250 provided by the embodiments of the present application. In some embodiments, step S250 can specifically include but is not limited to steps S510 to S530:

[0110] Step S510, determining the initial category weight according to the initial correlation category;

[0111] Step S520, determining the sample category weight according to the sample correlation category;

[0112] Step S530, determining the second loss value according to the first classification probability, the second classification probability, the initial category weight and the sample category weight.

[0113] In steps S510 to S530 of some embodiments, the initial relevance score and the sample relevance score are classified loss calculated by using a cross-entropy loss function, as shown in formula (2), to obtain a second loss value, denoted as class_loss. An initial class weight is determined according to the initial relevance category, and a sample class weight is determined according to the sample relevance category. In order to make the text search results at the front of the ranking be the results with stronger relevance, the weight factor corresponding to the preset category with stronger relevance can be increased, so that when the sample classification of this category is wrong, the loss of the model is increased, so that the model classifies the sample category with stronger relevance as correctly as possible. Then, the second loss value is determined according to the first classification probability, the second classification probability, the initial class weight and the sample class weight.

[0114] class_loss = -∑ x∈X a * p2(x) log(q2(x)) (2)

[0115] Wherein, a represents a weight factor corresponding to each preset category; p2(x) represents a pre-labeled initial relevance category value; q2(x) represents a classification probability in each preset category predicted by the model.

[0116] In step S260 of some embodiments, the model parameters of the initial text ranking model are adjusted according to the first loss value and the second loss value. Assuming that the initial relevance category is completely relevant, if the probability of predicting in the completely relevant category is greater, the corresponding loss function value will be smaller, otherwise, the function value will be greater. The adjusted initial text ranking model is continuously trained based on the training sample set until the first loss value and the second loss value meet the preset training end condition, to obtain a target text ranking model. The embodiments of the present application can effectively improve the accuracy of text ranking by training the model by combining the loss values of the relevance score and the relevance category prediction.

[0117] It should be noted that the preset training end condition can be that the total model loss value of the first loss value and the second loss value is less than a preset loss value threshold.

[0118] In step S130 of some embodiments, at least one query text pair of a target object can be obtained in actual application, and each query text pair includes an initial query text and a candidate search text of the initial query text.

[0119] Please refer to Figure 6 , Figure 7 is an optional flowchart of step S130 provided by the embodiments of the present application. In some embodiments, step S130 can specifically include but is not limited to steps S610 to S630:

[0120] Step S610, obtaining an initial query text of a target object;

[0121] Step S620, performing text matching on the initial query text according to a pre-obtained search text library to obtain at least one candidate search text;

[0122] Step S630, combining the initial query text and each candidate search text to obtain at least one query text pair.

[0123] In steps S610 to S630 of some embodiments, the server obtains an initial query text of a target object input by the user end, and the target object is used to represent a commodity or product to be searched. The initial query text corresponding to the target object can be input into a text box for searching, that is, text matching is performed on the initial query text according to a pre-obtained search text library to obtain at least one candidate search text. For example, when the input initial query text is “water cup”, the obtained candidate search texts can be “plastic water cup”, “glass water cup” and “insulated water cup”, and the candidate search texts are used to represent the description information of the target object. Then, the same initial query text is combined with each candidate search text to obtain at least one query text pair.

[0124] It should be noted that the pre-obtained search text library of the present application can be a search text library stored locally, or a search text library obtained by searching and matching through an ES (Elastic Search) server or the like via the Internet.

[0125] In step S140 of some embodiments, each query text pair is input into the trained target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a text ranking result.

[0126] Please refer to Figure 7 , Figure 8 is an optional flowchart of step S140 provided by the embodiments of the present application. In some embodiments, step S140 can specifically include but is not limited to steps S710 to S720:

[0127] Step S710, inputting the query text pair into the target text ranking model so that the target text ranking model processes the initial query text and the candidate search text to obtain a target relevance score;

[0128] Step S720, ranking the query text pair according to the target relevance score to obtain a text ranking result.

[0129] Specifically, the query text pair is input into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a target relevance score corresponding to each query text. The multiple candidate search texts are sorted in descending order according to the obtained target relevance scores to obtain a text ranking result.

[0130] It should be noted that in actual application, the text ranking result can be displayed on the interface of the user terminal device, so as to facilitate the user to quickly obtain the required query result information.

[0131] Please refer to Figure 9 The embodiment of the present application also provides a text ranking device, which can implement the above text ranking method. The device comprises:

[0132] The sample construction module 810 is configured to construct a training sample set, the training sample set comprising at least one training sample, each training sample comprising a sample query text, a sample search text, an initial relevance score and an initial relevance category;

[0133] The model training module 820 is configured to input the sample query text and the sample search text into a pre-constructed initial text ranking model, and adjust parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model.

[0134] The text pair acquisition module 830 is configured to acquire at least one query text pair of a target object, each query text pair comprising an initial query text and a candidate search text of the initial query text.

[0135] The ranking module 840 is configured to input the query text pair into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a text ranking result.

[0136] The specific implementation of the text ranking device is basically the same as the specific embodiment of the above text ranking method, and will not be repeated here.

[0137] The embodiment of the present application also provides an electronic device, which comprises a memory and a processor, the memory storing a computer program, and the processor implementing the above text ranking method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer and the like.

[0138] Please refer to Figure 9 , ​ The hardware structure of the electronic device of another embodiment is illustrated, which comprises:

[0139] The processor 910 can be implemented by a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0140] The memory 920 can be implemented by a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 920 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 920 and are called and executed by the processor 910 to implement the text sorting method of the embodiments of the present application.

[0141] The input / output interface 930 is configured to implement information input and output.

[0142] The communication interface 940 is configured to implement the communication interaction between the device and other devices. The communication can be implemented by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0143] The bus 950 is configured to transmit information between the components (for example, the processor 910, the memory 920, the input / output interface 930, and the communication interface 940) of the device.

[0144] The processor 910, the memory 920, the input / output interface 930, and the communication interface 940 are connected to each other by the bus 950 to realize the communication connection between the components in the device.

[0145] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by the processor to implement the above-mentioned text sorting method.

[0146] Memory, as used in the foregoing description, is used to store non-transitory software programs and non-transitory computer-executable programs. In addition, memory can include high-speed random access memory and can also include non- transitory memory such as at least one disk memory device, a flash memory device, or other non- transitory solid-state memory device. In some embodiments, memory can optionally include memory that is remotely located from the processor, such as memory that can be connected to the processor through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communications network, and combinations thereof.

[0147] The text sorting method, the text sorting device, the electronic equipment and the storage medium provided by the embodiment of the application are characterized in that a training sample set is constructed, the training sample set includes at least one training sample, each training sample includes a sample query text, a sample search text, an initial relevance score and an initial relevance category. The sample query text and the sample search text are input into a pre-constructed initial text sorting model, the sample query text and the sample search text are respectively processed by an encoding sub-model to obtain a first text feature corresponding to the sample query text and a second text feature corresponding to the sample search text. The first text feature and the second text feature are processed by a sorting sub-model to obtain a sample relevance score. The first text feature and the second text feature are processed by a classification sub-model to obtain a category probability set, the category probability set includes probability values of the sample relevance category being each preset category, and the sample relevance category is determined according to the category probability set. A first loss value is determined according to the initial relevance score and the sample relevance score; an initial category weight is determined according to the initial relevance category, a sample category weight is determined according to the sample relevance category, and a second loss value is determined according to the first classification probability, the second classification probability, the initial category weight and the sample category weight. Then, the model parameters of the initial text sorting model are adjusted according to the first loss value and the second loss value, the adjusted initial text sorting model is continuously trained based on the training sample set until the first loss value and the second loss value meet a preset training end condition, so as to obtain a target text sorting model. An initial query text of a target object is obtained, the initial query text is matched with a pre-acquired search text library to obtain at least one candidate search text. The initial query text and each candidate search text are combined to obtain at least one query text pair. Finally, the query text pair is input into the target text sorting model, so that the target text sorting model processes the initial query text and the candidate search text to obtain a target relevance score, the query text pair is sorted according to the target relevance score, and a text sorting result is obtained. The embodiment of the application performs product search sorting and relevance classification based on the BERT multi-task model, and the two sub-tasks share the underlying BERT encoder, that is, the same encoding sub-model is used to encode the sample query text and the sample search text, so that the obtained feature embedding information is more accurate, and the accuracy of the relevance score calculation is improved. Compared with the existing search sorting model, the embodiment of the application effectively utilizes the label information of the relevance category when constructing the deep learning sorting model, so that the relevance semantic representation information of the query text and the search text is more sufficient and accurate, thereby effectively improving the matching accuracy of the query text and the search text, and effectively improving the accuracy of the text sorting.

[0148] The embodiments described in the specification are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0149] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than shown in the figures, or combine certain steps, or different steps.

[0150] The device embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0151] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functional modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0152] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0153] It should be understood that, in the application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B, and A and B existing at the same time, wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, and c can be single or multiple.

[0154] In several embodiments provided in the application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0155] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0156] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0157] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0158] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method of ranking text, characterized by, The method comprises: constructing a training sample set comprising at least one training sample, each of the training samples comprising sample query text, sample search text, an initial relevance score, and an initial relevance category; inputting the sample query text and the sample search text into a pre-constructed initial text ranking model, and adjusting parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model; the initial text ranking model comprises an encoding sub-model, a ranking sub-model, and a classification sub-model; the inputting the sample query text and the sample search text into the pre-constructed initial text ranking model and adjusting parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model comprises: inputting the sample query text and the sample search text into the initial text ranking model, performing encoding processing on the sample query text and the sample search text respectively by the encoding sub-model to obtain first text features corresponding to the sample query text and second text features corresponding to the sample search text; performing feature processing on the first text features and the second text features by the ranking sub-model to obtain a sample relevance score; performing feature processing on the first text features and the second text features by the classification sub-model to determine a sample relevance category; determining a first loss value according to the initial relevance score and the sample relevance score; determining a second loss value according to the initial relevance category and the sample relevance category; adjusting model parameters of the initial text ranking model according to the first loss value and the second loss value, and continuing to train the adjusted initial text ranking model based on the training sample set until the first loss value and the second loss value meet a preset training end condition to obtain a target text ranking model; obtaining at least one query text pair of a target object, each of the query text pairs comprising an initial query text and a candidate search text of the initial query text; inputting the query text pairs into the target text ranking model to enable the target text ranking model to process the initial query text and the candidate search text to obtain a text ranking result.

2. The method of claim 1, wherein, The sample relevance category comprises a plurality of preset categories, and the determining a sample relevance category by the classification sub-model performing feature processing on the first text features and the second text features comprises: performing feature processing on the first text features and the second text features by the classification sub-model to obtain a category probability set, the category probability set comprising probability values of the sample relevance category being each of the preset categories; determining the sample relevance category according to the category probability set.

3. The method of claim 2, wherein, The classification sub-model comprises a first classification layer and a second classification layer, and the performing feature processing on the first text features and the second text features by the classification sub-model to obtain a category probability set comprises: The first text feature and the second text feature are processed by the first classification layer to obtain a first classification probability, the first classification probability being used to represent a probability value of the sample relevance category belonging to a first preset category; The first text feature and the second text feature are processed by the second classification layer to obtain a second classification probability, the second classification probability being used to represent a probability value of the sample relevance category belonging to a second preset category; A category probability set is obtained according to the first classification probability and the second classification probability.

4. The method of claim 3, wherein, The second loss value is determined according to the initial relevance category and the sample relevance category, including: An initial category weight is determined according to the initial relevance category; A sample category weight is determined according to the sample relevance category; A second loss value is determined according to the first classification probability, the second classification probability, the initial category weight and the sample category weight.

5. The method according to any one of claims 1 to 4, characterized in that, The at least one query text pair of the target object is obtained, including: An initial query text of the target object is obtained; Text matching is performed on the initial query text according to a pre-obtained search text library to obtain at least one candidate search text; The initial query text and each candidate search text are combined to obtain at least one query text pair.

6. The method according to any one of claims 1 to 4, characterized in that, The query text pair is input into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a text ranking result, including: The query text pair is input into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a target relevance score; The query text pair is sorted according to the target relevance score to obtain a text ranking result.

7. A text ranking apparatus characterized by comprising: The device includes: A sample construction module is configured to construct a training sample set, the training sample set including at least one training sample, each training sample including a sample query text, a sample search text, an initial relevance score and an initial relevance category. The model training module is configured to input the sample query text and the sample search text into a pre-constructed initial text ranking model, and adjust parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain a target text ranking model; the initial text ranking model includes an encoding sub-model, a ranking sub-model and a classification sub-model; the inputting the sample query text and the sample search text into the pre-constructed initial text ranking model and adjusting the parameters of the initial text ranking model according to the initial relevance score and the initial relevance category to obtain the target text ranking model includes: inputting the sample query text and the sample search text into the initial text ranking model, performing encoding processing on the sample query text and the sample search text respectively through the encoding sub-model to obtain a first text feature corresponding to the sample query text and a second text feature corresponding to the sample search text; performing feature processing on the first text feature and the second text feature through the ranking sub-model to obtain a sample relevance score; performing feature processing on the first text feature and the second text feature through the classification sub-model to determine a sample relevance category; determining a first loss value according to the initial relevance score and the sample relevance score; determining a second loss value according to the initial relevance category and the sample relevance category; adjusting model parameters of the initial text ranking model according to the first loss value and the second loss value, and continuing to train the adjusted initial text ranking model based on the training sample set until the first loss value and the second loss value meet a preset training end condition, so as to obtain the target text ranking model; The text pair obtaining module is configured to obtain at least one query text pair of a target object, each query text pair including an initial query text and a candidate search text of the initial query text; The ranking module is configured to input the query text pair into the target text ranking model, so that the target text ranking model processes the initial query text and the candidate search text to obtain a text ranking result.

8. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the method in any one of claims 1 to 6.

Images