Reordering method, related device, equipment and storage medium

Abstract

The invention discloses a reordering method, a related device, equipment and a storage medium, and the reordering method comprises the steps: obtaining a recommendation object set of a to-be-responded query, and determining a subset object number based on a target recommendation total number of the to-be-responded query; dividing a recommendation object set based on the subset object number to obtain a plurality of object subsets; reordering the object subsets based on the demand description of the to-be-responded query to obtain rearranged subsets of the object subsets; respectively selecting recommendation objects which are not more than the target recommendation total number from each rearranged subset in sequence, combining the recommendation objects as a new recommendation object set, and returning to the step of dividing the recommendation object set based on the subset object number to obtain a plurality of object subsets; and reordering the latest recommendation object set based on the demand description to at least obtain a target recommendation total number of sequentially arranged recommendation objects. According to the scheme, reordering can be realized in the face of mass data.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a reordering method and related devices, equipment and storage media. Background Art

[0002] In the fields of massive data retrieval, recommendation systems, and document question-answering systems, users usually find the most relevant content from a large number of candidate results. Traditional ranking algorithms generate an initial ranking result based on a specific scoring or relevance model for user reference.

[0003] However, due to the limitations of the initial sorting model or the complexity of a specific task, the initial sorting results may not meet user needs. At this time, re-ranking technology that deeply optimizes the initial sorting results is particularly important. Limited by the length of the text, the existing re-ranking technology still cannot cope well with massive data. In view of this, how to achieve re-ranking when facing massive data has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a reordering method and related devices, equipment and storage medium, which can realize reordering when facing massive data.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a reordering method, including: obtaining a set of recommended objects for a query to be responded to, and determining the number of subset objects based on the target total number of recommendations for the query to be responded to; wherein the number of subset objects is greater than the target total number of recommendations, and the recommended object set contains a number of recommended objects; dividing the recommended object set based on the number of subset objects to obtain a number of object subsets; reordering the object subsets based on the requirement description of the query to be responded to obtain a reordered subset of the object subsets; selecting recommended objects no more than the target total number of recommendations from each reordered subset in sequence, combining them as a new set of recommended objects, and returning to the step of dividing the recommended object set based on the number of subset objects to obtain a number of object subsets, until the total number of objects in the latest set of recommended objects is no greater than the number of subset objects; reordering the latest set of recommended objects based on the requirement description to obtain at least the target total number of recommended objects arranged in sequence.

[0006] In order to solve the above technical problems, the second aspect of the present application provides a reordering device, including: an acquisition and determination module, a set division module, a subset rearrangement module, a combination loop module and a final rearrangement module, the acquisition and determination module is used to obtain a set of recommended objects for a query to be responded to, and determine the number of subset objects based on the target total number of recommendations for the query to be responded to; wherein the number of subset objects is greater than the target total number of recommendations, and the recommended object set contains a number of recommended objects; the set division module is used to divide the recommended object set based on the number of subset objects to obtain a number of object subsets; the subset rearrangement module is used to reorder the object subsets based on the requirement description of the query to be responded to obtain a rearranged subset of the object subsets; the combination loop module is used to select recommended objects not more than the target total number of recommendations from each rearranged subset in sequence, combine them as a new recommended object set, and return to the step of dividing the recommended object set based on the number of subset objects to obtain a number of object subsets, until the total number of set objects of the latest recommended object set is not greater than the number of subset objects; the final rearrangement module is used to reorder the latest recommended object set based on the requirement description to obtain at least the target total number of recommended objects arranged in sequence.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which at least includes a memory and a processor coupled to each other, the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the reordering method in the above first aspect.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the reordering method of the first aspect.

[0009] The above scheme obtains the recommended object set of the query to be responded to, and determines the number of subset objects based on the target total number of recommendations for the query to be responded to, and the number of subset objects is greater than the target total number of recommendations. The recommended object set contains several recommended objects, and then divides the recommended object set based on the number of subset objects to obtain several object subsets, and reorders the object subsets based on the requirement description of the query to be responded to obtain a reordered subset of the object subset, thereby selecting recommended objects that are no more than the target total number of recommendations from each reordered subset in sequence, combining them as a new recommended object set, and returning the recommended object set divided based on the number of subset objects. The method comprises the following steps: first, recommending a set of objects, obtaining a plurality of object subsets, until the total number of objects in the latest recommended object set is no greater than the number of objects in the subset, and then reordering the latest recommended object set based on the requirement description to obtain at least the target total number of recommended objects arranged in order. Therefore, by dividing the subsets, each reordering is constrained within the data volume range limited by the number of objects in the subset, which can achieve data volume degradation in disguise, and combining the process of splitting, reordering, and merging in multiple cycles to achieve downgraded multi-way merge sorting, which helps to break through the limitations of existing reordering technologies caused by massive data. Therefore, reordering can be achieved when facing massive data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an embodiment of the reordering method of the present application; Figure 2a It is a process diagram of an embodiment of the reordering method of the present application; Figure 2b is a process diagram of another embodiment of the reordering method of the present application; Figure 3 It is a schematic diagram of the framework of an embodiment of the reordering device of the present application; Figure 4 It is a schematic diagram of the framework of an embodiment of the electronic device of the present application; Figure 5 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0011] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0012] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0013] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the fragment " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0014] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the reordering method of the present application. Specifically, it may include the following steps: Step S11: Obtain a set of recommended objects for the query to be responded to, and determine the number of subset objects based on the total number of target recommendations for the query to be responded to.

[0015] In the embodiment of the present disclosure, the number of subset objects is greater than the total number of target recommendations, and the recommended object set includes several recommended objects. For ease of description, the number of subset objects can be recorded as n, and the total number of target recommendations can be recorded as k. That is, in the embodiment of the present disclosure, the number of subset objects n>the total number of target recommendations k is satisfied.

[0016] In one implementation scenario, the query to be responded to may be triggered in a search engine scenario, and based on the requirement description of the query to be responded to, a set of recommended objects for the query to be responded to may be searched and obtained. Specifically, the requirement description of the query to be responded to may at least include key words and phrases input into the search engine. Accordingly, the recommended objects may at least include recommended web pages.

[0017] In an implementation scenario, the query to be responded to may be triggered in the recommendation system scenario, and based on the demand description of the query to be responded to, a set of recommended objects for the query to be responded to may be recommended. Specifically, the demand description of the query to be responded to may at least include the behavior information input into the recommendation system. Accordingly, the recommended objects may at least include recommended things. Exemplarily, when the behavior information includes shopping behavior, the recommended things may include recommended commodities; or, when the behavior information includes question-solving behavior, the recommended things may include recommended exercises. Of course, the above examples are only several possible applications of the recommendation system in actual applications, and other possible applications will not be given one by one here.

[0018] In one implementation scenario, the query to be responded to may be triggered in the scenario of the question-and-answer system, and based on the demand description of the query to be responded to, a set of recommended objects for the query to be responded to may be initially replied. Specifically, the demand description of the query to be responded to may at least include the question input into the question-and-answer system. Accordingly, the recommended objects may at least include recommended answers. For example, in the case where the question represents a request for travel suggestions, the recommended answers may include recommended travel plans; or, in the case where the question represents a request for computer assembly suggestions, the recommended answers may include computer assembly configuration plans. Of course, the above examples are only a few possible applications of the question-and-answer system in actual applications, and other possible applications will not be given one by one here.

[0019] It should be noted that the above examples are only several possible application scenarios of re-ranking, and do not limit the application of re-ranking in other possible scenarios. Other possible application scenarios will not be given examples one by one here. In addition, the method of obtaining the recommended object set can refer to the relevant description of obtaining the initial ranking results in possible application scenarios such as search engines, recommendation systems, and question-answering systems, and will not be repeated here.

[0020] In an implementation scenario, after obtaining the recommended object set and the target total number of recommendations, a positive integer slightly larger than the target total number of recommendations can be randomly selected as the number of subset objects. For example, if the total number of objects in the recommended object set is 100 and the target total number of recommendations is 3, the number of subset objects can be randomly set to a positive integer slightly larger than 3, such as 4, 5, 6, 7, 8, 9, 10, etc., and the specific value of the total number of subset objects is not limited here.

[0021] In another implementation scenario, after obtaining the recommended object set and the target total number of recommendations, the product of the target total number of recommendations and the preset multiple can be temporarily set as the number of subset objects. It should be noted that the preset multiple can be set appropriately smaller to prevent the target total number of recommendations from exceeding the total number of collection objects of the recommended object set after multiplying the preset multiple. Exemplarily, the preset multiple can be set to 2, 3, etc. After that, it is detected whether the temporary number of subset objects is less than the total number of collection objects of the recommended object set. If it is less, the aforementioned temporary total number of subset objects can be finally determined. If it is greater, the temporary number of subset objects can be further reduced. Exemplarily, still taking the total number of collection objects of the recommended object set as 100 and the target total number of recommendations as 3 as an example, the preset multiple can be set to 2, then the number of subset objects can be temporarily set to 6, and since the temporary number of subset objects 6 is much smaller than the total number of collection objects 100, the temporary number of subset objects 6 can be finally determined, that is, the final number of subset objects can be determined to be 6.

[0022] In another implementation scenario, after obtaining the recommended object set and the target total number of recommended objects, the target multiple can be determined based on the quotient of the total number of objects in the recommended object set and the target total number of recommended objects, and the target multiple is a positive integer. For example, still taking the total number of objects in the recommended object set as 100 and the target total number of recommended objects as 3 as an example, since the quotient of the total number of objects in the set and the target total number of recommended objects is 33, a value that is relatively close to the front (such as a preset ratio such as the top 5%, the top 10%) can be selected as the target multiple within the positive integer range of 1 to 33 as much as possible, such as 2, 3, 4, etc. can be selected as the target multiple. On this basis, the product of the target total number of recommended objects and the target multiple can be obtained as the number of subset objects. For example, in the above example, if 2 is selected as the target multiple, the product of the target total number of recommended objects 3 and the target multiple 2, that is, 6, can be obtained as the number of subset objects; or, if 3 is selected as the target multiple, the target total number of recommended objects 3 and the target multiple, that is, 9, can be obtained as the number of subset objects. Other situations can be deduced by analogy, and examples are not given one by one here. The above method obtains the quotient of the total number of set objects of the recommended object set and the target total number of recommendations, determines the target multiple, and then obtains the product of the target total number of recommendations and the target multiple as the number of subset objects, which can ensure that the object subsets subsequently divided according to the number of subset objects are as reasonable as possible.

[0023] Step S12: Divide the recommended object set based on the number of subset objects to obtain several object subsets.

[0024] Specifically, it is possible to detect whether the total number of set objects in the recommended object set is divisible by the number of subset objects. In response to the detection result including that it is divisible, the recommended object set can be directly divided into two halves based on the number of subset objects to obtain several object subsets. In response to the detection result including that it is not divisible, the recommended object set can be first divided into two halves based on the subset objects to obtain several object subsets, and then each recommended object in the recommended object set that has not been divided can be evenly distributed to each object subset. As a possible example, take the total number of set objects of the recommended object set as 100 and the number of subset objects as 10. Since the total number of set objects 100 can divide the number of subset objects 10, the recommended object set can be directly divided based on the number of subset objects 10 to obtain 10 object subsets, and each object subset contains 10 recommended objects; or, as another possible example, take the total number of set objects of the recommended object set as 100 and the number of subset objects as 9. Since the total number of set objects 100 cannot divide the number of subset objects 9, the recommended object set can be first divided based on the number of subset objects 9 to obtain 11 object subsets, and each object subset contains 9 recommended objects, and then the 1 recommended object that has not been divided is allocated to any (e.g., the first) object subset, that is, the first object subset contains 10 recommended objects, and the second to eleventh object subsets each contain 9 recommended objects. Without loss of generality, taking the recommended object set D containing m recommendations as an example, the subset division of the recommended object set D can be expressed as: …… (1) It should be noted that in the above formula (1), n ​​represents the number of subset objects. When m divides n, the object subset to contains n recommended objects; when m cannot divide n, the r recommended objects that are not divided after the first division can be spread over the first r object subsets, that is, the object subsets to contains n+1 recommended objects, and the object subset to In addition, in order to reduce the processing load as much as possible, before splitting the recommended object set for the first time, the recommendations in the recommended object set can be deduplicated.

[0025] Step S13: reordering the object subset based on the requirement description of the query to be responded to, to obtain a reordered subset of the object subset.

[0026] In an implementation scenario, a prompt instruction can be constructed based on the requirement description of the query to be responded to and the input object set, and the prompt instruction is used to instruct the large language model to sort the input object set, that is, the object subset, according to the relevance between them and the requirement description. On this basis, the prompt instruction can be input to the large language model to obtain the reordering result output by the large language model, that is, the reordered subset of the object subset. Exemplarily, the prompt template can be structured to construct the prompt instruction, such as: Given a query statement [fill in the requirement description of the query to be responded to here], and the following n recommended objects: [1]: [Fill in the specific content of recommended object 1] [1]: [Fill in the specific content of recommended object 2] …… [n]: [Fill in the specific content of the recommended object n] Please select the k recommended objects that are most relevant to the query statement from the above n recommended objects, according to the semantics. Similarity or answerability, generates a ranked list of the k most relevant objects in order.

[0027] It should be noted that the above prompt instruction is only a possible example in the actual application process, and the specific content of the prompt instruction is not limited here. As a possible example, the above prompt instruction is input into the large language model to obtain the reordering result: 1->3->8->2… It should be noted that the above re-ranking result indicates that the top k recommended objects are: recommended object 1, recommended object 3, recommended object 8, recommended object 2, etc. Of course, the above re-ranking result is only a possible example in the actual application process, and the specific content of the re-ranking result is not limited here. In addition, the large language model can be an open source large model such as LLAMA, Bloom, etc.; or, the large language model can also be obtained by fine-tuning the parameters of the open source large model based on a specific training set; or, the large language model can also be a custom large model, and the specific source of the large language model is not limited here.

[0028] In another implementation scenario, as described above, a prompt instruction can be constructed based on the requirement description of the query to be responded to and the input object set, and the prompt instruction is used to instruct the large language model to sort the recommended objects in the input object set according to their relevance to the requirement description. Figure 2a , Figure 2a 1 is a schematic diagram of the process of an embodiment of the reordering method of the present application. Figure 2a As shown, different from the above-mentioned implementation, a decoding word list can also be constructed based on at least the sorting label. It should be noted that the sorting label can at least include a digital number, such as 0 to 9. On this basis, a prompt instruction can be input to the large language model (such as can be referred to Figure 2a In the process, the prompt instruction is first input embedded to obtain the large language model based on the decoding vocabulary (i.e. Figure 2a It should be noted that the processing process of the large language model can refer to the aforementioned related description and will not be repeated here. In addition, the reasoning task of the large language model is usually implemented through a decoding method. The general idea is to select the token with the highest probability as the next decoding character according to a pre-prepared decoding vocabulary based on decoding algorithms such as greedy decoding and beam search decoding. In this implementation scenario, compared with the decoding vocabulary used in conventional decoding, which uses the character set of all texts of the full language, the vocabulary size is significantly reduced (the vocabulary size used for conventional decoding is as high as , after being constrained by this implementation scenario, only ), so on the one hand it can greatly reduce the decoding time of the large language model and improve the decoding performance of the large language model; on the other hand, because the decoding vocabulary is greatly compressed, the content of the massive background knowledge of the large language model can no longer be generated, so that the decoded characters are constrained within the given decoding vocabulary, which can greatly alleviate the hallucination problem and enhance the stability of the content generated by the large language model.

[0029] In a specific implementation scenario, the decoding vocabulary may also include a fallback response word for reordering, including but not limited to: yes, no, unknown, etc., so as to provide a fallback response when the large language model cannot perform reordering. For example, taking the application scenario of a search engine as an example, when each recommended object (i.e., recommended web page) in an object subset is empty (e.g., 404), the large language model may provide a fallback response of "unknown" when reordering the object subset. Other situations can be deduced by analogy, and examples are not given one by one here. In addition, in the case where the large language model gives a fallback response, as a possible example, the reordered subset can be regarded as the original object subset, that is, the order does not change after the reordering by the large language model this time; or, as another possible example, the object subset can also be deleted directly, that is, any recommended object in the object subset no longer participates in the subsequent reordering process. Of course, the above examples are only a few possible implementation methods in the case where the large language model gives a fallback response, and other possible implementation methods are not given examples one by one here.

[0030] In a specific implementation scenario, the decoding vocabulary may also include special symbols for indicating the order, including but not limited to: ']', '>', '-', etc. For example, the '->' in the above reordering result is composed of the special symbol '>' and the special symbol '-' in the decoding vocabulary.

[0031] It should be noted that, for ease of description, the decoding vocabulary can be expressed as .in, This means the aforementioned sorting label, This means the aforementioned reply words, This means the special symbols mentioned above.

[0032] Step S14: Select recommendation objects that are no more than the target total number of recommendations from each rearranged subset in sequence, combine them as a new recommendation object set, and return to the step of dividing the recommendation object set based on the number of subset objects to obtain several object subsets, until the total number of objects in the latest recommendation object set is no more than the number of subset objects.

[0033] Specifically, as a possible example, a target total number of recommended objects can be selected in sequence from the rearranged subsets, and then a new set of recommended objects can be combined based on the recommended objects selected from each rearranged subset. For example, a target total number of k recommended objects can be selected in sequence from m / n rearranged subsets and combined to obtain a new set of recommended objects containing k*m / n recommended objects. On this basis, it is possible to return to the aforementioned step S12 to iteratively perform splitting, reordering, merging and other process steps until the total number of objects in the latest recommended object set is no greater than the number of subset objects. It should be noted that in this embodiment, the iteration time complexity is ln(m)-ln(n), and its main time and performance overhead lies in the generation rate of the large language model, and the number of calls to the large language model is: ,in, Depending on the data size, up to Stop. After each split, the reordering of the object subsets can be performed in parallel. In addition, after the combination, they can be randomly shuffled and then used as a new recommended object set.

[0034] For ease of description, please refer to Figure 2b , Figure 2b FIG. 1 is a process diagram of another embodiment of the reordering method of the present application. Figure 2bAs shown in the figure, taking the total number of set objects m of the recommended object set as 26 and the total number of target recommended objects k as 3 as an example, the number of subset objects can be set to 6. For the convenience of description, the 26 recommended objects can be assigned numbers from 1 to 26 respectively. In the first split, the recommended objects 1 to 26 can be divided into 4 object subsets, among which the first and second object subsets contain 7 recommended objects, and the third and fourth object subsets each contain 6 recommended objects. On this basis, the 4 object subsets can be reordered to obtain their respective reordered subsets, and then the first k (3 in this example) recommended objects are selected from each reordered subset to obtain a new recommended object set. At this time, since the new recommended object set contains 12 recommended objects, which is still greater than the number of subset objects 6, it is necessary to continue splitting. In the second split, the first and second object subsets each contain 6 recommended objects. On this basis, we can reorder the two object subsets to obtain their own reordered subsets, and then select the first k (3 in this example) recommended objects in each reordered subset to combine them to obtain a new recommended object set. At this time, since the new recommended object set contains 6 recommended objects, which is not more than the number of subset objects 6, we can stop splitting. Of course, Figure 2b The example shown is only one possible example of reordering in actual application, and other possible situations will not be given one by one here.

[0035] Step S15: reorder the latest recommended object set based on the requirement description to obtain at least a target total number of recommended objects arranged in order.

[0036] For details, please refer to the above description of re-ranking using a large language model, which will not be repeated here. It should be noted that in the last re-ranking, the input object set of the large language model is the latest recommended object set. Figure 2b , the latest recommended object set is composed of: recommended object "4", recommended object "13", recommended object "2", recommended object "19", recommended object "21", recommended object "16". After re-sorting, the final result can be obtained: recommended object "4", recommended object "19", recommended object "13", recommended object "21", recommended object "2", recommended object "16". On this basis, at least the first k positions (in this case, the first 3 positions) can be selected and returned (i.e., recommended object "4", recommended object "19", recommended object "13") to respond to the query to be responded. Of course, the above example is only a possible example in the actual application process, and other possible situations will not be given one by one here.

[0037] The above scheme obtains the recommended object set of the query to be responded to, and determines the number of subset objects based on the target total number of recommendations for the query to be responded to, and the number of subset objects is greater than the target total number of recommendations. The recommended object set contains several recommended objects, and then divides the recommended object set based on the number of subset objects to obtain several object subsets, and reorders the object subsets based on the requirement description of the query to be responded to obtain a reordered subset of the object subset, thereby selecting recommended objects that are no more than the target total number of recommendations from each reordered subset in sequence, combining them as a new recommended object set, and returning the recommended object set divided based on the number of subset objects. The method comprises the following steps: first, recommending a set of objects, obtaining a plurality of object subsets, until the total number of objects in the latest recommended object set is no greater than the number of objects in the subset, and then reordering the latest recommended object set based on the requirement description to obtain at least the target total number of recommended objects arranged in order. Therefore, by dividing the subsets, each reordering is constrained within the data volume range limited by the number of objects in the subset, which can achieve data volume degradation in disguise, and combining the process of splitting, reordering, and merging in multiple cycles to achieve downgraded multi-way merge sorting, which helps to break through the limitations of existing reordering technologies caused by massive data. Therefore, reordering can be achieved when facing massive data.

[0038] See also Figure 3 , Figure 3 1 is a schematic diagram of a framework of an embodiment of the reordering device of the present application. The reordering device 30 includes: an acquisition determination module 31, a set division module 32, a subset rearrangement module 33, a combination loop module 34 and a final rearrangement module 35. The acquisition determination module 31 is used to obtain a set of recommended objects for the query to be responded to, and determine the number of subset objects based on the total number of target recommendations for the query to be responded to; wherein the number of subset objects is greater than the total number of target recommendations, and the recommended object set contains a number of recommended objects; the set division module 32 is used to divide the recommended object set based on the number of subset objects to obtain a number of object subsets; the subset rearrangement module 33 is used to determine the number of recommended objects based on the number of target recommendations for the query to be responded to. The description is required to reorder the object subsets to obtain a reordered subset of the object subsets; a combination loop module 34 is used to select recommended objects no more than the target total number of recommendations from each reordered subset in sequence, combine them as a new recommended object set, and return to the step of dividing the recommended object set based on the number of subset objects to obtain several object subsets, until the total number of objects in the latest recommended object set is no more than the number of subset objects; a final reordering module 35 is used to reorder the latest recommended object set based on the demand description to obtain at least the target total number of recommended objects arranged in sequence.

[0039] In the above scheme, the reordering device 30 obtains the recommended object set of the query to be responded to, and determines the number of subset objects based on the target total number of recommendations for the query to be responded to, and the number of subset objects is greater than the target total number of recommendations, and the recommended object set contains a number of recommended objects, and then divides the recommended object set based on the number of subset objects to obtain a number of object subsets, and reorders the object subsets based on the demand description of the query to be responded to obtain a reordered subset of the object subset, thereby selecting recommended objects that are no more than the target total number of recommendations from each reordered subset in sequence, combining them as a new recommended object set, and returning the recommended object set based on the subset objects. The number of recommended object sets is divided into several steps to obtain several object subsets, until the total number of objects in the latest recommended object set is not greater than the number of subset objects, and then the latest recommended object set is reordered based on the demand description to obtain at least the target total number of recommended objects arranged in order. Therefore, through subset division, each reordering is constrained within the data volume range limited by the number of subset objects, which can achieve data volume degradation in disguise, and combined with multiple rounds of cycles to execute the splitting, reordering, and merging process, it can achieve downgraded multi-way merge sorting, which helps to break through the limitations of existing reordering technologies caused by massive data. Therefore, reordering can be achieved when facing massive data.

[0040] In some disclosed embodiments, the acquisition and determination module 31 includes a multiple determination submodule, which is used to determine a target multiple based on the quotient of the total number of set objects in the recommended object set and the target total number of recommendations; wherein the target multiple is a positive integer; the acquisition and determination module 31 includes a total number determination submodule, which is used to obtain the product of the target total number of recommendations and the target multiple as the number of subset objects.

[0041] In some disclosed embodiments, the set partitioning module 32 includes a divisibility detection submodule, which is used to detect whether the total number of set objects in the recommended object set is divisible by the number of subset objects; the set partitioning module 32 includes a first partitioning submodule, which is used to, in response to the detection result including being divisible, directly divide the recommended object set in half based on the number of subset objects to obtain a number of object subsets; the set partitioning module 32 includes a second partitioning submodule, which is used to, in response to the detection result including being not divisible, first divide the recommended object set in half based on the number of subset objects to obtain a number of object subsets, and then amortize each recommended object in the recommended object set that has not been divided to each object subset.

[0042] In some disclosed embodiments, the combination loop module 34 includes an object selection submodule for sequentially selecting a target total number of recommended objects from the rearranged subsets; the combination loop module 34 includes an object combination submodule for combining a new set of recommended objects based on the recommended objects selected from each rearranged subset.

[0043] In some disclosed embodiments, the subset rearrangement module 33 includes a prompt construction submodule, which is used to construct a prompt instruction based on the requirement description of the query to be responded to and the input object set, and to construct a decoding vocabulary based at least on the sorting label; wherein the prompt instruction is used to instruct the large language model to sort each recommended object in the input object set according to their respective relevance to the requirement description; the subset rearrangement module 33 includes a result acquisition submodule, which is used to input the prompt instruction to the large language model to obtain the reordering result output by the large language model based on the decoding vocabulary.

[0044] In some disclosed embodiments, the sorting tag includes at least a digital number; and / or the decoding vocabulary also includes a response word for backing up the reordering; and / or the decoding vocabulary also includes special symbols for indicating the order of precedence.

[0045] In some disclosed embodiments, the demand description of the query to be responded to includes at least key words and phrases input into the search engine, and the recommended objects include at least recommended web pages; or, the demand description of the query to be responded to includes at least behavioral information input into the recommendation system, and the recommended objects include at least recommended things; or, the demand description of the query to be responded to includes at least questions input into the question-and-answer system, and the recommended objects include at least recommended answers.

[0046] See also Figure 4 , Figure 4 : is a schematic diagram of the framework of an embodiment of an electronic device of the present application. The electronic device 40 includes at least a memory 41 and a processor 42 coupled to each other, the memory 41 stores at least program instructions, and the processor 42 is used to execute the program instructions to implement the steps in any of the above-mentioned reordering method embodiments. For details, please refer to the aforementioned disclosed embodiments, which will not be repeated here. As a possible example, the electronic device 40 may include but is not limited to a server, etc., and the specific type of the electronic device 40 is not limited here.

[0047] Specifically, the processor 42 is used to control itself and the memory 41 to implement the steps in any of the above-mentioned reordering method embodiments. The processor 42 can also be called a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip with signal processing capabilities. The processor 42 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0048] In the above scheme, the electronic device 40 obtains the set of recommended objects for the query to be responded to, and determines the number of subset objects based on the total number of target recommendations for the query to be responded to, and the number of subset objects is greater than the total number of target recommendations, and the recommended object set contains a number of recommended objects, and then divides the recommended object set based on the number of subset objects to obtain a number of object subsets, and reorders the object subsets based on the requirement description of the query to be responded to obtain a reordered subset of the object subset, thereby selecting recommended objects that are no more than the total number of target recommendations from each reordered subset in sequence, combining them as a new set of recommended objects, and returning the recommended object set based on the subset objects. The number of recommended object sets is divided into several steps to obtain several object subsets, until the total number of objects in the latest recommended object set is not greater than the number of subset objects, and then the latest recommended object set is reordered based on the demand description to obtain at least the target total number of recommended objects arranged in order. Therefore, through subset division, each reordering is constrained within the data volume range limited by the number of subset objects, which can achieve data volume degradation in disguise, and combined with multiple rounds of cycles to execute the splitting, reordering, and merging process, it can achieve downgraded multi-way merge sorting, which helps to break through the limitations of existing reordering technologies caused by massive data. Therefore, reordering can be achieved when facing massive data.

[0049] See also Figure 5 , Figure 5 1 is a schematic diagram of a framework of an embodiment of a computer-readable storage medium 50 of the present application. The computer-readable storage medium 50 stores program instructions 51 that can be executed by a processor, and the program instructions 51 are used to implement the steps in any of the above-mentioned reordering method embodiments.

[0050] In the above scheme, the computer-readable storage medium 50 obtains a set of recommended objects for the query to be responded to, and determines the number of subset objects based on the total number of target recommendations for the query to be responded to, and the number of subset objects is greater than the total number of target recommendations, and the recommended object set contains a number of recommended objects, and then divides the recommended object set based on the number of subset objects to obtain a number of object subsets, and reorders the object subsets based on the requirement description of the query to be responded to obtain a reordered subset of the object subset, thereby selecting recommended objects that are no more than the total number of target recommendations from each reordered subset in sequence, combining them as a new recommended object set, and returning the recommended object set based on the subset. The recommended object set is divided according to the number of objects to obtain several object subsets, until the total number of objects in the latest recommended object set is not greater than the number of subset objects, and then the latest recommended object set is reordered based on the demand description to obtain at least the target total number of recommended objects arranged in order. Therefore, through subset division, each reordering is constrained within the data volume range limited by the number of subset objects, which can achieve data volume degradation in disguise, and combined with multiple rounds of cycles to execute the splitting, reordering, and merging process, it can achieve downgraded multi-way merge sorting, which helps to break through the limitations of existing reordering technologies caused by massive data. Therefore, reordering can be achieved when facing massive data.

[0051] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0052] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0053] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0054] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0055] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0056] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0057] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A reordering method, characterized in that: include: Obtain a set of recommended objects for a query to be responded to, and determine the number of subset objects based on a target total number of recommended objects for the query to be responded to; wherein the number of subset objects is greater than the target total number of recommended objects, and the set of recommended objects includes a number of recommended objects; Dividing the recommended object set based on the number of objects in the subset to obtain a plurality of object subsets; Reordering the object subset based on the requirement description of the query to be responded to, to obtain a reordered subset of the object subset; Selecting recommended objects that are no more than the target total number of recommendations from each of the rearranged subsets in order, combining them as a new recommended object set, and returning to the step of dividing the recommended object set based on the number of subset objects to obtain a plurality of object subsets, until the total number of objects in the latest recommended object set is no more than the number of subset objects; The latest set of recommended objects is reordered based on the requirement description to obtain at least the target total number of recommended objects arranged in sequence.

2. The method according to claim 1, characterized in that The determining the number of subset objects based on the total number of target recommendations for the query to be responded to includes: Determine a target multiple based on the quotient of the total number of set objects in the recommended object set and the target total number of recommendations; wherein the target multiple is a positive integer; The product of the target total number of recommendations and the target multiple is obtained as the number of subset objects.

3. The method according to claim 1, characterized in that The step of dividing the recommended object set based on the number of objects in the subset to obtain a plurality of object subsets includes: Detecting whether the total number of set objects of the recommended object set is evenly divisible by the number of subset objects; In response to the detection result including being divisible, directly dividing the recommended object set in half based on the number of objects in the subset to obtain the plurality of object subsets; In response to the detection result including being not divisible, the recommended object set is first divided equally based on the number of objects in the subset to obtain the plurality of object subsets, and then each of the recommended objects that have not been divided in the recommended object set is evenly distributed to each of the object subsets.

4. The method according to claim 1, characterized in that: The selecting, in order, from each of the rearranged subsets, recommendation objects that are no more than the total number of target recommendations and combining them as a new set of recommendation objects includes: Selecting the total number of recommendation objects of the target recommendation in order from the rearranged subset; Based on the recommended objects selected from each of the rearranged subsets, the new recommended object set is combined.

5. The method according to claim 1, characterized in that The reordering includes: Based on the requirement description of the query to be responded to and the input object set, a prompt instruction is constructed, and a decoding vocabulary is constructed based at least on the sorting label; wherein the prompt instruction is used to instruct the large language model to sort each of the recommended objects in the input object set according to their relevance to the requirement description; The prompt instruction is input to the large language model to obtain a reordering result output by the large language model according to the decoding vocabulary.

6. The method according to claim 5, characterized in that The sorting label at least includes a digital number; And / or, the decoding word list also includes reply words for backing up the reordering; And / or, the decoding vocabulary also includes special symbols for indicating the order of precedence.

7. The method according to any one of claims 1 to 6, characterized in that: The demand description of the query to be responded to includes at least a keyword sentence input into the search engine, and the recommended object includes at least a recommended web page; Alternatively, the demand description of the query to be responded to includes at least the behavior information input into the recommendation system, and the recommendation object includes at least the recommended thing; Alternatively, the demand description of the query to be responded to at least includes a question input into the question-answering system, and the recommended objects at least include recommended answers.

8. A reordering device, characterized in that: include: An acquisition and determination module is used to acquire a set of recommended objects for a query to be responded to, and determine the number of subset objects based on a target total number of recommended objects for the query to be responded to; wherein the number of subset objects is greater than the target total number of recommended objects, and the recommended object set includes a number of recommended objects; A set partitioning module, used for partitioning the recommended object set based on the number of objects in the subset to obtain a plurality of object subsets; A subset rearrangement module, configured to reorder the object subset based on the requirement description of the query to be responded to, to obtain a rearranged subset of the object subset; a combination loop module, configured to sequentially select recommendation objects not exceeding the target total number of recommendations from each of the rearranged subsets, combine them as a new recommendation object set, and return to the step of dividing the recommendation object set based on the number of subset objects to obtain a plurality of object subsets, until the total number of objects in the latest recommendation object set is not greater than the number of subset objects; The final re-arrangement module is used to re-arrange the latest set of recommended objects based on the requirement description to obtain at least the target total number of recommended objects arranged in sequence.

9. An electronic device, characterized in that: The system at least comprises a memory and a processor coupled to each other, wherein the memory at least stores program instructions, and the processor is used to execute the program instructions to implement the reordering method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the reordering method according to any one of claims 1 to 7.

Images