Sorting method and device, electronic equipment and storage medium

Abstract

The disclosure provides a ranking method, device, electronic equipment, storage medium and server, relates to the field of artificial intelligence, in particular to natural language processing and intelligent search technology, and can be applied to intelligent cloud, smart city and intelligent government scenarios. The implementation scheme is as follows: determining a ranking parameter of a recall result according to the relevance between a search statement and the recall result; determining a ranking score of the recall result based on the ranking parameter of the recall result; and ranking the recall results based on the ranking scores of the recall results. The embodiment of the disclosure can rank the recall results based on the relevance between the recall results and the search statement, quantitatively describe the relevance by using the ranking score, intuitively and simply describe the relevance degree between the recall results and the search statement, and then rank the recall results based on the relevance degree, so that more suitable recall results can be recommended.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically natural language processing and intelligent search technology, which can be applied in scenarios such as intelligent cloud, smart city, and smart government. Background Technology

[0002] Data retrieval based on a search query yields recall results. When there are many recall results, they need to be sorted and then displayed to the user in that order. The closer the recall result is to the search query, the higher it should be displayed to the user. Therefore, sorting the recall results is particularly important. Summary of the Invention

[0003] This disclosure provides a sorting method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of this disclosure, a sorting method is provided, comprising:

[0005] The ranking parameters of the recall results are determined based on the relevance between the search query and the recall results of the search query.

[0006] Based on the ranking parameters of the recall result, determine the ranking score of the recall result;

[0007] The recall results are ranked based on their ranking scores.

[0008] According to a second aspect of this disclosure, a sorting apparatus is provided, comprising:

[0009] The sorting parameter determination module is used to determine the sorting parameters of the recall result based on the relevance between the search statement and the recall result of the search statement.

[0010] The ranking score determination module is used to determine the ranking score of the recall result based on the ranking parameters of the recall result.

[0011] The sorting module is used to sort the recall results based on their sorting scores.

[0012] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0016] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the method of the first aspect described above.

[0017] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of the first aspect described above.

[0018] The solution provided in this embodiment can sort the recall results and search statements based on their relevance. It uses a ranking score to quantify the relevance, which can intuitively and easily describe the degree of relevance between the recall results and the search statements. Then, it can recommend more suitable recall results based on the degree of relevance.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is a schematic flowchart of a sorting method according to an embodiment of the present disclosure;

[0022] Figure 2 This is a flowchart illustrating a sorting method according to another embodiment of the present disclosure;

[0023] Figure 3 This is a flowchart illustrating a sorting method according to another embodiment of the present disclosure;

[0024] Figure 4 This is a flowchart illustrating a sorting method according to another embodiment of the present disclosure;

[0025] Figure 5 This is a schematic diagram illustrating the determination of a ranking score according to an embodiment of the present disclosure;

[0026] Figure 6 This is a schematic diagram illustrating the determination of a ranking score according to another embodiment of the present disclosure;

[0027] Figure 7 This is a schematic diagram of the structure of a sorting device according to an embodiment of the present disclosure;

[0028] Figure 8 This is a schematic diagram of the structure of a sorting device according to another embodiment of the present disclosure;

[0029] Figure 9This is a flowchart illustrating a sorting method according to an embodiment of the present disclosure in a specific application scenario.

[0030] Figure 10 This is an electronic device block diagram of a sorting method according to an embodiment of the present disclosure. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] Figure 1 This is a flowchart illustrating a sorting method according to an embodiment of the present disclosure. The method may include:

[0033] S101. Determine the sorting parameters of the recall result based on the relevance between the search statement and the recall result of the search statement.

[0034] S102. Based on the ranking parameters of the recall result, determine the ranking score of the recall result.

[0035] S103. Sort each recall result according to its ranking score.

[0036] In this embodiment, the search query can also be referred to as a search request, retrieval statement, or search purpose. The search query may include the specific content the user needs to search for. The results are ranked based on the recall and relevance of the search query. The relevance effectively describes the relationship between the two; the closer they are, the higher the relevance. This disclosure uses ranking parameters to describe relevance, making the description of relevance quantifiable. Then, ranking scores are used to quantify the relevance, providing a clear and simple description of the degree of relevance between the recall results and the search query. The higher the relevance, the closer the relationship between the recall results and the search query, and the higher the probability that it represents the desired recall result. Therefore, ranking based on relevance can recommend more suitable recall results, thereby improving search accuracy and user experience.

[0037] Figure 2 This is a flowchart illustrating a sorting method according to another embodiment of the present disclosure. The method of this embodiment includes one or more features of the sorting method embodiments described above. A search statement may include multiple words, which may be referred to as search terms. For ease of distinction, the words contained in the search statement in the embodiments of this disclosure are referred to as search terms, and the search terms contained in the recall results are referred to as recall terms.

[0038] In one possible implementation, for each recall result, the ranking parameters describing relevance may include at least one of the following: the importance of the recall term contained in the recall result, the confidence level of the attribute of the recall term, the field weight of the recall term, the matching degree between the recall result and the search statement, and the recall score of the recall result.

[0039] In this embodiment of the disclosure, the relevance between the recall results and the search terms is described from different dimensions. When multiple dimensions are used to describe the relevance between the recall results and the search terms, the degree of relevance between the two can be measured more comprehensively, thereby improving the accuracy of the ranking.

[0040] In one possible implementation, if the ranking parameter of the recall result includes the matching degree, in S101, the ranking parameter of the recall result is determined based on the relevance between the search statement and the recall result of the search statement, including: determining the matching degree between the recall result and the search statement based on the common string between the recall result and the search statement.

[0041] When performing a search based on a search query, the returned results often contain the search terms from the query. Therefore, the more search terms included, the higher the relevance of the results to the search query. Based on this, in this embodiment, using common strings allows for an accurate and convenient measurement of the character-level relevance between the results and the search query.

[0042] In one possible implementation, the match between the recall results and the search query can be represented by a first similarity score. This first similarity score can be determined as follows:

[0043] Determine the maximum string length in the recall result and the search query;

[0044] The first similarity is determined based on the length of the common string of the recall result and the length of the maximum string.

[0045] For example, the length of the search query string and the length of the retrieved results string are counted. Then, the maximum string length between these two lengths is selected. The length of the common string is then divided by this maximum string length to obtain the first similarity. Based on this, the first similarity can be expressed by formula (1):

[0046] sim1=lcs / max(query_size,content_size) (1)

[0047] In formula (1), sim1 represents the first similarity between the recall result and the search statement, lcs represents the length of the common string, query_size is the string length of the search statement, content_size represents the string length of the recall result, and max() represents taking the maximum value.

[0048] By using the maximum string length of the search query and the retrieved results, the matching degree between the retrieved results and the search query can be easily quantified, with low computational cost and easy implementation.

[0049] Furthermore, the first similarity between the recall results and the search statement indicates that they contain the same characters. However, different orders of the same characters in different statements will result in different intentions expressed by the statements. Therefore, in another embodiment of this disclosure, the matching degree between the recall results and the search statement can be comprehensively measured based on the first similarity and the character order. In implementation, such as Figure 2 As shown, based on the common string between the recall result and the search statement, the matching degree between the recall result and the search statement is determined, including:

[0050] S201. Determine the length of the common string between the recall result and the search statement, and sort the common string according to the character position in the recall result to obtain the string to be processed.

[0051] S202. Based on the length of the common string, determine the first similarity between the recall result and the search statement, and based on the edit distance between the string to be processed and the search statement, determine the second similarity between the recall result and the search statement.

[0052] The edit distance can be the Levenshtein distance. Based on the edit distance, a second similarity between the recall results and the search query can be measured at the character ranking level. The formula for determining the second similarity is shown in formula (2):

[0053] sim2=1-dic / lcs (2)

[0054] In formula (2), sim2 represents the second similarity, dic represents the edit distance, and lcs has the same meaning as in formula (1), that is, the length of the common string.

[0055] S203. Based on the first similarity and the second similarity, determine the matching degree between the recall result and the search statement.

[0056] The match between the recall results and the search query can be finally expressed as shown in formula (3):

[0057] Sim = sim1 + sim2 (3)

[0058] In formula (3), Sim represents the matching degree between the recall result and the search statement, sim1 represents the first similarity, and sim2 represents the second similarity.

[0059] In summary, the second similarity measure considers the differences in character ordering of common characters in the recall results and search statements, thereby distinguishing the expressive meaning brought about by different character orders. The first similarity measure measures the differences in common characters between the recall results and search statements. Therefore, by using both the first and second similarities, the relevance between the recall results and search statements can be measured from two dimensions: character matching degree and character order matching degree.

[0060] In one possible implementation, if the ranking parameter of the recall result includes the field weight, in S101, the ranking parameter of the recall result is determined according to the relevance between the search statement and the recall result of the search statement, including: determining the field weight of the recall term included in the recall result according to the preset weight corresponding to the recall term included in the recall result.

[0061] The preset weights for different terms can be set based on business needs. For example, in a public security scenario, matching an instant messaging account is less important than matching an ID card number; therefore, the preset weight for an ID card number is higher than that for an instant messaging account. Thus, the preset weights for different terms can be configured based on the needs of the business scenario, and the configured preset weights can be adjusted. This allows the field weights to adapt to different business scenario requirements, making them more flexible in use.

[0062] In this embodiment, the term weights are universal; however, the weights of the same search term in the same search query may not be differentiated based on the business scenario. Therefore, since the preset weights reflect the needs of the user's business scenario, the ranking results obtained based on field weights are more likely to meet the user's business needs. Different preset weights can be configured for the same field in different business scenarios. Under different business scenarios, the same recall results for the same search query may also differ due to differences in field weights. Thus, field weights can reflect the individual needs of different business scenarios. The field weights determined based on preset weights describe the relevance between the recall results and the search query from the perspective of application scenario requirements.

[0063] In one possible implementation, such as Figure 3 As shown, based on the preset weights corresponding to the recall terms included in the recall results, the field weights of the recall terms included in the recall results are determined, including:

[0064] S301. In the correspondence between preset fields and preset weights, find the preset weight corresponding to the recall term.

[0065] S302. If the recall result contains multiple recall terms, the sum of the preset weights of each recall term shall be determined as the weight of the field.

[0066] In this embodiment of the disclosure, when the recall results contain multiple recall terms, the field weight is determined by using the preset weights corresponding to the multiple recall terms respectively, thereby comprehensively describing the relevance between the recall results and the search statement in terms of the number of included recall terms and the importance of the preset weights.

[0067] In summary, based on field weights, we can measure the importance of the matched recall terms in the recall results, thereby filtering out recall results that are of concern to users and closely related to user scenarios.

[0068] In one possible implementation, a search query may include multiple search terms. Different search terms may have varying degrees of importance to the search query. For example, in the search query "a man named Zhang San," the search term "Zhang San" is more important than the search term "named," therefore, "Zhang San" is an important search term. Recall results containing important search terms should be ranked higher than those without. When multiple recall results contain important search terms, the importance of these important search terms to the search query may also differ. In such cases, it is necessary to distinguish which recall result contains the more important search term to improve ranking accuracy.

[0069] To measure which recall result contains more important recall terms, embodiments of this disclosure can determine the importance of the recall terms contained in the recall result. This importance is used to measure the significance of the recall terms included in the recall result. The importance of the recall terms contained in the recall result can effectively measure the relevance of the recall result to the search query. Based on this, when the ranking parameters of the recall result include this importance, in S101, the ranking parameters of the recall result are determined according to the relevance of the search query to the recall result of the search query, including: determining the importance of the recall terms contained in the recall result based on the word weights corresponding to the recall terms contained in the recall result.

[0070] Extracting search terms from a search query is called keyword extraction. These terms represent the search query to a certain extent. Determining term weights can be understood as obtaining a numerical sequence of the same length as the sentence. This sequence can contain two values: 0 and 1. This sequence is a 0-1 sequence. Positions with a value of 1 correspond to key search terms in the search query, while positions with a value of 0 are non-key search terms.

[0071] Beyond 01 sequences, numerical sequences can be further complicated, allowing search terms to be subdivided into different levels of importance. For example, when divided into 5 levels, different levels can be represented by 01234. 4 represents the most important, 3 the next most important, and so on, forming a hierarchical distribution of word importance.

[0072] In addition to different levels of importance, continuous values ​​can also be used to differentiate the importance of different words to the search query. For example, the continuous value range could be [0,1], with values ​​closer to 1 indicating greater importance of the search term.

[0073] In summary, after analyzing the search query to obtain the word weights of each search term, we can search for the search term in the recall results to obtain the recall terms contained in the recall results. Then, based on the word weights corresponding to the recall terms contained in the recall results, we can determine the importance of the recall terms contained in the recall results.

[0074] In this embodiment of the disclosure, word weights can accurately describe the importance of the recall words contained in the recall results, reflecting the relevance between the recall results and the search statement in terms of key search terms.

[0075] In one possible implementation, such as Figure 4 As shown, based on the word weights corresponding to the recalled words included in the recall results, the importance of the recalled words included in the recall results is determined, including:

[0076] S401. If the recall result contains multiple recall terms, obtain the word weight of each recall term.

[0077] S402. The sum of the word weights of each recall term is determined as the importance level.

[0078] For example, if the recall results include two recall terms, recall term 1 and recall term 2, with recall term 1 having a word weight of w1 and recall term 2 having a word weight of w2, then the importance of the recall terms included in the recall results is w = w1 + w2.

[0079] In this embodiment of the disclosure, the importance is described based on the sum of the word weights of the multiple recall terms included, which can comprehensively and accurately measure the relevance of the recall results and the search statement from the dimension of the importance of the search terms.

[0080] In one possible implementation, the attributes of a search term can be understood as the category of the search term. For example, in a public security scenario, personal information may include name, gender, ID number, instant messaging account, etc. Name, gender, etc., are each considered as an attribute. Of course, the attributes of search terms can be determined according to the business scenario, and this disclosure does not limit this.

[0081] The confidence level of an attribute can be understood as the probability that a search term belongs to the corresponding attribute. When the ranking parameters include the confidence level of the recall term's attributes, one possible implementation is to set regular expression conditions. Different regular expression conditions correspond to different confidence levels. For example, the confidence level corresponding to satisfying regular expression condition 1 is z1, and the confidence level corresponding to satisfying regular expression condition 2 is z2. The specific regular expression conditions can be set according to the business requirements of the application scenario, and this disclosure does not limit them.

[0082] Besides regular expression conditions, another implementation involves labeling the attributes of search terms in the search query and assigning corresponding confidence levels. A neural network model is then trained based on these labels, enabling it to recognize the attributes of search terms and provide confidence levels. After training, the neural network model can be used to identify the attributes and confidence levels of different search terms. Thus, the confidence level output by the neural network model can range from a continuous interval.

[0083] Attribute confidence represents the probability that a search term belongs to a corresponding attribute. Therefore, the probability that a recalled term belongs to a corresponding attribute reflects the difference in relevance between the recalled term and the search query. If the probability of the same recalled term belonging to a corresponding attribute differs across different recall results, the relevance between different recall results and the search query can be distinguished. Similarly, the confidence of an attribute of a recalled term in a search query can indirectly reflect the importance of the recalled term in the search query. Higher confidence indicates a more relevant recall result containing that term to the search query. Therefore, attribute-based confidence can effectively measure the relevance between recall results and the search query at the level of attribute importance.

[0084] In one possible implementation, while obtaining recall results based on the search query, each recall result has a corresponding recall score to describe the relevance between the recall result and the search query, thereby allowing the recall results to be ranked based on the recall score.

[0085] However, simply using recall scores for sorting has the following problems:

[0086] When a search query requires multiple retrievals, recall scores struggle to accurately describe the match between the retrieved results and the search query. For example, a search query for "Zhang San's backpack" might first retrieve "Zhang San" from the database, yielding the first result. Then, based on the first result, a second search for "backpack" would be performed, resulting in the final recall result. However, existing recall scores often score the first and second results separately, making it difficult to comprehensively evaluate the overall recall outcome. Furthermore, the scoring mechanism for recall scores is relatively simple, relying solely on ranking based on recall scores, and the ranking effect still needs improvement.

[0087] In view of this, in the embodiments of this disclosure, multiple parameters in the sorting parameters can be used to jointly describe the relevance between the recall results and the search statement.

[0088] For example, importance and recall score can be used together to determine the relevance between the two, or importance, match degree, and field weight can be used together to describe the relevance. That is, multiple parameters can be used to measure the relevance between the recall results and the search query, thereby measuring the relevance from multiple dimensions and improving the accuracy of ranking.

[0089] In one possible implementation, in S103, determining the ranking score of the recall result based on the ranking parameter of the recall result includes:

[0090] When the ranking parameters of the recall result include the importance, the recall score, the match degree, the confidence degree, and the field weight, the ranking score of the recall result is equal to the sum of the product of the importance, the recall score, the match degree, and the confidence degree, and the field weight; or,

[0091] When the ranking parameters of the recall result include the importance, the match degree, the recall score, and the field weight, the ranking score of the recall result is equal to the sum of the product of the importance, the recall score, and the match degree, and the field weight.

[0092] For example, combining Figure 5 As shown, when the ranking parameters include the matching degree, the confidence degree, and the field weight, the ranking score of the recall result is determined based on the following first ranking score determination formula, which is shown in formula (4):

[0093] Score = a * b * c * d + e (4)

[0094] Where 'a' represents the importance, 'b' represents the recall score, 'c' represents the match degree, 'd' represents the confidence degree, and 'e' represents the field weight.

[0095] For example, combining Figure 6 As shown, when the ranking parameter includes the matching degree and the field weight, the ranking parameter of the recall result is determined based on the following second ranking score determination formula, which is shown in formula (5):

[0096] Score = a * b * c + e (5)

[0097] Where 'a' represents the importance level, 'b' represents the recall score, 'c' represents the match degree, and 'e' represents the field weight.

[0098] In summary, match score, confidence score, and field weight describe the first type of relevance between the recall results and the search query. This relevance is universal, applicable across different business scenarios. Field weight addresses the specific user needs within a particular business scenario; therefore, summing the field weights demonstrates a focus on user needs. Thus, using both methods to determine the ranking score better addresses both generality and user-specific needs, resulting in ranking results that better meet user expectations.

[0099] Figure 7 This is a schematic diagram of a sorting device according to an embodiment of the present disclosure, the device may include:

[0100] The sorting parameter determination module 701 is used to determine the sorting parameters of the recall result based on the relevance between the search statement and the recall result of the search statement.

[0101] The ranking score determination module 702 is used to determine the ranking score of the recall result based on the ranking parameters of the recall result;

[0102] The sorting module 703 is used to sort each recall result based on the sorting score of each recall result.

[0103] Figure 8 This is a schematic diagram of a sorting apparatus according to another embodiment of the present disclosure. The apparatus of this embodiment includes one or more features of the sorting apparatus embodiments described above. In one possible implementation, the sorting parameters of the recall result include at least one of the following: the importance of the recall term included in the recall result, the confidence level of the attribute of the recall term, the field weight of the recall term, the matching degree between the recall result and the search statement, and the recall score of the recall result.

[0104] In one possible implementation, such as Figure 8 As shown, the sorting parameter determination module 701 includes:

[0105] The matching degree determination submodule 801 is used to determine the matching degree between the recall result and the search statement based on the common string between the recall result and the search statement when the sorting parameter of the recall result includes the matching degree.

[0106] In one possible implementation, the matching degree determination submodule 801 is used for:

[0107] Determine the length of the common string between the recall result and the search statement, and sort the common string according to the character position in the recall result to obtain the string to be processed;

[0108] Based on the length of the common string, the first similarity between the recall result and the search statement is determined, and the second similarity between the recall result and the search statement is determined based on the edit distance between the string to be processed and the search statement.

[0109] Based on the first similarity and the second similarity, the matching degree between the recall result and the search statement is determined.

[0110] In one possible implementation, the matching degree determination submodule 801 determines a first similarity between the recall result and the search statement based on the length of the common string, including:

[0111] Determine the maximum string length in the recall result and the search query;

[0112] The first similarity is determined based on the length of the common string and the length of the maximum string.

[0113] In one possible implementation, such as Figure 8 As shown, the sorting parameter determination module includes:

[0114] The field weight determination submodule 802 is used to determine the field weight of the recall terms contained in the recall result based on the preset weight corresponding to the recall terms contained in the recall result when the sorting parameters of the recall result include the field weight.

[0115] In one possible implementation, the field weight determination submodule 802 is used for:

[0116] In the correspondence between preset fields and preset weights, find the preset weight corresponding to the recall term;

[0117] If the recall result contains multiple recall terms, the sum of the preset weights of each recall term is determined as the weight of the field.

[0118] In one possible implementation, such as Figure 8 As shown, the sorting parameter determination module 701 includes:

[0119] The importance determination submodule 803 is used to determine the importance of the recall words contained in the recall result based on the word weights corresponding to the recall words contained in the recall result, when the ranking parameters of the recall result include the importance.

[0120] In one possible implementation, the importance determination submodule 803 is used for:

[0121] If the recall result contains multiple of the recalled terms, obtain the word weight of each of the recalled terms;

[0122] The importance level is determined by the sum of the word weights of each recall term.

[0123] In one possible implementation, the sorting score determination module 702 is used for:

[0124] When the ranking parameters of the recall result include the importance, the recall score, the match degree, the confidence degree, and the field weight, the ranking score of the recall result is equal to the sum of the product of the importance, the recall score, the match degree, and the confidence degree, and the field weight; or,

[0125] When the ranking parameters of the recall result include the importance, the match degree, the recall score, and the field weight, the ranking score of the recall result is equal to the sum of the product of the importance, the recall score, and the match degree, and the field weight.

[0126] This disclosure employs a ranking parameter to describe relevance, making the description of relevance quantifiable. Then, a ranking score is used to quantify the relevance, providing a clear and simple way to describe the degree of relevance between the retrieved results and the search query. The higher the relevance, the closer the relationship between the retrieved results and the search query, and the higher the probability that they represent the desired retrieved results. Therefore, ranking based on relevance allows for the recommendation of more suitable retrieved results, thereby improving search accuracy and user experience.

[0127] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0128] In a specific application scenario, such as Figure 9 As shown, the process of scoring the files of recalled personnel may include the following steps:

[0129] S901 retrieves the recall personnel files based on the search query.

[0130] S902, obtain the sorting parameters of the recalled personnel files, including: the importance of the recall terms contained in the recalled personnel files, the confidence of the attributes of the recall terms, the field weight of the recall terms, the matching degree between the recalled personnel files and the search statement, and the recall score of the recalled personnel files.

[0131] S903: Based on the ranking parameters, the files of recalled personnel are scored to obtain a ranking score.

[0132] S904, sort the files of each recalled personnel based on the ranking score.

[0133] S905 recommends and displays recall personnel files based on a ranking filter, highlighting key personnel. For example... Figure 9As shown, the top k files are selected as the recall personnel for Chaoyang staff. Alternatively, personnel with ranking scores higher than a threshold can be considered as key personnel.

[0134] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0135] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is provided. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0136] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0137] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes the sorting method described above. For example, in some embodiments, the sorting method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the sorting method may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the sorting method by any other suitable means (e.g., by means of firmware).

[0139] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0144] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0145] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A sorting method, comprising: Based on the relevance between the search query and the recall results, the ranking parameters of the recall results are determined, including: In the correspondence between preset fields and preset weights, find the preset weight corresponding to the recall term contained in the recall result; when the recall result contains multiple recall terms, determine the field weight of the recall term by summing the preset weights of each recall term; wherein, in different business scenarios, the preset weight for the same recall term is determined independently based on the requirements of the corresponding business scenario. The search statement is analyzed to obtain the word weight of each search term. The search term is searched in the recall results to obtain the recall terms contained in the recall results. If the recall results contain multiple recall terms, the word weight of each recall term is obtained. The sum of the word weights of each recall term is determined as the importance of the recall term. The word weight is used to distinguish the importance of different words to the search statement. When the ranking parameters of the recall results include the importance of the recalled terms contained in the recall results, the recall score of the recall results, the match degree between the recall results and the search query, the confidence degree of the attributes of the recalled terms, and the field weight of the recalled terms, the ranking score of the recall results is equal to the sum of the product of the importance, the recall score, the match degree and the confidence degree and the field weight; or, When the ranking parameters of the recall results include the importance, the matching degree, the recall score, and the field weight, the ranking score of the recall results is equal to the sum of the product of the importance, the recall score, and the matching degree, and the field weight. The recall results are ranked based on their ranking scores.

2. The method according to claim 1, wherein when the ranking parameter of the recall result includes the matching degree, determining the ranking parameter of the recall result based on the relevance between the search statement and the recall result of the search statement includes: The matching degree between the recall results and the search statement is determined based on the common string between the recall results and the search statement.

3. The method according to claim 2, wherein determining the matching degree between the recall result and the search statement based on the common string between the recall result and the search statement includes: Determine the length of the common string between the recall result and the search statement, and sort the common string according to the character position in the recall result to obtain the string to be processed; Based on the length of the public string, a first similarity between the recall result and the search statement is determined, and a second similarity between the recall result and the search statement is determined based on the edit distance between the string to be processed and the search statement; Based on the first similarity and the second similarity, the matching degree between the recall result and the search statement is determined.

4. The method according to claim 3, wherein determining the first similarity between the recall result and the search statement based on the length of the common string, includes: Determine the maximum string length in the recall results and the search query; The first similarity is determined based on the length of the common string and the length of the maximum string.

5. A sorting device, comprising: The sorting parameter determination module is used to determine the sorting parameters of the recall results based on the relevance between the search statement and the recall results of the search statement. The sorting parameter determination module includes: The field weight determination submodule is used to find the preset weight corresponding to the recall term contained in the recall result in the correspondence between preset fields and preset weights; when the recall result contains multiple recall terms, the sum of the preset weights of each recall term is determined as the field weight of the recall term; wherein, in different business scenarios, the preset weight for the same recall term is determined independently based on the requirements of the corresponding business scenario. The importance determination submodule is used to analyze the search statement to obtain the word weight of each search term, search for the search term in the recall results, obtain the recall terms contained in the recall results, and if the recall results contain multiple recall terms, obtain the word weight of each recall term; the sum of the word weights of all the recall terms is determined as the importance of the recall term; wherein, the word weight is used to distinguish the importance of different words to the search statement; The ranking score determination module is configured to, when the ranking parameters of the recall result include the importance of the recall terms contained in the recall result, the recall score of the recall result, the match degree between the recall result and the search statement, the confidence degree of the attributes of the recall terms, and the field weight of the recall terms, determine that the ranking score of the recall result is equal to the sum of the product of the importance, the recall score, the match degree and the confidence degree and the field weight; or, When the ranking parameters of the recall results include the importance, the matching degree, the recall score, and the field weight, the ranking score of the recall results is equal to the sum of the product of the importance, the recall score, and the matching degree, and the field weight. The sorting module is used to sort the recall results based on their sorting scores.

6. The apparatus according to claim 5, wherein the sorting parameter determination module comprises: The matching degree determination submodule is used to determine the matching degree between the recall result and the search statement based on the common string between the recall result and the search statement when the sorting parameter of the recall result includes the matching degree.

7. The apparatus according to claim 6, wherein the matching degree determination submodule is used for: Determine the length of the common string between the recall result and the search statement, and sort the common string according to the character position in the recall result to obtain the string to be processed; Based on the length of the public string, a first similarity between the recall result and the search statement is determined, and a second similarity between the recall result and the search statement is determined based on the edit distance between the string to be processed and the search statement; Based on the first similarity and the second similarity, the matching degree between the recall result and the search statement is determined.

8. The apparatus of claim 7, wherein the matching degree determination submodule is used to determine a first similarity between the recall result and the search statement based on the length of the common string, comprising: Determine the maximum string length in the recall results and the search query; The first similarity is determined based on the length of the common string and the length of the maximum string.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-4.

Images