Evaluation Method, Device, Computing Device and Medium for Search Algorithm

Abstract

Embodiments of the present disclosure provide a method, apparatus, computing device, and medium for evaluating a search algorithm. The method includes: displaying a search algorithm evaluation interface, which includes a search function area and a search result evaluation area; obtaining corresponding search results based on the search terms input in the search function area and displaying the search results; and outputting search algorithm evaluation data according to the evaluation operations on the search results in the search result evaluation area. According to the technical solution of the embodiments of the present disclosure, it is possible to output the evaluation data of the evaluated search algorithm based on the evaluation operations on the search results of the search algorithm, expose the problems of the search algorithm in advance, thereby guiding the update and iteration direction of the search algorithm, improving the accuracy and reliability of the search algorithm, and further reducing the phenomenon of poor user experience.

Description

Background Art

[0002] This section aims to provide background or context for the embodiments of the present disclosure described in the claims. The description herein is not admitted to be prior art merely by virtue of being included in this section.

[0003] Music platforms provide users with an ultimate audio-visual experience. Users can perform search operations for the required songs on the music platform, and the search algorithm of the music platform filters out relevant content based on the search terms input by the users and displays it to the users.

[0004] However, the search algorithm of the music platform does not always give good search results for every search, and there will be cases where the search results do not meet the user's needs.

[0005] Therefore, there is a great need for a method to evaluate the search algorithm of a music platform, so as to be able to evaluate the search algorithm of the music platform and expose the problems of the search algorithm in advance. Summary of the Invention

[0006] In the first aspect of the embodiments of the present disclosure, a method for evaluating a search algorithm is provided, including:

[0007] Display a search algorithm evaluation interface, where the search algorithm evaluation interface includes a search function area and a search result evaluation area;

[0008] Obtain corresponding search results based on the search terms input in the search function area, and display the search results;

[0009] Output search algorithm evaluation data according to the evaluation operations on the search results in the search result evaluation area.

[0010] In some exemplary embodiments of the present disclosure, before displaying the search algorithm evaluation interface, the evaluation method further includes:

[0011] Display an evaluation task configuration interface, where the evaluation task configuration interface includes configuration controls for the evaluation type of the search algorithm to be evaluated and configuration controls for the evaluation tab page to be evaluated;

[0012] In response to the configuration operations on the configuration controls for the evaluation type and the evaluation tab page to be evaluated, determine the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated;

[0013] Generate a search algorithm evaluation interface for the target evaluation tab page based on the target evaluation type and the target evaluation tab page.

[0014] After outputting the search algorithm evaluation data according to the evaluation operations on the search results in the search result evaluation area, the evaluation method further includes:

[0015] Generate a search algorithm evaluation report based on the evaluation data of the search algorithm.

[0016] In some exemplary embodiments of the present disclosure, the evaluation task configuration interface further includes at least one of the following: a configuration control for the evaluation rules corresponding to the evaluation type, a configuration control for evaluation samples, and a configuration control for the number of evaluations.

[0017] In some exemplary embodiments of the present disclosure, the output of the search algorithm evaluation data according to the evaluation operation on the search results in the search result evaluation area includes:

[0018] Determine the search algorithm evaluation data using the target evaluation rules according to the evaluation operation on the search results in the search result evaluation area;

[0019] Output the search algorithm evaluation data.

[0020] In some exemplary embodiments of the present disclosure, the evaluation operation includes a marking operation of the satisfaction degree of the search results, and the target evaluation rules reflect the scoring mechanism of the search results.

[0021] In some exemplary embodiments of the present disclosure, the search algorithm evaluation interface is determined based on the selected search algorithm evaluation type, and the selected search algorithm evaluation type includes at least one of the following: user experience evaluation, version evaluation, and competitor evaluation.

[0022] In some exemplary embodiments of the present disclosure, the search result evaluation area includes marking controls for the satisfaction degree of each scoring dimension of the search results; the determination of the search algorithm evaluation data using the target evaluation rules according to the evaluation operation on the search results in the search result evaluation area includes:

[0023] For each search term in each evaluation, in response to the marking operation on the marking controls for the satisfaction degree of each scoring dimension of the search results, use the target evaluation rules to determine the scoring values of each scoring dimension corresponding to the search term;

[0024] After completing the evaluations for the set number of evaluations, perform statistical analysis on the scoring values of each scoring dimension corresponding to all search terms to obtain the search algorithm evaluation data;

[0025] Display the search algorithm evaluation data.

[0026] In some exemplary embodiments of the present disclosure, the determination of the scoring values of each scoring dimension corresponding to the search term in response to the marking operation on the marking controls for the satisfaction degree of each scoring dimension of the search results includes:

[0027] Determine the satisfaction level of the search results for each scoring dimension in response to the marking operation of the marking control for the satisfaction of each scoring dimension of the search results;

[0028] Based on the determined satisfaction levels of the search results for each scoring dimension, match the scoring values for each scoring dimension from the target evaluation rules.

[0029] In some exemplary embodiments of the present disclosure, the statistical analysis of the scoring values for each scoring dimension corresponding to all search terms to obtain search algorithm evaluation data includes:

[0030] For each search term, match the scoring weights corresponding to each scoring dimension from the target evaluation rules;

[0031] Calculate the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights;

[0032] Count the number of satisfaction levels of the search results corresponding to each scoring dimension respectively to obtain a search result satisfaction distribution table;

[0033] Generate search algorithm evaluation data based on the target score and the search result satisfaction distribution table.

[0034] In some exemplary embodiments of the present disclosure, the evaluation method further includes:

[0035] Generate evaluation result information indicating whether the search algorithm passes the evaluation based on the target score;

[0036] Output the evaluation result information.

[0037] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type includes user experience evaluation, and the calculation of the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights includes:

[0038] Calculate the weighted total score value of all search terms according to the scoring values and scoring weights of each scoring dimension of the search results corresponding to each search term;

[0039] Calculate the average score value of each search term according to the weighted total score value to obtain the target score.

[0040] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type includes version evaluation or competitor evaluation, and the calculation of the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights includes:

[0041] Calculate the weighted total score value of all search terms based on the score values and score weights of each scoring dimension of the search results corresponding to each search term, and obtain the target score.

[0042] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type includes competitor evaluation, and generating search algorithm evaluation data based on the target score and the search result satisfaction distribution table includes:

[0043] Generate search algorithm evaluation data based on the target score and the search result satisfaction distribution table corresponding to each competitor version for the current evaluation version.

[0044] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type includes version evaluation, and the evaluation method further includes:

[0045] Based on the search result satisfaction distribution table, obtain the number of search result satisfaction levels corresponding to each scoring dimension;

[0046] For each scoring dimension, calculate the average value of the number of the first search result satisfaction levels and the number of the second search result satisfaction levels corresponding to each scoring dimension;

[0047] Calculate the sum of the number of the third search result satisfaction levels corresponding to each scoring dimension and the average value to obtain the total number of wins;

[0048] Obtain the total number of all search result satisfaction levels corresponding to each scoring dimension;

[0049] Calculate the ratio of the total number of wins to the total number to obtain the win rate of the scoring dimension of the current evaluation version relative to the reference version;

[0050] Output the win rate;

[0051] Wherein, the first search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version meet the search expectation, the second search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version do not meet the search expectation, and the third search result satisfaction level is the search result satisfaction level indicating that the current evaluation version can meet the search expectation while the reference version cannot meet the search expectation.

[0052] In some exemplary embodiments of the present disclosure, the search result evaluation area further includes a marking control for the problem types corresponding to each scoring dimension; the evaluation method further includes:

[0053] For each search term in each evaluation, in response to a marking operation on a marking control for a question type corresponding to each scoring dimension, determine the question types of each scoring dimension of the search results corresponding to the search term;

[0054] After completing the evaluations for the set number of evaluation times, perform statistical analysis on the question types of each scoring dimension of the search results corresponding to all search terms to obtain a question distribution table;

[0055] Add the question distribution table to the search algorithm evaluation data.

[0056] In some example embodiments of the present disclosure, the performing statistical analysis on the question types of each scoring dimension of the search results corresponding to all search terms to obtain a question distribution table includes:

[0057] For each scoring dimension of the search results corresponding to all search terms, respectively count the number of each question type corresponding to each scoring dimension to obtain a question distribution table.

[0058] In some example embodiments of the present disclosure, the evaluation method further includes:

[0059] When a command to create an evaluation rule is detected, display an evaluation rule configuration interface, which includes an evaluation type selection menu;

[0060] In response to a selection operation on the evaluation type selection menu, select the search algorithm evaluation type for the evaluation rule to be created;

[0061] Display an evaluation rule configuration sub-interface corresponding to the selected search algorithm evaluation type for the evaluation rule to be created. The evaluation rule configuration sub-interface includes a scoring dimension configuration area, an evaluation level configuration area, and a question type configuration area. The scoring dimension configuration area includes configuration controls for scoring dimensions and corresponding scoring weights. The evaluation level configuration area includes configuration controls for the satisfaction level of search results and corresponding scoring values. The question type configuration area includes configuration controls for question types;

[0062] In response to a configuration operation on the evaluation rule configuration sub-interface, generate an evaluation rule corresponding to the selected search algorithm evaluation type for the evaluation rule to be created.

[0063] In a second aspect of the embodiments of the present disclosure, there is provided an evaluation device for a search algorithm, including:

[0064] An evaluation interface display module for displaying a search algorithm evaluation interface, which includes a search function area and a search result evaluation area;

[0065] A search module, configured to obtain corresponding search results based on the detected search terms input in the search function area, and display the search results;

[0066] An evaluation module, configured to output search algorithm evaluation data according to the detected evaluation operations on the search results in the search result evaluation area.

[0067] In some exemplary embodiments of the present disclosure, the device further includes:

[0068] A report generation module, configured to generate a search algorithm evaluation report according to the search algorithm evaluation data.

[0069] In some exemplary embodiments of the present disclosure, the device further includes:

[0070] An evaluation task display module, configured to display an evaluation task configuration interface, where the evaluation task configuration interface includes configuration controls for the evaluation type of the search algorithm to be evaluated and configuration controls for the evaluation tab page to be evaluated;

[0071] A determination module, configured to determine the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated in response to configuration operations on the configuration controls for the evaluation type and the evaluation tab page to be evaluated;

[0072] An evaluation interface generation module, configured to generate a search algorithm evaluation interface for the target evaluation tab page based on the target evaluation type and the target evaluation tab page.

[0073] In some exemplary embodiments of the present disclosure, the device further includes:

[0074] A first evaluation rule display module, configured to display an evaluation rule configuration interface when an instruction to create an evaluation rule is detected, where the evaluation rule configuration interface includes an evaluation type selection menu;

[0075] An evaluation type determination module, configured to select the evaluation type of the search algorithm for which the evaluation rule is to be created in response to a selection operation on the evaluation type selection menu;

[0076] A second evaluation rule display module, configured to display an evaluation rule configuration sub-interface corresponding to the evaluation type of the search algorithm for which the evaluation rule to be created is selected, where the evaluation rule configuration sub-interface includes a scoring dimension configuration area, an evaluation level configuration area, and a question type configuration area, the scoring dimension configuration area includes configuration controls for scoring dimensions and corresponding scoring weights, the evaluation level configuration area includes configuration controls for the satisfaction level of the search results and corresponding scoring values, and the question type configuration area includes configuration controls for question types;

[0077] An evaluation rule generation module, configured to generate an evaluation rule corresponding to the evaluation type of the search algorithm of the selected to-be-created evaluation rule in response to a configuration operation for the evaluation rule configuration sub-interface.

[0078] In a third aspect of the embodiments of the present disclosure, a computing device is provided, including: a processor and a memory, the memory stores executable instructions, and the processor is configured to call the executable instructions stored in the memory to execute the method according to any one of the above first aspects.

[0079] In a fourth aspect of the embodiments of the present disclosure, a medium is provided, on which a program is stored, and when the program is executed by a processor, the method according to any one of the above first aspects is implemented.

[0080] According to the technical solution of the embodiments of the present disclosure, on the one hand, a search algorithm evaluation interface can be provided for evaluators, enabling evaluators to use the search algorithm evaluation interface to evaluate search results, and outputting evaluation data of the evaluated search algorithm based on the evaluation operations of the evaluators. Through this evaluation data, problems of the search algorithm can be pre-exposed, thereby guiding the update and iteration direction of the search algorithm, improving the accuracy and reliability of the search algorithm, and reducing the phenomenon of poor user experience; on the other hand, before the new version of the search algorithm is launched, the search effect of the search algorithm can be pre-known through the technical solution of the embodiments of the present disclosure, providing a reference basis for whether to launch. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understandable. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, wherein:

[0082] Figure 1 Schematically shows a flowchart of a method for evaluating a search algorithm according to some embodiments of the present disclosure;

[0083] Figure 2 Schematically shows a schematic diagram of a search algorithm evaluation interface corresponding to user experience evaluation according to some embodiments of the present disclosure;

[0084] Figure 3 Schematically shows a schematic diagram of the display effect of search results of version evaluation according to some embodiments of the present disclosure;

[0085] Figure 4 Schematically shows a schematic diagram of the display effect of search results of competitor evaluation according to some embodiments of the present disclosure;

[0086] Figure 5A flowchart of a method for generating a search algorithm evaluation interface according to some embodiments of the present disclosure is schematically shown;

[0087] Figure 6 A flowchart of a method for creating an evaluation rule according to some embodiments of the present disclosure is schematically shown;

[0088] Figure 7 A schematic diagram of an evaluation rule configuration interface according to some embodiments of the present disclosure is schematically shown;

[0089] Figure 8 A flowchart of a method for determining search algorithm evaluation data using a target evaluation rule according to some embodiments of the present disclosure is schematically shown;

[0090] Figure 9 A schematic diagram of an evaluation task configuration interface according to some embodiments of the present disclosure is schematically shown;

[0091] Figure 10 A flowchart of a method for statistically analyzing the scoring values of each scoring dimension corresponding to all search terms according to some embodiments of the present disclosure is schematically shown;

[0092] Figure 11 A flowchart of a method for obtaining the winning rate of the current evaluation version relative to the reference version according to some embodiments of the present disclosure is schematically shown;

[0093] Figure 12 A flowchart of a method for obtaining search algorithm evaluation data according to some embodiments of the present disclosure is schematically shown;

[0094] Figure 13 A schematic diagram of the display effect of a search algorithm evaluation report for user experience evaluation according to some embodiments of the present disclosure is schematically shown;

[0095] Figure 14 A schematic diagram of a search algorithm evaluation report for competitor evaluation according to some embodiments of the present disclosure is schematically shown;

[0096] Figure 15 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown;

[0097] Figure 16 A block diagram of the structure of an evaluation device for a search algorithm according to some embodiments of the present disclosure is schematically shown;

[0098] Figure 17 A block diagram of the structure of an electronic device according to some embodiments of the present disclosure is schematically shown.

[0099] In the accompanying drawings, the same or corresponding reference numerals denote the same or corresponding parts. Detailed Implementation Modes

[0100] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, rather than to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to convey the scope of the present disclosure fully to those skilled in the art.

[0101] Those skilled in the art know that the embodiments of the present disclosure can be implemented as a system, device, equipment, method, or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms: completely hardware, completely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0102] According to the embodiments of the present disclosure, a method, device, computing device, and medium for evaluating a search algorithm are provided.

[0103] In this article, it should be understood that the terms involved:

[0104] A search algorithm refers to presenting content related to the search term input by the user to the user according to a certain display effect.

[0105] Evaluation, that is, assessment test. In the embodiments of the present disclosure, the scheme is described by taking the assessment test of the search algorithm effect of a music software as an example. Based on the scheme of the present disclosure, it can assist the evaluator in scoring and recording the search results, and obtaining relevant evaluation data after statistical analysis of the scoring and recording results.

[0106] User experience evaluation is a way of long-term quality tracking of a search algorithm, and the execution cycle can be, for example, several months.

[0107] Version evaluation is a way of comparing the version to be released with the currently released and stably running version to obtain the effect of the search algorithm of the version to be released. It is carried out in daily business iterations and has a short execution cycle. It is a way of short-term quality improvement of the search algorithm.

[0108] Competitor evaluation, also known as competing product evaluation. Taking a music software as an example, it is to horizontally compare the search function of its own music software with that of competing products, score and record the search results of its own music software, and obtain relevant evaluation data after statistical analysis of the scoring and recording results, and promote the optimization and iteration of its own search algorithm by comparing with competing products.

[0109] In addition, the number of any elements in the accompanying drawings is for illustration rather than limitation, and any naming is only for distinction without any limiting meaning.

[0110] Reference will now be made in detail to several representative embodiments of the present disclosure to explain the principles and spirit of the present disclosure. Summary of the Invention

[0112] The inventor of the present invention has found that the search algorithms of existing software do not give good search results for every search term, and there are some cases where user needs are not met. Taking a music software as an example, for users, when performing a search operation, if the search results do not match the expectations, it will reduce their willingness to continue operating the music software, such as continuing to play, collect, etc., and they cannot quickly query the required content through the search function, losing the meaning of setting up the search function. Therefore, the accuracy of the search algorithm is very important for software applications.

[0113] Based on the above, the basic idea of the present disclosure is: to provide a method, device, computing device, and medium for evaluating a search algorithm, display a search algorithm evaluation interface, which includes a search function area and a search result evaluation area, obtain corresponding search results based on the search terms input in the search function area, display the search results, and output search algorithm evaluation data according to the evaluation operations on the search results in the search result evaluation area. According to the technical solutions of the embodiments of the present disclosure, on the one hand, a search algorithm evaluation interface can be provided for evaluators, enabling evaluators to evaluate the search results using this search algorithm evaluation interface and output the evaluation data of the evaluated search algorithm based on the evaluation operations. Through this evaluation data, the problems existing in the search algorithm can be pre-exposed, thereby guiding the update and iteration direction of the search algorithm, improving the accuracy and reliability of the search algorithm, and reducing the phenomenon of poor user experience; on the other hand, before the new version of the search algorithm is launched, the search effect of this search algorithm can be pre-known through the technical solutions of the embodiments of the present disclosure, providing a reference basis for whether to launch it.

[0114] After introducing the basic principles of the present disclosure, the various non-limiting embodiments of the present disclosure will be specifically introduced below.

[0115] Overview of Application Scenarios

[0116] It should be noted that the following application scenarios are only shown for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0117] Exemplarily, the technical solution of the embodiment of the present disclosure can be applied to the scenario of evaluating the search algorithm of a music software to know the search effect of the search algorithm. For example, for a music software that has been launched and released, its overall quality of the search algorithm can be tracked irregularly. Using the technical solution of the embodiment of the present disclosure, the evaluator can input search terms in the provided search algorithm evaluation interface, and then perform evaluation operations such as scoring the displayed search results and marking the satisfaction level. Based on this evaluation operation, evaluation data of the evaluated search algorithm can be obtained. The evaluator can know information such as the overall quality of the search algorithm, whether it can meet the user's expectations, the degree of meeting the expectations, and existing problems according to this evaluation data, so as to guide the update and iteration direction of the search algorithm.

[0118] For another example, for a new version of the music software or a newly developed search algorithm, the solution of the embodiment of the present disclosure can be used to compare the search results of, for example, the new version of the search algorithm to be launched with the search results of the currently launched and running search algorithm, obtain evaluation data of the new version of the search algorithm relative to the currently launched and running search algorithm, and use this evaluation data as a reference basis for whether the new version of the search algorithm is launched.

[0119] For yet another example, the solution of the embodiment of the present disclosure can be used to evaluate both the search function of its own music software and the search function of competing products, and make a horizontal comparison of the evaluation results, learn from each other's strengths and weaknesses through comparison with competitors, and promote the optimization and iteration of its own search algorithm.

[0120] Exemplary Method

[0121] Next, in combination with the above application scenarios, with reference to Figure 1 to describe the evaluation method of the search algorithm according to the exemplary embodiment of the present disclosure, this evaluation method may include steps S110 to S130. The execution subject of the evaluation method of the search algorithm provided by the embodiment of the present disclosure can be a device with computing and processing functions, such as a mobile phone, a computer, a server, etc. It should be noted that the above application scenarios are only shown for the convenience of understanding the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0122] Referring to Figure 1 as shown, in step S110, a search algorithm evaluation interface is displayed, and the search algorithm evaluation interface includes a search function area and a search result evaluation area.

[0123] In an exemplary embodiment of the present disclosure, the search algorithm evaluation interface can be determined based on the selected search algorithm evaluation type. Exemplarily, the selected search algorithm evaluation type may include at least one of the following: user experience evaluation, version evaluation, and competitor evaluation.

[0124] Taking the evaluation type of the selected search algorithm as user experience evaluation as an example, with reference to Figure 2 , a schematic diagram of a search algorithm evaluation interface corresponding to user experience evaluation according to some embodiments of the present disclosure is schematically shown. The search algorithm evaluation interface may include a search function area 21 and a search result evaluation area 22. The search function area 21 provides function controls for inputting search terms. For example, optional search terms may be presented in a list form in the search function area 21, and the user can input a search term by selecting a certain search term therein; for another example, an entry for inputting a search term may be provided through a function control for inputting a search term, and the user can directly input the term to be searched through this function control. The search function area 21 may also provide function controls for evaluation task information, such as Figure 2 's "Task Information" menu. The user can view the created evaluation tasks through this menu, select the required evaluation tasks for evaluation, or view the current task progress or evaluation results of a certain evaluation task by selecting a certain evaluation task. A function control for adding a new evaluation task may also be set in the "Task Information" menu. After selecting this function control, the process of creating an evaluation task can be entered. For example, an evaluation task configuration interface can be called up for the user to create an evaluation task. The search result evaluation area 22 may provide function controls for evaluating search results, such as mark controls for the satisfaction levels of each scoring dimension of the search results, and mark controls for the problem types corresponding to each scoring dimension, etc. Exemplarily, as Figure 2 shows, satisfaction mark controls 221 corresponding to each scoring dimension such as "Best Match", "Sorting", and "First Module" are provided in the search result evaluation area 22. The evaluator can mark the satisfaction level of the corresponding scoring dimension through the satisfaction mark control 221 according to the search results displayed in the search result display area 23. The "First Module" here is just a naming method and can refer to the first label module on the search result page. For example, Figure 2 's "You may be interested" module shown. Correspondingly, the "single song" can be called the "second module". It should be noted that this is only for illustration and does not limit the present disclosure. In actual applications, any required controls can be set according to the idea of the present disclosure, such as a problem type mark control for recording the problems existing in the search results, a mark control for recording the number of search results, etc.

[0125] For version evaluation and competitor evaluation, exemplarily, their search algorithm evaluation interfaces can be analogous to Figure 2, it can also include a search function area and a search result evaluation area. The search function area can provide function controls for inputting search terms, and the search result evaluation area can provide corresponding function controls for evaluating search results, such as scoring search results and recording satisfaction levels. The difference is that for version evaluation, the search results of the current evaluated version and the reference version can be displayed in the search result display area to facilitate comparison and evaluation. An exemplary display effect can be referred to Figure 3 , in Figure 3 , the search results of the current evaluated version (group name: t) and the reference version (group name: c) can be displayed in the search result display area 33 shown. Users can conduct evaluations in the search result evaluation area 32 based on the horizontal comparison of the search results. For example, compare the "You may be interested" module in the search results and record the comparison results; for competitor evaluation, the search results of its own music software and the search results of each competitor can be displayed in the search result display area for comparison and evaluation. An exemplary display effect can be referred to Figure 4 , in Figure 4 , the search results of its own music software (group name: t), the search results of competitor c1, and the search results of competitor c2 can be displayed in the search result display area 43 shown. Users can conduct a horizontal comparison of the search results and record the comparison results.

[0126] In some exemplary embodiments of the present disclosure, the search algorithm evaluation type can be selected when creating an evaluation task. Before performing step S110, the evaluation method of the search algorithm can further include the step of generating a search algorithm evaluation interface. Refer to Figure 5 , which can specifically include the following steps S510 to S530:

[0127] In step S510, an evaluation task configuration interface is displayed. The evaluation task configuration interface can include configuration controls for the evaluation type of the search algorithm to be evaluated and configuration controls for the tab pages to be evaluated.

[0128] In some exemplary embodiments of the present disclosure, the evaluation type can include at least one of user experience evaluation, version evaluation, and competitor evaluation. The evaluator can, according to the evaluation requirements, select or input the required evaluation type through the configuration control for the evaluation type of the search algorithm to be evaluated, such as selecting version evaluation. The search function page of the music software provides various types of tab pages, such as comprehensive, single song, singer, playlist, video, podcast, etc. The evaluator can set one or more tab pages to be evaluated through the configuration control for the tab pages to be evaluated.

[0129] In step S520, in response to the configuration operations on the configuration controls for the evaluation type and the tab pages to be evaluated, the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated are determined.

[0130] In step S530, a search algorithm evaluation interface for the target evaluation tag page is generated based on the target evaluation type and the target evaluation tag page.

[0131] For example, in the evaluation task configuration interface, if the evaluator selects the user experience evaluation through the configuration control of the evaluation type and selects the single song tag page through the configuration control of the tag page to be evaluated, then based on the selected user experience evaluation and the single song tag page, a search algorithm evaluation interface for the single song tag page can be generated. An exemplary display effect can be referred to Figure 2 . When the evaluator enters a search term in the search function area of the search algorithm evaluation interface, related single songs can be displayed in a certain order in the search result display area. The evaluator can perform evaluation operations such as marking the satisfaction level of the search results and recording the information of interest in the search result evaluation area.

[0132] In step S120, corresponding search results are obtained based on the search term entered in the search function area, and the search results are displayed.

[0133] The search algorithm of the music software can index songs, playlists, etc. After the evaluator enters a search term in the search function area, the search algorithm can perform intent analysis on the search term entered by the user, obtain a roughly sorted result from the data resource library according to the intent analysis result, and then adjust the result according to the set adjustment algorithm, such as adding the song popularity value, the new song attention index, etc., so as to screen out a certain number of search results for display.

[0134] In step S130, search algorithm evaluation data is output according to the evaluation operations on the search results in the search result evaluation area.

[0135] In some exemplary embodiments of the present disclosure, the search algorithm evaluation data can be determined based on the evaluation operations on the search results in the search result evaluation area by using the target evaluation rules, and then the search algorithm evaluation data is output. In an exemplary embodiment of the present disclosure, the evaluation operation may include a marking operation of the satisfaction level of the search results, and the target evaluation rules can reflect the scoring mechanism of the search results.

[0136] For each type of evaluation, one or more evaluation rules can be corresponding. In one implementation, one type of evaluation can correspond to one evaluation rule, and this evaluation rule can be used as the target evaluation rule for this type of evaluation. In another implementation, one type of evaluation can correspond to multiple evaluation rules, and one evaluation rule can be selected from these multiple evaluation rules as the target evaluation rule for this type of evaluation. For example, the evaluator can select the evaluation type of the search algorithm to be evaluated through the evaluation task configuration interface described in step S510. This type of evaluation can correspond to a default target evaluation rule. When the evaluator selects the evaluation type, the corresponding target evaluation rule is also selected. For another example, configuration controls for the evaluation rules corresponding to the evaluation type can also be provided in the evaluation task configuration interface. The evaluator can use this configuration control to select an evaluation rule as the target evaluation rule for the selected evaluation type.

[0137] In an exemplary embodiment of the present disclosure, the evaluation rule can be established in advance before creating the evaluation task, or can be established when creating the evaluation task when it is detected that the selected evaluation type has no corresponding evaluation rule. For example, when it is detected that the selected evaluation type has no corresponding evaluation rule, a thread for creating the evaluation rule is automatically started, or a prompt message indicating that the selected evaluation type has no corresponding evaluation rule is output, and the user decides whether to create the corresponding evaluation rule. When the user determines to create the corresponding evaluation rule, a thread for creating the evaluation rule is started.

[0138] Refer to Figure 6 , which schematically shows a method for creating an evaluation rule in an exemplary embodiment of the present disclosure. This method can include the following steps S610 to step S640:

[0139] In step S610, when an instruction for creating an evaluation rule is detected, an evaluation rule configuration interface is displayed. The evaluation rule configuration interface includes an evaluation type selection menu.

[0140] In step S620, in response to a selection operation on the evaluation type selection menu, the evaluation type of the search algorithm for the evaluation rule to be created is selected.

[0141] In step S630, an evaluation rule configuration sub-interface corresponding to the evaluation type of the search algorithm for the evaluation rule to be created and selected is displayed. This evaluation rule configuration sub-interface can include a scoring dimension configuration area, an evaluation level configuration area, and a question type configuration area. The scoring dimension configuration area can include configuration controls for scoring dimensions and corresponding scoring weights. The evaluation level configuration area can include configuration controls for the satisfaction level of search results and corresponding scoring values. The question type configuration area can include configuration controls for question types.

[0142] Such as Figure 7As shown, an evaluation rule configuration interface is exemplarily shown. The evaluation rule configuration interface includes an evaluation type selection menu 71. The evaluation type selection menu 71 provides optional evaluation types: user experience evaluation, version evaluation, and competitor evaluation. Users can select the search algorithm evaluation type for the evaluation rule to be created through the evaluation type selection menu 71. For example, if the user experience evaluation is selected, the evaluation rule configuration sub-interface corresponding to the user experience evaluation can be displayed in the preset area of the evaluation rule configuration interface. An exemplary display effect can be referred to Figure 7 . In Figure 7 In the evaluation rule configuration sub-interface shown, evaluators can configure the scoring dimensions and the corresponding scoring weights through the configuration controls for the scoring dimensions and the corresponding scoring weights in the scoring dimension configuration area 72. For example, increase or decrease the scoring dimensions, set the scoring weights of the scoring dimensions, etc.; evaluators can configure the satisfaction level of the search results and the corresponding scoring values through the configuration controls for the satisfaction level of the search results and the corresponding scoring values in the evaluation level configuration area 73. For example, increase or decrease the satisfaction level of the search results, set the scoring values of the satisfaction level of the search results, etc., and can also set the processing type corresponding to the satisfaction level of the search results. For example, for the case where the search results are completely satisfied, it can be set that the search results pass, and for the case where the search results are completely not satisfied, it can be set that the search results do not pass, etc.; evaluators can configure the optional problem types through the configuration controls for the problem types in the problem type configuration area 74. The problem types can be used to reflect the problems that exist when the search results do not meet the expectations. For example, the search results not meeting the expectations may be due to copyright problems, intent recognition problems, sorting problems, relevance problems, timeliness problems, single song problems, playlist problems, etc. Evaluators can select relevant problems from these problem types in the evaluation interface to mark the problem types according to the search results.

[0143] It should be noted that this is only for illustration and does not uniquely limit the present disclosure. The configuration items of the evaluation rules can be set according to the actual situation under the guidance of the basic idea of the present disclosure. For example, it can only include the scoring dimension configuration area 72 and the evaluation level configuration area 73, or can only include the problem type configuration area 74, etc. The present disclosure does not make special limitations on this.

[0144] For the version evaluation and the competitor evaluation, the configuration items in the corresponding evaluation rule configuration sub-interfaces can be analogous to the user experience evaluation, which will not be elaborated here.

[0145] In step S640, in response to the configuration operation for the evaluation rule configuration sub-interface, an evaluation rule corresponding to the selected search algorithm evaluation type of the evaluation rule to be created is generated.

[0146] Taking the user experience evaluation as an example, an exemplary configuration result of its evaluation rule can be referred toFigure 7 , according to Figure 7 the configured results shown, the evaluation rules corresponding to the user experience evaluation can be generated. The evaluation rules can include the corresponding relationship between the scoring dimensions and the scoring weights, the corresponding relationship between the satisfaction level of the search results and the scoring values, and can also include the selectable question types. After the evaluator performs the evaluation operation on the search results in the search result evaluation area, the evaluation data of the search algorithm can be determined according to the evaluation operation and the corresponding evaluation rules.

[0147] In an exemplary embodiment of the present disclosure, the search result evaluation area may include marking controls for the satisfaction levels of each scoring dimension of the search results. Referring to Figure 8 shown, the method for determining the evaluation data of the search algorithm using the target evaluation rules may include the following steps S810 to S830:

[0148] In step S810, for each search term in each evaluation, in response to the marking operation on the marking control for the satisfaction level of each scoring dimension of the search results, the scoring values of each scoring dimension corresponding to each search term are determined using the target evaluation rules.

[0149] In an exemplary embodiment, the scoring values of each scoring dimension corresponding to each search term can be determined by the following method: in response to the marking operation on the marking control for the satisfaction level of each scoring dimension of the search results, determine the satisfaction level of the search results for each scoring dimension; based on the determined satisfaction level of the search results for each scoring dimension, match the scoring values of each scoring dimension from the target evaluation rules.

[0150] Taking the user experience evaluation as an example, referring to Figure 2 shown, the evaluator can evaluate according to the search results displayed in the search result display area 23 with reference to certain criteria, such as new songs need to be ranked in the top three, those without copyright cannot be ranked in the top three, etc., and use the satisfaction marking control 221 to mark the satisfaction levels of scoring dimensions such as "Best Match", "Sorting", "First Module", etc. For example, according to the search results displayed in the search result display area 23, mark the satisfaction level of "Best Match" as "Completely Unsatisfied", mark the satisfaction level of "Sorting" as "Basically Satisfied", etc. After completing the marking of the scoring dimensions and determining, according to this marking, in combination with Figure 7 the evaluation rules shown, match the scoring value of "Best Match" as -2, match the scoring value of "Sorting" as 1, etc.

[0151] In step S820, after completing the evaluations for the set number of evaluations, statistical analysis is performed on the scoring values of each scoring dimension corresponding to all search terms to obtain the evaluation data of the search algorithm.

[0152] Exemplarily, referring toFigure 2 As shown, all search terms used for evaluation can be displayed in a list in the search function area 21. After evaluating all these search terms, it is considered that one evaluation is completed. For any evaluation type, one evaluation or multiple evaluations can be performed. The number of evaluations can be the default number or set by the user, such as when creating an evaluation task. For example, the above evaluation task configuration interface can also include a configuration control for the number of evaluations, and the user can set the number of evaluations through this configuration control for the number of evaluations.

[0153] Exemplarily, the search terms displayed in the search function area 21 can be provided by an evaluation sample. This evaluation sample can be a pre-set evaluation sample. For any evaluation type, this evaluation sample is used to generate the search terms displayed in the search function area 21, or there can be a corresponding evaluation sample for each evaluation type. After the evaluator selects the evaluation type through the configuration control for the evaluation type of the search algorithm to be evaluated, the evaluation sample corresponding to this evaluation type is also selected. The evaluation sample can also be configured by the evaluator, such as when creating an evaluation task. For example, the above evaluation task configuration interface can also include a configuration control for the evaluation sample, and the user can select the evaluation sample through this configuration control for the evaluation sample. The terms in the evaluation sample can be collected online. For example, 200 terms are generated as the evaluation sample by extracting popular terms, moderately popular terms, and unpopular terms in a ratio of 5:3:2; among them, the popular terms can be the terms with a click-through rate ranking in the top 10%, the moderately popular terms can be the terms with a click-through rate ranking between 40% and 60%, and the unpopular terms can be the terms with a click-through rate ranking in the bottom 10%. The content and values here are only for exemplary illustration and are not used to limit the present disclosure.

[0154] For example, Figure 9 Schematically shows an exemplary evaluation task configuration interface of an embodiment of the present disclosure. The evaluation task configuration interface can also include a configuration control 93 for the operating environment. The user can set the environment in which the evaluation runs through this configuration control 93 for the operating environment. This operating environment can be an online environment or a test environment. The search algorithm can be released online after passing the evaluation in the test environment. In Figure 9In the evaluation task configuration interface shown, the user can set the name of the evaluation task to be created through the configuration control 91 of the evaluation task name, can set the evaluation type for evaluation through the configuration control 92 of the evaluation type of the search algorithm to be evaluated. For example, if the set type is "version evaluation", the user can select the samples required for evaluation through the configuration control 95 of the evaluation samples to provide optional search terms, can set the evaluation rules corresponding to the selected evaluation type through the configuration control 96 of the evaluation rules, can set the tab page for evaluation through the configuration control 97 of the tab page to be evaluated, and can set the required number of evaluations through the configuration control 98 of the number of evaluations. Additionally, for the same search algorithm, multiple groups of experimental evaluations may be conducted, and the user can also set the experimental group number through the experimental group configuration control 94 in the evaluation task configuration interface to distinguish the evaluation results of each group. Correspondingly, the process of generating the search algorithm evaluation interface can be based on Figure 9 the content configured in the evaluation task configuration interface shown.

[0155] In an exemplary embodiment of the present disclosure, referring to Figure 10 shown, the score values of each scoring dimension corresponding to all search terms can be statistically analyzed through the following steps S821 to S824 to obtain search algorithm evaluation data:

[0156] In step S821, for each search term, match the scoring weights corresponding to each scoring dimension from the target evaluation rules.

[0157] Taking the user experience evaluation as an example, referring to Figure 7 shown, the target evaluation rules include the corresponding relationship between the scoring dimensions and the scoring weights. For example, the scoring weight corresponding to "best match" can be matched as 0.2, the scoring weight corresponding to "sorting" can be matched as 0.2, the scoring weight corresponding to "the first module" can be matched as 0.3, the scoring weight corresponding to "the second module" can be matched as 0.2, etc. from the corresponding relationship between the scoring dimensions and the scoring weights.

[0158] In step S822, calculate the target score according to the score values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights.

[0159] The search algorithm evaluation type can include at least one of user experience evaluation, version evaluation, and competitor evaluation.

[0160] Exemplarily, for the user experience evaluation, calculating the target score according to the score values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights can include the following process: calculate the weighted total score value of all search terms according to the score values of each scoring dimension of the search results corresponding to each search term and the scoring weights; calculate the average score value of each search term according to the weighted total score value to obtain the target score.

[0161] Exemplarily, for version evaluation and competitor evaluation, calculating the target score based on the score values of each scoring dimension and the matched scoring weights of the search results corresponding to each search term may include the following process: calculating the weighted total score value of all search terms based on the score values of each scoring dimension and the scoring weights of the search results corresponding to each search term to obtain the target score.

[0162] For example, assume that there are two search terms in total: search term A and search term B; three scoring dimensions are set in the target evaluation rule: scoring dimension 1 to scoring dimension 3; the scoring weights corresponding to scoring dimension 1 to scoring dimension 3 are p1, p2, and p3 respectively; for search term A, assume that the score values of scoring dimension 1 to scoring dimension 3 are 0, 2, and 1 respectively; for search term B, assume that the score values of scoring dimension 1 to scoring dimension 3 are -1, 1, and -2 respectively. Then the weighted total score value M of all search terms = (0×p1 + 2×p2 + 1×p3) + (-1×p1 + 1×p2 - 2×p3), and the average score value of each search term is M / 2.

[0163] In step S823, count the number of each search result satisfaction level corresponding to each scoring dimension respectively to obtain the search result satisfaction distribution table.

[0164] Exemplarily, taking the user experience evaluation as an example, assume that the scoring dimensions include "best match", "sorting", "first module", and "second module", and the search result satisfaction levels include five levels: "fully satisfied", "basically dissatisfied", "fully dissatisfied", "basically satisfied", and "unclear intention"; assume that there are 5 search terms in total: term A, term B, term C, term D, and term E, and two evaluations are conducted. The results of the two evaluations can be referred to Table 1 below:

[0165] Table 1

[0166]

[0167] Statistical analysis of the evaluation results shown in Table 1 reveals that for the scoring dimension "Best Match", the numbers of the corresponding satisfaction levels of search results "Fully Satisfied", "Basically Satisfied", "Basically Unsatisfied", "Fully Unsatisfied", and "Unclear Intent" (other scoring dimensions are described in this order) are 6, 2, 2, 0, and 0 respectively; for the scoring dimension "Sorting", the numbers of the corresponding satisfaction levels of search results are 2, 2, 2, 1, and 3 respectively; for "the first module", the numbers of the corresponding satisfaction levels of search results are 3, 5, 0, 1, and 1 respectively; for "the second module", the numbers of the corresponding satisfaction levels of search results are 4, 4, 0, 2, and 0 respectively. The search result satisfaction distribution table obtained according to this statistical result can be as shown in Table 2:

[0168] Table 2

[0169]

[0170] It should be noted that this is only an example for illustration and is not used to limit the present disclosure. The scoring dimension, the satisfaction level of search results, etc. can be determined according to actual applications without violating the basic idea of the present disclosure.

[0171] Exemplarily, for version evaluation, the numbers of the corresponding satisfaction levels of search results for each scoring dimension can be counted by analogy with user experience evaluation to obtain the corresponding search result satisfaction distribution table. For competitor evaluation, by analogy with user experience evaluation, statistics can be separately performed on the current evaluation version and each competitor version to obtain the search result satisfaction distribution tables respectively corresponding to the current evaluation version and each competitor version.

[0172] In step S824, search algorithm evaluation data is generated based on the target score and the search result satisfaction distribution table.

[0173] Exemplarily, for user experience evaluation, the average score value obtained in step S822 and the search result satisfaction distribution table shown in Table 2 obtained in step S823 can be combined to generate the corresponding search algorithm evaluation data. For version evaluation, the weighted total score value obtained in step S822 and the search result satisfaction distribution table can be combined to generate the corresponding search algorithm evaluation data. For competitor evaluation, search algorithm evaluation data can be generated based on the target scores and the search result satisfaction distribution tables respectively corresponding to the current evaluation version and each competitor version, where the target score can be the weighted total score value.

[0174] In an exemplary embodiment of the present disclosure, for version evaluation, the search effect of the evaluated search algorithm can also be reflected by the winning rate of the current evaluation version relative to the reference version. Refer to Figure 11As shown, the evaluation method of the search algorithm may further include the step of obtaining the winning rate of the current evaluation version relative to the reference version, which may specifically include the following steps S1110 to S1160:

[0175] In step S1110, based on the search result satisfaction distribution table, obtain the number of each search result satisfaction level corresponding to each scoring dimension.

[0176] Exemplarily, the search result satisfaction distribution table for version evaluation may be analogous to Table 2 above. According to the search result satisfaction distribution table for version evaluation, the number of each search result satisfaction level corresponding to each scoring dimension of version evaluation can be obtained.

[0177] In step S1120, for each scoring dimension, calculate the average value of the number of the first search result satisfaction level and the number of the second search result satisfaction level corresponding to each scoring dimension.

[0178] In the embodiments of the present disclosure, the search result satisfaction level may be established according to whether the current evaluation version and the reference version can both meet the search expectation. For example, the first search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version meet the search expectation, the second search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version cannot meet the search expectation, and the third search result satisfaction level is the search result satisfaction level indicating that the current evaluation version can meet the search expectation while the reference version cannot meet the search expectation.

[0179] In step S1130, calculate the sum of the number of the third search result satisfaction level and the average value corresponding to each scoring dimension to obtain the total number of wins.

[0180] In step S1140, obtain the total number of all search result satisfaction levels corresponding to each scoring dimension.

[0181] In step S1150, calculate the ratio of the total number of wins to the total number to obtain the winning rate of each scoring dimension of the current evaluation version relative to the reference version.

[0182] In step S1160, output the winning rate. This winning rate can be displayed as part of the search algorithm evaluation data.

[0183] For example, an exemplary search result satisfaction distribution table for version evaluation is shown in Table 3 below, including four search result satisfaction levels: "all not satisfied", "lost", "won", and "all satisfied". Among them, "all satisfied", "all not satisfied", and "won" correspond to the first, second, and third search result satisfaction levels respectively as described above.

[0184] Table 3

[0185] Satisfaction Level of Search Results All Not Satisfied Defeated Won All Satisfied Best Match 14 23 22 41

[0186] According to the content of Table 3, the winning rate can be calculated as 49.5% using the formula "(the number of wins + (the number of all satisfied + the number of all not satisfied) / 2) / the total number", where the total number is the total number of all search result satisfaction levels.

[0187] The quality of the search algorithm of the current evaluation version can be evaluated by the winning rate. When the winning rate exceeds the set reference value (such as 55%), it indicates that the search algorithm of the current evaluation version is better than the reference version. The search algorithm of the current evaluation version can be used to replace the search algorithm of the reference version to improve the accuracy and reliability of the search results, thereby enhancing the user's search experience.

[0188] In step S830, display the search algorithm evaluation data.

[0189] The search algorithm evaluation data can be generated based on the target score and the search result satisfaction distribution table. The search result satisfaction distribution table can reflect the search expectation situation of each scoring dimension of the search results, and the target score can reflect the quality of the search algorithm of the current evaluation.

[0190] For user experience evaluation, when the target score exceeds the set score reference value, it indicates that the search algorithm of the current evaluation can meet the user's search expectation, and the current user experience evaluation passes. For version evaluation, the target score of the current evaluation version can be compared with the target score of the reference version. If the target score of the current evaluation version is higher than the target score of the reference version, it indicates that the search algorithm of the current evaluation version is better than the reference version. The search algorithm of the current evaluation version can be used to replace the search algorithm of the reference version, or increase the usage traffic of the search algorithm of the current evaluation version to improve the accuracy and reliability of the search results, thereby enhancing the user's search experience. For competitor evaluation, the target score of the current evaluation version can be compared with the target score of each competitor version. The one with a higher target score indicates a better search algorithm, which can be used to guide the optimization of the current evaluation version.

[0191] In some exemplary embodiments of the present disclosure, evaluation result information indicating whether the search algorithm passes the evaluation can also be generated based on the target score, and then the evaluation result information can be output. For example, when conducting a user experience evaluation, when the target score exceeds the set score reference value, evaluation passed result information is generated and output. In this way, it can be intuitively known that the current user experience evaluation passes.

[0192] In the above method steps, the search effect of the search algorithm can be evaluated using the target score. In some exemplary embodiments of the present disclosure, statistical analysis can also be performed on the problems of the search algorithm reflected by the search results, and the analysis results can be presented to the user.

[0193] Exemplarily, as shown in Figure 2 the search result evaluation area 22 may also include marking controls 222 for problem types corresponding to each scoring dimension. The evaluator can mark the problems reflected by the search results through the marking controls. For example, there are copyright problems in the search results, the sorting of the search results has problems, the problem of unclear intent recognition exists in the search results, the single module in the search results cannot meet the expectations, the relevance of the search results is not high, etc. These problem types can be enumerated, associated with the marking controls for problem types corresponding to each scoring dimension, and optional problem types are provided. The evaluator can select the relevant problem types through the marking control 222 to achieve the marking of the problem types. Alternatively, the user can also directly input the relevant problem types through the marking controls for problem types corresponding to each scoring dimension. Correspondingly, as shown in Figure 12 the method for obtaining search algorithm evaluation data may further include the following steps S1210 to step S1230:

[0194] In step S1210, for each search term in each evaluation, in response to the marking operation on the marking control for the problem type corresponding to each scoring dimension, determine the problem types of each scoring dimension of the search results corresponding to the search term.

[0195] For example, as shown in Figure 2As shown, for each scoring dimension such as "Best Match", "Sorting", and "First Module", there is a corresponding tagging control 222 for the question type. According to the search results displayed in the search result display area 23, for example, if the evaluator finds that the "Best Match" scoring dimension of the search results does not meet the search expectation and there is a copyright issue, the question type can be marked as "Copyright Issue" through the tagging control 222 corresponding to the "Best Match" question type, and then it can be determined that the question type corresponding to "Best Match" is "Copyright Issue". For each scoring dimension, the tagging operation of the question type can be performed through the corresponding tagging control 222 for its question type, and the question types of each scoring dimension can be obtained based on the tagging operation. It can be understood that for each scoring dimension, the corresponding question may be one or more, and one or more question types can be marked through the tagging control 222 for the question type.

[0196] In step S1220, after completing the evaluation for the set number of evaluations, statistical analysis is performed on the question types of each scoring dimension of the search results corresponding to all search terms to obtain a problem distribution table.

[0197] For any evaluation type, one evaluation or multiple evaluations can be performed. The number of evaluations can be the default number or set by the user, such as when creating an evaluation task.

[0198] In an example implementation, for each scoring dimension of the search results corresponding to all search terms, the number of each question type corresponding to each scoring dimension can be separately counted to obtain a problem distribution table.

[0199] For example, taking the user experience evaluation as an example, assume that the scoring dimensions include "Best Match", "Sorting", "First Module", and "Second Module", and the marked question types include "Intent Recognition Issue", "Copyright Issue", "Sorting Issue", and "Relevance Issue"; assume there are a total of 5 search terms: Term A, Term B, Term C, Term D, and Term E, and a total of two evaluations are performed. For each search term in each evaluation, the question type corresponding to each scoring dimension can be marked. The question type corresponding to each scoring dimension may be one or more. After obtaining the evaluation results, the process of counting the number of each question type corresponding to each scoring dimension based on the evaluation results can be analogous to the process of counting the number of satisfaction levels of each search result in step S823, which will not be elaborated here. The problem distribution table obtained through statistical analysis may be, for example, Table 4 as follows:

[0200] Table 4

[0201]

[0202]

[0203] It should be noted that the examples provided here are for illustrative purposes only and are not used to limit the present disclosure. Without departing from the basic idea of the technical solution of the present disclosure, the scoring dimensions, question types, and actual statistical results can all be obtained according to actual applications.

[0204] In step S1230, the question distribution table is added to the search algorithm evaluation data.

[0205] The question distribution table can also be regarded as part of the search algorithm evaluation data and, together with the above-mentioned target scores and the satisfaction degree distribution table of search results, evaluate the quality of the search algorithm. The question distribution table can intuitively reflect the problems existing in each scoring dimension of the search results. For example, the "best match" dimension of the search results fails to meet expectations mainly due to copyright issues and intent recognition problems. Based on this, targeted improvements can be made to the search algorithm to improve its accuracy and reliability, and further enhance the search experience of music software users.

[0206] In an exemplary embodiment of the present disclosure, after obtaining the search algorithm evaluation data, the evaluation method of the search algorithm may further include the step of generating a search algorithm evaluation report based on the search algorithm evaluation data. The search algorithm evaluation report can be output, such as displayed or printed.

[0207] Exemplarily, the target scores obtained in step S822 and the satisfaction degree distribution table of search results obtained in step S823 can be combined in a set format to generate a search algorithm evaluation report. For example, a column of target scores can be added to the satisfaction degree distribution table of search results, or the target scores can be used as a separate part to be associated with the satisfaction degree distribution table of search results to obtain the search algorithm evaluation report. The display form of the satisfaction degree distribution table of search results in the search algorithm evaluation report can be, for example, Table 2 and Table 3 above. The present disclosure does not make special limitations on the specific display form of the satisfaction degree distribution table of search results in the search algorithm evaluation report.

[0208] In one implementation, the search algorithm evaluation report may further include the question distribution table as described in step S1220. Correspondingly, the target scores, the satisfaction degree distribution table of search results, and the question distribution table can be associated to obtain a search algorithm evaluation report in a set format and then presented to the user. For example, an exemplary display effect of the search algorithm evaluation report for user experience evaluation can be referred to Figure 13 。

[0209] In one implementation, the search algorithm evaluation report may further include evaluation result information indicating whether the search algorithm passes the evaluation generated based on the target scores. Through this evaluation result information, it can be directly known whether the currently evaluated search algorithm passes.

[0210] In one implementation, for version evaluation, the search algorithm evaluation report may further include the winning rate of the current evaluated version relative to the reference version. Correspondingly, the target score, the satisfaction degree distribution table of search results, and the winning rate can be associated to obtain a search algorithm evaluation report in a set format, and then presented to the user.

[0211] In one implementation, for competitor evaluation, the search algorithm evaluation data of the current evaluated version and each competitor version can be compared in the same dimension. For example, in combination with Figure 4 As shown, the "single song" modules in the three results are compared, and then the results are presented in the search algorithm evaluation report in a comparative form. For example, the report area can be divided according to the comparison dimension, and the target scores corresponding to the current evaluated version and each competitor version are placed in the same area, and the problem distribution tables corresponding to them are placed in the same area, etc. Refer to Figure 14 As shown, an exemplary search algorithm evaluation report for competitor evaluation is schematically shown, and the target score, the score distribution diagram, and the problem distribution diagram can be respectively presented in a comparative manner in a set area. The score distribution diagram among them can be output by comparing in the form of score proportion. The score proportion can be, for example, the proportion of the number of each score value among all the score values corresponding to the search terms. For example, the number of each search result satisfaction level can be obtained through the satisfaction degree distribution table of search results. Each search result satisfaction level corresponds to a score value. For example, the four levels of "fully satisfied", "basically satisfied", "completely dissatisfied", and "unclear intention" respectively correspond to the score values 2, 1, -2, and 0. In this way, the number of each score value can be obtained, and then the proportion of the number of each score value in the total number of score values can be calculated to obtain the score proportion. Of course, the score proportion can also be in other forms. For example, the score values can be divided into three categories: good, general, and poor. The score values higher than the first threshold are classified into the good category, the score values lower than the second threshold are classified into the poor category, and the rest are classified into the general category, and the proportion of each category of score values is obtained.

[0212] Exemplarily, the score distribution diagram may further include a comparison diagram after comparison according to the score proportion. Exemplarily, the problem distribution diagram can be output by comparing in the form of the proportion of problem types. The search algorithm evaluation report is output in a comparative format, which can intuitively reflect the advantages and disadvantages of the current evaluated version relative to each competitor version, so as to guide the improvement of the search algorithm and improve the accuracy and reliability of the search algorithm.

[0213] It should be noted that the format shown in the search algorithm evaluation report is only for illustration, and in actual applications, it can also be other formats. The present disclosure does not make special limitations on this.

[0214] Exemplary Medium

[0215] After introducing the methods of the exemplary embodiments of the present disclosure, next, the media of the exemplary embodiments of the present disclosure will be described.

[0216] In some possible embodiments, various aspects of the present disclosure may also be implemented as a medium having program code stored thereon, which is used to implement the steps in the evaluation method of the search algorithm according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification when the program code is executed by a processor of a device.

[0217] In some possible embodiments, when the processor of the device executes the program code, it may be used to implement the following steps: Step S110, display a search algorithm evaluation interface, where the search algorithm evaluation interface includes a search function area and a search result evaluation area; Step S120, obtain corresponding search results based on the search terms input in the search function area and display the search results; Step S130, output search algorithm evaluation data according to the evaluation operations on the search results in the search result evaluation area.

[0218] Reference Figure 15 As shown, a program product 1500 for implementing the above data processing method according to an embodiment of the present disclosure is described, which may be a portable compact disc read-only memory and includes program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0219] It should be noted that: the above medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disc read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0220] The readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0221] The program code contained on a readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical fiber cables, radio frequency signals, etc., or any suitable combination of the above.

[0222] The program code for performing the operations of the present disclosure can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network or a wide area network, or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0223] Exemplary Device

[0224] After introducing the medium of the exemplary embodiments of the present disclosure, next, reference is made to Figure 16 the evaluation device of the search algorithm of the exemplary embodiments of the present disclosure will be described.

[0225] Referring to Figure 16 As shown, the evaluation device 1600 of the search algorithm includes: an evaluation interface display module 1610 for displaying an evaluation interface of the search algorithm, which includes a search function area and a search result evaluation area; a search module 1620 for obtaining corresponding search results based on the detected search terms input in the search function area and displaying the search results; an evaluation module 1630 for outputting search algorithm evaluation data according to the detected evaluation operations on the search results in the search result evaluation area.

[0226] In some example embodiments of the present disclosure, the evaluation device 1600 of the search algorithm further includes: a report generation module for generating a search algorithm evaluation report according to the search algorithm evaluation data.

[0227] In some exemplary embodiments of the present disclosure, the evaluation device 1600 of the search algorithm further includes: an evaluation task display module for displaying an evaluation task configuration interface, where the evaluation task configuration interface includes configuration controls for the evaluation type of the search algorithm to be evaluated and configuration controls for the tab page to be evaluated; a determination module for determining the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated in response to configuration operations on the configuration controls for the evaluation type and the tab page to be evaluated; and an evaluation interface generation module for generating a search algorithm evaluation interface for the target evaluation tab page based on the target evaluation type and the target evaluation tab page.

[0228] In some exemplary embodiments of the present disclosure, the evaluation task configuration interface further includes at least one of the following: a configuration control for the evaluation rule corresponding to the evaluation type, a configuration control for the evaluation sample, and a configuration control for the number of evaluations.

[0229] In some exemplary embodiments of the present disclosure, the evaluation module 1630 is specifically configured to determine search algorithm evaluation data using the target evaluation rule based on an evaluation operation on the search results in the search result evaluation area, and output the search algorithm evaluation data.

[0230] In some exemplary embodiments of the present disclosure, the evaluation operation may include a marking operation for the satisfaction degree of the search results, and the target evaluation rule reflects the scoring mechanism of the search results.

[0231] In some exemplary embodiments of the present disclosure, the search algorithm evaluation interface is determined based on the selected search algorithm evaluation type, and the selected search algorithm evaluation type may include at least one of the following: user experience evaluation, version evaluation, and competitor evaluation.

[0232] In some exemplary embodiments of the present disclosure, the search result evaluation area may include marking controls for the satisfaction degree of each scoring dimension of the search results; the evaluation module 1630 may include: a scoring value determination unit for, for each search term in each evaluation, determining the scoring values of the respective scoring dimensions corresponding to the search term using the target evaluation rule in response to a marking operation on the marking controls for the satisfaction degree of each scoring dimension of the search results; a first analysis unit for statistically analyzing the scoring values of the respective scoring dimensions corresponding to all search terms after completing the evaluations for the set number of evaluations to obtain search algorithm evaluation data; and a display unit for displaying the search algorithm evaluation data.

[0233] In some exemplary embodiments of the present disclosure, the scoring value determination unit is specifically configured to: determine the satisfaction degree level of the search results for each scoring dimension in response to a marking operation on the marking controls for the satisfaction degree of each scoring dimension of the search results; and match the scoring values of the respective scoring dimensions from the target evaluation rule based on the determined satisfaction degree levels of the search results for each scoring dimension.

[0234] In some exemplary embodiments of the present disclosure, the first analysis unit is specifically configured to: for each search term, match the scoring weights corresponding to each scoring dimension from the target evaluation rules; calculate a target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights; respectively count the number of search results satisfaction levels corresponding to each scoring dimension to obtain a search results satisfaction distribution table; and generate search algorithm evaluation data based on the target score and the search results satisfaction distribution table.

[0235] In some exemplary embodiments of the present disclosure, the evaluation module 1630 may further include: an evaluation result information generation unit, configured to generate evaluation result information indicating whether the search algorithm passes the evaluation based on the target score; and an evaluation result information output unit, configured to output the evaluation result information.

[0236] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type may include user experience evaluation. When calculating the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights, the first analysis unit is specifically configured to: calculate the weighted total scoring value of all search terms according to the scoring values of each scoring dimension of the search results corresponding to each search term and the scoring weights; and calculate the average scoring value of each search term according to the weighted total scoring value to obtain the target score.

[0237] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type may include version evaluation or competitor evaluation. When calculating the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights, the first analysis unit is specifically configured to: calculate the weighted total scoring value of all search terms according to the scoring values of each scoring dimension of the search results corresponding to each search term and the scoring weights to obtain the target score.

[0238] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type includes competitor evaluation. The first analysis unit is specifically configured to generate search algorithm evaluation data based on the target scores and the search results satisfaction distribution tables respectively corresponding to the current evaluation version and each competitor version.

[0239] In some exemplary embodiments of the present disclosure, the selected search algorithm evaluation type may include version evaluation. The first analysis unit may further be used to: obtain the number of search result satisfaction levels corresponding to each scoring dimension based on the search result satisfaction degree distribution table; for each scoring dimension, calculate the average value of the number of the first search result satisfaction levels corresponding to each scoring dimension and the number of the second search result satisfaction levels; calculate the sum of the number of the third search result satisfaction levels corresponding to each scoring dimension and the average value to obtain the total number of wins; obtain the total number of all search result satisfaction levels corresponding to each scoring dimension; calculate the ratio of the total number of wins to the total number, to obtain the winning rate of the scoring dimension of the current evaluation version relative to the reference version; output the winning rate; wherein, the first search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version meet the search expectation, the second search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version do not meet the search expectation, and the third search result satisfaction level is the search result satisfaction level indicating that the current evaluation version can meet the search expectation while the reference version cannot meet the search expectation.

[0240] In some exemplary embodiments of the present disclosure, the search result evaluation area may further include a marking control for the problem type corresponding to each scoring dimension; the evaluation module 1630 may further include: a problem type determination unit, configured to, for each search term in each evaluation, in response to a marking operation on the marking control for the problem type corresponding to each scoring dimension, determine the problem type of each scoring dimension of the search result corresponding to the search term; a second analysis unit, configured to, after completing the evaluation for the set number of evaluation times, perform statistical analysis on the problem types of each scoring dimension of the search results corresponding to all search terms, obtain a problem distribution table, and add the problem distribution table to the search algorithm evaluation data.

[0241] In some exemplary embodiments of the present disclosure, the second analysis unit is specifically configured to, for each scoring dimension of the search results corresponding to all search terms, respectively count the number of each problem type corresponding to each scoring dimension to obtain a problem distribution table.

[0242] In some exemplary embodiments of the present disclosure, the evaluation device 1600 of the search algorithm may further include:

[0243] A first evaluation rule display module, configured to display an evaluation rule configuration interface when detecting an instruction to create an evaluation rule, and the evaluation rule configuration interface includes an evaluation type selection menu;

[0244] An evaluation type determination module, configured to, in response to a selection operation on the evaluation type selection menu, select the search algorithm evaluation type for which the evaluation rule to be created is to be determined;

[0245] The second evaluation rule display module is used to display the evaluation rule configuration sub-interface corresponding to the search algorithm evaluation type of the to-be-created evaluation rule selected, and the evaluation rule configuration sub-interface includes a scoring dimension configuration area, an evaluation level configuration area, and a question type configuration area. The scoring dimension configuration area includes configuration controls for scoring dimensions and corresponding scoring weights. The evaluation level configuration area includes configuration controls for the satisfaction level of search results and corresponding scoring values. The question type configuration area includes configuration controls for question types;

[0246] The evaluation rule generation module is used to generate an evaluation rule corresponding to the search algorithm evaluation type of the to-be-created evaluation rule selected in response to a configuration operation on the evaluation rule configuration sub-interface.

[0247] Since each functional module of the evaluation device for the search algorithm in the exemplary embodiments of the present disclosure corresponds to the steps in the exemplary embodiments of the above-mentioned search algorithm evaluation method, for the details not disclosed in the device embodiments of the present disclosure, please refer to the embodiments of the above-mentioned search algorithm evaluation method of the present disclosure.

[0248] Exemplary Computing Device

[0249] After introducing the methods, media, and devices of the exemplary embodiments of the present disclosure, next, a computing device according to another exemplary embodiment of the present disclosure will be introduced.

[0250] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0251] In some possible embodiments, the computing device according to the embodiments of the present disclosure may at least include at least one processor and at least one memory. Among them, the memory stores program code, and when the program code is executed by the processor, the processor executes the steps in the search algorithm evaluation method according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processor can execute the steps shown in Figure 1 : Step S110, display a search algorithm evaluation interface, where the search algorithm evaluation interface includes a search function area and a search result evaluation area; Step S120, obtain corresponding search results based on the search terms input in the search function area and display the search results; Step S130, output search algorithm evaluation data according to the evaluation operation on the search results in the search result evaluation area.

[0252] Next, refer toFigure 17 Describe the electronic device 1700 according to an exemplary embodiment of the present disclosure. Figure 17 The illustrated electronic device 1700 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present disclosure.

[0253] As Figure 17 shown, the electronic device 1700 is presented in the form of a general-purpose computing device. The components of the electronic device 1700 may include, but are not limited to: at least one of the above-mentioned processing units 1710, at least one of the above-mentioned storage units 1720, and a bus 1730 that connects different system components (including the storage unit 1720 and the processing unit 1710).

[0254] The bus 1730 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the multiple bus architectures.

[0255] The storage unit 1720 may include a readable medium in the form of volatile memory, such as RAM (Random Access Memory) 1721 and / or cache memory 1722, and may further include ROM (Read-Only Memory) 1723.

[0256] The storage unit 1720 may further include a program / utility 1725 having a set (at least one) of program modules 1724. Such program modules 1724 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0257] The electronic device 1700 can also communicate with one or more external devices 1740 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 1700, and / or communicate with any device that enables the electronic device 1700 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 1750. The electronic device 1700 further includes a display unit 1770, which is connected to the input / output (I / O) interface 1750 for display. Moreover, the electronic device 1700 can also communicate with one or more networks (such as a local area network, a wide area network, and / or a public network, such as the Internet) through the network adapter 1760. As shown in the figure, the network adapter 1760 communicates with other modules of the electronic device 1700 through the bus 1730. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 1700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems, etc.

[0258] It should be noted that although several units or subunits of the music popularity prediction device are mentioned in the above detailed description, this division is only exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0259] In addition, although the operations of the method of the present disclosure are described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.

[0260] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division of each aspect does not mean that the features in these aspects cannot be combined for benefits. This division is only for the convenience of expression. The present disclosure aims to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for evaluating a search algorithm, characterized in that, Including: Display a search algorithm evaluation interface, which includes a search function area and a search result evaluation area in the search algorithm evaluation interface; The search result evaluation area includes marking controls for the satisfaction levels of each scoring dimension of the search results; When a command to create an evaluation rule is detected, display an evaluation rule configuration interface, which includes an evaluation type selection menu in the evaluation rule configuration interface; in response to a selection operation on the evaluation type selection menu, select the search algorithm evaluation type for the evaluation rule to be created; display an evaluation rule configuration sub-interface corresponding to the selected search algorithm evaluation type for the evaluation rule to be created, and the evaluation rule configuration sub-interface includes a scoring dimension configuration area, an evaluation level configuration area, and a question type configuration area. The scoring dimension configuration area includes configuration controls for scoring dimensions and corresponding scoring weights. The evaluation level configuration area includes configuration controls for the satisfaction levels of search results and corresponding scoring values. The question type configuration area includes configuration controls for question types; in response to a configuration operation on the evaluation rule configuration sub-interface, generate an evaluation rule corresponding to the selected search algorithm evaluation type for the evaluation rule to be created; Obtain corresponding search results based on the search terms entered in the search function area and display the search results; For each search term in each evaluation, in response to a marking operation on the marking controls for the satisfaction levels of each scoring dimension of the search results, determine the satisfaction levels of the search results for each scoring dimension; based on the determined satisfaction levels of the search results for each scoring dimension, match the scoring values for each scoring dimension from the target evaluation rule; after completing the evaluation for the set number of evaluation times, perform statistical analysis on the scoring values for each scoring dimension corresponding to all search terms to obtain search algorithm evaluation data; display the search algorithm evaluation data; output the search algorithm evaluation data.

2. The evaluation method according to claim 1, wherein After outputting the search algorithm evaluation data according to the evaluation operation on the search results in the search result evaluation area, the evaluation method further includes: Generate a search algorithm evaluation report based on the search algorithm evaluation data.

3. The evaluation method according to claim 1, wherein Before displaying the search algorithm evaluation interface, the evaluation method further includes: Display an evaluation task configuration interface, which includes configuration controls for the evaluation type of the search algorithm to be evaluated and configuration controls for the evaluation tab page to be evaluated; In response to a configuration operation on the configuration controls for the evaluation type and the evaluation tab page to be evaluated, determine the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated; Generate a search algorithm evaluation interface for the target evaluation tab page based on the target evaluation type and the target evaluation tab page.

4. The evaluation method according to claim 3, wherein The evaluation task configuration interface further includes at least one of the following: configuration controls for the evaluation rules corresponding to the evaluation type, configuration controls for evaluation samples, and configuration controls for the number of evaluation times.

5. The evaluation method according to claim 1, characterized in that, The target evaluation rule reflects the scoring mechanism of the search results.

6. The evaluation method according to claim 1, wherein The search algorithm evaluation interface is determined based on the selected search algorithm evaluation type, and the selected search algorithm evaluation type includes at least one of the following: user experience evaluation, version evaluation, and competitor evaluation.

7. The evaluation method according to claim 1, wherein Performing statistical analysis on the scoring values of each scoring dimension corresponding to all search terms to obtain search algorithm evaluation data, including: For each search term, matching the scoring weights corresponding to each scoring dimension from the target evaluation rules; Calculating the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights; Counting the number of each search result satisfaction level corresponding to each scoring dimension respectively to obtain a search result satisfaction distribution table; Generating search algorithm evaluation data based on the target score and the search result satisfaction distribution table.

8. The evaluation method according to claim 7, wherein It also includes: Generating evaluation result information indicating whether the search algorithm passes the evaluation based on the target score; Outputting the evaluation result information.

9. The evaluation method according to claim 7, wherein The selected search algorithm evaluation type includes user experience evaluation, and the calculating the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights includes: Calculating the weighted total score value of all search terms according to the scoring values of each scoring dimension of the search results corresponding to each search term and the scoring weights; Calculating the average score value of each search term according to the weighted total score value to obtain the target score.

10. The evaluation method according to claim 7, wherein The selected search algorithm evaluation type includes version evaluation or competitor evaluation, and the calculating the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights includes: Calculating the weighted total score value of all search terms according to the scoring values of each scoring dimension of the search results corresponding to each search term and the scoring weights to obtain the target score.

11. The evaluation method according to claim 7, characterized in that, The selected search algorithm evaluation type includes competitor evaluation, and the generating search algorithm evaluation data based on the target score and the search result satisfaction distribution table includes: Generating search algorithm evaluation data based on the target score and the search result satisfaction distribution table corresponding to the current evaluation version and each competitor version respectively.

12. The evaluation method according to claim 7, characterized in that The selected search algorithm evaluation type includes version evaluation, and the evaluation method also includes: Based on the search result satisfaction distribution table, obtaining the number of each search result satisfaction level corresponding to each scoring dimension; For each scoring dimension, calculating the average value of the number of the first search result satisfaction level and the number of the second search result satisfaction level corresponding to each scoring dimension; Calculating the sum of the number of the third search result satisfaction level corresponding to each scoring dimension and the average value to obtain the total number of wins; Obtaining the total number of all search result satisfaction levels corresponding to each scoring dimension; Calculating the ratio of the total number of wins to the total number to obtain the win rate of the scoring dimension of the current evaluation version relative to the reference version; Outputting the win rate; Among them, the satisfaction level of the first search result is the satisfaction level of the search result indicating that both the current evaluation version and the reference version meet the search expectation. The satisfaction level of the second search result is the satisfaction level of the search result indicating that neither the current evaluation version nor the reference version meets the search expectation. The satisfaction level of the third search result is the satisfaction level of the search result indicating that the current evaluation version can meet the search expectation while the reference version cannot meet the search expectation.

13. The evaluation method according to claim 1, wherein The search result evaluation area further includes marking controls for the problem types corresponding to each scoring dimension; the evaluation method further includes: For each search term in each evaluation, in response to a marking operation on the marking control for the problem type corresponding to each scoring dimension, determine the problem types of each scoring dimension of the search result corresponding to the search term; After completing the evaluation for the set number of evaluations, statistically analyze the problem types of each scoring dimension of the search results corresponding to all search terms to obtain a problem distribution table; Add the problem distribution table to the search algorithm evaluation data.

14. The evaluation method according to claim 13, characterized in that The statistically analyzing the problem types of each scoring dimension of the search results corresponding to all search terms to obtain a problem distribution table includes: For each scoring dimension of the search results corresponding to all search terms, respectively count the number of each problem type corresponding to each scoring dimension to obtain a problem distribution table.

15. An evaluation device for a search algorithm, characterized in that Includes: An evaluation interface display module for displaying a search algorithm evaluation interface, where the search algorithm evaluation interface includes a search function area and a search result evaluation area; The search result evaluation area includes marking controls for the satisfaction levels of each scoring dimension of the search result; A first evaluation rule display module for displaying an evaluation rule configuration interface when an instruction to create an evaluation rule is detected. The evaluation rule configuration interface includes an evaluation type selection menu; an evaluation type determination module for selecting the search algorithm evaluation type of the evaluation rule to be created in response to a selection operation on the evaluation type selection menu; a second evaluation rule display module for displaying an evaluation rule configuration sub-interface corresponding to the selected search algorithm evaluation type of the evaluation rule to be created. The evaluation rule configuration sub-interface includes a scoring dimension configuration area, an evaluation level configuration area, and a problem type configuration area. The scoring dimension configuration area includes configuration controls for the scoring dimension and the corresponding scoring weight. The evaluation level configuration area includes configuration controls for the satisfaction level of the search result and the corresponding score value. The problem type configuration area includes configuration controls for the problem type; an evaluation rule generation module for generating an evaluation rule corresponding to the selected search algorithm evaluation type of the evaluation rule to be created in response to a configuration operation on the evaluation rule configuration sub-interface A search module for obtaining a corresponding search result based on the search term input in the search function area and displaying the search result; An evaluation module for determining search algorithm evaluation data using a target evaluation rule according to the evaluation operation on the search result in the search result evaluation area and outputting the search algorithm evaluation data; includes: A scoring value determination unit, which is used to, for each search term in each evaluation, in response to a marking operation of a marking control for the satisfaction degree of each scoring dimension of the search results, determine the satisfaction degree level of the search results for each scoring dimension; based on the determined satisfaction degree levels of the search results for each scoring dimension, match the scoring values for each scoring dimension from the target evaluation rules; a first analysis unit, which is used to, after completing the evaluations for the set number of evaluations, perform statistical analysis on the scoring values for each scoring dimension corresponding to all search terms to obtain search algorithm evaluation data; A display unit, which is used to display the search algorithm evaluation data.

16. The evaluation device according to claim 15, characterized in that, The evaluation device for the search algorithm further includes: A report generation module, which is used to generate a search algorithm evaluation report according to the search algorithm evaluation data.

17. The evaluation device according to claim 16, wherein The evaluation device for the search algorithm further includes: An evaluation task display module, which is used to display an evaluation task configuration interface, and the evaluation task configuration interface includes a configuration control for the evaluation type of the search algorithm to be evaluated and a configuration control for the evaluation tab page to be evaluated; A determination module, which is used to, in response to a configuration operation for the configuration control of the evaluation type and the configuration control of the evaluation tab page to be evaluated, determine the target evaluation type and the target evaluation tab page of the search algorithm to be evaluated; An evaluation interface generation module, which is used to generate a search algorithm evaluation interface for the target evaluation tab page based on the target evaluation type and the target evaluation tab page.

18. The evaluation device according to claim 17, wherein The evaluation task configuration interface further includes at least one of the following: a configuration control for the evaluation rules corresponding to the evaluation type, a configuration control for the evaluation samples, and a configuration control for the number of evaluations.

19. The evaluation device according to claim 15, wherein The target evaluation rules reflect the scoring mechanism of the search results.

20. The evaluation device according to claim 15, wherein The search algorithm evaluation interface is determined based on the selected search algorithm evaluation type, and the selected search algorithm evaluation type includes at least one of the following: user experience evaluation, version evaluation, and competitor evaluation.

21. The evaluation device according to claim 15, wherein The first analysis unit is specifically used for: For each search term, match the scoring weights corresponding to each scoring dimension from the target evaluation rules; Calculate the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights; Respectively count the number of satisfaction degree levels of the search results corresponding to each scoring dimension to obtain a search result satisfaction degree distribution table; Generate search algorithm evaluation data based on the target score and the search result satisfaction degree distribution table.

22. The evaluation device according to claim 21, wherein The evaluation module further includes: An evaluation result information generation unit, which is used to generate evaluation result information indicating whether the search algorithm passes the evaluation based on the target score; An evaluation result information output unit, which is used to output the evaluation result information.

23. The evaluation device according to claim 21, wherein The selected search algorithm evaluation type includes user experience evaluation. When the first analysis unit calculates the target score according to the scoring values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights, it is specifically used for: Calculate the weighted total scoring value of all search terms according to the scoring values of each scoring dimension of the search results corresponding to each search term and the scoring weights; Calculate the average scoring value of each search term according to the weighted total scoring value to obtain the target score.

24. The evaluation device according to claim 21, wherein The selected search algorithm evaluation types include version evaluation or competitor evaluation. When calculating the target score according to the score values of each scoring dimension of the search results corresponding to each search term and the matched scoring weights, the first analysis unit is specifically used for: Calculating the weighted total score value of all search terms according to the score values of each scoring dimension of the search results corresponding to each search term and the scoring weights to obtain the target score.

25. The evaluation device according to claim 21, wherein The selected search algorithm evaluation type includes competitor evaluation. The first analysis unit is specifically used for: Generating search algorithm evaluation data based on the target scores corresponding to the current evaluation version and each competitor version and the search result satisfaction distribution table.

26. The evaluation device according to claim 21, characterized in that The selected search algorithm evaluation type includes version evaluation. The first analysis unit is also used for: Based on the search result satisfaction distribution table, obtaining the number of each search result satisfaction level corresponding to each scoring dimension; For each scoring dimension, calculating the average value of the number of the first search result satisfaction levels corresponding to each scoring dimension and the number of the second search result satisfaction levels; Calculating the sum of the number of the third search result satisfaction levels corresponding to each scoring dimension and the average value to obtain the total number of wins; Obtaining the total number of all search result satisfaction levels corresponding to each scoring dimension; Calculating the ratio of the total number of wins to the total number to obtain the winning rate of the scoring dimension of the current evaluation version relative to the reference version; Outputting the winning rate; Wherein, the first search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version meet the search expectation, the second search result satisfaction level is the search result satisfaction level indicating that both the current evaluation version and the reference version do not meet the search expectation, and the third search result satisfaction level is the search result satisfaction level indicating that the current evaluation version can meet the search expectation while the reference version cannot meet the search expectation.

27. The evaluation device according to claim 15, wherein The search result evaluation area also includes marking controls for the problem types corresponding to each scoring dimension; the evaluation module further includes: A problem type determination unit, configured to, for each search term in each evaluation, in response to a marking operation on the marking controls for the problem types corresponding to each scoring dimension, determine the problem types of each scoring dimension of the search results corresponding to the search term; A second analysis unit, configured to, after completing the evaluations for the set number of evaluations, perform statistical analysis on the problem types of each scoring dimension of the search results corresponding to all search terms to obtain a problem distribution table; Adding the problem distribution table to the search algorithm evaluation data.

28. The evaluation device according to claim 27, wherein The second analysis unit is used for: For each scoring dimension of the search results corresponding to all search terms, respectively counting the number of each problem type corresponding to each scoring dimension to obtain a problem distribution table.

29. A computing device, comprising: A processor and a memory, where the memory stores executable instructions, and the processor is configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 14.

30. A medium having a program stored thereon, the program, when executed by a processor, implementing the method according to any one of claims 1 to 14.

Images