Test methods, media, devices and computing equipment

Abstract

The embodiments of the present disclosure provide a testing method, medium, apparatus and computing device. The method includes: displaying a test interface, the test interface includes an input box for inputting a search object, a first control for determining a search algorithm and a search control for initiating a search; in response to a first interactive operation directed to the search control, displaying the search results of the target search object under the target search algorithm, and the index data of the search results; determining the type of the corresponding search results according to the index data, the type includes expected search results and / or unexpected search results. The scheme of the present disclosure can obtain the index data of the search results by initiating a search according to the search object and the search algorithm, and determine the type of the search results based on the index data, without the need for manual observation and verification of the search results, which is time-saving and efficient.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of Internet technology, and more specifically, embodiments of the present disclosure relate to a testing method, medium, apparatus, and computing device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] In the process of updating and iterating Internet products, it is necessary to experiment with various design schemes and control the direction of updating and optimization based on the experimental data.

[0004] Taking search algorithms as an example, it is usually necessary to observe the online effects of various search algorithms after the experiment, so as to determine the optimal search algorithm to apply to the product. At present, in the process of observing the online effects of search algorithms, it is usually necessary to configure the experimental identifier and environment to obtain the search results of the search algorithm. Then, the relevant indicator data of the search results is obtained through database query, and finally the developers observe and verify the search results based on the relevant indicator data to determine the online effect of the search algorithm.

[0005] In the above solution, the search results need to be manually observed and verified based on relevant indicator data, which is time-consuming and inefficient. Summary of the invention

[0006] The present disclosure provides a testing method, medium, device and computing equipment to solve the problem that manual observation and verification of search results based on relevant indicator data is time-consuming and inefficient.

[0007] In a first aspect, an embodiment of the present disclosure provides a testing method, including:

[0008] Displaying a test interface, the test interface comprising an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search;

[0009] In response to a first interactive operation directed to the search control, displaying search results of a target search object under a target search algorithm and index data of the search results, wherein the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control;

[0010] The type of the corresponding search result is determined according to the indicator data, and the type includes expected search results and / or unexpected search results.

[0011] In a possible implementation manner, the test interface further includes a storage control for storing the unexpected search results, and the test method further includes:

[0012] In response to the type of the search result being an unexpected search result, the storage control prompts to save the unexpected search result.

[0013] In a possible implementation, the prompting, through the storage control, to save the unexpected search results includes:

[0014] In response to a second interactive operation directed to the storage control, the unexpected search result is saved.

[0015] In a possible implementation, the testing method further includes:

[0016] In response to the first interactive operation, a similarity calculation control is displayed, wherein the similarity calculation control is used to initiate calculation of similarities of search results under each of the target search algorithms.

[0017] In a possible implementation, the testing method further includes:

[0018] In response to a third interactive operation directed to the similarity calculation control, displaying a setting interface on the test interface, the setting interface including a module selection box and a comparison search algorithm selection box;

[0019] In response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

[0020] In a possible implementation, determining the type of the corresponding search result according to the indicator data includes:

[0021] If it is determined according to the indicator data that the search result does not meet the first rule requirement of the target search algorithm, then determining the type to be the unexpected search result;

[0022] If it is determined according to the indicator data that the search result meets the first rule requirement, the type is determined according to the indicator data, the first rule requirement and the second rule requirement.

[0023] In a possible implementation manner, determining the type according to the indicator data and the first rule requirement and the second rule requirement includes:

[0024] If the first rule requirement does not conflict with the second rule requirement, determining the type according to the indicator data and the second rule requirement;

[0025] If the first rule requirement conflicts with the second rule requirement, the type is determined to be the expected search result.

[0026] In a possible implementation manner, determining the type according to the indicator data and the second rule requirement includes at least one of the following:

[0027] Determining the type according to the number of clicks on the search result, wherein the indicator data includes the number of clicks;

[0028] Determining the type according to the relevance between the target search object and the search result;

[0029] The type is determined according to the type and quantity of the search results, and the indicator data includes the type and quantity of the search results.

[0030] In a possible implementation, in response to the fourth interactive operation directed to the similarity calculation control, displaying on the test interface the similarity of search results of the corresponding target modules under the first target search algorithm and the second target search algorithm, includes:

[0031] Obtaining a first search result of a target module corresponding to the first target search algorithm and a second search result of a target module corresponding to the second target search algorithm;

[0032] Obtaining a first ranked list according to the identifier of each search result in the first search result, and obtaining a second ranked list according to the identifier of each search result in the second search result;

[0033] Acquire the similarity according to the first sorted list and the second sorted list;

[0034] The similarity is displayed on the test interface.

[0035] In a possible implementation manner, acquiring the similarity according to the first sorted list and the second sorted list includes:

[0036] Obtain the number of intersections of the first d elements in the first sorted list and the second sorted list, and the weights corresponding to the d elements, where d is 1, 2, 3, ... in sequence;

[0037] The similarity is obtained according to the number of the intersections and the weight.

[0038] In a possible implementation manner, obtaining the similarity according to the number of the intersections and the weight includes:

[0039]

[0040] Wherein, R is the similarity, and w d is the weight corresponding to d, w d =(1-p)p d-1 , where p is a preset value, where p is greater than 0 and less than 1, where I d is the number of intersections among the first d elements.

[0041] In a possible implementation manner, before displaying the search results of the target search object under the target search algorithm and the index data of the search results, the testing method further includes:

[0042] Obtaining an identifier of the target search algorithm; obtaining the search results and the indicator data according to the target search object and the identifier; or,

[0043] Obtain the website address corresponding to the target search algorithm; and obtain the search results and the indicator data according to the target search object and the website address.

[0044] In a second aspect, an embodiment of the present disclosure provides a testing device, including:

[0045] A first display module, used to display a test interface, wherein the test interface includes an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search;

[0046] A second display module, configured to display, in response to a first interactive operation directed to the search control, search results of a target search object under a target search algorithm, and indicator data of the search results, wherein the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control;

[0047] A processing module is used to determine the type of corresponding search results according to the indicator data, and the type includes expected search results and / or unexpected search results.

[0048] In a possible implementation manner, the test interface further includes a storage control for storing the unexpected search results, and the processing module is further configured to:

[0049] In response to the type of the search result being an unexpected search result, the storage control prompts to save the unexpected search result.

[0050] In a possible implementation manner, the processing module is further configured to:

[0051] In response to a second interactive operation directed to the storage control, the unexpected search result is saved.

[0052] In a possible implementation manner, the second display module is further used for:

[0053] In response to the first interactive operation, a similarity calculation control is displayed, wherein the similarity calculation control is used to initiate calculation of similarities of search results under each of the target search algorithms.

[0054] In a possible implementation manner, the second display module is further used for:

[0055] In response to a third interactive operation directed to the similarity calculation control, displaying a setting interface on the test interface, the setting interface including a module selection box and a comparison search algorithm selection box;

[0056] In response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

[0057] In a possible implementation manner, the processing module is specifically used to:

[0058] If it is determined according to the indicator data that the search result does not meet the first rule requirement of the target search algorithm, then determining the type to be the unexpected search result;

[0059] If it is determined according to the indicator data that the search result meets the first rule requirement, the type is determined according to the indicator data, the first rule requirement and the second rule requirement.

[0060] In a possible implementation manner, the processing module is specifically used to:

[0061] If the first rule requirement does not conflict with the second rule requirement, determining the type according to the indicator data and the second rule requirement;

[0062] If the first rule requirement conflicts with the second rule requirement, the type is determined to be the expected search result.

[0063] In a possible implementation manner, the processing module is specifically used for at least one of the following:

[0064] Determining the type according to the number of clicks on the search result, wherein the indicator data includes the number of clicks;

[0065] Determining the type according to the relevance between the target search object and the search result;

[0066] The type is determined according to the type and quantity of the search results, and the indicator data includes the type and quantity of the search results.

[0067] In a possible implementation manner, the second display module is further used for:

[0068] Obtaining a first search result of a target module corresponding to the first target search algorithm and a second search result of a target module corresponding to the second target search algorithm;

[0069] Obtaining a first ranked list according to the identifier of each search result in the first search result, and obtaining a second ranked list according to the identifier of each search result in the second search result;

[0070] Acquire the similarity according to the first sorted list and the second sorted list;

[0071] The similarity is displayed on the test interface.

[0072] In a possible implementation manner, the second display module is further used for:

[0073] Obtain the number of intersections of the first d elements in the first sorted list and the second sorted list, and the weights corresponding to the d elements, where d is 1, 2, 3, ... in sequence;

[0074] The similarity is obtained according to the number of the intersections and the weight.

[0075] In a possible implementation manner, the similarity is:

[0076]

[0077] Wherein, R is the similarity, and w d is the weight corresponding to d, w d =(1-p)p d-1 , where p is a preset value, where p is greater than 0 and less than 1, where I d is the number of intersections among the first d elements.

[0078] In a possible implementation manner, before displaying the search results of the target search object under the target search algorithm and the index data of the search results, the processing module is further used to:

[0079] Obtaining an identifier of the target search algorithm; obtaining the search results and the indicator data according to the target search object and the identifier; or,

[0080] Obtain the website address corresponding to the target search algorithm; and obtain the search results and the indicator data according to the target search object and the website address.

[0081] In a third aspect, an embodiment of the present disclosure provides a computing device, including: at least one processor and a memory;

[0082] The memory stores computer-executable instructions;

[0083] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the testing method as described in any one of the first aspects.

[0084] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the test method as described in any one of the first aspects is implemented.

[0085] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes a computer program; when the computer program is executed, the test method described in any one of the first aspects is implemented.

[0086] The test method, medium, device and computing device provided by the embodiment of the present disclosure first display a test interface, which includes an input box for inputting a search object, a first control for determining a search algorithm and a search control for initiating a search, and then in response to a first interactive operation facing the search control, display the search results of the target search object under the target search algorithm and the index data of the search results, wherein the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control; finally, determine the type of the corresponding search result according to the index data, and determine whether the search result is an expected search result or an unexpected search result. The scheme of the embodiment of the present disclosure does not require a database query, but after inputting a search object and selecting a search algorithm, the index data of the search result can be obtained by initiating a search, and the type of the search result is determined based on the index data, thereby determining the effect of the search algorithm according to the type of the search result, and there is no need to manually observe and verify the search results, which is time-consuming and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0088] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present disclosure;

[0089] Figure 2 A flow chart of a testing method provided in an embodiment of the present disclosure;

[0090] Figure 3 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 1 ;

[0091] Figure 4 A schematic diagram of a search algorithm provided for an embodiment of the present disclosure;

[0092] Figure 5 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 1 ;

[0093] Figure 6 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 2 ;

[0094] Figure 7 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 3 ;

[0095] Figure 8 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 2 ;

[0096] Fig. 9 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 3 ;

[0097] Fig.10 A schematic diagram of a setting interface provided in an embodiment of the present disclosure;

[0098] Fig.11 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 4 ;

[0099] Fig.12 A schematic diagram of a program product provided by an embodiment of the present disclosure;

[0100] Fig.13 A schematic diagram of the structure of a testing device provided in an embodiment of the present disclosure;

[0101] Fig.14A schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0102] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0103] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0104] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0105] According to an embodiment of the present disclosure, a testing method, a medium, an apparatus and a computing device are proposed.

[0106] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0107] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0109] For Internet products, the update and iteration of the entire APP system requires the establishment of a set of metrics to control the direction of the entire process optimization. Among them, AB test (comparative experiment) is a commonly used tool for variable control and optimization direction selection.

[0110] AB test is also called bucket test or diversion test. In network analysis, AB test is a random experiment, usually with two variants, such as A and B, which usually represent two different design solutions. For example, when setting a search control on the interface of an APP, you need to choose the location of the search control setting. Solution A sets the search control location on the left side of the interface, and Solution B sets the search control location in the middle of the interface.

[0111] When other conditions are the same, we design Plan A and Plan B respectively, and then launch the corresponding APPs of Plan A and Plan B to a certain number of users, that is, some users use Plan A and some users use Plan B. Then, we record the user's usage through logs, obtain user interaction data, and analyze relevant indicators. On the one hand, we can select the plan that better meets the expected goals from Plan A and Plan B based on relevant indicators, and finally switch all traffic to the plan that meets the expected goals; on the other hand, we can get the effect of Plan A and / or Plan B based on relevant indicators, so as to optimize and improve Plan A and / or Plan B in the future.

[0112] In the disclosed embodiments, AB test is mainly aimed at different search algorithms, and different search algorithms correspond to different solutions. After different search algorithms are launched, corresponding users are assigned to different search algorithms, and then relevant indicators are analyzed based on user interaction data. On the one hand, according to the relevant indicators of each search algorithm, the search algorithm that meets the expected goals can be selected from each search algorithm and applied to the product. On the other hand, the effect of each search algorithm can be obtained according to the relevant indicators of each search algorithm, so as to optimize the system recommendation strategy based on the effect of each search algorithm.

[0113] After the experiment is completed, it is usually necessary to observe the online effect of each search algorithm, so that a better search algorithm can be selected from each search algorithm and applied to the product according to the online effect, or the shortcomings of the search algorithm can be optimized accordingly according to the online effect, etc. The inventors have found that the online effect of the search algorithm is currently obtained mainly by querying the relevant index data of the search algorithm through the database, and then the developer determines the online effect of the search algorithm according to the relevant index data. The above scheme requires manual observation and verification of the online effect of the search algorithm according to the relevant index data, which is time-consuming and inefficient.

[0114] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0115] Application Scenario Overview

[0116] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure, such as Figure 1 As shown, after the experiment is completed, after the search object "forest bird" is input, the corresponding search results are obtained according to the target search algorithm, and the search results are shown in interface 10.

[0117] After obtaining the search results under the target search algorithm, it is necessary to further verify the rationality of the search results. If it is reasonable, it means that the search result is a good case, that is, the expected search result. If it is unreasonable, it means that the search result is a bad case, that is, an unexpected search result.

[0118] After obtaining the corresponding search results according to the target search algorithm, it may not be possible to directly judge whether it is the expected search result or the unexpected search result based on the search results. It is necessary to make a comprehensive judgment based on other relevant indicator data. Taking the search results as songs as an example, the indicator data may include the click-through rate, click volume, comment volume, number of songs and other data.

[0119] At present, after obtaining the search results, if it is necessary to obtain relevant indicator data to judge the search result type, it is necessary to obtain the indicator data through database query, and then manually observe and verify according to the indicator data to obtain the type of search result. The above method is time-consuming and inefficient. Based on this, the embodiment of the present disclosure provides a testing method to solve the above problem.

[0120] Exemplary Methods

[0121] Combine the following Figure 1 For application scenarios, refer to Figure 2 To describe the test method according to the exemplary embodiment of the present disclosure. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0122] Figure 2 A flow chart of the test method provided in the embodiment of the present disclosure is as follows: Figure 2 As shown, the method may include:

[0123] S21, displaying a test interface, the test interface including an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search.

[0124] In the disclosed embodiment, after conducting experiments on different search algorithms, it is necessary to observe and verify the online effects of each search algorithm. Therefore, the corresponding test tool of the present invention can be opened first to display the test interface.

[0125] The test interface includes an input box, a first control, and a search control. The user can manually enter the search object in the input box, or enter the search object in the input box by manipulating the input device. The first control is used to determine the search algorithm, which can be a search algorithm to be applied to a new product, a search algorithm commonly used in products of the same type, or other search algorithms for comparison. The number of search algorithms determined in the first control can be one or more. The search control is used to initiate a search.

[0126] Figure 3 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 1 ,like Figure 3 As shown, the test interface 30 includes an input box 31, a first control 32 and a search control 33. The search object can be input in the input box 31, for example, Figure 3 , the search object entered is "forest birds".

[0127] After the search object is input in the input box 31, the search algorithm can be determined in the first control 32. The search algorithm determined in the first control 32 can be a single search algorithm or a group of search algorithms. The first control 32 can be implemented in the form of an input box or a selection box.

[0128] When the first control 32 is implemented in the form of an input box, the search algorithm of the first control 32 can be determined by entering the search algorithm in the input box. When the first control 32 is implemented in the form of a selection box, the search algorithm can be selected from the available options by pulling down the selection box. Figure 4 Provide explanation.

[0129] Figure 4 A schematic diagram of a search algorithm provided in an embodiment of the present disclosure, such as Figure 4 As shown in FIG. 1 , two implementation forms of the first control 32 are respectively illustrated. Figure 4 In the example, the number of search algorithms selected in the first control 32 is taken as 1 for introduction, wherein the selectable search algorithms include first type algorithm A, first type algorithm B, first type algorithm C, second type algorithm D, second type algorithm E and second type algorithm F.

[0130] Assume that the search algorithm that the user wishes to select is the first type of algorithm A. When the implementation form of the first control 32 is Figure 4In the form of input box 41 in the input box 41, the corresponding text can be input in the input box 41 to determine the search algorithm. For example, when "first category" is input in the input box 41, three algorithms are displayed on the test interface, namely "first category algorithm A", "first category algorithm B" and "first category algorithm C". The user selects the first category algorithm A from these three options to realize the selection of the search algorithm.

[0131] When the first control 32 is implemented in the form of a selection box, such as Figure 4 As shown, the selection box 42 can be pulled down to display all available search algorithms on the test interface, including first-category algorithm A, first-category algorithm B, first-category algorithm C, second-category algorithm D, second-category algorithm E, and second-category algorithm F. The user selects the first-category algorithm A from the six options to select the search algorithm.

[0132] S22, in response to the first interactive operation oriented to the search control, the search results of the target search object under the target search algorithm and the index data of the search results are displayed, the target search object is the search object in the input box, and the target search algorithm is the search algorithm selected in the first control.

[0133] After inputting a search object through the input box for inputting a search object and determining a search algorithm through the first control, a search may be initiated through a first interactive operation directed to the search control. The first interactive operation may be, for example, an operation of clicking the search control.

[0134] After the search is initiated, the search results of the target search object under the target search algorithm and the index data of the search results are displayed on the test interface according to the input search object and the selected search algorithm. Among them, the target search object is the search object entered in the input box, and the target search algorithm is the search algorithm selected in the first control. Figure 3 For example, the target search object is “forest bird” input in the input box 31 , and the target search algorithm is the search algorithm selected in the first control 32 .

[0135] The search results are the results obtained by searching the target search object according to the target search algorithm. Taking music apps as an example, the target search object may be a song name, singer name, etc., and the search results obtained by searching the target search object according to the target search algorithm may be singles, MVs, playlists, albums, etc., and the number of search results is one or more.

[0136] For example, Figure 3 In the figure, the search results corresponding to the target search object "forest bird" under three target search algorithms are illustrated. Figure 3The search results of the left example include a playlist "

Single Pure Enjoyment

[0137] The metric data of the search results is data related to the search results. Taking the search results as songs as an example, the metric data of the songs can, for example, include the click-through rate, comment volume, play volume, and so on, and can also include the type and quantity of the songs, and so on. Taking Figure 3 the search results on the left in the middle as an example, the metric data of the search results can, for example, include the type of the search results (2 types, namely playlists and singles), the play volume of the single (for example, the play volume of single 1 "Forest (Mr-if I am)" is 54469), the comment volume of the single (for example, the comment volume of single 1 "Forest (Mr-if I am)" is 308223), and so on.

[0138] S23. Determine the corresponding type of the search results according to the metric data. The types include expected search results and / or unexpected search results.

[0139] After obtaining the metric data of the search results, the corresponding type of the search results can be determined according to the metric data. The types of the search results include expected search results and / or unexpected search results.

[0140] Expected search results (goodcase) are reasonable and better-than-expected search results, and unexpected search results (badcase) are unreasonable and non-expected search results. Taking Figure 3Taking the search algorithm on the left as an example, if the search algorithm is set to rank the search results with the highest number of views at the front, and the number of views for "Forest (Mr-if I am)" is 54469, and the number of views for "Norwegian Forest (Wu Bai)" is 93518, which is greater than the number of views for "Forest (Mr-if I am)", but ranked behind "Forest (Mr-if I am)", then the search results of the search algorithm on the left are unexpected search results (bad case).

[0141] The test method provided by the embodiment of the present disclosure first displays a test interface, which includes an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search. Then, in response to a first interactive operation facing the search control, the search results of the target search object under the target search algorithm and the index data of the search results are displayed. The target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control. Finally, the type of the corresponding search result is determined according to the index data, and it is determined whether the search result is an expected search result or an unexpected search result. The scheme of the embodiment of the present disclosure does not require a database query. Instead, after inputting a search object and selecting a search algorithm, the index data of the search result can be obtained by initiating a search, and the type of the search result is determined based on the index data, thereby determining the effect of the search algorithm according to the type of the search result. There is no need to manually observe and verify the search results, which is time-consuming and efficient.

[0142] The solution of the present application is described in detail below with reference to the accompanying drawings.

[0143] After obtaining the search results of the target search object under the target search algorithm and the index data of the search results, it is necessary to determine the type of the search results according to the index data.

[0144] Optionally, before displaying the index data of the search results, the identifier of the target search algorithm can be obtained, and the search results and index data can be obtained based on the target search object and the identifier. In this embodiment, it is mainly aimed at a clear target search algorithm, for example, a target search algorithm with a definite operating logic, or a target search algorithm with corresponding code. After conducting experiments based on the target search algorithm, the system stores the search results and index data corresponding to the target search algorithm. Therefore, the search results and index data can be obtained based on the target search object and the identifier.

[0145] Optionally, before displaying the index data of the search results, the website corresponding to the target search algorithm can be obtained, and then the search results and index data are obtained according to the target search object and the website. In this embodiment, it is mainly aimed at unclear target search algorithms, for example, target search algorithms used in the same type of products.

[0146] When determining the type of search results, it is necessary to make a determination in combination with the first rule requirement and / or the second rule requirement.

[0147] The first rule requirement is a specific rule requirement for the target search algorithm. Taking the music APP as an example, free original songs, copyright-free songs and VIP original songs can be searched in the music APP. If the specific rule requirement of a search algorithm is to arrange the free original songs in front of the VIP original songs, and in the search results obtained by the search algorithm, the VIP original songs are arranged in front of the free original songs, it means that the search results do not meet the specific rule requirements of the search algorithm. For different target search algorithms, the corresponding first rule requirements may be different. For example, taking the music APP as an example, the first rule requirement of some search algorithms is that the search results are all free original songs. As long as the search results are all free original songs, it is considered that the first rule requirement is met. If the search results include non-free original songs, it is considered that the first rule requirement is not met; the first rule requirement of some search algorithms is that the free original songs are arranged in front of the copyright-free songs. As long as the free original songs are arranged in front of the copyright-free songs, it is considered that the first rule requirement is met. If the free original songs are arranged behind the copyright-free songs, it is considered that the first rule requirement is not met, and so on.

[0148] When determining the type of search results, if it is determined based on the index data of the search results that the search results do not meet the first rule requirements of the target search algorithm, then regardless of whether the search results meet the second rule requirements, the type of the search results is determined to be an unexpected search result.

[0149] If it is determined based on the index data of the search results that the search results meet the first rule requirement of the target search algorithm, it is also necessary to further determine the type of the search results based on the index data, the first rule requirement and the second rule requirement.

[0150] Specifically, since the first rule requirement and the second rule requirement may be different, in some cases, the first rule requirement and the second rule requirement may conflict.

[0151] When the first rule requirement and the second rule requirement conflict, any search result cannot meet both the first rule requirement and the second rule requirement due to the conflict. In this case, the first rule requirement specific to the target search algorithm is used as the main rule requirement. Since the search result meets the first rule requirement and the first rule requirement and the second rule requirement conflict, the type of the search result is determined to be the expected search result.

[0152] When the first rule requirement and the second rule requirement do not conflict, then since the search result meets the first rule requirement, if the search result meets the second rule requirement, the type of the search result is considered to be the expected search result, and if the search result does not meet the second rule requirement, the type of the search result is considered to be the unexpected search result. That is, the type of the search result needs to be determined based on the indicator data and the second rule requirement.

[0153] One possible implementation is that the indicator data includes the number of clicks on the search results, and the type of the search results is determined based on the number of clicks on the search results. In this implementation, the second rule requirement may be that search results with higher popularity are prioritized at the front, while less popular search results are prioritized at the back, wherein the popularity can be reflected by the number of clicks on the search results.

[0154] Figure 5 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 1 ,like Figure 5 As shown, interface 50 illustrates the search results obtained by searching according to the target search algorithm. The search results include 5 singles, from top to bottom: "Dad (singer A)", "Dad (singer B)", "Dad (singer C)", "Dad (singer D)" and "Dad (singer E)". At the same time, interface 50 illustrates the number of clicks for each single.

[0155] The number of clicks indicates the number of times a user clicks on a single, which to a certain extent reflects the popularity of a single. The popularity of a single is positively correlated with the number of clicks on the single. From the index data of the search results, it can be learned that the number of clicks on "Dad (singer C)" is significantly higher than the number of clicks on other singles, but the single is ranked at the back in the search results, while the singles "Dad (singer A)" and "Dad (singer B)" with lower clicks are ranked at the front. At this time, it is considered that the search result does not meet the requirements of the second rule, and the type of the search result is determined to be an unexpected search result.

[0156] One possible implementation method is to determine the type of search results based on the correlation between the target search object and the search results. In this embodiment, the second rule requirement may be to prioritize the recall of search results with higher correlation with the target search object. In the field of search, recall refers to the process of screening out search results related to the search object from a large number of results. Taking the recall of songs as an example, a large number of songs can be regarded as a song pool, and there are a large number of songs in the song pool. After the search object (such as the name of a singer or a song) is entered, 5 related songs are screened out from the song pool. The process of screening out these 5 related songs is the recall of songs under the search object, and the recalled songs are these 5 related songs.

[0157] Figure 6 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 2 ,like Figure 6 As shown, the interface 60 illustrates the search results obtained by searching according to the target search algorithm. Among them, the target search object is "DAN", and the search results include 5 singles, from top to bottom, "Love That Never Lost Connection", "Not Called a Hero", "Love Like Tides", "Xiao Yu" and "Tou Toutiao". The above 5 singles have a low correlation with the target search object "DAN", and the music library includes a single of the same name that has a high correlation with "DAN", but the single is not recalled in the search results. At this time, it is considered that the search result does not meet the requirements of the second rule, and the type of the search result is determined to be an unexpected search result.

[0158] One possible implementation is that the indicator data includes the types and quantity of search results, and the type of search results is determined based on the types and quantity of search results. In this embodiment, the second rule requirement may be that when the types and quantity of search results reach a certain threshold, the search results are considered to meet the second rule requirement, and when the types and quantity of search results do not reach a certain threshold, the search results are considered not to meet the second rule requirement. The threshold of the types and quantity of search results may be a pre-set value, such as greater than or equal to 2 types and greater than or equal to 6 numbers. The threshold of the types and quantity of search results may also be a threshold determined based on the types and quantities of search results corresponding to other search algorithms. For example, other search algorithms may be search algorithms used in products of the same type.

[0159] For example, if the target search algorithm in the embodiment of the present disclosure is a search algorithm applied in a music APP, then the types and number of search results obtained after entering the target search object in other types of music APPs can be obtained. The threshold of the types and number of search results can be set to a certain proportion of the types and number of search results corresponding to other search algorithms. When the types and number of search results corresponding to the target search algorithm reach a certain proportion of the types and number of search results corresponding to other search algorithms, it is considered that the search results meet the requirements of the second rule, otherwise, it is considered that the search results do not meet the requirements of the second rule. Taking the ratio of one-half as an example, when the type of search results corresponding to the target search algorithm is 1 and the number is 2, if the types of search results corresponding to other search algorithms are 3 (half of which is 1.5) and the number is 10 (half of which is 5), the types and number of search results do not reach the specified threshold (1.5 types, 5), and do not meet the requirements of the second rule.

[0160] Figure 7 Schematic diagram of determining the type of search results provided by the embodiment of the present disclosure Figure 3 ,like Figure 7As shown, the interface 71 illustrates the search results obtained by searching according to the target search algorithm, and the type of search results is 1 and the number is 1. The interface 72 illustrates the search results obtained by searching other search algorithms, and the type of search results is 2 and the number is 5. The types and number of search results of the target search algorithm are significantly smaller than the types and numbers of search results of other search algorithms. At this time, it is considered that the search results do not meet the requirements of the second rule, and the type of search results is determined to be an unexpected search result.

[0161] After determining the type of search results, if the search results are unexpected, you can also save and report the search results. Figure 8 Make an introduction.

[0162] Figure 8 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 2 ,like Figure 8 As shown, the test interface 30 includes an input box 31, a first control 32 and a search control 33. The search object can be input in the input box 31, for example, Figure 3 , the search object entered is "forest birds".

[0163] After responding to the first interactive operation towards the search control 33, the search results of the target search object under the target search algorithm will be displayed on the test interface 30. The test interface 30 also includes a storage control for saving unexpected search results.

[0164] For example, Figure 8 In the example, the test interface 30 includes search results corresponding to three target search algorithms, and the search results on the right are unexpected search results. The test interface 30 also includes a storage control 81. Optionally, on the test interface 30, each search result corresponding to the target search algorithm corresponds to a storage control. In response to the type of the search result being an unexpected search result, the storage control corresponding to the unexpected search result prompts the unexpected search result to be saved. Figure 8 As shown in , the storage control 81 can be highlighted, bolded, or displayed in other ways to prompt the saving of unexpected search results.

[0165] At this time, the user can operate the storage control 81, and the operation can be operated by controlling an input device (such as clicking with a mouse), or by touch-clicking, etc. In response to the second interactive operation directed to the storage control 81, the unexpected search result can be saved. For example, the unexpected search result can be reported to the Figure 8 What search results are included in the unexpected search results, the relevant indicator data of each search result, etc., so as to be used for subsequent improvement and optimization of the search algorithm.

[0166] Fig. 9 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 3 ,like Fig. 9 As shown, the test interface 30 includes an input box 31, a first control 32 and a search control 33. The search object can be input in the input box 31, for example, Figure 3 , the search object entered is "forest birds".

[0167] After responding to the first interactive operation towards the search control 33 , a similarity calculation control 91 is displayed on the test interface 30 , and the similarity calculation control 91 is used to initiate calculation of similarities of search results under each target search algorithm.

[0168] The user may perform a third interactive operation on the similarity calculation control 91. The third interactive operation may be, for example, a touch-click operation on the similarity calculation control 91. According to the third interactive operation, the similarity of the search results under each target search algorithm may be obtained.

[0169] Specifically, in response to the third interactive operation directed to the similarity calculation control 91 , a setting interface is displayed on the test interface 30 . Fig.10 A schematic diagram of a setting interface provided in an embodiment of the present disclosure, such as Fig.10 As shown, the setting interface 100 includes a module selection box 101 and a comparison search algorithm selection box 102 .

[0170] The module selection box 101 is used to select the module of the search algorithm for comparison. For example, for a music APP, the search results may include MV, playlist, album, single, etc. These search results belong to different categories, that is, different modules. The module selection box 101 can be used to determine the module to be compared, for example, Fig.10 In the example, the selected module is single song. In subsequent comparisons, only the single songs in the search results are compared for similarity.

[0171] The comparison search algorithm selection box 102 is used to select a target search algorithm for comparison. For example, in group c, a target search algorithm to be compared can be selected, and in group t, a search algorithm to be compared with the algorithm selected in group c can be selected.

[0172] In response to the fourth interactive operation oriented to the similarity calculation control 91, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface 30, wherein the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

[0173] For example, Fig.10In the example, the target search algorithm selected by group c is identified as 3304175487 (corresponding to Fig. 9 The target search algorithm on the left, that is, algorithm A), then the selected algorithm is algorithm A, and algorithm A is the first target search algorithm; the identifier of the target search algorithm selected by group t is 398600744 (corresponding to Fig. 9 The target search algorithm in the middle, namely algorithm B) and 3969029504 (corresponding to Fig. 9 The target search algorithm on the right, namely algorithm C), algorithm B and algorithm C are the second target search algorithms. If the target module is a single song, then the subsequent calculation is the similarity of the single song in the search results corresponding to algorithm B and algorithm A, and the similarity of the single song in the search results corresponding to algorithm C and algorithm A.

[0174] For the compared first target search algorithm and second target search algorithm, a first search result of the target module corresponding to the first target search algorithm and a second search result of the target module corresponding to the second target search algorithm can be obtained. The first search result includes one or more search results, and the second search result includes one or more search results.

[0175] Different search results are distinguished by different identifiers, and the same search results have the same identifier. Therefore, after obtaining the first search result and the second search result, the identifiers of each search result in the first search result and the identifiers of each search result in the second search result can be obtained.

[0176] Further, according to the identifier of each search result in the first search result, a first sorted list is obtained.

[0177] S, wherein the elements included in the first sorted list S are the labels of the search results in the first search result.

[0178] The order of the elements in the first sorted list S corresponds to the order of the search results in the first search result.

[0179] Similarly, based on the identifier of each search result in the second search result, a second sorted list T is obtained, wherein the elements included in the second sorted list T are the identifiers of each search result in the second search result, and the arrangement order of the elements in the second sorted list T corresponds to the arrangement order of each search result in the second search result.

[0180] Taking the target module as a single module as an example, the first search result includes multiple singles from front to back, namely single 1, single 2, and single 3, then the first sorted list S = [single id1, single id2, single id3, ...]. The second search result includes multiple singles from front to back, namely single 4, single 5, and single 6, then the second sorted list T = [single id4, single id5, single id6, ...]. Among them, S and T are both sorted lists of infinite length. In S, except for the identifiers of multiple singles in the first search result, the following elements are filled with empty elements. In T, except for the identifiers of multiple singles in the second search result, the following elements are filled with empty elements.

[0181] According to the first sorting list S and the second sorting list T, the similarity of the search results of the target modules corresponding to the first target search algorithm and the second target search algorithm can be obtained.

[0182] Specifically, the number of intersections of the first d elements in the first sorted list S and the second sorted list T and the weight corresponding to d may be obtained first, where d is 1, 2, 3, ... in sequence.

[0183] Then, the similarity is obtained according to the number of intersections and the weight corresponding to d. The similarity can be obtained by the following formula:

[0184]

[0185] Among them, R is the similarity, w d is the weight corresponding to d, w d =(1-p)p d-1 , p is the preset value, p is greater than 0 and less than 1, I d is the number of intersections between the first d elements of the first sorted list S and the second sorted list T.

[0186] I d =S 1:d ∩T 1:d ,

[0187] Among them, S 1:d is the set of all elements from the 1st element to the dth element (including the 1st element and the dth element) in the first sorted list S, T 1:d It is the set consisting of all elements from the 1st element to the dth element (including the 1st element and the dth element) in the second sorted list T, and ∩ represents the intersection.

[0188] Fig.11 Schematic diagram of the test interface provided by the embodiment of the present disclosure Figure 4 ,like Fig.11As shown, after clicking the similarity control 91, the test interface 30 displays that the similarity between algorithm B and algorithm A in the single song module is 0%, and the similarity between algorithm C and algorithm A in the single song module is 90%.

[0189] The test method provided by the embodiment of the present disclosure first displays a test interface, which includes an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search. Then, in response to a first interactive operation facing the search control, the search results of the target search object under the target search algorithm and the index data of the search results are displayed. The target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control. Finally, the type of the corresponding search result is determined according to the index data, and it is determined whether the search result is an expected search result or an unexpected search result. The scheme of the embodiment of the present disclosure does not require a database query. Instead, after inputting a search object and selecting a search algorithm, the index data of the search result can be obtained by initiating a search, and the type of the search result is determined based on the index data, thereby determining the effect of the search algorithm according to the type of the search result. There is no need to manually observe and verify the search results, which is time-consuming and efficient.

[0190] Exemplary Media

[0191] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Fig.12 A storage medium according to an exemplary embodiment of the present disclosure is described.

[0192] Fig.12 A schematic diagram of a program product provided by an embodiment of the present disclosure, referring to Fig.12 As shown, a program product 120 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0193] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0194] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable signal medium may also be any readable medium other than a readable storage medium.

[0195] Program code for performing the disclosed operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0196] Exemplary Devices

[0197] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Fig.13 A test device according to an exemplary embodiment of the present disclosure is described, which is used to implement a method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be described in detail here.

[0198] Fig.13 A schematic diagram of a test device provided in an embodiment of the present disclosure is shown in FIG. Fig.13 As shown, including:

[0199] A first display module 131, used to display a test interface, wherein the test interface includes an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search;

[0200] A second display module 132 is used to display the search results of the target search object under the target search algorithm and the index data of the search results in response to the first interactive operation directed to the search control, the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control;

[0201] The processing module 133 is used to determine the type of the corresponding search result according to the indicator data, and the type includes expected search results and / or unexpected search results.

[0202] In a possible implementation manner, the test interface further includes a storage control for storing the unexpected search results, and the processing module 133 is further used to:

[0203] In response to the type of the search result being an unexpected search result, the storage control prompts to save the unexpected search result.

[0204] In a possible implementation manner, the processing module 133 is further configured to:

[0205] In response to a second interactive operation directed to the storage control, the unexpected search result is saved.

[0206] In a possible implementation manner, the second display module 132 is further used for:

[0207] In response to the first interactive operation, a similarity calculation control is displayed, wherein the similarity calculation control is used to initiate calculation of similarities of search results under each of the target search algorithms.

[0208] In a possible implementation manner, the second display module 132 is further used for:

[0209] In response to a third interactive operation directed to the similarity calculation control, displaying a setting interface on the test interface, the setting interface including a module selection box and a comparison search algorithm selection box;

[0210] In response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

[0211] In a possible implementation manner, the processing module 133 is specifically configured to:

[0212] If it is determined according to the indicator data that the search result does not meet the first rule requirement of the target search algorithm, then determining the type to be the unexpected search result;

[0213] If it is determined according to the indicator data that the search result meets the first rule requirement, the type is determined according to the indicator data, the first rule requirement and the second rule requirement.

[0214] In a possible implementation manner, the processing module 133 is specifically configured to:

[0215] If the first rule requirement does not conflict with the second rule requirement, determining the type according to the indicator data and the second rule requirement;

[0216] If the first rule requirement conflicts with the second rule requirement, the type is determined to be the expected search result.

[0217] In a possible implementation manner, the processing module 133 is specifically used for at least one of the following:

[0218] Determining the type according to the number of clicks on the search result, wherein the indicator data includes the number of clicks;

[0219] Determining the type according to the relevance between the target search object and the search result;

[0220] The type is determined according to the type and quantity of the search results, and the indicator data includes the type and quantity of the search results.

[0221] In a possible implementation manner, the second display module 132 is further configured to:

[0222] Obtaining a first search result of a target module corresponding to the first target search algorithm and a second search result of a target module corresponding to the second target search algorithm;

[0223] Obtaining a first ranked list according to the identifier of each search result in the first search result, and obtaining a second ranked list according to the identifier of each search result in the second search result;

[0224] Acquire the similarity according to the first sorted list and the second sorted list;

[0225] The similarity is displayed on the test interface.

[0226] In a possible implementation manner, the second display module 132 is further configured to:

[0227] Obtain the number of intersections of the first d elements in the first sorted list and the second sorted list, and the weights corresponding to the d elements, where d is 1, 2, 3, ... in sequence;

[0228] The similarity is obtained according to the number of the intersections and the weight.

[0229] In a possible implementation manner, the similarity is:

[0230]

[0231] Wherein, R is the similarity, and w dis the weight corresponding to d, w d =(1-p)p d-1 , where p is a preset value, where p is greater than 0 and less than 1, where I d is the number of intersections among the first d elements.

[0232] In a possible implementation manner, before displaying the search results of the target search object under the target search algorithm and the index data of the search results, the processing module 133 is further used to:

[0233] Obtaining an identifier of the target search algorithm; obtaining the search results and the indicator data according to the target search object and the identifier; or,

[0234] Obtain the website address corresponding to the target search algorithm; and obtain the search results and the indicator data according to the target search object and the website address.

[0235] The testing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar and will not be described in detail here.

[0236] Exemplary Computing Devices

[0237] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, reference is made to Fig.14 A computing device according to an exemplary embodiment of the present disclosure is described.

[0238] Fig.14 The computing device 140 shown is merely an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0239] Fig.14 A schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure, such as Fig.14 As shown, the computing device 140 is in the form of a general computing device. The components of the computing device 140 may include but are not limited to: at least one processing unit 141, at least one storage unit 142, and a bus 143 connecting different system components (including the processing unit 141 and the storage unit 142).

[0240] The bus 143 includes a data bus, a control bus, and an address bus.

[0241] The storage unit 142 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1421 and / or a cache memory 1422 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 1423 .

[0242] The storage unit 142 may also include a program / utility 1425 having a set (at least one) of program modules 1424, such program modules 1424 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0243] The computing device 140 may also communicate with one or more external devices 144 (e.g., a keyboard, a pointing device, etc.). Such communication may be performed via an input / output (I / O) interface 145. In addition, the computing device 140 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 146. Fig.14 As shown, network adapter 146 communicates with other modules of computing device 140 via bus 143. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computing device 140, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0244] It should be noted that, although several units / modules or sub-units / modules of the test device are mentioned in the above detailed description, such division is merely exemplary and not mandatory. In fact, according to an embodiment of the present disclosure, the features and functions of two or more units / modules described above may be embodied in one unit / module. Conversely, the features and functions of one unit / module described above may be further divided to be embodied by multiple units / modules.

[0245] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0246] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A testing method comprising: Displaying a test interface, the test interface comprising an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search; In response to a first interactive operation directed to the search control, displaying search results of a target search object under a target search algorithm and index data of the search results, wherein the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control; Determine the type of the corresponding search result according to the indicator data; the type includes expected search results and / or unexpected search results; If it is determined according to the indicator data that the search result does not meet the first rule requirement of the target search algorithm, then determining the type to be the unexpected search result; If it is determined according to the indicator data that the search result meets the first rule requirement, then determining the type according to the indicator data, and the first rule requirement and the second rule requirement; the first rule requirement and the second rule requirement are set according to the indicator and target search algorithm; If the first rule requirement does not conflict with the second rule requirement, the type is determined according to the indicator data and the second rule requirement; if the search result meets the second rule requirement, the type of the search result is determined to be the expected search result; if the first rule requirement does not conflict with the second rule requirement, and the search result does not meet the second rule requirement, the type of the search result is considered to be an unexpected search result; If the first rule requirement conflicts with the second rule requirement, and the search result satisfies the first rule requirement, determining the type to be the expected search result; The determining the type according to the indicator data and the second rule requirement includes at least one of the following: Determining the type according to the number of clicks on the search result, wherein the indicator data includes the number of clicks; Determining the type according to the relevance between the target search object and the search result; The type is determined according to the type and quantity of the search results, and the indicator data includes the type and quantity of the search results.

2. The testing method according to claim 1, wherein the testing interface further comprises a storage control for storing the unexpected search results, and the testing method further comprises: In response to the type of the search result being an unexpected search result, the storage control prompts to save the unexpected search result.

3. The testing method according to claim 2, wherein the prompting of saving the unexpected search results through the storage control comprises: In response to a second interactive operation directed to the storage control, the unexpected search result is saved.

4. The testing method according to claim 1, further comprising: In response to the first interactive operation, a similarity calculation control is displayed, wherein the similarity calculation control is used to initiate calculation of similarities of search results under each of the target search algorithms.

5. The testing method according to claim 4, further comprising: In response to a third interactive operation directed to the similarity calculation control, displaying a setting interface on the test interface, the setting interface including a module selection box and a comparison search algorithm selection box; In response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

6. The testing method according to claim 5, wherein in response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, comprising: Obtaining a first search result of a target module corresponding to the first target search algorithm and a second search result of a target module corresponding to the second target search algorithm; Obtaining a first ranked list according to the identifier of each search result in the first search result, and obtaining a second ranked list according to the identifier of each search result in the second search result; Acquire the similarity according to the first sorted list and the second sorted list; The similarity is displayed on the test interface.

7. The testing method according to claim 6, wherein obtaining the similarity according to the first sorted list and the second sorted list comprises: Obtain the number of intersections of the first d elements in the first sorted list and the second sorted list, and the weights corresponding to the d elements, where d is 1, 2, 3, ... in sequence; The similarity is obtained according to the number of the intersections and the weight.

8. The testing method according to claim 7, wherein obtaining the similarity according to the number of the intersections and the weight comprises: Wherein, R is the similarity, and w d is the weight corresponding to d, w d =(1-p)p d-1 , where p is a preset value, where p is greater than 0 and less than 1, where I d is the number of intersections among the first d elements.

9. The testing method according to any one of claims 1 to 5, before displaying the search results of the target search object under the target search algorithm and the index data of the search results, the testing method further comprises: Obtaining an identifier of the target search algorithm; According to the target search object and the identifier, obtaining the search result and the indicator data; or, Obtain the website address corresponding to the target search algorithm; and obtain the search results and the indicator data according to the target search object and the website address.

10. A testing device comprising: A first display module, used to display a test interface, wherein the test interface includes an input box for inputting a search object, a first control for determining a search algorithm, and a search control for initiating a search; A second display module, configured to display, in response to a first interactive operation directed to the search control, search results of a target search object under a target search algorithm, and indicator data of the search results, wherein the target search object is the search object input in the input box, and the target search algorithm is the search algorithm selected in the first control; A processing module, configured to determine the type of the corresponding search result according to the indicator data; the type includes expected search results and / or unexpected search results; If it is determined according to the indicator data that the search result does not meet the first rule requirement of the target search algorithm, then determining the type to be the unexpected search result; If it is determined according to the indicator data that the search result meets the first rule requirement, then the type is determined according to the indicator data, and the first rule requirement and the second rule requirement; the first rule requirement and the second rule requirement are set according to the indicator and the target search algorithm; If the first rule requirement does not conflict with the second rule requirement, the type is determined according to the indicator data and the second rule requirement; if the search result meets the second rule requirement, the type of the search result is determined to be the expected search result; if the first rule requirement does not conflict with the second rule requirement, and the search result does not meet the second rule requirement, the type of the search result is considered to be an unexpected search result; If the first rule requirement conflicts with the second rule requirement, and the search result satisfies the first rule requirement, determining the type to be the expected search result; The processing module is specifically used for at least one of the following: Determining the type according to the number of clicks on the search result, wherein the indicator data includes the number of clicks; Determining the type according to the relevance between the target search object and the search result; The type is determined according to the type and quantity of the search results, and the indicator data includes the type and quantity of the search results.

11. The testing device according to claim 10, wherein the testing interface further comprises a storage control for storing the unexpected search results, and the processing module is further configured to: In response to the type of the search result being an unexpected search result, the storage control prompts to save the unexpected search result.

12. The testing device according to claim 11, wherein the processing module is further configured to: In response to a second interactive operation directed to the storage control, the unexpected search result is saved.

13. The testing device according to claim 10, wherein the second display module is further used for: In response to the first interactive operation, a similarity calculation control is displayed, wherein the similarity calculation control is used to initiate calculation of similarities of search results under each of the target search algorithms.

14. The testing device according to claim 13, wherein the second display module is further used for: In response to a third interactive operation directed to the similarity calculation control, displaying a setting interface on the test interface, the setting interface including a module selection box and a comparison search algorithm selection box; In response to the fourth interactive operation directed to the similarity calculation control, the similarity of the search results of the corresponding target modules under the first target search algorithm and the second target search algorithm is displayed on the test interface, the first target search algorithm and the second target search algorithm are the target search algorithms selected in the comparison search algorithm selection box, and the target module is the module selected in the module selection box.

15. The testing device according to claim 14, wherein the second display module is further used for: Obtaining a first search result of a target module corresponding to the first target search algorithm and a second search result of a target module corresponding to the second target search algorithm; Obtaining a first ranked list according to the identifier of each search result in the first search result, and obtaining a second ranked list according to the identifier of each search result in the second search result; Acquire the similarity according to the first sorted list and the second sorted list; The similarity is displayed on the test interface.

16. The testing device according to claim 15, wherein the second display module is further used for: Obtain the number of intersections of the first d elements in the first sorted list and the second sorted list, and the weights corresponding to the d elements, where d is 1, 2, 3, ... in sequence; The similarity is obtained according to the number of the intersections and the weight.

17. The testing device according to claim 16, wherein the similarity is: in, The R is the similarity, the w d is the weight corresponding to d, w d =(1-p)p d-1 , where p is a preset value, where p is greater than 0 and less than 1, where I d is the number of intersections among the first d elements.

18. The testing device according to any one of claims 10 to 14, before displaying the search results of the target search object under the target search algorithm and the index data of the search results, the processing module is further used to: Obtaining an identifier of the target search algorithm; obtaining the search results and the indicator data according to the target search object and the identifier; or, Obtain the website address corresponding to the target search algorithm; and obtain the search results and the indicator data according to the target search object and the website address.

19. A computing device comprising: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the testing method according to any one of claims 1 to 9.

20. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the testing method according to any one of claims 1 to 9 is implemented.

21. A computer program product, comprising a computer program; when the computer program is executed, the testing method according to any one of claims 1 to 9 is implemented.

Images