Evaluation method for location search input prompt service, electronic device, and program product

Abstract

The evaluation method, electronic device, and program product for location retrieval input suggestion services disclosed herein relate to map retrieval technology. The method includes: acquiring sample data, which includes the desired location and the user's input trajectory when inputting the name of the desired location; sending a retrieval request to a first suggestion service to be evaluated in debug mode based on the input trajectory, and acquiring first intermediate strategy data and a first retrieval result when the first suggestion service processes the retrieval request; and evaluating the first suggestion service based on the first intermediate strategy data, the first retrieval result, and the desired location. This disclosure not only provides the evaluation result of the first suggestion service but also the evaluation results of each strategy set within the first suggestion service, thus providing richer evaluation indicators and offering developers directions for improving the first suggestion service.

Description

Technical Field

[0001] This disclosure relates to map retrieval technology in data processing technology, and more particularly to an evaluation method, electronic device and program product for a location retrieval input prompt service. Background Technology

[0002] Location search input suggestion service (Suggestion POI search, Sug search) refers to online suggested terms based on keywords. Users enter keywords in the input box, and the system can provide corresponding online suggested terms.

[0003] The accuracy of online suggested terms corresponding to keywords retrieved by the system directly impacts user experience. Therefore, existing technologies obtain historical logs, calculate the user input step length based on the data in these logs, and use this step length (the number of times the user enters the information) as a core metric for the Sug search service. For example, the Sug search function can be evaluated based on the user input step length when it finds suggested terms that match the user's needs.

[0004] However, this implementation method can only determine the quality of the Sug search function, but it cannot determine the cause of the poor search function when the Sug search function is poor. Summary of the Invention

[0005] This disclosure provides an evaluation method, electronic device, and program product for location search input suggestion services, in order to more accurately evaluate the merits of Sug search functions.

[0006] According to a first aspect of this disclosure, an evaluation method for a location retrieval input suggestion service is provided, comprising:

[0007] Obtain sample data, which includes the desired location and the user's input trajectory when entering the name of the desired location; the input trajectory is used to characterize the text content entered by the user in the search box at each step when entering the name of the desired location.

[0008] According to the input trajectory, a retrieval request is sent to the first prompting service to be evaluated in debug mode, and the first intermediate strategy data and the first retrieval result are obtained when the first prompting service processes the retrieval request; the first intermediate strategy data includes data obtained by the first prompting service when processing the retrieval request based on multiple strategies;

[0009] The first prompt service is evaluated based on the first intermediate strategy data, the first search results, and the requested location.

[0010] According to a second aspect of this disclosure, an evaluation device for a location search input suggestion service is provided, comprising:

[0011] The acquisition unit is used to acquire sample data, which includes the required location and the input trajectory of the user when entering the name of the required location; the input trajectory is used to characterize the text content entered by the user in the search box at each step when entering the name of the required location.

[0012] The sending unit is configured to send a retrieval request to the first prompt service to be evaluated in debug mode based on the input trajectory;

[0013] The acquisition unit is further configured to acquire first intermediate strategy data and first search result when the first prompting service processes the search request; the first intermediate strategy data includes data obtained when the first prompting service processes the search request based on multiple strategies;

[0014] An evaluation unit is used to evaluate the first prompt service based on the first intermediate strategy data, the first search result, and the required location.

[0015] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform an evaluation method for a location retrieval input suggestion service as described in the first aspect.

[0019] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform an evaluation method for a location retrieval input prompt service as described in the first aspect.

[0020] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.

[0021] The evaluation method, electronic device, and program product for location retrieval input suggestion service disclosed herein include: acquiring sample data, which includes the desired location and the user's input trajectory when inputting the name of the desired location; the input trajectory is used to characterize the text content entered by the user in the search box at each step when inputting the name of the desired location; sending a search request to a first suggestion service to be evaluated in debug mode according to the input trajectory, and acquiring first intermediate strategy data and a first search result when the first suggestion service processes the search request; the first intermediate strategy data includes data obtained by the first suggestion service when processing the search request based on multiple strategies; and evaluating the first suggestion service based on the first intermediate strategy data, the first search result, and the desired location. The evaluation method, electronic device, and program product for location retrieval input suggestion service disclosed herein can acquire the results of the first suggestion service recalling suggested terms based on the text content after the user inputs text content at each step in the search box, as well as the results of the first suggestion service recalling suggested terms based on the text content using different strategies. By analyzing this data, not only the evaluation results of the first suggestion service can be obtained, but also the evaluation results of each strategy set in the first suggestion service can be obtained. Therefore, richer evaluation indicators can be provided, thereby providing researchers with directions for improving the first suggestion service.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0024] Figure 1 A diagram illustrating an exemplary embodiment of the interface operation;

[0025] Figure 2 A schematic flowchart illustrating an evaluation method for a location retrieval input suggestion service, as shown in an exemplary embodiment of this disclosure;

[0026] Figure 3 A schematic flowchart illustrating an evaluation method for a location retrieval input suggestion service, as shown in another embodiment of this disclosure;

[0027] Figure 4 This is a schematic flowchart illustrating an evaluation method for a location search input suggestion service, as shown in yet another embodiment of the present disclosure.

[0028] Figure 5 A schematic diagram of the structure of an evaluation device for a location search input suggestion service, as shown in an exemplary embodiment of the present disclosure;

[0029] Figure 6 Structural schematic diagram of an evaluation device for location retrieval input prompt service shown in another exemplary embodiment of the present disclosure;

[0030] Figure 7 It is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. Detailed implementation manners

[0031] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] Figure 1 It is an interface operation diagram shown in an exemplary embodiment.

[0033] As Figure 1 shown, there is an input box 11 for inputting information in the map navigation software. The user can input keywords in the input box 11, so that the map navigation software queries locations corresponding to the keywords.

[0034] Generally, there is an input trajectory when the user inputs keywords. For example, the location that the user hopes to search consists of multiple words. For example, if the location that the user needs to search is "Wudaokou Shopping Center", the input trajectory of the user inputting this keyword may be "Five" -> "Wudao" -> "Wudaokou" -> "Wudaokou Shopping" -> "Wudaokou Shopping Center".

[0035] Every time the user inputs content in the input box, the retrieval service can determine the suggested locations according to the currently input content. For example, when the user inputs "Five", the service can recall the suggested locations based on "Five", sort them, and feedback several suggested locations ranked at the front to the user terminal, so that the user terminal displays these suggested locations. When the user continues to input, for example, inputs "Wudao", the service can feedback the suggested locations corresponding to "Wudao".

[0036] Among them, the process of the user inputting text is the input trajectory, and one step is added each time a character is input. For example, when inputting "Five", it is step 1, and when inputting "Wudao", the step is 2.

[0037] Specifically, the accuracy of the Sug retrieval service in feedbacking suggested locations to the user terminal directly affects the user experience. Therefore, it is necessary to evaluate the function of the Sug retrieval service in feedbacking suggested locations to determine whether the Sug retrieval service can accurately and quickly recommend suggested locations to the user.

[0038] Based on user input patterns, the search service can determine the suggested locations returned by the user after each text input. This allows the service to determine the user input step size when the suggested locations sent to the user's terminal include the desired location. The shorter the step size, the better the search service's location recommendation function. For example, if a user inputs "Wudaokou" and the search service recommends "Wudaokou Shopping Center" to the user's terminal, then the input step size for recalling the user's primary desired location, "Wudaokou Shopping Center," is 3.

[0039] In practical applications, the search service has multiple internal strategies. These strategies are used to recall and rank suggested locations, which are then pushed to the user's terminal. For example, if the search service has five strategies, the first strategy is used to process keywords and recall 100 suggested locations from a pre-defined location database. The second strategy is then used to rank these key locations. The third strategy adjusts the ranking of the suggested locations, and so on. Finally, the last strategy is used to rank the suggested locations, and the top N suggested locations are pushed to the user's terminal.

[0040] Therefore, the suggested locations and their order ultimately recommended to the user's terminal are directly related to the various strategies set in the search system.

[0041] If the retrieval service is evaluated based solely on the step size when recalling users' primary demand locations, it can only assess whether the service can quickly recall suggested locations that match user needs, but it cannot identify the strategies that lead to recall failures or slow recall speeds. Therefore, it cannot provide developers with more detailed evaluation results, nor can it directly guide them to improve the retrieval system.

[0042] To address the aforementioned technical issues, the solution provided in this disclosure involves sending a request to the first prompt service in debug mode during testing. This allows the acquisition of intermediate policy data generated by the first prompt service when processing the request. Specifically, this includes intermediate policy data generated when the service processes the request using each policy. By analyzing this data, policy-level metrics within the service can be obtained, enabling a more granular evaluation of the first prompt service.

[0043] Figure 2 This is a flowchart illustrating an evaluation method for a location retrieval input suggestion service, as shown in an exemplary embodiment of this disclosure.

[0044] like Figure 2 As shown, the evaluation method for the location search input suggestion service provided in this disclosure includes:

[0045] Step 201: Obtain sample data. The sample data includes the required location and the input trajectory when the user enters the name of the required location. The input trajectory is used to represent the text content entered in the search box at each step when the user enters the name of the required location.

[0046] Among them, the method provided by this disclosure can be executed by an electronic device with computing capabilities, such as a computer. This computer can be the background server of the Sug search service. For example, it can be an evaluation server used to evaluate the function of the Sug search service.

[0047] Specifically, the evaluation server can obtain multiple sample data, use these sample data to test the Sug search service, and evaluate the Sug search service based on the test results.

[0048] Furthermore, the sample data can be obtained by processing historical logs. Historical logs can be obtained, and based on the historical logs, the input trajectory when the user enters the location name can be obtained. For example, if the user enters "di" in the first step, "tie" in the second step, and "zhan" in the third step, then the user input trajectory is "di" -> "subway" -> "subway station".

[0049] In actual application, the required location can also be obtained from historical logs. Specifically, the word used when querying the location can be used as the name of the required location. For example, after the user enters "subway station" and then operates the user terminal to send a location search request to the background server, "subway station" can be used as the name of the required location for this search by the user, and then the required location can be determined. Another example is that after the user enters "Tsinghua", selects "Tsinghua University" from the multiple suggested locations displayed on the user terminal, and operates the user terminal to send a location search request to the background server, then "Tsinghua University" can be used as the required location for this search by the user.

[0050] Among them, historical logs can be obtained in advance, and the historical logs that meet the requirements can be screened out based on certain conditions, and then the data in the historical logs can be extracted. Specifically, the input trajectory when the user enters the name of the required location in each historical log can be extracted, and the required location in this historical log can also be extracted.

[0051] Specifically, the historical logs that meet the requirements can be screened based on conditions such as time conditions and the range of the retrieved locations.

[0052] Step 202: Send a search request to the first hint service to be evaluated in the debug mode according to the input trajectory, and obtain the first intermediate policy data and the first search result when the first hint service processes the search request. The first intermediate policy data includes the data obtained when the first hint service processes the search request based on multiple policies.

[0053] The first suggestion service refers to the Sug search service that is yet to be evaluated.

[0054] Furthermore, the first notification service can be set up in either a test environment or a preview machine. The test environment is used to test services that are under development, while the preview machine is used to test services that are about to be launched.

[0055] In practical applications, the evaluation server can send a retrieval request to the test environment or preview machine, so that the first suggestion service can process the request.

[0056] Specifically, a retrieval request can be sent based on the input trajectory. Multiple search terms can be determined based on the input trajectory, thereby constructing a retrieval request corresponding to each search term. This allows the first suggestion service to process each search term and recall the corresponding suggested terms. The first suggestion service can store the first retrieval results obtained by processing the search terms in a preset database, such as a Redis (Remote Dictionary Server) database.

[0057] Specifically, when the first suggestion service processes each search term, it will process the search term based on a preset strategy. In order to obtain the first intermediate strategy data obtained when processing search terms using different strategies, the evaluation server can send a search request to the first suggestion service to be evaluated in debug mode.

[0058] Furthermore, the debug mode of the first prompt service can be set to 1, thereby enabling the first prompt service to receive retrieval requests in debug mode.

[0059] In practical applications, after receiving a search request in debug mode, the First Suggestion Service processes the request based on various internally configured strategies. Specifically, it retrieves suggested locations based on the search terms included in the request and sorts these suggested locations. Simultaneously, after processing the search terms based on preset strategies, the First Suggestion Service can store the processing results in a database, such as the Redis database mentioned above.

[0060] The processing results of different strategies can include the recalled suggested locations and their ranking. For example, the first suggestion service processes the search terms based on strategy 1, which can recall 100 suggested locations. These 100 suggested locations and their ranking results can then be stored in the database. Next, strategy 2 is used to process the search terms and the recalled suggested locations, and these suggested locations are ranked. Strategy 3 and other strategies can be used to adjust and re-rank the ranking. Based on the final ranking result, the top N suggested locations can be selected and fed back to the user terminal.

[0061] Specifically, the evaluation server can obtain the first retrieval results corresponding to each retrieval request and the first intermediate policy data from the database, so as to evaluate the first hint service using these data.

[0062] Step 203: Evaluate the first hint service according to the first intermediate policy data, the first retrieval results, and the demand location.

[0063] Furthermore, the evaluation server can construct retrieval requests using multiple input trajectories. Therefore, the first intermediate policy data and the first retrieval results corresponding to multiple retrieval requests can be obtained. And multiple sample data can be used for testing. Therefore, these data can be used to determine the effect of the first hint service in recalling the demand locations in the sample data.

[0064] In actual application, the first retrieval result is the final recall result of the first hint service. The first hint service can be evaluated as a whole using the first retrieval result. For example, if a retrieval request is archived, and its corresponding demand location is "Wumart Supermarket", and "Wumart Supermarket" is not included in the first retrieval results of this retrieval request, it can be considered that the overall recall ability of the first hint service is poor.

[0065] Among them, the first intermediate policy data is the result of the first hint service processing the retrieval request using different policies. Therefore, based on this data, the recall effects of different policies can be determined. For example, if a retrieval request is archived, and its corresponding demand location is "Wumart Supermarket", and the search term in the currently constructed retrieval request is "Wu", and Policy 1 recalls the demand location "Wumart Supermarket", then it is considered that Policy 1 meets the requirements. If the demand location is ranked in a relatively forward position when sorting the already recalled recommended locations based on Policy 2, it can be considered that the recall effect of Policy 2 is relatively good. If the demand location is ranked in a relatively backward position when further optimizing the sorting result based on Policy 3, it can be determined that the recall effect of Policy 3 is poor.

[0066] Specifically, when constructing a retrieval request according to the input trajectory, the input content corresponding to different input step lengths is different, and the constructed retrieval requests are also different. Correspondingly, the first intermediate policy data and the first retrieval results are also different. The input step length required to recall a demand location can be determined according to the input step length corresponding to the retrieval request. For example, the step length required to recall "Tsinghua University" is 1. The recall effect of the first hint service can be evaluated according to the input step length required to recall the demand location using the first hint service. The recall effect of the policy can also be evaluated according to the input step length required to recall the demand location using a policy.

[0067] The first prompt service can be evaluated using the retrieval request corresponding to each sample data, the first retrieval result, and the first intermediate strategy data, thus enabling the evaluation of the service using a large amount of data.

[0068] In this way, not only can the retrieval effect of the first prompt service itself be evaluated, but also the recall effect of each strategy within the first prompt service can be evaluated, thus obtaining more granular evaluation results.

[0069] The evaluation method for a location search input suggestion service provided in this disclosure includes: acquiring sample data, which includes the desired location and the user's input trajectory when inputting the name of the desired location; the input trajectory is used to characterize the text content entered by the user in the search box at each step when inputting the name of the desired location; sending a search request to a first suggestion service to be evaluated in debug mode according to the input trajectory, and acquiring first intermediate strategy data and first search results when the first suggestion service processes the search request; the first intermediate strategy data includes data obtained by the first suggestion service when processing the search request based on multiple strategies; and evaluating the first suggestion service based on the first intermediate strategy data, the first search results, and the desired location. The method provided in this disclosure can obtain the results of the first suggestion service recalling suggested terms based on the text content after the user inputs text content at each step in the search box, as well as the results of the first suggestion service recalling suggested terms based on the text content using different strategies. By analyzing this data, not only the evaluation results of the first suggestion service can be obtained, but also the evaluation results of each strategy set in the first suggestion service can be obtained. Therefore, richer evaluation indicators can be provided, thereby providing developers with directions for improving the first suggestion service.

[0070] Figure 3 This is a schematic flowchart illustrating an evaluation method for a location retrieval input suggestion service, as shown in another embodiment of this disclosure.

[0071] like Figure 3 As shown, the evaluation method for the location search input suggestion service provided in this disclosure includes:

[0072] Step 301: Obtain the location search input suggestion service to process the content entered by the user and query the historical logs generated by the suggested locations.

[0073] The location search input suggestion service can be an online service. When a user searches for a location using navigation software, the online location search input suggestion service can process the user's input, query suggested locations, and provide feedback to the user's terminal.

[0074] Specifically, the server for the location search input suggestion service can also generate and store logs when querying suggested locations based on the user's input.

[0075] Further, the evaluation server can obtain these historical logs, which may include the input trajectory when storing the user input content, and may also store the recommended location finally selected by the user.

[0076] For example, after the user enters the content and selects Location A, Location A can be used as the recommended location. Each time the user enters text content in the input box, it can form the user's input trajectory.

[0077] In an optional implementation, conditions for filtering historical logs can be set to reduce the amount of data processing. For example, historical logs for the most recent 1 month can be filtered. In this way, the problem of evaluation failure caused by excessive data volume can be avoided.

[0078] For example, in the prior art, online logs are pulled every hour, and the online logs generated within 1 hour are analyzed to evaluate whether the recommended location recalled by the online service is accurate. However, in this case, due to the extremely large amount of log data within each hour, evaluation failures often occur. By filtering the logs, the amount of data to be processed can be reduced, thus avoiding evaluation failures.

[0079] Step 302, obtain the recommended location according to the historical logs, and the input trajectory when the user enters the name of the recommended location.

[0080] Further, the evaluation server can process the obtained historical logs to extract sample data. Specifically, the recommended location that the user hopes to retrieve and the input trajectory when the user enters the name of the recommended location can be extracted from the historical logs. For example, the input trajectory when the user enters the name of the recommended location may be "Five" -> "Wudao" -> "Wudaokou" -> "Wudaokou Shopping" -> "Wudaokou Shopping Center".

[0081] Through this implementation, based on the historical logs generated by the online location retrieval input prompt service, the recommended location that the user hopes to retrieve and the real input trajectory when the user enters the name of the recommended location can be extracted to construct sample data, and then these sample data can be used to evaluate the first prompt service to be evaluated, which can be more in line with the real retrieval situation, and thus evaluate the first prompt service more accurately.

[0082] In an optional implementation manner, the location retrieval input prompt service that generates the historical logs and the first prompt service can be the same service. In this implementation manner, the method provided by the present disclosure can be used to evaluate the online set location retrieval input prompt service.

[0083] Step 303: Determine multiple search terms for the sample data based on the input trajectory, and send the search terms to the first prompt service in debug mode.

[0084] The user input trajectory includes the input content corresponding to each step. For example, if the first step is to input "five", the second step is to input "five roads", and the third step is to input "five intersections", then the search terms can be determined based on the input content at each step. For example, the first search term is "five", the second search term is "five roads", and the third search term is "five intersections". For each sample data, at least one search term corresponding to that sample data can be determined.

[0085] Specifically, the evaluation server can send each search term to the first suggestion service one by one, so that the first suggestion service can process each search term separately.

[0086] In practical applications, when a user enters the name of a desired location into navigation software, the suggested locations displayed by the software will update as more information is entered. Specifically, the online SUG service of the navigation software is based on the current content in the input box.

[0087] Therefore, by sending search terms corresponding to the user's input trajectory to the first prompt service one by one, it is possible to reconstruct the process by which the first prompt service provides suggested locations to the user terminal based on changes in the user's input.

[0088] When the first prompt service processes any search term, it does so based on various internally set strategies. Therefore, the result of processing the search term based on the preset strategies directly affects the overall recall result of the first prompt service.

[0089] In order to obtain the recall or ranking results obtained by the first suggestion service when processing search terms based on each strategy, the solution provided in this disclosure involves the evaluation server sending each search term to the first suggestion service in debug mode. This enables the first suggestion service to store the first intermediate strategy data generated by processing search terms based on each strategy.

[0090] Specifically, the first suggestion service has a debug mode configuration parameter. Setting it to 1 indicates that debug mode requests can be received, while 0 indicates that debug mode requests cannot be received. Setting this configuration parameter to 1 allows the evaluation server to send debug requests containing search terms to the first suggestion service, enabling the service to process the search terms.

[0091] Step 304: Obtain the first retrieval results when the first hint service processes each search term, and the first intermediate policy data corresponding to the policy and the search term generated when the first hint service processes each search term based on multiple policies.

[0092] Furthermore, the first hint service can process each search term based on multiple policies. For example, for search term 1, the entry policy can be used to process search term 1 to recall N1 suggested locations, then the first policy is used to sort the N1 recalled suggested locations, then the second policy is used to re-sort the sorted suggested locations, and then the third policy is used to re-sort the sorted suggested locations, and so on. Finally, the sorting result of these N1 suggested locations is obtained, and the top N suggested locations are fed back to the user terminal.

[0093] In actual application, the N suggested locations finally fed back to the user terminal are the first retrieval results when the first hint service processes search term 1.

[0094] Among them, the recall results or sorting results obtained based on different policies are the first intermediate policy data. For example, it can include the recall result of using the entry policy to process search term 1, and can also include the sorting result of using the first policy to sort the N1 suggested locations, and can also include the sorting result of using the second policy to re-sort the N1 suggested locations, etc.

[0095] In this way, the recall results or sorting results obtained when each policy processes the search term can be obtained, so that these results can be used to evaluate each preset policy in the first hint service to obtain a more fine-grained evaluation result.

[0096] Step 305: Obtain the input step length corresponding to the search term.

[0097] Specifically, the search terms determined based on the user's input trajectory have an input step length, and the evaluation server can also obtain the input step length corresponding to each search term. For example, if the user enters the search term in one step, the input step length of this search term is 1. Another example is that if the user enters the search term in three steps, the input step length of this search term is 3.

[0098] For example, in the above example, when the input trajectory is "five" -> "five roads" -> "Wudaokou" -> "Wudaokou purchase" -> "Wudaokou shopping" -> "Wudaokou shopping center" -> "Wudaokou Shopping Mall", the input step length of "five" is 1, the input step length of "five roads" is 2, the input step length of "Wudaokou" is 3, the input step length of "Wudaokou purchase" is 4, the input step length of "Wudaokou shopping" is 5, the input step length of "Wudaokou shopping center" is 6, and the input step length of "Wudaokou Shopping Mall" is 7.

[0099] Step 306: Determine the shortest service recall step length required for the first prompt service to rank the demand location in the sample data before the specified position according to the first retrieval result corresponding to each retrieval term of the sample data.

[0100] Furthermore, the evaluation server can comprehensively evaluate the retrieval effect of the first prompt service as a whole based on the first retrieval results of each retrieval term of multiple sample data.

[0101] In actual application, when the first prompt service feeds back suggested locations to the user terminal, the higher the ranking of the demand location, the better the user experience. Therefore, it can be determined the service recall step length required for the first prompt service to rank the demand location desired by the user before the specified position.

[0102] Among them, the specified position can be preset, for example, it can be 3. For example, it can be determined that when the user searches for the demand location, the step length of the retrieval term required for the first prompt service to rank the demand location among the top three.

[0103] A sample data has multiple retrieval terms, and each retrieval term has an input step length. When feeding back suggested locations based on the retrieval terms, if the suggested location is ranked higher and the input step length of the corresponding retrieval term is shorter, it indicates that the recall effect of the first prompt service is better. The solution provided by this disclosure can obtain the shortest service recall step length required to rank the demand location before the specified position when using a sample data for testing. For example, when the demand location is "Wudaokou Shopping Center" and the retrieval terms are respectively "Five", "Wudao", "Wudaokou", "Wudaokou Shopping", "Wudaokou Shopping Center", if the retrieval term is "Wudaokou" and Wudaokou Shopping Center ranks third in the first retrieval result, and when the retrieval terms are respectively "Wudaokou Shopping", "Wudaokou Shopping Center", Wudaokou Shopping Center ranks first in the first retrieval result, then the retrieval term required for the first prompt service to rank the demand location among the top three is 3.

[0104] When processing multiple retrieval terms of the same sample data, there may be a scenario where the demand location ranks among the top three in the first retrieval results corresponding to multiple retrieval terms. In this scenario, the shortest input step length among these retrieval terms is determined as the service recall step length required for the demand location to be ranked before the specified position.

[0105] The service recall step length can reflect the ability of the first prompt service to recall the demand location. Therefore, the determined service recall step length can be used to comprehensively evaluate the recall effect of the first prompt service.

[0106] In one alternative implementation, when testing with sample data, there may be cases where recall fails. For example, if the first prompt service does not rank the required location in the top three even when the input step size is the largest, the service recall step size of the sample data can be set to -1.

[0107] Step 307: Based on the first intermediate strategy data corresponding to the strategy and the search terms, determine the shortest strategy recall step length corresponding to each strategy.

[0108] Furthermore, for a search term, the first suggestion service will process the search term based on each strategy to obtain first intermediate strategy data, which may include, for example, recall results and ranking results.

[0109] For example, using entry strategy 0 to process search term 1 can yield recall results. Similarly, using strategy 1 to rank the suggested locations yields the first ranked result, and so on. Using strategy N to rerank the suggested locations yields the Nth ranked result. For another search term, recall or ranking results can also be obtained based on the processing of that search term using various strategies.

[0110] For example, the following correspondence can be obtained between strategies, search terms, and first intermediate strategy data (including recall results and ranking results): entry strategy 0 - search term 1 - recall result 01, entry strategy 0 - search term 2 - recall result 02, strategy 1 - search term 1 - ranking result 11, strategy 1 - search term 2 - ranking result 12, etc.

[0111] When testing with sample data, the shortest strategy recall step size can be determined based on the first intermediate strategy data that corresponds to each search term in the strategy and the sample data. The strategy recall step size characterizes the shortest step required to process each search term in the sample data using this strategy, recall suggested locations, or sort suggested locations, and place the desired location before a preset position.

[0112] For example, when using an entry strategy to recall suggested locations for search term 1, the strategy recall step size required to recall the desired locations is as follows: The entry strategy sorts locations in a pre-defined location database based on the search term and selects the top N1 locations as suggested locations. N1 can be, for example, 100, 200, etc., and the specific value is not limited.

[0113] When processing multiple search terms for a sample of data, there may be a situation where the required location can be successfully recalled when processing multiple search terms based on the entry strategy. In this case, the shortest input step size among the search terms that successfully recall the suggested location is determined as the strategy recall step size.

[0114] For example, when using strategy 1 to sort suggested locations for search term 1, what is the strategy recall step size required to rank the desired location within the top 50? For instance, the first suggestion service uses strategy 1 to sort the recalled N1 suggested locations.

[0115] When processing multiple search terms in the sample data, if a certain strategy can be used to process multiple search terms and the desired location can be ranked before the specified sorting, then the shortest input step length among these search terms can be determined as the strategy recall step length.

[0116] In one optional implementation, the strategies set in the first suggestion service include a recall strategy and a ranking strategy. For example, the first entry strategy used to process search terms is a recall strategy, which is used to recall suggested locations based on the search terms.

[0117] When determining the recall step size for a recall strategy, the shortest first recall step size required to recall the desired locations in the sample data can be determined based on the first intermediate strategy data corresponding to each search term in the recall strategy and sample data.

[0118] For each piece of sample data, the shortest first recall step size required for the recall strategy to successfully recall the desired location can be determined. For example, if a sample data is recalled using a search term with a step size of 1 but the desired location is not recalled, but the desired location is successfully recalled using a search term with a step size of 2, then the first recall step size for the recall strategy to process the sample data is 2.

[0119] When determining the recall step size for a ranking strategy, the shortest second recall step size required for the ranking strategy to rank the desired locations in the sample data before a preset position is determined based on the first intermediate strategy data corresponding to the ranking strategy and each search term in the sample data.

[0120] For each sample data point, the shortest second recall step size can be determined to prioritize the desired location when processing it with a ranking strategy. For example, for a sample data point with a step size of 1 for a search term, if ranking strategy 1 sorts the suggested locations already recalled for that search term and ranks the desired location within the top 50, then ranking strategy 1 has a second recall step size of 1. If ranking strategy 2 sorts the suggested locations already recalled for that search term and ranks the desired location after 30, but if ranking strategy 2 sorts the suggested locations already recalled for a sample data point with a step size of 2 and ranks the desired location before 30, then ranking strategy 2 has a second recall step size of 2.

[0121] When determining the policy recall step length for a sorting policy, the shortest third recall step length required for the policy recall demand location can also be determined according to the first intermediate policy data corresponding to the sorting policy and each search term of the sample data.

[0122] Some sorting policies also have a filtering logic, which filters some locations in the processing result of the previous policy, and it is possible to filter out the demand location. Therefore, the filtering logic of the sorting policy can be measured by the third recall step length.

[0123] Specifically, the third recall step length of a policy can be determined according to the intermediate policy data obtained by the previous policy of a policy for processing the search term and the intermediate policy data obtained by the current policy for processing the search term.

[0124] For example, when the user inputs "Wudao", the demand location is recalled in the entry policy at this time, but the demand location may be filtered at sorting policy 1. As a result, when the user inputs the three characters of "Wudaokou", sorting policy 1 recalls the demand location. Then, the shortest recall step length of sorting policy 1 is 3.

[0125] Through this implementation method, different methods can be used to obtain the corresponding policy recall step lengths for different policies, so as to be able to evaluate different types of policies from different perspectives, making the evaluation results more comprehensive, and also being able to obtain the evaluation indicators of different types of policies through different methods, making the obtained indicators more accurate.

[0126] Step 308, evaluate the first prompt service according to the service recall step length and the policy recall step length corresponding to each policy.

[0127] The service recall step length can evaluate the overall recall effect of the first prompt service, and the policy recall step length corresponding to each policy can evaluate the recall effect of the policy. Therefore, based on these two parts of data, the first prompt service can be evaluated more accurately.

[0128] In the solution provided by the present disclosure, not only can the recall effect of the first prompt service itself be evaluated, but also the recall effects of each policy set in the service can be evaluated, so as to provide a more fine-grained evaluation result to provide an improvement direction for the service to the R & D personnel.

[0129] According to each service recall step length, determine the service average step length when the first prompt service ranks the demand location before the specified position; among them, the service average step length is used to evaluate the overall recall effect of the first prompt service, and the shorter the service average step length, the better the recall effect of the first prompt service.

[0130] When testing the first prompt service using each sample data, the shortest service recall step length required to rank the demand location in the sample data before the specified position can be obtained. Therefore, the average service step length can be determined by using the service recall step length corresponding to each sample data.

[0131] Developers can refer to the average service step size to determine the recall effect of the first-pickup service. The shorter the average service step size, the better the recall effect of the first-pickup service. That is, the less information is required, the better the first-pickup service can place the requested location before the specified location.

[0132] When processing each search term in sample data using a strategy, the shortest strategy recall step length corresponding to that strategy can be determined. For example, it could be the shortest first recall step length required to recall the demand location in the sample data, or the shortest second recall step length required to rank the demand location before a preset position. Therefore, the average strategy step length can be determined by using multiple shortest recall step lengths corresponding to a strategy.

[0133] Researchers can refer to the average step size of a strategy to determine its recall effectiveness. A shorter average step size indicates better recall; that is, the strategy requires less input to rank the requested location before the preset position.

[0134] This implementation method can obtain the overall service average step size for evaluating the first prompt service, as well as the strategy average step size for evaluating each strategy. Therefore, the solution provided in this disclosure can provide quantitative indicators at the service level and quantitative indicators at the strategy level, enabling R&D personnel to quickly locate the cause of service problems and thus guide them to optimize each strategy in the service.

[0135] Steps 309 and 310 may also be included after step 304.

[0136] Step 309: Based on the first search results corresponding to each search term in the sample data, determine the service recall rate of the first suggestion service that ranks the demand location in the sample data before the specified position; wherein, the service recall rate is used to evaluate the overall recall effect of the first suggestion service, and the higher the service recall rate, the better the recall effect of the first suggestion service.

[0137] Specifically, the first suggestion service can be used to process each search term in the sample data to obtain the first search results. For example, the first search results may include multiple suggested locations and a ranking of the suggested locations. Based on the first search results, the service recall rate that ranks the desired locations in the sample data before the specified locations in the final results obtained by the first suggestion service processing the search terms can be determined.

[0138] For example, if the first suggestion service processes a total of M search terms, and among the m first search results, the required location corresponding to the search term is ranked before the specified location, then the service recall rate is m / M.

[0139] Researchers can use service recall rate to determine the effectiveness of the first-pickup service. A higher service recall rate indicates a better recall effect, meaning the service has a higher probability of prioritizing the requested location over its intended location in the suggested locations list during recall.

[0140] Step 310: Based on the first intermediate strategy data corresponding to each search term in the strategy and sample data, determine the strategy recall rate when processing the search terms using the strategy; wherein, the strategy recall rate is used to evaluate the recall effect of the strategy, and the higher the strategy recall rate, the better the recall effect of the strategy.

[0141] Specifically, when the first suggestion service processes each search term in the sample data, it does so based on various internally set strategies. For example, for a search term, it first uses a recall strategy to process the search term, then uses a first ranking strategy to rank the suggested locations retrieved by the search term, and then uses a next ranking strategy to re-rank the suggested locations.

[0142] When processing a search term using a recall strategy, first intermediate strategy data can be stored. This data determines whether the desired location in the sample data can be successfully recalled in the current processing result. Similarly, when processing a search term using a ranking strategy, first intermediate strategy data can be stored. This data determines whether the desired location in the sample data can be ranked before a second specified position in the current processing result. For example, when processing a search term using the first ranking strategy, can the desired location be ranked in the top 50? Or, for example, when processing a search term using the second ranking strategy, can the desired location be ranked in the top 30?

[0143] Using this data, we can determine the strategy recall rate for each strategy when processing search terms. Specifically, this can include the strategy recall rate for successfully recalling desired locations, and it can also include the strategy recall rate for placing desired locations in the sample data before a second specified position.

[0144] For each sorting strategy, a corresponding second specified position can be set. For example, the second specified position for the first sorting strategy is 50, and the second specified position for the second sorting strategy is 30.

[0145] Developers can use strategy recall rate to determine the recall effectiveness of each strategy in the first prompt service. The higher the strategy recall rate, the better the recall effect of the strategy. That is, when recalling, the strategy has a higher probability of placing the demand location before the second preset position, or has a higher probability of successfully recalling the demand location.

[0146] This implementation method allows us to obtain the overall service recall rate for evaluating the first prompt service, as well as the strategy recall rate for evaluating each strategy. Therefore, the solution provided in this disclosure can provide quantitative indicators at the service level and at the strategy level, enabling developers to quickly locate the causes of service problems and thus guide them to optimize the strategies in the service.

[0147] Optionally, the solution provided in this disclosure may also include steps 311 and 312.

[0148] Step 311: Based on the first search result, determine the failed search request for the location of the recall request that failed, and determine the first target strategy data corresponding to the failed search request in the first intermediate strategy data.

[0149] The above-described scheme allows for the identification of metrics for evaluating the overall recall effectiveness of the first-prompt service, such as the service recall step size and service recall rate. It also allows for the identification of metrics for evaluating the recall effectiveness of individual strategies, such as the strategy recall step size and strategy recall rate. Furthermore, the scheme provided in this disclosure can analyze individual cases of recall identification on a case-by-case basis.

[0150] Specifically, failed search requests that failed to retrieve the required location can be determined based on the first search result. For example, if the first search result does not include the required location corresponding to the search term after a search request is sent based on the first search term, then the request is considered a failed search request.

[0151] The first target strategy data for a failed retrieval request can be obtained from each of the first intermediate strategy data. For example, after a failed retrieval request is sent, the first prompt service processes the failed retrieval request using the entry strategy and obtains one piece of intermediate strategy data, which is the first target strategy data. After the first prompt service processes the failed retrieval request using the first sorting strategy, it obtains another piece of intermediate strategy data, which is another first target strategy data.

[0152] Step 312: Based on the first target retrieval data, determine the target strategy that caused the retrieval failure.

[0153] The first target retrieval data can characterize the results obtained by the first prompt service in processing failed retrieval requests using different strategies. These results determine the final retrieval failure. Therefore, the target strategy that caused the retrieval failure can be determined through this data.

[0154] Specifically, we can determine the recall or ranking results of the desired location by using the primary target data and the processing results of different strategies. For example, if recalling the desired location fails, we can assume that the recall strategy is the cause of the failure. If a ranking strategy places the desired location in a lower position, then the ranking strategy may be the reason for the failure.

[0155] In this way, the cause of failure for each failed search request can be identified, thus providing R&D personnel with more precise directions for improvement.

[0156] In one optional implementation, the recall result of the first prompt service based on each strategy to recall the demand location can be determined according to the first target retrieval data, wherein the recall result includes success or failure; then, according to the order in which the first prompt service executes strategies when processing the retrieval request, the first strategy with a failure result is determined as the target strategy that caused the retrieval failure.

[0157] In practical applications, the first suggestion service uses strategies to process search terms one by one. For example, if there are 5 strategies, the search terms are processed according to the order of the strategies, and finally the top N suggested locations are obtained and fed back to the user terminal. For example, the top 10 suggested locations are fed back to the user terminal.

[0158] Therefore, if the top-ranked strategies have poor recall performance, it may directly lead to retrieval failure.

[0159] Based on this, the solution provided in this disclosure can determine the recall result of the first prompt service recalling the required locations based on each strategy by retrieving data from each first target, and the recall result includes success or failure.

[0160] For the recall strategy, if the requested location is successfully recalled based on the recall strategy, the recall result is considered successful; otherwise, it is considered a failure. For the ranking strategy, if the requested location is ranked before the second specified position based on the ranking strategy, the recall is considered successful; otherwise, the recall is considered a failure.

[0161] Then, following the order in which the first prompt service processes the search request, the first strategy that resulted in a failed retrieval is identified as the target strategy that caused the search failure.

[0162] In this way, based on the data obtained from retrieval using each strategy and the order in which the strategies are executed, the target strategy that caused the retrieval failure can be accurately located, providing R&D personnel with more precise optimization directions.

[0163] Figure 4 This is a schematic flowchart illustrating an evaluation method for a location search input suggestion service, as shown in another embodiment of the present disclosure.

[0164] The method provided in this disclosure can be used with Figure 3 The illustrated embodiments can be combined or executed independently.

[0165] like Figure 4 As shown, the evaluation method for the location search input suggestion service provided in this disclosure includes:

[0166] Step 401: Obtain sample data, which includes the desired location and the user's input trajectory when entering the name of the desired location. The input trajectory is used to represent the text content entered by the user in the search box at each step when entering the name of the desired location.

[0167] The implementation of step 401 is similar to that of step 201, and will not be described again.

[0168] Step 402: In debug mode, send a retrieval request to the first prompt service to be evaluated according to the input trajectory, and obtain the first intermediate strategy data and the first retrieval result when the first prompt service processes the retrieval request; the first intermediate strategy data includes the data obtained by the first prompt service when processing the retrieval request based on multiple strategies.

[0169] The implementation method of step 402 is similar to that of step 202, and will not be described again.

[0170] Step 403: Send a retrieval request to the second prompting service in debug mode according to the input trajectory, and obtain the second intermediate strategy data and the second retrieval result when the second prompting service processes the retrieval request; the second intermediate strategy data includes the data obtained by the second prompting service when processing the retrieval request based on multiple strategies.

[0171] The second suggestion service could be a location search input suggestion service for online applications, for example, it could be... Figure 3 The location search input suggestion service mentioned in step 301.

[0172] In the solution provided in this disclosure, in addition to sending a search request to the first prompting service using sample data, a search request is also sent to the second prompting service. By comparing the search results generated by the two prompting services with the intermediate strategy data, the reason why the recall effect of the first prompting service is reduced compared to the second prompting service can be determined.

[0173] In a practical application scenario, for example, the first prompt service can be an optimized service or a service updated by the R&D personnel. Through the solution provided in this disclosure, it is possible to compare whether the recall effect of the first prompt service is improved compared with the second prompt service in actual online application, thereby providing optimization direction for users to optimize the first prompt service.

[0174] Step 404: Evaluate the first prompt service based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the required location.

[0175] The evaluation server can determine the indicators for evaluating the recall effect of the first prompt service based on the first intermediate strategy data, the first search results, and the demand location obtained from the first prompt service. It can also determine the indicators for evaluating the recall effect of the second prompt service based on the second intermediate strategy data, the second search results, and the demand location obtained from the second prompt service. By comparing the indicators of the two services, it can determine the reasons why the recall effect of the first prompt service is lower than that of the second prompt service, or the reasons why the recall effect of the first prompt service is not significantly improved compared to the second prompt service.

[0176] Specifically, if based on Figure 3 The illustrated embodiment sends a retrieval request to the service. For example, the retrieval request includes multiple sub-requests corresponding to the search terms, which are determined based on the input trajectory. The first retrieval result and the second retrieval result respectively include the retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data respectively include the intermediate strategy data corresponding to each search term.

[0177] The first suggestion service is evaluated based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the required location, including:

[0178] Obtain the input step size corresponding to the search term;

[0179] Based on the first search results corresponding to each search term, determine the first service recall step size required for the location of the first prompt service recall demand; based on the second search results corresponding to each search term, determine the second service recall step size required for the location of the second prompt service recall demand.

[0180] Based on the first intermediate strategy data corresponding to the strategy and the search terms, determine the recall step size of each first strategy corresponding to each strategy; based on the second intermediate strategy data corresponding to the strategy and the search terms, determine the recall step size of each second strategy corresponding to each strategy.

[0181] The first prompt service is evaluated based on the first service recall step size, the second service recall step size, the first strategy recall step size corresponding to each strategy, and the second strategy recall step size.

[0182] For specific methods on determining the service recall step size and strategy recall step size, please refer to... Figure 3 The illustrated embodiment. Both the first strategy recall step size and the second strategy recall step size can include a first recall step size, a second recall step size, and a third recall step size.

[0183] This implementation method allows for comparison of the recall step size of the first service and the recall step size of the second service, thereby determining whether the overall recall effect of the first prompt service is improved compared to the second prompt service. It also allows for comparison of the recall step size of the first strategy and the recall step size of the second strategy corresponding to each strategy, thereby determining whether the recall effect of the first prompt service is improved compared to the second prompt service for the same strategy.

[0184] In general, different versions of the notification service have the same strategy structure, such as five strategies. However, the specific implementation of each strategy may differ, leading to different feedback suggestion locations. Therefore, the above implementation method can determine whether updating the internal implementation of the same strategy improves the recall effect.

[0185] The table below shows a comparison of service recall step size and policy recall step size in an exemplary embodiment:

[0186]

[0187] Where base represents the second suggestion service and test represents the first suggestion service. The shortest recall step size of sorting strategy one is the average of the third recall step size of sorting strategy one.

[0188] In an alternative implementation, steps 402 and 403 may be followed by:

[0189] Step 405: Based on the first search results corresponding to each search term in the sample data, determine the first service recall rate of the first suggestion service that ranks the demand location in the sample data before the specified position; based on the second search results corresponding to each search term in the sample data, determine the second service recall rate of the second suggestion service that ranks the demand location in the sample data before the specified position.

[0190] Step 406: Determine the first strategy recall rate when processing the search terms using the strategy based on the first intermediate strategy data corresponding to each search term in the strategy and sample data; determine the second strategy recall rate when processing the search terms using the strategy based on the second intermediate strategy data corresponding to each search term in the strategy and sample data.

[0191] Step 407: Evaluate the first prompt service based on the first service recall rate, the second service recall rate, the first strategy recall rate, and the second strategy recall rate.

[0192] For details, please refer to... Figure 3 The illustrated embodiment determines a first service recall rate, a second service recall rate, a first strategy recall rate, and a second strategy recall rate.

[0193] The table below shows a comparison of service recall rate and policy recall rate according to an exemplary embodiment:

[0194]

[0195] This implementation method allows for comparison of the recall rates of the first and second services, thereby determining whether the overall recall effect of the first notification service is improved compared to the second notification service. It also allows for comparison of the recall rates of the first and second strategies corresponding to each strategy, thereby determining whether the recall effect of the first notification service is improved compared to the second notification service for the same strategy.

[0196] In one optional implementation, if there is a failed retrieval request in the first prompt service, intermediate strategy data obtained by the second prompt service in processing the corresponding request can also be obtained. By comparing this data, the reason for the failure of the first prompt service can be determined more accurately.

[0197] The table below shows the comparison results of intermediate strategy data for failed retrieval requests, as illustrated in an exemplary embodiment:

[0198]

[0199] This shows that the reason why the current version of the prompt service failed to retrieve information is that the entry strategy did not recall the required location.

[0200] In the table, 1 indicates that the strategy recall was successful, or the strategy ranked the demand location before the specified position, while -1 indicates that the strategy recall failed, or the strategy ranked the demand location after the specified position.

[0201] Figure 5 This is a schematic diagram illustrating the structure of an evaluation device for a location search input prompt service, as shown in an exemplary embodiment of this disclosure.

[0202] like Figure 5 As shown, the evaluation device 500 for the location search input suggestion service provided in this disclosure includes:

[0203] The acquisition unit 510 is used to acquire sample data, which includes the required location and the input trajectory when the user enters the name of the required location; the input trajectory is used to characterize the text content entered by the user in the search box at each step when entering the name of the required location.

[0204] Sending unit 520 is used to send a retrieval request to the first prompt service to be evaluated in debug mode according to the input trajectory;

[0205] The acquisition unit 510 is further configured to acquire first intermediate strategy data and first search result when the first prompting service processes the search request; the first intermediate strategy data includes data obtained when the first prompting service processes the search request based on multiple strategies;

[0206] Evaluation unit 530 is used to evaluate the first prompt service based on the first intermediate strategy data, the first search result, and the required location.

[0207] The evaluation device for the location search input suggestion service disclosed herein can obtain the results of the first suggestion service recalling suggested words based on the text content after the user enters text content in the search box at each step, as well as the results of the first suggestion service recalling suggested words based on the text content using different strategies. By analyzing this data, not only the evaluation results of the first suggestion service can be obtained, but also the evaluation results of each strategy set in the first suggestion service can be obtained. Therefore, it can provide richer evaluation indicators, thereby providing researchers with directions for improving the first suggestion service.

[0208] Figure 6 This is a schematic diagram illustrating the structure of an evaluation device for a location retrieval input prompt service, as shown in another exemplary embodiment of this disclosure.

[0209] like Figure 6 As shown, in the evaluation device 600 for the location search input suggestion service provided in this disclosure, the acquisition unit 610 and Figure 5 The acquisition unit 510 shown is similar to the sending unit 620. Figure 5 The sending unit 520 shown is similar to the evaluation unit 630. Figure 5 The evaluation unit 530 shown is similar.

[0210] Specifically, the sending unit 620 is used to determine at least one search term of the sample data based on the input trajectory, and send the search term to the first prompt service in debug mode;

[0211] The acquisition unit 610 is specifically used to acquire the first search results when the first prompt service processes each of the search terms, and the first intermediate strategy data generated by the first prompt service when processing each of the search terms based on multiple strategies, corresponding to the strategies and the search terms.

[0212] The evaluation unit 630 includes:

[0213] The acquisition module 631 is used to acquire the input step size corresponding to the search term;

[0214] The service step size determination module 632 is used to determine, based on the first search result corresponding to each of the search terms in the sample data, the shortest service recall step size required for the first prompt service to rank the demand location in the sample data before the specified position;

[0215] The strategy step size determination module 633 is used to determine the shortest strategy recall step size corresponding to each strategy based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data.

[0216] The evaluation module 634 is used to evaluate the first prompt service based on the service recall step size and the strategy recall step size corresponding to each strategy.

[0217] The strategy includes a recall strategy and a ranking strategy, and the strategy step size determination module 633 is specifically used for:

[0218] Based on the first intermediate strategy data corresponding to each search term in the recall strategy and the sample data, determine the shortest first recall step length required by the recall strategy to recall the demand locations in the sample data.

[0219] Based on the ranking strategy and the first intermediate strategy data corresponding to each search term in the sample data, determine the shortest second recall step length required by the ranking strategy to rank the demand locations in the sample data before a preset position, and the shortest third recall step length required by the strategy to recall the demand locations.

[0220] Specifically, the evaluation module 634 is used for:

[0221] Based on the service recall step lengths, the average service step length for the first prompt service to place the demand location before the specified position is determined; wherein, the average service step length is used to evaluate the overall recall effect of the first prompt service, and the shorter the average service step length, the better the recall effect of the first prompt service.

[0222] Based on the shortest recall step length corresponding to each strategy, the average strategy step length is determined; wherein, the average strategy step length is used to evaluate the strategy, and the shorter the average strategy step length, the better the recall effect of the strategy.

[0223] The evaluation unit 630 includes:

[0224] The service recall rate determination module 635 is used to determine the service recall rate of the first prompt service in placing the required location in the sample data before a specified position based on the first search results corresponding to each of the search terms in the sample data; wherein, the service recall rate is used to evaluate the overall recall effect of the first prompt service, and the higher the service recall rate, the better the recall effect of the first prompt service.

[0225] The strategy recall rate determination module 636 is used to determine the strategy recall rate when processing the search terms using the strategy based on the first intermediate strategy data corresponding to each search term of the strategy and the sample data; wherein, the strategy recall rate is used to evaluate the recall effect of the strategy, and the higher the strategy recall rate, the better the recall effect of the strategy.

[0226] The apparatus further includes a failure request processing unit 640, the failure request processing unit 640 comprising:

[0227] The target data determination module 641 is used to determine the failed retrieval request that failed to recall the required location based on the first retrieval result, and to determine the first target strategy data corresponding to the failed retrieval request in the first intermediate strategy data;

[0228] The target strategy determination module 642 is used to determine the target strategy that caused the retrieval failure based on the first target retrieval data.

[0229] The target strategy determination module 642 is specifically used for:

[0230] Based on the first target retrieval data, the first prompt service determines the recall result of recalling the required location based on each strategy, wherein the recall result includes success or failure;

[0231] According to the order in which the first prompt service executes strategies when processing the retrieval request, the first strategy that fails to retrieve the results is identified as the target strategy that caused the retrieval failure.

[0232] The sending unit 620 is also configured to send a retrieval request to the second prompting service in debug mode according to the input trajectory;

[0233] The acquisition unit 610 is further configured to acquire second intermediate strategy data and second search results when the second prompting service processes the search request; the second intermediate strategy data includes data obtained when the second prompting service processes the search request based on multiple strategies;

[0234] The evaluation unit 630 includes a comparison evaluation module 637, used for:

[0235] The first prompt service is evaluated based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the requested location.

[0236] The retrieval request includes multiple sub-requests corresponding to the search terms, which are determined based on the input trajectory; the first retrieval result and the second retrieval result each include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data each include intermediate strategy data corresponding to each search term;

[0237] The comparison and evaluation module 635 is specifically used for:

[0238] Obtain the input step size corresponding to the search term;

[0239] Based on the first search result corresponding to each of the search terms, a first service recall step size is determined for the first prompt service to rank the desired location in the sample data before the specified position; based on the second search result corresponding to each of the search terms, a second service recall step size is determined for the second prompt service to rank the desired location in the sample data before the specified position.

[0240] Based on the first intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest first strategy recall step size corresponding to each strategy; based on the second intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest second strategy recall step size corresponding to each strategy.

[0241] The first prompt service is evaluated based on the first service recall step size, the second service recall step size, the first strategy recall step size corresponding to each strategy, and the second strategy recall step size.

[0242] The retrieval request includes multiple sub-requests corresponding to the search terms, which are determined based on the input trajectory; the first retrieval result and the second retrieval result each include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data each include intermediate strategy data corresponding to each search term;

[0243] The comparison and evaluation module 635 is specifically used for:

[0244] Based on the first search results corresponding to each of the search terms in the sample data, a first service recall rate is determined for the first suggestion service to rank the desired location in the sample data before a specified position; based on the second search results corresponding to each of the search terms in the sample data, a second service recall rate is determined for the second suggestion service to rank the desired location in the sample data before a specified position.

[0245] Based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a first strategy recall rate is determined when the strategy is used to process the search terms; based on the second intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a second strategy recall rate is determined when the strategy is used to process the search terms.

[0246] The first prompt service is evaluated based on the first service recall rate, the second service recall rate, the first strategy recall rate, and the second strategy recall rate.

[0247] The acquisition unit 610 includes:

[0248] The log acquisition module 611 is used to acquire the historical logs generated by the location retrieval input suggestion service after processing the content input by the user and querying the suggested locations.

[0249] The sample extraction module 612 is used to obtain the suggested location and the input trajectory when the user enters the name of the suggested location based on the historical log.

[0250] This disclosure provides an evaluation method, electronic device, and program product for location search input suggestion services, applied to map search technology in data processing technology, to more accurately evaluate the merits of Sug search functions.

[0251] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0252] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0253] According to embodiments of this disclosure, this disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, at least one processor of an electronic device being able to read the computer program from the readable storage medium, and the at least one processor executing the computer program causing the electronic device to perform the scheme provided in any of the above embodiments.

[0254] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0255] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0256] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0257] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the evaluation method for a location retrieval input suggestion service. For example, in some embodiments, the evaluation method for a location retrieval input suggestion service may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the evaluation method for the location retrieval input suggestion service described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured, by any other suitable means (e.g., by means of firmware), to perform an evaluation method for the location retrieval input prompt service.

[0258] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0259] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0260] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0261] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0262] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0263] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the management difficulties and weak business scalability inherent in traditional physical hosts and VPS (Virtual Private Server) services. Servers can also be servers for distributed systems or servers integrated with blockchain technology.

[0264] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0265] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An evaluation method for a location search input suggestion service, comprising: Acquire sample data, which includes the desired location and the user's input trajectory when entering the name of the desired location; The input trajectory is used to represent the text content entered by the user in the search box at each step when entering the name of the desired location; Based on the input trajectory, a retrieval request is sent to the first prompting service to be evaluated in debug mode, and the first intermediate strategy data and the first retrieval result are obtained when the first prompting service processes the retrieval request. The first intermediate strategy data includes data obtained when the first prompting service processes the search request based on multiple strategies; The first prompt service is evaluated based on the first intermediate strategy data, the first search results, and the requested location.

2. The method according to claim 1, wherein, The step of sending a retrieval request to the first prompting service to be evaluated in debug mode based on the input trajectory, and obtaining the first intermediate strategy data and the first retrieval result when the first prompting service processes the retrieval request, includes: Based on the input trajectory, at least one search term is determined from the sample data, and the search term is sent to the first prompt service in debug mode; The system obtains the first search results when the first prompting service processes each of the search terms, and the first intermediate strategy data generated by the first prompting service when it processes each of the search terms based on multiple strategies, corresponding to the strategies and the search terms.

3. The method according to claim 2, wherein, The step of evaluating the first suggestion service based on the first intermediate strategy data, the first search result, and the requested location includes: Obtain the input step size corresponding to the search term; Based on the first search result corresponding to each of the search terms in the sample data, determine the shortest service recall step required for the first prompt service to rank the desired location in the sample data before the specified position. Based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, determine the shortest strategy recall step size corresponding to each strategy; The first prompt service is evaluated based on the service recall step size and the strategy recall step size corresponding to each strategy.

4. The method according to claim 3, wherein, The strategy includes a recall strategy and a ranking strategy. The step of determining the shortest strategy recall step size corresponding to each strategy based on the strategy and the first intermediate strategy data corresponding to each search term in the sample data includes: Based on the first intermediate strategy data corresponding to each search term in the recall strategy and the sample data, determine the shortest first recall step length required by the recall strategy to recall the demand locations in the sample data. Based on the ranking strategy and the first intermediate strategy data corresponding to each search term in the sample data, determine the shortest second recall step length required by the ranking strategy to rank the demand locations in the sample data before a preset position, and the shortest third recall step length required by the strategy to recall the demand locations.

5. The method according to claim 3, wherein, The step of evaluating the first prompt service based on the service recall step size and the strategy recall step size corresponding to each strategy includes: Based on the service recall step lengths, the average service step length for the first prompt service to place the demand location before the specified position is determined; wherein, the average service step length is used to evaluate the overall recall effect of the first prompt service, and the shorter the average service step length, the better the recall effect of the first prompt service. Based on the shortest recall step length corresponding to each strategy, the average strategy step length is determined; wherein, the average strategy step length is used to evaluate the strategy, and the shorter the average strategy step length, the better the recall effect of the strategy.

6. The method according to claim 2, wherein, The step of evaluating the first suggestion service based on the first intermediate strategy data, the first search result, and the requested location includes: Based on the first search results corresponding to each of the search terms in the sample data, the service recall rate of the first prompt service in ranking the required location in the sample data before the specified position is determined; wherein, the service recall rate is used to evaluate the overall recall effect of the first prompt service, and the higher the service recall rate, the better the recall effect of the first prompt service. Based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a strategy recall rate is determined when the strategy is used to process the search terms; wherein, the strategy recall rate is used to evaluate the recall effect of the strategy, and the higher the strategy recall rate, the better the recall effect of the strategy.

7. The method according to any one of claims 1-6, further comprising: Based on the first search result, a failed search request that failed to recall the required location is determined, and a first target strategy data corresponding to the failed search request is determined in the first intermediate strategy data; Based on the first target retrieval data, determine the target strategy that caused the retrieval failure.

8. The method according to claim 7, wherein, The step of determining the target strategy that caused the retrieval failure based on the first target retrieval data includes: Based on the first target retrieval data, the first prompt service determines the recall result of recalling the required location based on each strategy, wherein the recall result includes success or failure; According to the order in which the first prompt service executes strategies when processing the retrieval request, the first strategy that fails to retrieve the results is identified as the target strategy that caused the retrieval failure.

9. The method according to any one of claims 1-6 and 8, further comprising: According to the input trajectory, a retrieval request is sent to the second prompting service in debug mode, and the second intermediate strategy data and the second retrieval result are obtained when the second prompting service processes the retrieval request; the second intermediate strategy data includes data obtained by the second prompting service when processing the retrieval request based on multiple strategies; The step of evaluating the first suggestion service based on the first intermediate strategy data, the first search result, and the requested location includes: The first prompt service is evaluated based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the requested location.

10. The method according to claim 9, wherein the retrieval request includes multiple sub-requests corresponding to the search terms, the search terms are determined based on the input trajectory; the first retrieval result and the second retrieval result respectively include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data respectively include intermediate strategy data corresponding to each search term; The evaluation of the first prompt service based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the requested location includes: Obtain the input step size corresponding to the search term; Based on the first search result corresponding to each of the search terms, determine the first service recall step size required for the first prompt service to rank the desired location in the sample data before the specified position; Based on the second search result corresponding to each of the search terms, determine the second service recall step size required for the second prompt service to rank the desired location in the sample data before the specified position; Based on the first intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest first strategy recall step size corresponding to each strategy; based on the second intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest second strategy recall step size corresponding to each strategy. The first prompt service is evaluated based on the first service recall step size, the second service recall step size, the first strategy recall step size corresponding to each strategy, and the second strategy recall step size.

11. The method according to claim 9, wherein the retrieval request includes multiple sub-requests corresponding to the search terms, the search terms are determined based on the input trajectory; the first retrieval result and the second retrieval result respectively include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data respectively include intermediate strategy data corresponding to each search term; The evaluation of the first prompt service based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the requested location includes: Based on the first search results corresponding to each of the search terms in the sample data, a first service recall rate is determined for the first suggestion service to rank the desired location in the sample data before a specified position; based on the second search results corresponding to each of the search terms in the sample data, a second service recall rate is determined for the second suggestion service to rank the desired location in the sample data before a specified position. Based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a first strategy recall rate is determined when the strategy is used to process the search terms; based on the second intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a second strategy recall rate is determined when the strategy is used to process the search terms. The first prompt service is evaluated based on the first service recall rate, the second service recall rate, the first strategy recall rate, and the second strategy recall rate.

12. The method according to any one of claims 1-6, 8, and 10-11, wherein, The acquisition of sample data includes: The location search input suggestion service processes the user's input and queries the historical logs generated by the suggested locations; The suggested location is obtained from the historical logs, along with the user's input trajectory when entering the name of the suggested location.

13. An evaluation device for a location search input suggestion service, comprising: The acquisition unit is used to acquire sample data, which includes the required location and the input trajectory when the user enters the name of the required location; The input trajectory is used to represent the text content entered by the user in the search box at each step when entering the name of the desired location; The sending unit is configured to send a retrieval request to the first prompt service to be evaluated in debug mode based on the input trajectory; The acquisition unit is further configured to acquire first intermediate strategy data and first search result when the first prompting service processes the search request; the first intermediate strategy data includes data obtained when the first prompting service processes the search request based on multiple strategies; An evaluation unit is used to evaluate the first prompt service based on the first intermediate strategy data, the first search result, and the required location.

14. The apparatus according to claim 13, wherein, The sending unit is specifically used to determine at least one search term for the sample data based on the input trajectory, and to send the search term to the first prompting service in debug mode; The acquisition unit is specifically used to acquire the first search results when the first prompt service processes each of the search terms, and the first intermediate strategy data generated by the first prompt service when processing each of the search terms based on multiple strategies, corresponding to the strategies and the search terms.

15. The apparatus according to claim 14, wherein, The evaluation unit includes: The acquisition module is used to acquire the input step size corresponding to the search term; The service step size determination module is used to determine the shortest service recall step size required for the first prompt service to rank the demand location in the sample data before the specified position based on the first search result corresponding to each of the search terms in the sample data. The strategy step size determination module is used to determine the shortest strategy recall step size corresponding to each strategy based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data. The evaluation module is used to evaluate the first prompt service based on the service recall step size and the strategy recall step size corresponding to each strategy.

16. The apparatus according to claim 15, wherein, The strategy includes a recall strategy and a ranking strategy, and the strategy step size determination module is specifically used for: Based on the first intermediate strategy data corresponding to each search term in the recall strategy and the sample data, determine the shortest first recall step length required by the recall strategy to recall the demand locations in the sample data. Based on the ranking strategy and the first intermediate strategy data corresponding to each search term in the sample data, determine the shortest second recall step length required by the ranking strategy to rank the demand locations in the sample data before a preset position, and the shortest third recall step length required by the strategy to recall the demand locations.

17. The apparatus according to claim 15, wherein, The evaluation module is specifically used for: Based on the service recall step lengths, the average service step length for the first prompt service to place the demand location before the specified position is determined; wherein, the average service step length is used to evaluate the overall recall effect of the first prompt service, and the shorter the average service step length, the better the recall effect of the first prompt service. Based on the shortest recall step length corresponding to each strategy, the average strategy step length is determined; wherein, the average strategy step length is used to evaluate the strategy, and the shorter the average strategy step length, the better the recall effect of the strategy.

18. The apparatus according to claim 14, wherein, The evaluation unit includes: The service recall rate determination module is used to determine the service recall rate of the first prompt service in placing the required location in the sample data before a specified position based on the first search results corresponding to each of the search terms in the sample data; wherein, the service recall rate is used to evaluate the overall recall effect of the first prompt service, and the higher the service recall rate, the better the recall effect of the first prompt service. The strategy recall rate determination module is used to determine the strategy recall rate when processing the search terms using the strategy, based on the first intermediate strategy data corresponding to each search term of the strategy and the sample data; wherein, the strategy recall rate is used to evaluate the recall effect of the strategy, and the higher the strategy recall rate, the better the recall effect of the strategy.

19. The apparatus according to any one of claims 13-18, further comprising a failure request processing unit, the failure request processing unit comprising: The target data determination module is used to determine the failed retrieval request that failed to recall the required location based on the first retrieval result, and to determine the first target strategy data corresponding to the failed retrieval request in the first intermediate strategy data; The target strategy determination module is used to determine the target strategy that caused the retrieval failure based on the first target retrieval data.

20. The apparatus according to claim 19, wherein, The target strategy determination module is specifically used for: Based on the first target retrieval data, the first prompt service determines the recall result of recalling the required location based on each strategy, wherein the recall result includes success or failure; According to the order in which the first prompt service executes strategies when processing the retrieval request, the first strategy that fails to retrieve the results is identified as the target strategy that caused the retrieval failure.

21. The apparatus according to any one of claims 13-18, 20, The sending unit is also configured to send a retrieval request to the second prompting service in debug mode based on the input trajectory; The acquisition unit is further configured to acquire second intermediate strategy data and second search results when the second prompting service processes the search request; the second intermediate strategy data includes data obtained by the second prompting service when processing the search request based on multiple strategies; The evaluation unit includes a comparison evaluation module, used for: The first prompt service is evaluated based on the first intermediate strategy data, the first search result, the second intermediate strategy data, the second search result, and the requested location.

22. The apparatus according to claim 21, wherein the retrieval request includes a plurality of sub-requests corresponding to a search term, the search term being determined based on the input trajectory; the first retrieval result and the second retrieval result respectively include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data respectively include intermediate strategy data corresponding to each search term; The comparison and evaluation module is specifically used for: Obtain the input step size corresponding to the search term; Based on the first search result corresponding to each of the search terms, determine the first service recall step size required for the first prompt service to rank the desired location in the sample data before the specified position; Based on the second search result corresponding to each of the search terms, determine the second service recall step size required for the second prompt service to rank the desired location in the sample data before the specified position; Based on the first intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest first strategy recall step size corresponding to each strategy; based on the second intermediate strategy data corresponding to each search term of the strategy and the sample data, determine the shortest second strategy recall step size corresponding to each strategy. The first prompt service is evaluated based on the first service recall step size, the second service recall step size, the first strategy recall step size corresponding to each strategy, and the second strategy recall step size.

23. The apparatus according to claim 21, wherein the retrieval request includes a plurality of sub-requests corresponding to the search terms, the search terms being determined based on the input trajectory; the first retrieval result and the second retrieval result respectively include retrieval results corresponding to each search term, and the first intermediate strategy data and the second intermediate strategy data respectively include intermediate strategy data corresponding to each search term; The comparison and evaluation module is specifically used for: Based on the first search results corresponding to each of the search terms in the sample data, a first service recall rate is determined for the first suggestion service to rank the desired location in the sample data before a specified position; based on the second search results corresponding to each of the search terms in the sample data, a second service recall rate is determined for the second suggestion service to rank the desired location in the sample data before a specified position. Based on the first intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a first strategy recall rate is determined when the strategy is used to process the search terms; based on the second intermediate strategy data corresponding to each of the search terms in the strategy and the sample data, a second strategy recall rate is determined when the strategy is used to process the search terms. The first prompt service is evaluated based on the first service recall rate, the second service recall rate, the first strategy recall rate, and the second strategy recall rate.

24. The apparatus according to any one of claims 13-18, 20, and 22-23, wherein, The acquisition unit includes: The log acquisition module is used to acquire historical logs generated by the location search input suggestion service after processing the user's input. The sample extraction module is used to obtain the suggested location based on the historical logs, as well as the input trajectory when the user enters the name of the suggested location.

25. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

27. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-12.

Images