Navigation application test system, navigation application test method and equipment

Abstract

The embodiment of the invention provides a navigation application test system, a navigation application test method and equipment. The navigation application test system comprises a data persistence layer and a business service layer, the data persistence layer is used for storing document information; wherein the document information comprises an evaluation index system document; and the business service layer is used for generating a test report of the to-be-tested navigation application according to the document information, the sample data of the to-be-tested navigation application and the first reference data. The navigation application test system is used for achieving the effects of improving the navigation application test efficiency and reducing the test labor cost.

Description

Technical Field

[0001] The present application relates to the field of automated testing technology, and in particular to a navigation application testing system, a navigation application testing method and a device. Background Art

[0002] In the process of providing navigation function services to users based on the navigation application, it is necessary to test the navigation function of the navigation application.

[0003] In the prior art, based on a manual processing method, the navigation function record data of the navigation application actually used by the user is manually evaluated and processed to obtain the function test result of the navigation application.

[0004] However, in the above method, the manual processing method is adopted, the test operation is cumbersome and the test time is long, which reduces the efficiency of navigation application testing and also leads to high labor costs for testing. Summary of the invention

[0005] The embodiments of the present application provide a navigation application testing system, a navigation application testing method and a device to improve the efficiency of navigation application testing and reduce the testing manpower cost.

[0006] In a first aspect, an embodiment of the present application provides a navigation application testing system, the navigation application testing system comprising: a data persistence layer and a business service layer;

[0007] The data persistence layer is used to store document information; wherein the document information includes evaluation index system documents;

[0008] The business service layer is used to generate a test report for the navigation application to be tested based on the document information, sample data of the navigation application to be tested and first reference data; wherein the sample data represents the actual navigation function record of the navigation application to be tested; the first reference data represents the expected navigation function record of the navigation application to be tested; and the test report includes the navigation function quality test result of the navigation application to be tested.

[0009] In a possible implementation, the business service layer is specifically used to: obtain application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservices in the navigation application to be tested; the user behavior data characterizes the user's navigation function usage behavior of the navigation application to be tested; based on each of the user behavior data, generate the sample data and the first reference data; and classify and store the sample data and the first reference data in the data persistence layer; and obtain the sample data and the first reference data from the data persistence layer.

[0010] In a possible implementation, the business service layer is specifically used to: perform feature processing on each of the user behavior data to obtain feature data corresponding to each of the user behavior data; and generate the sample data according to each of the user behavior data and each of the feature data.

[0011] In a possible implementation, the business service layer is further specifically used to: process each of the user behavior data to obtain at least one first reference value in the first reference data; obtain second reference data of at least one target navigation application through a preset interface; wherein the second reference data represents the expected navigation function record of the target navigation application; vote on each of the second reference data to obtain at least one second reference value in the first reference data; process key data in each of the user behavior data to obtain at least one third reference value in the first reference data.

[0012] In a possible implementation, the business service layer is further used to: if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, then function repair processing is performed on the navigation application to be tested, and the step of generating a test report of the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

[0013] In a possible implementation, the navigation application testing system further includes a data cleaning and noise reduction layer; the data cleaning and noise reduction layer is used to update the sample data and the document information according to the test report;

[0014] The business service layer is used to perform test comparison processing on the updated sample data and the first true value reference data according to the updated document information to obtain an updated test report.

[0015] In a possible implementation, the navigation application testing system further includes a web page display layer; the web page display layer is used to display the test report and test data of the navigation application to be tested; wherein the test data includes the document information, the sample data and the first reference data.

[0016] In a possible implementation, the navigation application testing system further includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0017] In a second aspect, an embodiment of the present application provides a navigation application testing method, the method being applied to a business service layer of a navigation application testing system; the navigation application testing system further includes a data persistence layer; the method comprising:

[0018] Acquire the document information stored in the data persistence layer; wherein the document information includes the evaluation index system document;

[0019] Acquire sample data and first reference data of the navigation application to be tested; wherein the sample data represents actual navigation function records of the navigation application to be tested; and the first reference data represents expected navigation function records of the navigation application to be tested;

[0020] According to the document information, the sample data and the first reference data are evaluated and compared to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

[0021] In a possible implementation manner, obtaining sample data and first reference data of the navigation application to be tested includes:

[0022] Acquire application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservice in the navigation application to be tested;

[0023] Generate the sample data and the first reference data according to each of the user behavior data; and classify and store the sample data and the first reference data in the data persistence layer;

[0024] The sample data and the first reference data are obtained from the data persistence layer.

[0025] In a possible implementation, generating the sample data according to each of the user behavior data includes:

[0026] Performing feature processing on each of the user behavior data to obtain feature data corresponding to each of the user behavior data;

[0027] The sample data is generated according to each of the user behavior data and each of the feature data.

[0028] In a possible implementation, generating the first reference data according to each of the user behavior data includes:

[0029] Processing each of the user behavior data to obtain at least one first reference value in the first reference data;

[0030] Acquire, through a preset interface, second reference data of at least one target navigation application; wherein the second reference data represents an expected navigation function record of the target navigation application;

[0031] Performing voting on each of the second reference data to obtain at least one second reference value in the first reference data;

[0032] The key data in each of the user behavior data are processed to obtain at least one third reference value in the first reference data.

[0033] In one possible implementation, the method includes:

[0034] If it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, function repair processing is performed on the navigation application to be tested, and the step of generating a test report for the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

[0035] In a possible implementation, the navigation application testing system further includes a data cleaning and noise reduction layer; after determining that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, it also includes:

[0036] Based on the data cleaning noise reduction layer, and according to the test report, the sample data and the document information are updated;

[0037] According to the updated document information, a test comparison process is performed on the updated sample data and the first reference data to obtain an updated test report.

[0038] In a possible implementation, the navigation application testing system further includes a web page display layer; the method includes:

[0039] Based on the web page display layer, the test report and test data of the navigation application to be tested are displayed; wherein the test data includes the document information, the sample data and the first reference data.

[0040] In a possible implementation, the navigation application testing system further includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0041] In a third aspect, an embodiment of the present application provides a navigation application testing device, which is applied to a business service layer of a navigation application testing system; the navigation application testing system also includes a data persistence layer; the device includes:

[0042] A first acquisition module, used to acquire document information stored in the data persistence layer; wherein the document information includes an evaluation indicator system document;

[0043] A second acquisition module is used to acquire sample data and first reference data of the navigation application to be tested; wherein the sample data represents actual user behavior based on the navigation application to be tested; and the first reference data represents expected user behavior based on the navigation application to be tested;

[0044] The test module is used to evaluate and compare the sample data and the first reference data according to the document information to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

[0045] In a possible implementation, the second acquisition module is specifically used to: acquire application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservices in the navigation application to be tested; generate the sample data and the first reference data according to each of the user behavior data; and classify and store the sample data and the first reference data in the data persistence layer; and acquire the sample data and the first reference data from the data persistence layer.

[0046] In a possible implementation, the second acquisition module is specifically used to: perform feature processing on each of the user behavior data to obtain feature data corresponding to each of the user behavior data; and generate the sample data according to each of the user behavior data and each of the feature data.

[0047] In a possible implementation, the second acquisition module is further specifically used to: process each of the user behavior data to obtain at least one first reference value in the first reference data; obtain second reference data of at least one target navigation application through a preset interface; wherein the second reference data represents the expected navigation function record of the target navigation application; vote on each of the second reference data to obtain at least one second reference value in the first reference data; process the key data in each of the user behavior data to obtain at least one third reference value in the first reference data.

[0048] In a possible implementation, the device is also used for: if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, then function repair processing is performed on the navigation application to be tested, and the step of generating a test report of the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

[0049] In a possible implementation, the navigation application testing system also includes a data cleaning and denoising layer; after the device is used to determine that the test report represents an abnormality in the navigation function test of the navigation application to be tested, the device is also used to: based on the data cleaning and denoising layer, update the sample data and the document information according to the test report; based on the updated document information, test and compare the updated sample data and the first reference data to obtain an updated test report.

[0050] In a possible implementation, the navigation application testing system also includes a web page display layer; the device is also used to: based on the web page display layer, display the test report and test data of the navigation application to be tested; wherein the test data includes the document information, the sample data and the first reference data.

[0051] In a possible implementation, the navigation application testing system further includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0052] In a fourth aspect, an embodiment of the present application provides an electronic device cluster, including: at least one memory and at least one processor;

[0053] at least one of said memories stores computer-executable instructions;

[0054] At least one of the processors executes at least one computer-executable instruction stored in the memory, so that at least one of the processors executes the above second aspect and / or various possible implementations of the second aspect.

[0055] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the above second aspect and / or various possible implementations of the second aspect.

[0056] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above second aspect and / or various possible implementation methods of the second aspect.

[0057] The navigation application testing system, navigation application testing method and device provided in the embodiments of the present application are based on the business service layer of the navigation application testing platform, and define an evaluation index system document. According to the evaluation index system document, the actual navigation function record of the navigation application to be tested and the expected navigation function record of the navigation application to be tested are tested and compared to obtain the navigation function quality test result of the navigation application to be tested; further, by comparing the actual navigation function record of the navigation application to be tested with the expected navigation function record of the navigation application to be tested, the navigation application testing process is automated, which can achieve the effect of improving the navigation application testing efficiency and reducing the testing labor cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0059] Figure 1A schematic diagram of an application scenario provided for this application;

[0060] Figure 2 A structural block diagram of a navigation application testing system provided in an embodiment of the present application;

[0061] Figure 3 A structural block diagram of another navigation application testing system provided in an embodiment of the present application;

[0062] Figure 4 A flowchart of a navigation application testing method provided in an embodiment of the present application;

[0063] Figure 5 A flowchart of an automated testing process for a navigation application provided in an embodiment of the present application;

[0064] Figure 6 A flowchart of another navigation application testing method provided in an embodiment of the present application;

[0065] Figure 7 A basic technical structure diagram of a navigation quality evaluation system provided in an embodiment of the present application;

[0066] Figure 8 A flowchart of another navigation application automation testing process provided by an embodiment of the present application;

[0067] Fig. 9 A schematic diagram of an example of an API acquisition result provided in an embodiment of the present application;

[0068] Fig.10 A schematic diagram of the structure of a navigation application testing device provided in an embodiment of the present application;

[0069] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0070] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0071] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0072] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, user behavior data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0073] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, user behavior data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0074] It should be noted that the present application can be used in the field of automated testing technology, and can also be used in any field other than automated testing technology, and the application field of the present application is not limited.

[0075] Figure 1 A schematic diagram of an application scenario provided for this application, such as Figure 1 As shown, the specific application scenario of the present application includes an electronic device 101 and a tester 102. The tester 102 obtains the navigation function record data of the navigation application to be tested that has been actually used by the user through the electronic device 101, and performs manual evaluation and processing on the navigation function record data based on the electronic device 101 to obtain the functional test results of the navigation application.

[0076] In combination with the above scenarios, it can be seen that in the prior art, the manual processing method has the problem of cumbersome test operations and long test time, which reduces the efficiency of navigation application testing and also leads to technical problems of high test labor costs.

[0077] The navigation application testing solution provided in the present application solves the technical problems of low navigation application testing efficiency and high testing labor cost by comparing the actual navigation function records of the navigation application to be tested with the expected navigation function records of the navigation application to be tested, and automates the navigation application testing process.

[0078] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0079] Figure 2 A structural block diagram of a navigation application test system provided in an embodiment of the present application, such as Figure 2 As shown, the navigation application testing system includes: a data persistence layer and a business service layer.

[0080] The data persistence layer is used to store document information; the document information includes the evaluation indicator system document.

[0081] The business service layer is used to generate a test report for the navigation application to be tested based on document information, sample data of the navigation application to be tested and first reference data; wherein the sample data represents the actual navigation function record of the navigation application to be tested; the first reference data represents the expected navigation function record of the navigation application to be tested; and the test report includes the navigation function quality test results of the navigation application to be tested.

[0082] Exemplarily, the execution subject of this embodiment may be an electronic device, hereinafter referred to as the device. The tester develops and deploys a navigation application test system through the device, and the navigation application test system includes: a data persistence layer and a business service layer, wherein the data persistence layer and the business service layer may be software abstraction layers in the navigation application test system, such as encapsulating the functions of one or more components to be responsible for processing and executing the corresponding business logic. The data persistence layer is used to store document information generated by evaluators directly customizing the device, wherein the document information includes an evaluation indicator system document, and the evaluation indicator system document may include but is not limited to the core functional experience, evaluation indicators, evaluation methods, scoring standards, problem classification and report templates. The business service layer is used to obtain document information from the data persistence layer, and obtain sample data and first reference data of the navigation application to be tested, wherein the sample data represents the actual navigation function record of the navigation application to be tested; the first reference data represents the expected navigation function record of the navigation application to be tested; the document information, the sample data of the navigation application to be tested and the first reference data are automatically processed to generate a test report for the navigation application to be tested, so as to obtain a navigation function quality test result of the navigation application to be tested.

[0083] In this embodiment, a navigation application testing system is provided. The automated evaluation process relies on platform tools to achieve efficient and accurate comparison of sample results with reference data. Automated operations ensure the timeliness and continuity of the evaluation work, and can promptly discover problems that may arise in the navigation software at different stages, thereby achieving the effect of improving the efficiency of navigation application testing and reducing testing labor costs.

[0084] Figure 3 A structural block diagram of another navigation application test system provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the navigation application test system includes: data persistence layer and business service layer; the navigation application test system also includes data cleaning and noise reduction layer, web page display layer, and basic technology module. Among them, the data persistence layer and business service layer, data cleaning and noise reduction layer, web page display layer, and basic technology module are abstract layers in the software architecture of the navigation application test system, responsible for processing the corresponding business logic.

[0085] In one possible implementation, the business service layer is specifically used to: obtain application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservices in the navigation application to be tested; the user behavior data represents the user's navigation function usage behavior of the navigation application to be tested; generate sample data and first reference data based on each user behavior data; and classify and store the sample data and the first reference data in the data persistence layer; based on the business service layer, obtain the sample data and the first reference data from the data persistence layer.

[0086] Exemplarily, the business service layer can obtain the user behavior data generated by the microservices in the navigation application to be tested through the service gateway of the navigation application to be tested, so as to characterize the user's navigation function usage behavior of the navigation application to be tested, such as the navigation record data generated after the user uses the route navigation function, which contains the navigation recommended route and the actual travel trajectory data. The business service layer is used to call the preset data processing rules to recycle the obtained multiple user behavior data, for example, to perform data format conversion processing to obtain the corresponding sample data, and can perform model processing on the multiple user behavior data through the preset deep learning model to obtain the first reference data, and the obtained sample data and the first reference data are classified and stored in the data persistence layer. When the business service layer performs the evaluation task, the sample data and the first reference data are obtained from the data persistence layer.

[0087] In a possible implementation, the business service layer is specifically used to: perform feature processing on each user behavior data to obtain feature data corresponding to each user behavior data; and generate sample data according to each user behavior data and each feature data.

[0088] Specifically, the business service layer performs feature processing on these user behavior data through feature extraction algorithms to obtain feature data corresponding to each user behavior data. At the same time, the business service layer performs format conversion and other recycling processing on these user behavior data to obtain processed user behavior data, and merges the feature data and the processed user behavior data to obtain the final sample data.

[0089] In a possible implementation, the business service layer is further specifically used to: process each user behavior data to obtain at least one first reference value in the first reference data; obtain second reference data of at least one target navigation application through a preset interface; wherein the second reference data represents the expected navigation function record of the target navigation application; vote on each second reference data to obtain at least one second reference value in the first reference data; process key data in each user behavior data to obtain at least one third reference value in the first reference data.

[0090] Exemplarily, the business service layer is also specifically used to recycle and process these user behavior data to obtain multiple reference values ​​corresponding to these user behavior data, namely, the first reference value, and obtain the expected navigation function record generated by each other navigation application, namely, the target navigation application, through the preset interface of the navigation application testing system, namely, the second reference data, and vote on each of the obtained second reference data through a voting processing algorithm to obtain multiple reference values ​​corresponding to the target open platforms of the multiple target navigation applications, namely, the second reference values; the business service layer is also used to extract data that cannot be automatically processed in the obtained user behavior data, namely, key data, and analyze, process and annotate these key data to obtain multiple reference values ​​corresponding to these key data, namely, the third reference value, and finally organize them to obtain the first reference data.

[0091] In a possible implementation, the business service layer is also used to: if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, then function repair processing is performed on the navigation application to be tested, and the step of generating a test report of the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

[0092] Exemplarily, after the business service layer is used to execute the automated evaluation phase, the business service layer is also used to, based on the content of the test report, perform functional repair processing on the navigation application to be tested if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal; after completing the problem repair, the business service layer re-executes the corresponding evaluation task of the sample set, that is, re-tests and compares the sample data and the first reference data according to the document information, and obtains a new test report of the navigation application to be tested, so as to verify the improvement of the functional quality after adjustment.

[0093] In a possible implementation, the navigation application testing system further includes a data cleaning and noise reduction layer; the data cleaning and noise reduction layer is used to update the sample data and document information according to the test report.

[0094] The business service layer is used to perform test comparison processing on the updated sample data and the first true value reference data according to the updated document information to obtain an updated test report.

[0095] Exemplarily, the navigation application testing platform also includes a data cleaning and denoising layer; after determining that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, the data cleaning and denoising layer is used to update the sample data and document information according to the test report to obtain updated document information and updated sample data; the business service layer is used to test and compare the updated sample data and the first reference data according to the updated document information to obtain an updated test report to complete a new round of testing.

[0096] In a possible implementation, the navigation application testing system further includes a web page display layer; the web page display layer is used to display the test report and test data of the navigation application to be tested; wherein the test data includes document information, sample data and first reference data.

[0097] Exemplarily, the navigation application testing system also includes a web page display layer; the web page display layer is used to display the test report and test data of the navigation application to be tested; wherein the test data includes document information, sample data and first reference data related to the navigation function evaluation.

[0098] In a possible implementation, the navigation application testing system further includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0099] Exemplarily, the navigation application testing platform also includes a basic technology module, which is used to provide navigation assistance services, such as vector maps, road services and other navigation assistance services; the basic technology module can also provide basic resources to provide a virtual operating environment for the aforementioned business service layer, data persistence layer, data cleaning and noise reduction layer, web page display layer, etc.

[0100] In this embodiment, on the basis of the above-mentioned embodiment, the automation of navigation function testing based on large samples can more comprehensively present the performance of navigation software in various scenarios. Through the automated evaluation process, a large number of samples can be efficiently processed to avoid the inefficiency and errors of manual processing. Moreover, in the iterative closed-loop process, samples or evaluation indicators can be dynamically adjusted based on the feedback of each evaluation result, so that the evaluation results are more in line with the actual situation, and the software problems of the navigation application can be discovered and solved in time, thereby effectively improving the navigation function service quality of the navigation application and improving the user experience.

[0101] Figure 4 A flowchart of a navigation application testing method provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the method is applied to the business service layer of the navigation application test system; the navigation application test system also includes a data persistence layer; the method includes:

[0102] 201. Obtain document information stored in a data persistence layer; wherein the document information includes an evaluation indicator system document.

[0103] Exemplarily, the execution subject of this embodiment can be an electronic device, hereinafter referred to as the device. The tester develops and deploys a navigation application test system through the device. The navigation application test system can be a navigation application test platform. As a unified capability output platform under the navigation application test environment, the navigation application test platform includes a business service layer and a data persistence layer. Based on the business service layer, the preset document information can be obtained from the data persistence storage space or in a customized way. The evaluator directly customizes and generates document information through the device, wherein the document information includes an evaluation index system document, which can include but is not limited to the core experience of the function, evaluation indicators, evaluation methods, scoring standards, problem classification and report templates, so as to realize the standardization and quantification of the navigation quality evaluation index system. Furthermore, the fuzzy navigation function test concept is transformed into a clear quantitative indicator, such as a clear scoring standard, so that the evaluation is no longer subjective and arbitrary; this method allows different personnel to evaluate based on a unified standard to ensure the objectivity and consistency of the evaluation results; at the same time, the quantified data is easy to compare and analyze, can accurately point out the problems of the navigation software, provide a clear direction for improvement, and enhance the operability of the evaluation and testing work.

[0104] For example, Figure 5 A flowchart of a navigation application automated testing process provided by an embodiment of the present application is shown in FIG. Figure 5As shown in the figure, when carrying out navigation software evaluation related work, testers need to clearly define the indicator system, which is the criterion for judging the quality of navigation software. In this process, testers, product developers, analysts, etc. can obtain user demand data through equipment. After data analysis by all parties, they can generate an initial indicator system document based on the equipment. All parties review and improve the initial indicator system document from different perspectives based on the equipment, such as product personnel from the perspective of product design, evaluators from the perspective of actual operation, and analysts from the perspective of data interpretation, so that the initial indicator system document is more comprehensive and more accurate, and then the corresponding evaluation indicators, evaluation methods, scoring standards, problem classification methods and report templates are obtained to obtain a more accurate evaluation indicator system document, and the evaluation indicator system document is stored for subsequent access and query processing.

[0105] 202. Acquire sample data and first reference data of the navigation application to be tested; wherein the sample data represents actual navigation function records of the navigation application to be tested; and the first reference data represents expected navigation function records of the navigation application to be tested.

[0106] For example, in combination Figure 5 Through the business service layer in the navigation application test platform, the device can directly obtain the sample data of the navigation application to be tested from the sample library pre-stored in the data storage space of the device by evaluators and engineers, or directly from the navigation application to be tested, and then obtain the reference data corresponding to the sample data, that is, the first reference data; wherein, the sample data represents the actual navigation function record of the navigation application to be tested, and the first reference data is the expected output result, that is, the first reference data represents the expected navigation function record of the navigation application to be tested, and is used to verify whether the navigation function is correctly executed as designed.

[0107] 203. According to the document information, the sample data and the first reference data are evaluated and compared to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

[0108] For example, in combination Figure 5, the device uses the first reference data as a comparison benchmark according to the evaluation indicators and evaluation methods designed in the evaluation indicator system document in the determined document information, so as to automatically evaluate and compare the acquired sample data and the first reference data, and then obtain the test results of the navigation application to be tested, that is, the navigation function quality test results of the navigation application to be tested. Product personnel, analysts, engineers and other role personnel analyze the test results through the equipment, analyze and feedback the evaluation results. In this process, we use the platform tools of the navigation application testing platform to automatically analyze problem samples, classify problems and output reports to obtain the corresponding test reports, and finally form analysis reports and problem feedback, which become the final results of a round of navigation function quality evaluation and provide important basis for subsequent improvement and optimization.

[0109] In this embodiment, a navigation application testing method is provided, in which the automated evaluation link relies on platform tools to achieve efficient and accurate comparison of sample results with reference data; periodic automated operations ensure the timeliness and continuity of the evaluation work, and can promptly discover problems that may arise in the navigation software at different times, thereby achieving the effect of improving the efficiency of navigation application testing and reducing testing labor costs; in addition, by defining the evaluation indicator system document in the document information to achieve standardization and quantification of the indicator system, the evaluation results can be made more objective and accurate, and can accurately point out the advantages and disadvantages of the navigation application, which is conducive to targeted improvement of the navigation function quality of the navigation application to be tested.

[0110] Figure 6 A flowchart of another navigation application testing method provided in an embodiment of the present application is shown as follows: Figure 6 As shown, the method is applied to the business service layer of the system; the navigation application test system also includes a data persistence layer; the method includes:

[0111] 301. Obtain document information stored in a data persistence layer; wherein the document information includes an evaluation indicator system document.

[0112] Exemplarily, this step may refer to step 201 and will not be described in detail here.

[0113] 302. Obtain application data from a service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by a microservice in the navigation application to be tested.

[0114] Exemplarily, the navigation application test platform of the device also includes a data persistence layer, a business service layer based on the navigation application test platform, and access to the service gateway of the navigation application to be tested through the navigation application test platform, so that the business service layer can obtain the user behavior data generated by the microservice in the navigation application to be tested through the service gateway of the navigation application to be tested, so as to characterize the user's navigation function usage behavior of the navigation application to be tested, such as map viewing records and traffic condition search record data generated after the user uses the map search function in the navigation application to be tested, and navigation record data containing navigation recommended routes and actual travel trajectory data generated after using the route navigation function. For example, Figure 7 A basic technical structure diagram of a navigation quality evaluation system provided in an embodiment of the present application, such as Figure 7 As shown in the figure, multiple self-services are deployed in the microservice of the navigation application to be tested, including map service, search service, reverse geography, suggestion, route calculation service, tram service, remote broadcast, route matching service, estimated time of arrival (ETA) service, road condition service, event service, along-the-way search, etc. When providing navigation services to users, this sub-service will generate basic data in user behavior data such as spatial database (SD data) and lane-level data, and store them in the log library through log management, including trajectory data, log data, business tracking points, operation logs, etc. The business service layer based on the navigation application test platform is connected to the service gateway of the navigation application to be tested through the navigation application test platform, so that the business service layer can obtain the corresponding user behavior data from the log library in the microservice of the navigation application to be tested through the service gateway of the navigation application to be tested, for processing.

[0115] 303. Generate sample data and first reference data according to each user's behavior data; and classify and store the sample data and the first reference data in a data persistence layer.

[0116] Among them, the navigation application testing platform also includes a data persistence layer.

[0117] Exemplarily, the navigation application testing platform also includes a data persistence layer. The device recycles the obtained multiple user behavior data through the business service layer, wherein all user behavior data can be recycled to obtain corresponding sample data and first reference data; or, a part of the user behavior data can be recycled, such as through format conversion processing to obtain corresponding sample data, and another part of the user behavior data can be recycled through a preset deep learning model to obtain corresponding first reference data. Based on the business service layer, the obtained sample data and the first reference data are classified and stored in the sample library and reference library in the data persistence layer; for example, in combination with Figure 5 The data persistence layer stores ETA sample data, Point of Interest (POI) sample data, as well as itinerary data, link data, deviation data, start and end points, click data, trajectory data, etc.

[0118] In one example, generating sample data according to each user's behavior data in step 303 includes the following steps:

[0119] The first step is to perform feature processing on each user behavior data to obtain feature data corresponding to each user behavior data.

[0120] The second step is to generate sample data based on each user's behavior data and each feature data.

[0121] For example, Figure 8 A flowchart of another navigation application automated testing process provided by an embodiment of the present application is shown in FIG. Figure 8 As shown, based on the platform tools in the business service layer, after obtaining a large amount of user behavior data, the evaluators and engineers can perform feature processing on these user behavior data according to the demand characteristics through the feature processing algorithm of the equipment to obtain the feature data corresponding to each user behavior data. They can also perform format conversion and other recycling processing on these user behavior data to obtain the processed user behavior data. The business service layer merges the processed user behavior data and feature data to obtain the final sample data. In turn, a large number of samples can be obtained, which can more comprehensively cover different user groups and usage scenarios, effectively making up for the problem of small sample size and insufficient representativeness of traditional road test; at the same time, the platform tools provide strong support for the complex data processing process, improving work efficiency and sample quality.

[0122] In one example, generating first reference data according to each user behavior data in step 303 includes:

[0123] Step 1: Process each user behavior data to obtain at least one first reference value in the first reference data.

[0124] Step 2: Obtain second reference data of at least one target navigation application through a preset interface; wherein the second reference data represents an expected navigation function record of the target navigation application.

[0125] Step 3: Voting is performed on each second reference data to obtain at least one second reference value in the first reference data.

[0126] Step 4: Process the key data in each user behavior data to obtain at least one third reference value in the first reference data.

[0127] Specifically, combined Figure 7 , 8 Based on the platform tools in the business service layer, after obtaining a large amount of user behavior data, the evaluators and engineers can recycle and process these user behavior data to obtain multiple reference values ​​corresponding to these user behavior data, namely the first reference value; they can also obtain the expected navigation function records generated by each target navigation application from multiple other navigation applications with navigation functions, namely the target navigation application, through the preset interface of the navigation application test platform, such as the application programming interface (API), and vote on each of the obtained second reference data through a voting processing algorithm to obtain multiple reference values ​​corresponding to the multiple target navigation applications, namely the second reference value; in addition, the key data in each of the obtained user behavior data is extracted. For example, when processing user behavior data, if it is found that there are special circumstances such as data errors in the data processing process that cause the data processing process to be unable to continue, it is necessary to screen out the data in the user behavior data that cannot be automatically processed, namely the key data. The evaluators and engineers can analyze and process these key data based on electronic equipment and mark them to obtain multiple reference values ​​corresponding to these key data, namely the third reference value, and finally sort out the first reference data. The reliability of reference data is guaranteed by a multi-source data acquisition method based on the recovery of user behavior data, voting on API acquisition results, and manual annotation. User behavior data reflects actual usage, voting on API acquisition results provides reference for the same navigation functions of different navigation applications, and manual annotation involves evaluators and engineers using their professional knowledge and skills to ensure the accuracy of reference data when processing key data and complex situations. Platform tools ensure the stability and efficiency of data processing.

[0128] For example, Fig. 9 A schematic diagram of an example of an API acquisition result provided in an embodiment of the present application, such as Fig. 9As shown, through the API interface of the navigation application testing platform, data is acquired from multiple target navigation applications (target navigation application 1, target navigation application 2, target navigation application 3), and the expected navigation function record of each target navigation application is obtained, including search service, route calculation service, and suggestion service, that is, the second reference data is obtained, and through the voting processing algorithm, the content of each obtained second reference data is parsed and voted, to obtain multiple reference values ​​corresponding to the multiple target navigation applications, that is, the second reference values.

[0129] 304. Obtain sample data and first reference data from the data persistence layer.

[0130] For example, in combination Figure 5 When the business service layer needs to perform automated evaluation work, the sample data and the first reference data can be obtained from the data persistence layer for processing.

[0131] 305. According to the document information, the sample data and the first reference data are evaluated and compared to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

[0132] Exemplarily, this step may refer to step 203 and will not be described in detail here.

[0133] 306 . If it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, a function repair process is performed on the navigation application to be tested, and step 305 is executed again.

[0134] For example, in combination Figure 8 , based on the business service layer, after executing the automated evaluation link, according to the content of the test report, if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, for example, when the navigation function quality evaluation is low, the gap between the low-scoring samples in the sample set and the navigation function quality of other target navigation applications can be traced, and the defects and errors of the online service can be adjusted in a targeted manner to repair the function of the navigation application to be tested; after the problem is repaired, the corresponding evaluation task of the sample set is re-executed, that is, the sample data and the first reference data are tested and compared according to the document information again, and a new test report of the navigation application to be tested is obtained to verify the improvement of the function quality after adjustment. In this process, the platform tools are used to automatically analyze the problem samples, preliminarily screen the problem classification, and the product personnel, engineers, and analysts use the platform tools in the equipment to perform manual assisted sampling to refine the problem, output the test report, and feed back the analysis results to the automated evaluation link, which can form a dynamic optimization mechanism, so that the evaluation process is transformed from a one-way one-time operation to a system that continuously improves itself, guides and optimizes the direction of automated evaluation, and forms a closed loop of continuous iteration and optimization.

[0135] In one example, the navigation application testing system further includes a data cleaning and noise reduction layer; after determining that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, step 306 further includes:

[0136] The first step is to update the sample data and document information based on the data cleaning and noise reduction layer and the test report.

[0137] The second step is to perform a test comparison process on the updated sample data and the first reference data based on the business service layer and the updated document information to obtain an updated test report.

[0138] Exemplarily, the navigation application testing platform also includes a data cleaning and noise reduction layer; as daily usage scenarios expand and change, and map data is updated over time, the analysis and classification methods of samples in the navigation function quality evaluation process also need to be updated accordingly. After determining that the test report represents an abnormality in the navigation function test of the navigation application to be tested, based on the data cleaning and noise reduction layer, according to the test report, the sample data and document information are updated to obtain updated document information and updated sample data; based on the business service layer, according to the updated document information, the updated sample data and the first reference data are tested and compared to obtain an updated test report to complete a new round of testing.

[0139] For example, combining Figure 7 , 8 When the evaluation results in the test report show obvious abnormalities or the evaluation results of all parties tend to be close, it means that the classification logic of the current sample set is wrong or tends to fail. It is necessary to recheck and adjust the collection and classification logic of the sample set, such as noise processing, itinerary splitting, start and end point aggregation, log deduplication, log aggregation, log classification, etc. Iteratively adjust the indicator system according to the problem distribution, guide and optimize the automated evaluation, and form a closed loop of continuous iteration and optimization.

[0140] 307. Based on the web page display layer, display the test report and test data of the navigation application to be tested; wherein the test data includes document information, sample data and first reference data.

[0141] Among them, the navigation application testing system also includes a web page display layer.

[0142] For example, in combination Figure 7 The navigation application testing platform also includes a web page display layer. Based on the web page display layer, users can view the test report and test data of the navigation application to be tested, including the test results of different navigation function evaluation aspects, as well as document information, sample data and first reference data related to different navigation function evaluation aspects in the test data.

[0143] For example, combining Figure 7 Based on the web display layer, the evaluation systems of different navigation function evaluation aspects such as POI evaluation system, Quickest Estimated Time of Arrival (QETA) evaluation system, route calculation evaluation system, and reachability evaluation system can be displayed first. Users can click on the corresponding functional modules to further see more specific navigation function evaluation aspects under each evaluation system. These more specific navigation function evaluation aspects can be established, managed and executed based on the business service layer. The corresponding evaluation tasks include the first optimal rate evaluation of the route, the reasonable rate evaluation of the route, the consistency rate evaluation of the route, the reasonable rate evaluation of the route segment, the POI expiration evaluation, the POI freshness evaluation, the first hit evaluation of the POI, the POI satisfaction evaluation, the POI sample data management, the ETA sample data management, the QETA evaluation, and the terminal reachability evaluation. After executing each evaluation task, users can query the test report and test data of each evaluation task based on the web display layer. In addition, based on the functions of the business service layer, various evaluation tasks (execution time, frequency) can be generated, managed, and executed. For example, an automatic monitoring regression mechanism can be established to automatically conduct evaluation and monitoring every day. When the evaluation score deteriorates beyond the set threshold, the monitoring platform can automatically alarm and block the online access to ensure that the online effect does not regress, thereby achieving the effect of positive iteration and automatic online access.

[0144] In one example, the navigation application testing system also includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0145] Exemplarily, the navigation application testing platform also includes a basic technology module, which is used to provide navigation assistance services, for example, in combination with Figure 5 The basic technology module can provide navigation assistance services including vector map, path matching service, path similarity, distance calculation, route sorting, POI duplicate detection, POI sorting, spatial search, spatial overlap, message push, etc. In addition, when executing evaluation tasks at the business service layer, the basic technology module will be called to execute the evaluation tasks.

[0146] In this embodiment, on the basis of the above-mentioned embodiment, the automation of navigation function testing based on large samples can more comprehensively present the performance of navigation software in various scenarios. Through the automated evaluation process, a large number of samples can be efficiently processed to avoid the inefficiency and errors of manual processing. Moreover, in the iterative closed-loop process, samples or evaluation indicators can be dynamically adjusted based on the feedback of each evaluation result, so that the evaluation results are more in line with the actual situation, and the software problems of the navigation application can be discovered and solved in time, thereby effectively improving the navigation function service quality of the navigation application and improving the user experience.

[0147] Fig.10A schematic diagram of the structure of a navigation application testing device provided in an embodiment of the present application is shown in FIG. Fig.10 As shown, the device is applied to the business service layer of the navigation application test system; the navigation application test system also includes a data persistence layer; the device includes:

[0148] The first acquisition module 401 is used to acquire document information stored in the data persistence layer; wherein the document information includes the evaluation index system document;

[0149] The second acquisition module 402 is used to acquire sample data and first reference data of the navigation application to be tested based on the business service layer; wherein the sample data represents actual user behavior based on the navigation application to be tested; and the first reference data represents expected user behavior based on the navigation application to be tested;

[0150] The testing module 403 is used to evaluate and compare the sample data and the first reference data according to the document information to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

[0151] The device can also be directly an implementation form of the business service layer; wherein the first acquisition module 401, the second acquisition module 402, and the test module 403 are all specific implementations or components in the business service layer, which are usually responsible for processing the logic corresponding to more specific business functions or fields.

[0152] In a possible implementation, the second acquisition module 402 is specifically used to: acquire application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservices in the navigation application to be tested; generate sample data and first reference data according to each user behavior data; and classify and store the sample data and the first reference data in the data persistence layer; and acquire the sample data and the first reference data from the data persistence layer.

[0153] In a possible implementation, the second acquisition module 402 is specifically used to: perform feature processing on each user behavior data to obtain feature data corresponding to each user behavior data; and generate sample data according to each user behavior data and each feature data.

[0154] In a possible implementation, the second acquisition module 402 is further specifically used to: process each user behavior data to obtain at least one first reference value in the first reference data; obtain second reference data of at least one target navigation application through a preset interface; wherein the second reference data represents the expected navigation function record of the target navigation application; vote on each second reference data to obtain at least one second reference value in the first reference data; process key data in each user behavior data to obtain at least one third reference value in the first reference data.

[0155] In a possible implementation, the device is also used for: if it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, then function repair processing is performed on the navigation application to be tested, and the step of generating a test report of the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

[0156] In a possible implementation, the navigation application testing system also includes a data cleaning and denoising layer; after the device is used to determine that the test report represents an abnormality in the navigation function test of the navigation application to be tested, the device is also used to: based on the data cleaning and denoising layer, update the sample data and document information according to the test report; based on the business service layer, test and compare the updated sample data and the first reference data according to the updated document information to obtain an updated test report.

[0157] In a possible implementation, the navigation application testing system further includes a web page display layer; the device is further used to: based on the web page display layer, display the test report and test data of the navigation application to be tested; wherein the test data includes document information, sample data and first reference data.

[0158] In a possible implementation, the navigation application testing system further includes a basic technology module; the basic technology module is used to provide navigation assistance services.

[0159] The device of this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principles are the same and will not be repeated here.

[0160] The embodiment of the present application also provides an electronic device cluster, which includes at least one memory and at least one processor; wherein the at least one memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the at least one memory, so that the at least one processor executes the method provided in the above embodiment. Exemplarily, the electronic device cluster includes at least one electronic device, Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.11As shown, the electronic device includes: a memory 501 and a processor 502; the memory 501 is a memory for storing instructions executable by the processor 502.

[0161] The processor 502 is configured to execute the method provided in the above embodiment.

[0162] The electronic device 500 further includes a receiver 503 and a transmitter 504. The receiver 503 is used to receive instructions and data sent by other devices, and the transmitter 504 is used to send instructions and data to external devices.

[0163] The specific implementation process of the processor can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0164] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0165] An embodiment of the present application also provides a chip for executing instructions, which is used to execute the technical solution of the processing method in the above embodiment.

[0166] The embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the computer executes the technical solution of the processing method of the above embodiment.

[0167] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0168] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist in a device as discrete components.

[0169] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the processing method in the above embodiment can be implemented.

[0170] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0171] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0172] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0173] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0174] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0177] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0178] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0179] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A navigation application testing system, characterized in that: The navigation application testing system includes: a data persistence layer and a business service layer; The data persistence layer is used to store document information; wherein the document information includes evaluation index system documents; The business service layer is used to generate a test report for the navigation application to be tested based on the document information, sample data of the navigation application to be tested and first reference data; wherein the sample data represents the actual navigation function record of the navigation application to be tested; the first reference data represents the expected navigation function record of the navigation application to be tested; and the test report includes the navigation function quality test result of the navigation application to be tested.

2. The system according to claim 1, characterized in that The business service layer is specifically used to: obtain application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservice in the navigation application to be tested; the user behavior data represents the user's navigation function usage behavior of the navigation application to be tested; Generate the sample data and the first reference data according to each of the user behavior data; and classify and store the sample data and the first reference data in the data persistence layer; Based on the business service layer, the sample data and the first reference data are obtained from the data persistence layer.

3. The system according to claim 2, characterized in that The business service layer is specifically used for: Performing feature processing on each of the user behavior data to obtain feature data corresponding to each of the user behavior data; The sample data is generated according to each of the user behavior data and each of the feature data.

4. The system according to claim 2, characterized in that The business service layer is also specifically used for: Processing each of the user behavior data to obtain at least one first reference value in the first reference data; Acquire, through a preset interface, second reference data of at least one target navigation application; wherein the second reference data represents an expected navigation function record of the target navigation application; Performing voting on each of the second reference data to obtain at least one second reference value in the first reference data; The key data in each of the user behavior data are processed to obtain at least one third reference value in the first reference data.

5. The system according to claim 1, characterized in that The business service layer is also used for: If it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, function repair processing is performed on the navigation application to be tested, and the step of generating a test report for the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

6. The system according to claim 1, characterized in that The navigation application testing system also includes a data cleaning and noise reduction layer; The data cleaning and noise reduction layer is used to update the sample data and the document information according to the test report; The business service layer is used to perform test comparison processing on the updated sample data and the first true value reference data according to the updated document information to obtain an updated test report.

7. The system according to claim 1, characterized in that The navigation application testing system also includes a web page display layer; The web page display layer is used to display the test report and test data of the navigation application to be tested; wherein the test data includes the document information, the sample data and the first reference data.

8. The system according to any one of claims 1 to 7, characterized in that: The navigation application testing system also includes a basic technology module; The basic technical module is used to provide navigation assistance services.

9. A navigation application testing method, characterized in that: The method is applied to the business service layer of the navigation application test system; the navigation application test system also includes a data persistence layer; the method includes: Acquire the document information stored in the data persistence layer; wherein the document information includes the evaluation index system document; Acquire sample data and first reference data of the navigation application to be tested; wherein the sample data represents actual navigation function records of the navigation application to be tested; and the first reference data represents expected navigation function records of the navigation application to be tested; According to the document information, the sample data and the first reference data are evaluated and compared to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

10. The method according to claim 9, characterized in that Acquiring sample data and first reference data of the navigation application to be tested, including: Acquire application data from the service gateway of the navigation application to be tested; wherein the application data includes user behavior data generated by the microservice in the navigation application to be tested; the user behavior data represents the user's navigation function usage behavior of the navigation application to be tested; Generate the sample data and the first reference data according to each of the user behavior data; and classify and store the sample data and the first reference data in the data persistence layer; The sample data and the first reference data are obtained from the data persistence layer.

11. The method according to claim 9 or 10, characterized in that: The method comprises: If it is determined that the test report indicates that the navigation function test of the navigation application to be tested is abnormal, function repair processing is performed on the navigation application to be tested, and the step of generating a test report for the navigation application to be tested based on the document information, sample data of the navigation application to be tested and the first reference data is re-executed.

12. A navigation application testing device, characterized in that: The device is applied to the business service layer of the navigation application test system; The navigation application testing system also includes a data persistence layer; the device includes: A first acquisition module, used to acquire document information stored in the data persistence layer; wherein the document information includes an evaluation indicator system document; A second acquisition module is used to acquire sample data and first reference data of the navigation application to be tested; wherein the sample data represents actual user behavior based on the navigation application to be tested; and the first reference data represents expected user behavior based on the navigation application to be tested; The test module is used to evaluate and compare the sample data and the first reference data according to the document information to obtain a test report of the navigation application to be tested; wherein the test report includes a navigation function quality test result of the navigation application to be tested.

13. An electronic equipment cluster, characterized in that: include: at least one memory and at least one processor; at least one of said memories stores computer-executable instructions; At least one of the processors executes at least one computer-executable instruction stored in the memory, so that at least one of the processors performs the method according to any one of claims 9 to 11.

14. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 9 to 11; and / or, the computer program product includes a computer program, which, when executed by a processor, implements the method as described in any one of claims 9 to 11.

Images