Application test method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses an application testing method and device, computer equipment and a computer readable storage medium, relates to the technical field of automatic testing, and realizes comprehensive, efficient and intelligent testing of a to-be-tested application through cooperative work of a point burying technology, large language model analysis and a testing operation component. In the testing process, not only are functions and performance expressions of an application level paid attention to, but also user operation is truly simulated through the testing operation component, so that testing goes deep into a bottom hardware level, and the effectiveness of testing and the accuracy of a testing result are remarkably improved. The method comprises the steps of determining a to-be-tested application, performing point burying processing on the to-be-tested application, and obtaining point burying data generated when a user executes operation on the to-be-tested application; analyzing and processing the burying point data based on a large language model to generate test coordinate data; and controlling the test operation component to simulate a user to execute test operation on the to-be-tested application according to the test coordinate data to obtain an application test result of the to-be-tested application.

Description

Technical Field

[0001] The present application relates to the field of automated testing technology, and in particular to an application testing method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] With the rapid development of mobile Internet technology, stylus-based interactive apps (Applications) are increasingly used on mobile devices. These apps greatly enrich the user's interactive experience by providing precise handwriting, drawing, note-taking, and graffiti functions, and are widely used in learning, work, artistic creation, and other fields. However, while pursuing functional richness and improving user experience, ensuring the stability and reliability of apps has become an important challenge for developers.

[0003] In the related technology, the stability test of stylus interactive APP is carried out by adopting UI (User Interface) automation script. By simulating user operations, a series of predefined test processes are automatically executed to check indicators such as APP's UI interaction process, function implementation and response performance.

[0004] In the process of implementing the relevant technology, the applicant realizes that the relevant technology has at least the following technical problems:

[0005] UI automation scripts mainly focus on application-level testing, which simulates user input and checks the state changes of UI elements to determine whether the APP functions as expected. However, the uniqueness of stylus-interactive APPs lies in their close interaction with hardware (such as stylus, touch screen, etc.). In actual use, the performance and compatibility of hardware and subtle differences in user operations may have a significant impact on the stability of the APP. UI automation scripts cannot fully cover these potential hardware problems. Moreover, the user input simulated by UI automation scripts often lacks authenticity and diversity, and it is difficult to fully simulate the complex operation scenarios and emergencies of users in actual use. This may lead to a large deviation between the test results and the actual user experience, affecting the effectiveness and accuracy of the test. Summary of the invention

[0006] In view of this, the present application provides an application testing method, apparatus, computer device and computer-readable storage medium, the main purpose of which is to solve the problem that the current UI automation scripts cannot fully cover these potential hardware problems. The user input simulated by the UI automation scripts often lacks authenticity and diversity, and it is difficult to fully simulate the complex operation scenarios and emergencies encountered by users during actual use, which may lead to a large deviation between the test results and the actual user experience, affecting the effectiveness and accuracy of the test.

[0007] According to the first aspect of the present application, there is provided an application testing method, the method comprising:

[0008] Determine an application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when a user performs an operation on the application to be tested;

[0009] Parsing and processing the buried point data based on the large language model to generate test coordinate data;

[0010] The control test operation component simulates a user performing a test operation on the application to be tested according to the test coordinate data, and obtains an application test result of the application to be tested.

[0011] Optionally, the determining the application to be tested, performing tracking processing on the application to be tested, and obtaining tracking data generated when the user performs an operation on the application to be tested includes:

[0012] In response to an application test request, the application indicated by the application test request is used as the application to be tested, and the tracking point parameters carried by the application test request are obtained, where the tracking point parameters are used to indicate the location and type of the tracking point to be tested;

[0013] According to the tracking point parameters, a data collection point is set in the application to be tested, tracking point processing of the application to be tested is completed, and the application to be tested after the tracking point processing is deployed so that the user can use the application to be tested after the tracking point processing;

[0014] The buried point data accumulated during the operation of the application to be tested is periodically collected and acquired, and the buried point data is generated after the data collection point is triggered when the user uses the application to be tested.

[0015] Optionally, parsing and processing the buried point data based on the large language model to generate test coordinate data includes:

[0016] Inputting the buried point data into the large language model, parsing the buried point data based on the large language model to obtain a plurality of high-frequency pixel coordinate groups;

[0017] Using the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups;

[0018] The large language model is used to perform interpolation processing on the plurality of plane coordinate groups, and the interpolated plurality of plane coordinate groups are used as the test coordinate data.

[0019] Optionally, the parsing of the buried point data based on the large language model to obtain a plurality of high-frequency pixel coordinate groups includes:

[0020] Determine a field name for indicating coordinate data in the buried point data, and obtain a preset frequency condition;

[0021] Constructing coordinate output prompt words with reference to the field name and the frequency condition;

[0022] The coordinate output prompt word is used to guide the large language model to parse the buried point data, so that the large language model outputs the multiple high-frequency pixel coordinate groups.

[0023] Optionally, the using the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups includes:

[0024] Query preset device parameters and coordinate units, wherein the device parameters include device screen size and device screen resolution;

[0025] Constructing coordinate conversion prompt words with reference to the device parameters and the coordinate units;

[0026] The coordinate transformation prompt word is used to guide the large language model to perform coordinate transformation on each of the plurality of high-frequency pixel coordinate groups, and the large language model outputs the plurality of plane coordinate groups.

[0027] Optionally, the control test operation component simulates a user performing a test operation on the application to be tested according to the test coordinate data to obtain an application test result of the application to be tested, including:

[0028] Establishing data transmission with the test operation component, and sending the test coordinate data to the test operation component through a preset transmission protocol, so that the test operation component simulates the user's operation on the application to be tested according to the test coordinate data;

[0029] During the process of the test operation component simulating the operation, the operation response result of the application to be tested is continuously obtained, and the application test result is generated based on the obtained operation response result.

[0030] Optionally, the method further comprises:

[0031] Controlling the test operation component to continuously simulate the user's operation according to the test coordinate data, and counting the simulation time until the simulation time reaches a time threshold, thereby obtaining a plurality of application test results;

[0032] Count the ratio of the number of operation response results indicating failure in the multiple application test results to the total number of operation response results, analyze the application performance data in the multiple application test results, obtain stability index data of the application to be tested, and output the stability index data.

[0033] According to a second aspect of the present application, an application testing device is provided, the device comprising:

[0034] An acquisition module is used to determine an application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when a user performs an operation on the application to be tested;

[0035] A parsing module, used to parse and process the buried point data based on a large language model to generate test coordinate data;

[0036] The simulation module is used to control the test operation component to simulate the user performing a test operation on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested.

[0037] Optionally, the acquisition module is used to respond to an application test request, take the application indicated by the application test request as the application to be tested, and obtain the tracking point parameters carried by the application test request, wherein the tracking point parameters are used to indicate the location and type of the tracking point to be tested; according to the tracking point parameters, set data collection points in the application to be tested, complete the tracking point processing of the application to be tested, and deploy the application to be tested after the tracking point processing so that the user can use the application to be tested after the tracking point processing; periodically collect and obtain the tracking point data accumulated during the operation of the application to be tested, wherein the tracking point data is generated after the data collection point is triggered when the user uses the application to be tested.

[0038] Optionally, the parsing module is used to input the buried point data into the large language model, parse the buried point data based on the large language model to obtain multiple high-frequency pixel coordinate groups; use the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups; interpolate the multiple plane coordinate groups through the large language model, and use the multiple plane coordinate groups after interpolation as the test coordinate data.

[0039] Optionally, the parsing module is used to determine the field name used to indicate the coordinate data in the buried point data and obtain a preset frequency condition; construct a coordinate output prompt word with reference to the field name and the frequency condition; and use the coordinate output prompt word to guide the large language model to parse the buried point data so that the large language model outputs the multiple high-frequency pixel coordinate groups.

[0040] Optionally, the parsing module is used to query preset device parameters and coordinate units, the device parameters including device screen size and device screen resolution; construct coordinate conversion prompts with reference to the device parameters and the coordinate units; use the coordinate conversion prompts to guide the large language model to perform coordinate conversion on each of the multiple high-frequency pixel coordinate groups, and the large language model outputs the multiple plane coordinate groups.

[0041] Optionally, the simulation module is used to establish data transmission with the test operation component, and send the test coordinate data to the test operation component through a preset transmission protocol, so that the test operation component simulates the user's operation on the application to be tested according to the test coordinate data; during the process of the test operation component simulating the operation, continuously obtain the operation response result of the application to be tested, and generate the application test result based on the obtained operation response result.

[0042] Optionally, the device further comprises:

[0043] The simulation module is further used to control the test operation component to continuously simulate the user's operation according to the test coordinate data, and to count the simulation time until the simulation time reaches a time threshold, thereby obtaining multiple application test results;

[0044] The output module is used to count the proportion of the number of operation response results indicating failure in the multiple application test results to the total number of operation response results, and to analyze the application performance data in the multiple application test results to obtain stability index data of the application to be tested, and output the stability index data.

[0045] According to a third aspect of the present application, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0046] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0047] Through the above technical scheme, the present application provides an application testing method, device, computer equipment and computer-readable storage medium. The present application determines the application to be tested, performs tracking processing on the application to be tested, and obtains the tracking data generated when the user performs operations on the application to be tested, analyzes and processes the tracking data based on the large language model, generates test coordinate data, and controls the test operation component to simulate the user to perform test operations on the application to be tested according to the test coordinate data, so as to obtain the application test results of the application to be tested. Through the collaborative work of tracking technology, large language model analysis and automated test operation components, comprehensive, efficient and intelligent testing of the application to be tested is achieved. During the testing process, not only the functions and performance at the application level are paid attention to, but also the user operations are truly simulated through the test operation components, so that the test goes deep into the underlying hardware level, which significantly improves the effectiveness of the test and the accuracy of the test results.

[0048] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0050] Figure 1 A schematic diagram of an application testing method provided in an embodiment of the present application is shown;

[0051] Figure 2A A schematic diagram of another application testing method provided in an embodiment of the present application is shown;

[0052] Figure 2B A schematic diagram of a buried point data provided by an embodiment of the present application is shown;

[0053] Figure 2C A schematic diagram of a pixel coordinate group provided in an embodiment of the present application is shown;

[0054] Figure 2D A schematic diagram of a plane coordinate group provided in an embodiment of the present application is shown;

[0055] Figure 2E A schematic diagram of an interpolation result provided in an embodiment of the present application is shown;

[0056] Figure 2FA schematic diagram of a flow chart of another application testing method provided in an embodiment of the present application is shown;

[0057] Figure 3 A schematic diagram of the structure of an application testing device provided in an embodiment of the present application is shown;

[0058] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0060] The present application embodiment provides an application testing method, such as Figure 1 As shown, the method includes:

[0061] 101. Determine an application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when a user performs operations on the application to be tested.

[0062] The embodiments of the present application can be applied to the application testing system. The developer can specify the application to be tested as the application to be tested on the application testing system, so that the application testing system executes the technical solution in the embodiments of the present application and completes the test of the application to be tested. The application to be tested can be a stylus interactive APP. The stylus interactive APP refers to an application that interacts with the screen through a stylus (or a touch pen). They usually provide rich handwriting, drawing, notes, graffiti and other functions, and have a good user interaction experience. For example, some built-in APPs on learning machines, after the user triggers the APP by long pressing the stylus, draws a pattern to circle the content (or draws a horizontal line under the target content, etc.), the APP can automatically obtain the circled content, and perform relevant processing, and the APP can be used as the application to be tested. In the embodiments of the present application, the application to be tested is a stylus interactive APP as an example for explanation. The actual application process does not limit the application type of the application to be tested.

[0063] Furthermore, buried data is a data collection method, which refers to data collection points pre-set at specific locations (i.e., "buried points") in an application (such as an APP) to record user behavior data within the application. Specifically, an in-depth analysis can be conducted on the application to be tested to determine key user interaction points, such as button clicks, page jumps, data input, etc., and these interaction points can be buried. The buried technology allows the user's operating behavior at these interaction points to be captured without affecting the user's normal experience, and detailed buried data can be generated. In this way, the test target can be accurately located, the pertinence and efficiency of the test can be improved, and user behavior can be fully captured through buried data, providing a rich data source for subsequent data analysis and test operations.

[0064] 102. Parse and process the buried point data based on the large language model to generate test coordinate data.

[0065] In an embodiment of the present application, after acquiring the buried point data, the application test system will use a large language model to deeply analyze these buried point data. The large language model (LLM) is a deep learning model trained with a large amount of text data, which is designed to understand and generate human language. It has powerful natural language processing capabilities and context understanding capabilities. In an embodiment of the present application, a large language model is used to identify the user's operation intentions, and the corresponding test coordinate data is generated according to the operation description.

[0066] The test coordinate data is used to indicate what type of operation (such as clicking, sliding, inputting, etc.) the user has performed at the corresponding operation position (such as a button in the application) and the specific operation content (such as input text content, sliding distance, etc.), so that the user's actual operation can be simulated according to the test coordinate data in the future to implement the test of the application to be tested. The embodiment of the present application uses a large language model to improve the accuracy and efficiency of data analysis, reduce manual intervention, and the generated test coordinate data is accurate and reliable, providing a solid foundation for subsequent automated testing.

[0067] 103. Control the test operation component to simulate the user performing a test operation on the application to be tested according to the test coordinate data, and obtain an application test result of the application to be tested.

[0068] In an embodiment of the present application, based on the test coordinate data generated by the large language model, the application testing system simulates the user's real operation by controlling the test operation component, and performs a series of automated tests on the application to be tested. In an embodiment of the present application, the application to be tested is tested end-to-end based on the test operation component, that is, the entire application usage process from the user's perspective is simulated to ensure that the application to be tested can run correctly throughout the business logic and process. This testing method covers the entire process from the user opening the application to be tested, to completing a specific task or function, and finally exiting the application to be tested.

[0069] The test operation component may be a test robot, a mechanical arm, etc. The embodiment of the present application is described by taking the test operation component as a mechanical arm as an example. Specifically, when selecting a mechanical arm, a four-degree-of-freedom mechanical arm may be selected, which has the advantages of lightweight structure, high positioning accuracy, high torque, etc., and supports a variety of wireless control methods, such as HTTP protocol communication, so that the test coordinate data can be sent to the mechanical arm through the HTTP protocol to complete the control of the mechanical arm.

[0070] The embodiments of the present application mainly conduct stability testing on the application to be tested. Stability testing refers to the process of checking whether the application to be tested can run stably and whether it can work normally in various usage scenarios. Its main purpose is to detect defects and vulnerabilities in the application to be tested, and provide developers with directions for improvement and optimization, thereby improving the quality of the APP and user experience. Therefore, during the test process, the application testing system will record the performance of the application in real time, including response time, interface changes, data processing results, etc., and finally form a detailed application test result.

[0071] In this way, the application testing solution proposed in the embodiment of the present application realizes automated testing of the application, reduces labor costs, and improves testing efficiency; moreover, by simulating real user operations, the performance of the application in actual usage scenarios is more accurately evaluated, and the test results are detailed and reliable, providing strong data support for application optimization.

[0072] The method provided in the embodiment of the present application determines the application to be tested, performs tracking processing on the application to be tested, and obtains the tracking data generated when the user performs operations on the application to be tested, parses and processes the tracking data based on the large language model, generates test coordinate data, and controls the test operation component to simulate the user performing test operations on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested. Through the coordinated work of tracking technology, large language model analysis and automated test operation components, a comprehensive, efficient and intelligent test of the application to be tested is achieved. During the test process, not only the functions and performance at the application level are paid attention to, but also the user operations are truly simulated through the test operation components, so that the test goes deep into the underlying hardware level, which significantly improves the effectiveness of the test and the accuracy of the test results.

[0073] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, the embodiment of the present application provides another application test method, such as Figure 2A As shown, the method includes:

[0074] 201. In response to an application test request, the application indicated by the application test request is used as the application to be tested, and the tracking parameters carried by the application test request are obtained. According to the tracking parameters, data collection points are set in the application to be tested, the tracking processing of the application to be tested is completed, and the application to be tested after the tracking processing is deployed so that users can use the application to be tested after the tracking processing.

[0075] The embodiments of the present application can be applied to an application testing system, and developers can specify the application to be tested as the application to be tested on the application testing system, so that the application testing system executes the technical solution in the embodiments of the present application and completes the test of the application to be tested. When the developer has an application testing requirement, an application test request can be initiated in the application testing system. In response to the application test request, the application testing system will parse the application test request and determine the application to be tested specified in the application test request. At the same time, the application testing system will extract the embedded point parameters carried from the application test request. The embedded point parameters are used to indicate the location and type of the embedded point. The embedded point parameters include but are not limited to interface element identifiers (such as button IDs, text box names, etc.) and operation types (such as page browsing, button clicks, form submissions, page loading time, etc.).

[0076] Based on the parsed tracking parameters, the application testing system will set the corresponding data collection points in the source code or configuration file of the application to be tested. The data collection points will be triggered when the user performs a specific operation to collect relevant operation data. Then, the application testing system will integrate the configured tracking logic into the code of the application to be tested to ensure that the tracking code can be executed correctly when the application is running and does not affect the normal function of the application.

[0077] Then, the application testing system will compile and package the application to be tested containing the embedding code to generate a deployable application package, ensuring that the embedding code is tightly integrated with other parts of the application to be tested and can run normally in the actual application environment; the constructed application package is then deployed to the target environment (such as a test server, simulator or real device). The deployment process includes steps such as installing, configuring and starting the application to ensure that the application to be tested can run normally in the target environment.

[0078] After deployment, the application test system can start monitoring the data collected by tracking. At this point, users or testers can use the application to be tested after tracking in the normal way, and the application test system will collect operation data in real time in the background. The collected data will also be used for tracking data in subsequent performance evaluation, troubleshooting, and user experience optimization.

[0079] In this way, by dynamically setting the tracking parameters, the monitoring scope and depth can be flexibly adjusted to meet different testing needs; moreover, by setting data collection points, the collected data can be ensured to be true and accurate, reducing errors; in addition, the tracking technology strives to minimize the impact on application performance and user experience, ensuring that the normal use of the application by users or testers is not affected while testing.

[0080] 202. Periodically collect and obtain the tracking data accumulated during the operation of the application to be tested.

[0081] In an embodiment of the present application, after the tracking processing of the application to be tested is completed and successfully deployed, the tracking data will be generated when the user uses the application to be tested and triggers the data collection point. Next, the application testing system will periodically collect and obtain the tracking data accumulated during the operation of the application to be tested, ensuring the timeliness and completeness of the test data, and providing strong support for subsequent performance evaluation, problem diagnosis, etc.

[0082] First, it is necessary to define an appropriate collection cycle based on the test requirements and application characteristics. The length of the collection cycle should take into account factors such as the test purpose, data volume, system resource consumption, and real-time requirements. For example, for applications that require real-time monitoring, the collection cycle may be set to a few minutes or hours; while for applications that do not change frequently, the collection cycle can be appropriately extended to one day or longer.

[0083] After determining the collection cycle, you can configure the corresponding collection task, specifically setting the execution time, execution frequency, collection scope (such as specific users, specific functional modules, etc.) and storage location of the collected data. The collection task can be implemented through automated testing tools, scripts or background services to ensure that it is automatically started and executed at the specified time.

[0084] When the collection task is executed, the application test system will extract the tracking data from the application to be tested according to the preset collection scope and rules. The collected tracking data can be stored in the log file or database of the application to be tested or transmitted to the remote server through the network API. In addition, during the collection process, the application test system needs to ensure the integrity and accuracy of the data to avoid missing or erroneous data collection.

[0085] Taking the application to be tested as a stylus interactive APP as an example, the collected tracking data may be Figure 2B Data in the form shown, Figure 2B The field named "real_bbox" records the pixel data drawn by the user with the stylus after activating the APP.

[0086] 203. Input the buried point data into the large language model, and parse the buried point data based on the large language model to obtain multiple high-frequency pixel coordinate groups.

[0087] The applicant recognizes that the application to be tested generates a large amount of point-of-sale data every day, among which the high-frequency data is more typical and representative. Therefore, the large language model can be used to analyze the point-of-sale data and extract the high-frequency data. Therefore, in the embodiment of the present application, before inputting the point-of-sale data into the large language model, it is necessary to determine the field name used to indicate the coordinate data in the point-of-sale data and obtain the preset frequency condition. The field indicated by the field name is used to record the pixel coordinate information during user interaction. Figure 2B For example, the field named "real_bbox" records the pixel coordinate information, and the determined field name is "real_bbox"; further, in order to extract the high-frequency pixel coordinate group, a frequency condition will be pre-set in the application test system. The frequency condition can be a specific value (such as more than 100 clicks) or a relative proportion (such as the top 10% of the coordinate points). The frequency condition can be set based on the test requirements and data analysis goals.

[0088] After determining the field name and frequency conditions, you can refer to the field name and frequency conditions to construct coordinate output prompts. The coordinate output prompts need to clearly guide the large language model on how to output high-frequency pixel coordinate groups. Then, the application test system will input the constructed coordinate output prompts into the large language model to guide the large language model to parse the buried point data. The large language model outputs multiple high-frequency pixel coordinate groups with its powerful natural language processing capabilities and context understanding capabilities.

[0089] In the actual application process, taking the field name "real_bbox" as an example, the designed coordinate output prompt word can be the prompt word, as shown below:

[0090] "TrackingData, compare the coordinate group data on the right side of each real_bbox equal sign, and randomly return a coordinate group data that appears frequently (that is, the coordinate group on the right side of the real_bbox equal sign). Please return the result directly, and be careful not to return any other text content."

[0091] In the above coordinate output prompt, TrackingData refers to each tracking point data (tens of thousands of tracking point data, each of which has a real_bbox field). The prompt is transmitted to LLM through the interface for processing. LLM will complete the extraction operation of the coordinate group and return the result to obtain multiple high-frequency pixel coordinate groups. Figure 2B The processing results of the buried point data shown are as follows Figure 2C As shown, it includes multiple high-frequency pixel coordinate groups. The large language model is used to parse the buried point data and extract the high-frequency pixel coordinate groups, which greatly reduces manual intervention and error rate. The extracted high-frequency pixel coordinate groups provide strong support for subsequent user behavior analysis, performance optimization and other work.

[0092] 204. Use a large language model to perform coordinate transformation on multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups.

[0093] The applicant recognizes that since the buried point data stores pixel coordinate data, and the test operating components such as robotic arms use distance unit data in mm, in an embodiment of the present application, the application test system will use a large language model to perform coordinate conversion on multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups.

[0094] Before performing coordinate conversion, you first need to query and obtain the specific parameters and coordinate units of the device where the application to be tested is running. This is crucial for accurately converting pixel coordinates into plane coordinates. Among them, device parameters include device screen size and device screen resolution. The device screen size indicates the physical size of the device screen, usually in inches or centimeters; the device screen resolution refers to the number of pixels displayed on the screen, usually expressed in the form of "horizontal pixels * vertical pixels". The horizontal pixels can be represented by X, and the vertical pixels can be represented by Y, such as 1920*1080. The coordinate unit is used to indicate the unit of the generated plane coordinate group. Since the distance unit used by test operating components such as robotic arms is millimeters, the coordinate unit can be millimeters.

[0095] Then, the application test system will refer to the queried device parameters and coordinate units to construct the coordinate transformation prompt words. The coordinate transformation prompt words need to clearly specify how to transform the high-frequency pixel coordinate group into the plane coordinate group. The application test system will use the coordinate transformation prompt words to guide the large language model to perform coordinate transformation on each of the multiple high-frequency pixel coordinate groups, and the large language model will output multiple plane coordinate groups.

[0096] In the actual application process, continue with the above Figure 2C Taking the example shown as an example, the coordinate transformation prompt word designed can be a prompt word, as shown below:

[0097] "PixelData, given that the tablet's screen size is A and the screen resolution is X*Y, convert the above coordinate group data into data in mm, retain the original format, and do not retain decimals. Please return the processing result directly, and be careful not to return any other text content."

[0098] In the above coordinate conversion prompts, PixelData refers to Figure 2C The high-frequency pixel coordinate group in ; A refers to the device screen size, such as 14 inches; X*Y refers to the device screen resolution, such as 2880*1800. The large language model is targeted at Figure 2C The conversion results returned by the high-frequency pixel coordinate group in can be specifically as follows Figure 2D As shown, Figure 2D That is, a series of converted plane coordinate groups are displayed. The embodiment of the present application converts pixel coordinates into plane coordinates with more general significance through coordinate conversion, which is convenient for cross-device execution and makes the test process of the application to be tested closer to user operation.

[0099] 205. Perform interpolation processing on the multiple plane coordinate groups through the large language model, and use the multiple plane coordinate groups after interpolation as test coordinate data.

[0100] The applicant recognizes that the buried point data stores only part of the pixel data. In order to enable the test operation components to move more smoothly, the data needs to be properly interpolated to make the transition between points smoother, and this part of the operation can also be processed by LLM. Therefore, in an embodiment of the present application, the application test system will interpolate multiple plane coordinate groups through a large language model, and use the multiple plane coordinate groups after interpolation as test coordinate data.

[0101] Specifically, for the interpolation process, a prompt word can also be constructed, and the prompt word can be specifically as follows:

[0102] "DistanceData, interpolate the above coordinate data and retain the original format. Please return the result directly. Be careful not to return any other text content."

[0103] Continue with the above Figure 2D Taking the plane coordinate group in as an example, in the above prompt word, DistanceData refers to Figure 2D The plane coordinate group included in . Among them, the form of a chart is used to Figure 2D The results of displaying the plane coordinate group in can be seen in Figure 2E In Figure ①, Figure 2D The results of interpolating the plane coordinates in the graph can be seen in Figure 2E Figure ② in the figure. Figure 2E From Figure ① and Figure ②, we can see that the data points in Figure ② are denser and the transition between points is smoother, which is conducive to the smooth operation of the test operation components.

[0104] 206. Control the test operation component to simulate the user performing a test operation on the application to be tested according to the test coordinate data, and obtain an application test result of the application to be tested.

[0105] In an embodiment of the present application, after obtaining the test coordinate data required for the actual operation of the test operation component, the application testing system will establish data transmission with the test operation component, and send the test coordinate data to the test operation component through a preset transmission protocol, so that the test operation component can simulate the user's operation on the test application according to the test coordinate data.

[0106] At the same time, during the process of simulating the operation of the test operation component, the application test system will continuously obtain the operation response results of the application to be tested, and generate application test results based on the obtained operation response results. Specifically, during the test process, the running results of the APP will be asserted to determine whether the APP is successfully processed. Assertion processing is a key link in determining whether the application behavior meets expectations. The assertion rules may include checking whether the expected elements appear on the interface, whether the attributes of the elements meet expectations (such as checking whether the text content, color, size, etc. are consistent with expectations), verifying whether the data processed by the application is consistent with the input data or expected results, etc. The application test system will generate corresponding application test results based on the results of the assertion processing.

[0107] In the process of actual application, in order to ensure the stability and reliability of the APP, a long test run is required to comprehensively evaluate its performance. Therefore, in an embodiment of the present application, the stability index data of the APP is obtained through a long test run. Specifically, the application testing system will control the test operation component to continuously simulate the user's operation according to the test coordinate data, and count the simulation time until the simulation time reaches the time threshold, and obtain multiple application test results to ensure that the test runs long enough to fully expose the potential problems of the application to be tested. The time threshold can be set according to the complexity of the application to be tested and the test requirements, which may be a few hours, a few days or even longer. After the simulation time reaches the time threshold, the application testing system will stop the simulation operation and collect all application test results generated during this period. The application testing system will collect statistics on multiple application test results, and count the ratio of the number of operation response results indicating failure in multiple application test results to the total number of operation response results. The ratio can intuitively reflect the stability performance of the application to be tested during long-term operation. In addition to counting the failure ratio, the application testing system will also analyze the application performance data in multiple application test results to obtain the stability index data of the application to be tested, and output the stability index data. The analysis content includes but is not limited to response time, resource utilization (such as CPU, memory), frame rate stability, etc., so as to evaluate the performance and stability of the application to be tested in different scenarios.

[0108] It should be noted that after completing the above statistics and analysis, the application testing system can output the stability index data as a test report or a visual chart to more intuitively understand the relationship and trend between the data. Specifically, it can generate a failed operation ratio chart, a performance trend chart, a crash rate and a non-response rate chart, etc. The failed operation ratio chart is used to show the proportion of failed operations to all operations, which can intuitively show the stability performance of the APP in different test stages; the performance trend chart is used to show the changing trend of the performance data of the APP during long-term operation, such as the growth of response time over time, the fluctuation of resource utilization, etc.; the crash rate and non-response rate chart uses bar charts, line charts, etc. to show the crash rate and non-response rate of the APP in different test scenarios, helping the development team to quickly locate the problem area, which not only provides the development team with intuitive feedback on the stability of the APP, but also can serve as an important evaluation basis before the product is released.

[0109] In summary, the technical solutions of the embodiments of the present application are summarized as follows:

[0110] like Figure 2FAs shown, the application to be tested is processed with embedded points, and the accumulated embedded point data is obtained. The embedded point data is analyzed and the pixel coordinate group is extracted through LLM. The pixel coordinate group is converted to obtain the plane coordinate group, and then the plane coordinate group is interpolated. The robot arm is controlled to operate according to the interpolated plane coordinate group to simulate the user operation. Finally, by asserting the running result of the application to be tested, it is determined whether the application to be tested is successfully processed, and the above process is repeated. After long-term operation, the stability index data of the APP can be obtained.

[0111] The method provided in the embodiment of the present application determines the application to be tested, performs tracking processing on the application to be tested, and obtains the tracking data generated when the user performs operations on the application to be tested, parses and processes the tracking data based on the large language model, generates test coordinate data, and controls the test operation component to simulate the user performing test operations on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested. Through the coordinated work of tracking technology, large language model analysis and automated test operation components, a comprehensive, efficient and intelligent test of the application to be tested is achieved. During the test process, not only the functions and performance at the application level are paid attention to, but also the user operations are truly simulated through the test operation components, so that the test goes deep into the underlying hardware level, which significantly improves the effectiveness of the test and the accuracy of the test results.

[0112] Further, as Figure 1 In the specific implementation of the method, the present application embodiment provides an application testing device, such as Figure 3 As shown, the device includes: an acquisition module 301, a parsing module 302 and a simulation module 303.

[0113] The acquisition module 301 is used to determine the application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when the user performs operations on the application to be tested;

[0114] The parsing module 302 is used to parse and process the buried point data based on the large language model to generate test coordinate data;

[0115] The simulation module 303 is used to control the test operation component to simulate the user performing a test operation on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested.

[0116] In a specific application scenario, the acquisition module 301 is used to respond to an application test request, take the application indicated by the application test request as the application to be tested, and obtain the tracking point parameters carried by the application test request, where the tracking point parameters are used to indicate the location and type of the tracking point to be tested; according to the tracking point parameters, set data collection points in the application to be tested, complete the tracking point processing of the application to be tested, and deploy the application to be tested after the tracking point processing so that the user can use the application to be tested after the tracking point processing; periodically collect and obtain the tracking point data accumulated during the operation of the application to be tested, where the tracking point data is generated after the data collection point is triggered when the user uses the application to be tested.

[0117] In a specific application scenario, the parsing module 302 is used to input the buried point data into the large language model, parse the buried point data based on the large language model to obtain multiple high-frequency pixel coordinate groups; use the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups; interpolate the multiple plane coordinate groups through the large language model, and use the multiple plane coordinate groups after interpolation as the test coordinate data.

[0118] In a specific application scenario, the parsing module 302 is used to determine the field name used to indicate coordinate data in the buried point data and obtain a preset frequency condition; construct a coordinate output prompt word with reference to the field name and the frequency condition; and use the coordinate output prompt word to guide the large language model to parse the buried point data so that the large language model outputs the multiple high-frequency pixel coordinate groups.

[0119] In a specific application scenario, the parsing module 302 is used to query preset device parameters and coordinate units, where the device parameters include the device screen size and the device screen resolution; construct coordinate conversion prompts with reference to the device parameters and the coordinate units; use the coordinate conversion prompts to guide the large language model to perform coordinate conversion on each of the multiple high-frequency pixel coordinate groups, and the large language model outputs the multiple plane coordinate groups.

[0120] In a specific application scenario, the simulation module 303 is used to establish data transmission with the test operation component, and send the test coordinate data to the test operation component through a preset transmission protocol, so that the test operation component simulates the user's operation on the application to be tested according to the test coordinate data; during the process of the test operation component simulating the operation, the operation response result of the application to be tested is continuously obtained, and the application test result is generated based on the obtained operation response result.

[0121] In a specific application scenario, the device also includes:

[0122] The simulation module 303 is further used to control the test operation component to continuously simulate the user's operation according to the test coordinate data, and to count the simulation time until the simulation time reaches a time threshold, thereby obtaining multiple application test results;

[0123] The output module is used to count the proportion of the number of operation response results indicating failure in the multiple application test results to the total number of operation response results, and to analyze the application performance data in the multiple application test results to obtain stability index data of the application to be tested, and output the stability index data.

[0124] The device provided in the embodiment of the present application determines the application to be tested, performs tracking processing on the application to be tested, and obtains the tracking data generated when the user performs operations on the application to be tested, parses and processes the tracking data based on the large language model, generates test coordinate data, and controls the test operation component to simulate the user's execution of test operations on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested. Through the coordinated work of tracking technology, large language model analysis and automated test operation components, a comprehensive, efficient and intelligent test of the application to be tested is achieved. During the test process, not only the functions and performance at the application level are paid attention to, but also the user operations are truly simulated through the test operation components, so that the test goes deep into the underlying hardware level, which significantly improves the effectiveness of the test and the accuracy of the test results.

[0125] It should be noted that for other corresponding descriptions of the functional units involved in an application test device provided in an embodiment of the present application, reference can be made to Figure 1 and FIG. 2A to FIG. 2F The corresponding description in will not be repeated here.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0127] The above embodiments and the technical features in the embodiments can be arbitrarily combined with each other. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0128] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0129] In an exemplary embodiment, see Figure 4 , and also provides a computer device, which includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device, wherein each functional unit can communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the application test method in the above embodiment.

[0130] A computer-readable storage medium stores a computer program, which implements the steps of the application testing method when executed by a processor.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0132] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0133] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0134] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0135] The above disclosure only discloses several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by technicians in this field should fall within the scope of protection of the present application.

Claims

1. An application testing method, characterized in that: include: Determine an application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when a user performs an operation on the application to be tested; Parsing and processing the buried point data based on the large language model to generate test coordinate data; The control test operation component simulates a user performing a test operation on the application to be tested according to the test coordinate data, and obtains an application test result of the application to be tested.

2. The method according to claim 1, characterized in that The determining of the application to be tested, performing tracking processing on the application to be tested, and obtaining tracking data generated when the user performs an operation on the application to be tested includes: In response to an application test request, the application indicated by the application test request is used as the application to be tested, and the tracking point parameters carried by the application test request are obtained, where the tracking point parameters are used to indicate the location and type of the tracking point to be tested; According to the tracking point parameters, a data collection point is set in the application to be tested, tracking point processing of the application to be tested is completed, and the application to be tested after the tracking point processing is deployed so that the user can use the application to be tested after the tracking point processing; The buried point data accumulated during the operation of the application to be tested is periodically collected and acquired, and the buried point data is generated after the data collection point is triggered when the user uses the application to be tested.

3. The method according to claim 1, characterized in that The step of parsing and processing the buried point data based on the large language model to generate test coordinate data includes: Inputting the buried point data into the large language model, parsing the buried point data based on the large language model to obtain a plurality of high-frequency pixel coordinate groups; Using the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups; The large language model is used to perform interpolation processing on the plurality of plane coordinate groups, and the interpolated plurality of plane coordinate groups are used as the test coordinate data.

4. The method according to claim 3, characterized in that The step of parsing the buried point data based on the large language model to obtain a plurality of high-frequency pixel coordinate groups includes: Determine a field name for indicating coordinate data in the buried point data, and obtain a preset frequency condition; Constructing coordinate output prompt words with reference to the field name and the frequency condition; The coordinate output prompt word is used to guide the large language model to parse the buried point data, so that the large language model outputs the multiple high-frequency pixel coordinate groups.

5. The method according to claim 3, characterized in that: The using the large language model to perform coordinate transformation on the multiple high-frequency pixel coordinate groups to obtain multiple plane coordinate groups includes: Query preset device parameters and coordinate units, wherein the device parameters include device screen size and device screen resolution; Constructing coordinate conversion prompt words with reference to the device parameters and the coordinate units; The coordinate transformation prompt word is used to guide the large language model to perform coordinate transformation on each of the plurality of high-frequency pixel coordinate groups, and the large language model outputs the plurality of plane coordinate groups.

6. The method according to claim 1, characterized in that The control test operation component simulates a user performing a test operation on the application to be tested according to the test coordinate data to obtain an application test result of the application to be tested, including: Establishing data transmission with the test operation component, and sending the test coordinate data to the test operation component through a preset transmission protocol, so that the test operation component simulates the user's operation on the application to be tested according to the test coordinate data; During the process of the test operation component simulating the operation, the operation response result of the application to be tested is continuously obtained, and the application test result is generated based on the obtained operation response result.

7. The method according to claim 1, characterized in that The method further comprises: Controlling the test operation component to continuously simulate the user's operation according to the test coordinate data, and counting the simulation time until the simulation time reaches a time threshold, thereby obtaining a plurality of application test results; Count the ratio of the number of operation response results indicating failure in the multiple application test results to the total number of operation response results, analyze the application performance data in the multiple application test results, obtain stability index data of the application to be tested, and output the stability index data.

8. An application testing device, characterized in that: include: An acquisition module is used to determine an application to be tested, perform tracking processing on the application to be tested, and obtain tracking data generated when a user performs an operation on the application to be tested; A parsing module, used to parse and process the buried point data based on a large language model to generate test coordinate data; The simulation module is used to control the test operation component to simulate the user performing a test operation on the application to be tested according to the test coordinate data, so as to obtain the application test result of the application to be tested.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images