Interaction system reliability test method and related device

Abstract

The invention discloses an interactive system reliability test method and a related device, and relates to the field of software testing, and the method comprises the following steps: obtaining burying point data; extracting target parameters in the burying point data; constructing a test script based on the target parameter; and testing the to-be-tested system based on the constructed test script. According to the scheme, the test script is constructed by adopting the buried point data, the to-be-tested system is tested by adopting the test script, and the buried point data is used for representing manual actual operation, so that the test scheme can be equivalent to manual test, the test process can better fit the manual actual operation, and the test efficiency is improved. According to the method, the probability of finding abnormity in the testing process can be remarkably improved, and moreover, due to the fact that the burying point data is the data actually operated by the user, the common functions of the user of the to-be-tested system can be meticulously tested by adopting the testing scheme, so that the common functions of the user have higher reliability.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to an interactive system reliability testing method and related devices. Background Art

[0002] In order to meet user needs, the system's functions need to be gradually expanded during the maintenance process. In order to ensure the reliability of the expanded system, test cases need to be used to test the expanded system.

[0003] In existing solutions, an automated testing system is usually used to test the system. During the automated testing process, the scope of the test cases is fixed, resulting in much fewer bugs found in automated testing tasks than in manual testing, and both the scenario coverage and verification points are highly dependent on the personal ability of automated testing. For example, after the applicant's experiment, the results of the most recent normalized test of System A were: the test task executed 85.5% of the automated cases in all the cases, and only 6.25% of the total number of bugs were found, with a relatively low input-output ratio, and the remaining bugs were all found by manual testing. This shows that the test results of automatically automating the test cases are not ideal. Summary of the invention

[0004] In view of the above problems, the present application provides an interactive system reliability testing method and related devices to achieve the purpose of efficient automated testing. The specific scheme is as follows:

[0005] A reliability testing method for an interactive system, comprising:

[0006] Get buried point data;

[0007] Extracting target parameters from the buried point data;

[0008] Building a test script based on the target parameters;

[0009] The system to be tested is tested based on the constructed test scripts.

[0010] Optionally, in the above interactive system reliability testing method, extracting target parameters from the tracking data includes:

[0011] Extracting target buried point data from the buried point data, wherein the target buried point data includes buried point data with an occurrence frequency higher than a preset value and / or buried point data with abnormalities in a system response process;

[0012] Target parameters are extracted from the target buried point data.

[0013] Optionally, in the above interactive system reliability test method, obtaining tracking data includes:

[0014] Obtain the tracking data corresponding to the target test range from the tracking database.

[0015] Optionally, in the above interactive system reliability test method, obtaining the tracking data corresponding to the target test range from the tracking database includes:

[0016] Obtain from the tracking point database the tracking point data with a frequency greater than a preset value within the target test range and / or the tracking point data in which abnormalities have occurred during the response of the system to be tested.

[0017] Optionally, in the above interactive system reliability testing method, the step of obtaining a test script that matches the target parameter includes:

[0018] Obtaining an automated test script fragment that matches the target parameter;

[0019] The automated test script fragments are combined to obtain a test script that matches the target parameters.

[0020] Optionally, in the above interactive system reliability testing method, obtaining an automated test script fragment matching the target parameter includes:

[0021] Determine an automated test script fragment that matches the target parameter based on a preset mapping table, wherein the preset mapping table stores a mapping relationship between the target parameter and the automated test script fragment;

[0022] Alternatively, a large model is used to determine an automated test script segment that matches the target parameter.

[0023] Optionally, in the above interactive system reliability testing method, testing the system to be tested based on the constructed test script includes:

[0024] Determine a priority that matches the data type of the acquired buried point data, wherein the data type includes but is not limited to abnormal buried point data and high-frequency buried point data;

[0025] Based on the priorities corresponding to the various buried point data, the constructed test script is used to test the system to be tested in a descending order of priority.

[0026] Optionally, in the above interactive system reliability test method, determining the priority that matches the data type of the acquired buried data includes:

[0027] When the acquired buried point data is buried point data in which an abnormality occurs during the response process of the interactive system, the priority of the buried point data is the first priority;

[0028] When the acquired burying point data is burying point data with a current frequency higher than a preset value, the priority of the burying point data is the second priority;

[0029] The priority level of the first priority is higher than that of the second priority, and, among multiple buried point data belonging to the same priority level, the priority level of buried point data with a high occurrence frequency is higher than the priority level of buried point data with a low occurrence frequency.

[0030] Optionally, in the above interactive system reliability testing method, the target parameters include:

[0031] A combination of one or more of the purpose, preconditions and input data of each burying point event in the burying point data.

[0032] Optionally, in the above interactive system reliability testing method, testing the system to be tested based on the constructed test script includes:

[0033] Acquire automated testing resources that match the target parameters;

[0034] The system to be tested is tested based on the test script constructed using automated testing resources.

[0035] An interactive system reliability testing device, comprising:

[0036] A buried point data collection unit, used to obtain buried point data;

[0037] A target parameter extraction unit, used to extract the target parameters from the buried point data;

[0038] A test script construction unit, used for constructing a test script based on the target parameters;

[0039] The test unit is used to test the system to be tested based on the constructed test scripts.

[0040] An electronic device, characterized in that it comprises at least one processor and a memory connected to the processor, wherein:

[0041] The memory is used to store computer programs;

[0042] The processor is used to execute the computer program so that the electronic device can implement any one of the interactive system reliability testing methods described above.

[0043] By means of the above-mentioned technical scheme, the above-mentioned scheme provided by the present application, constructs a test script by adopting the buried data, and then uses the test script to test the system to be tested. Since the buried data is used to characterize actual manual operations, this test scheme can be equivalent to manual testing. As a result, the test process can be made more in line with actual manual operations, and the probability of discovering anomalies during the test process can be significantly improved. Moreover, since the buried data is the data of real user operations, this test scheme can be used to conduct detailed tests on the user's commonly used functions of the system to be tested, so that the user's commonly used functions have higher reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other features, advantages and aspects of the embodiments disclosed in the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0045] Figure 1 A schematic diagram of an implementation system architecture of an interactive system reliability testing method provided in this application;

[0046] Figure 2 A schematic diagram of a terminal structure provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a server structure provided in an embodiment of the present application;

[0048] Figure 4 A flowchart of an interactive system reliability testing method provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of a buried point data provided for this application;

[0050] Figure 6 A schematic diagram of a test process for completing a test when testing a system to be tested according to an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the structure of an interactive system reliability testing device provided in an embodiment of the present application;

[0052] Figure 8 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0054] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0055] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0056] This application can be applied in the field of software testing. Taking the interactive system reliability testing solution as an example, multiple application scenarios implemented in products will be introduced below.

[0057] First, the application scenario of this application is introduced.

[0058] This application can be applied to, but is not limited to, applications with system detection functions or cloud services provided by cloud-side servers, etc., which are introduced below:

[0059] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0060] Among them, an interactive system reliability test application can be installed on the terminal 100. The above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the interactive interface and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters and return the processing results to the terminal 100.

[0061] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0062] Next describe Figure 1The product form of the mid-terminal 100;

[0063] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0064] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0065] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0066] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0067] Among them, the input device 132 can receive input data and the like.

[0068] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 can be used to display the interface of the configuration data, the processing results, etc.

[0069] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0070] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0071] Among them, the memory 120 can be used to store software codes related to the interactive system reliability testing method, the processor 170 can execute the steps of the interactive system reliability testing method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to implement corresponding functions.

[0072] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0073] In this embodiment of the present application, the RF unit 110 can send data to the server 200 and receive processing results sent by the server 200.

[0074] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0075] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0076] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0077] Although not shown, the terminal 100 may also include a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0078] Next describe Figure 1 The product form of the server 200;

[0079] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0080] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0081] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0082] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard disk (HDD), or a solid state drive (SSD).

[0083] The memory 204 may be used to store software codes related to the interactive system reliability testing method, and the processor 202 may execute the steps of the interactive system reliability testing method for a chip, and may also schedule other units to implement corresponding functions.

[0084] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0085] In order to solve the above problems pointed out in the background technology, the embodiment of the present application provides an interactive system reliability testing method that can accurately identify system problems. The interactive system reliability testing method of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0086] Reference Figure 4 , Figure 4 The following is a flow chart of an interactive system reliability testing method provided in an embodiment of the present application. Figure 4 As shown, an interactive system reliability testing method provided in an embodiment of the present application may include steps S401 to S404, and these steps are described in detail below.

[0087] S401, obtaining buried point data;

[0088] Tracking, also known as event tracking, is a commonly used data collection method. It refers to attaching the data collection program code to the functional program code at the "operation node" where data needs to be collected, and capturing, processing and uploading user behaviors or events on the operation node. Tracking can include front-end tracking and back-end tracking. Front-end tracking: By implanting code in the front-end page or button, it monitors the user's clicks, browsing, sliding and other behaviors, and sends the data to the server in real time. This method is mainly suitable for the collection of behavioral data of direct interaction between users and the interface. Back-end tracking: Record user operation behaviors or system events on the server side, such as API calls, database operations, etc. Back-end tracking can make up for some back-end processing logic data that cannot be obtained by front-end tracking.

[0089] Event Tracking Data or User Behavior Data refers to user behavior information captured and recorded through pre-set data collection points (commonly referred to as "tracking points") in applications, websites or software systems. These data points are carefully designed to monitor and track user operations, interactions or events in specific scenarios, including but not limited to clicks, browsing, purchases, searches, swipes, form filling, video watching and other events.

[0090] In the present application, a tracking point can be set in advance at a specific location in the target system, and user behaviors or events can be captured through the tracking point. The captured user behaviors or events can be pre-stored in a tracking point database. When the tracking point data needs to be obtained, the tracking point data can be directly obtained from the tracking point database.

[0091] In the technical solution disclosed in this embodiment, the buried point data can be classified and stored in the buried point database based on the type, usage scenario, frequency, whether it is an abnormal response, etc. of the captured buried point data, so that the buried point data can be quickly captured when used subsequently.

[0092] S402, extracting target parameters from the buried point data;

[0093] The acquired buried point data can be understood as an event. There will be multiple target parameters in an event. These target parameters are key data for constructing test cases. Therefore, after acquiring the buried point data, the buried point data is parsed to obtain the target parameters corresponding to each buried point data. The parsing and extraction process of the target parameters can be performed before the acquired buried point data is stored in the buried point database. After the extraction is completed, the buried point data and its corresponding target parameters can be stored together in the buried point database. The extraction process of the target parameters can also be completed in the buried point database. Of course, it can also be obtained by parsing the buried point data after the buried point data is extracted in this step.

[0094] Therefore, in this step, the target parameters of the buried point data extracted may refer to target parameters matching the buried point data extracted from the buried point database, or may refer to target parameters corresponding to the buried point data obtained by parsing the acquired buried point data.

[0095] like Figure 5 As shown, Figure 5This is a structural diagram of a tracking data provided by this application; the tracking data is tracking data that depends on system A, where vehvoice_002_0031 can be understood as a tracking event in the tracking data. Multiple target parameters are stored in a tracking event. Trackid and tracking events are associated with each other through Record id and Trackid. Taking the user Query (user inquiry, expressing the user's instructions to the smart voice assistant) "Open the map" as an example, the target parameters of each tracking event are associated and obtained, and the following information can be obtained:

[0096]

[0097] According to the above target parameters, it can be concluded that: in the same dialogue interaction (the same Track ID), the user only inputs one command. The test content of the test case built based on the target parameter is: when the central control screen is on the desktop, the main driver wakes up the assistant tool with the main wake-up word, and the control assistant tool "opens the map".

[0098] S403, constructing a test script based on the target parameters;

[0099] Each point-of-care data obtained can be used as a test case, and there is a clear correspondence between the target parameters in the test case and the test script fragment. During the software testing process, the test case describes in detail how to test the software application, including the purpose, preconditions, test steps, input data, and expected results of the test, etc., wherein the purpose, preconditions, and input data can all be extracted from the point-of-care data as target parameters, the test steps can be determined by the test script, and the expected results can be determined based on the point-of-care events. The test script is a code fragment used to automate these test steps. It performs specific test actions according to the description of the test case and verifies whether the test results meet expectations. By maintaining the correspondence between the target parameters in the test case and the test script fragment, the test team can ensure the accuracy and consistency of the test process, thereby improving the effectiveness and reliability of the test. Thus, after determining the target parameters, a test script constructed with the target parameters can be obtained.

[0100] Each test script is composed of multiple automated test script fragments. An automated script fragment refers to a definable part of a test script. For example, a test script is to wake up the voice on the desktop, say "Today's weather", and verify that the weather card is displayed normally. The execution action of returning to the desktop can be defined as an automated test script fragment.

[0101] In the technical solution disclosed in this embodiment, a mapping relationship between target parameters and automated test script fragments can be established in advance, and the relationship can be stored in a preset mapping table. After the target parameters corresponding to the buried data are determined, the automated test script fragments that match the target parameters can be obtained based on the pre-established mapping relationship, and then the test script can be constructed based on the automated test script fragments. Of course, a large model based on natural language processing technology can also be used to determine the automated test script fragments that match the target parameters, and then the various automated test script fragments are spliced ​​according to the rule definition to form a complete test script. By using natural language processing technology, it is possible to parse the user's instructions and preconditions in the buried data, select the corresponding automated test script fragments through matching logic, and convert them into complete automated scripts.

[0102] In this embodiment, the existing capabilities of the automation platform can also be reasonably utilized to match appropriate automated test resources through the target parameters in the buried data to ensure that the dependencies to be tested can be successfully executed on the corresponding resources. The so-called dependencies here refer to the scene resources executed by the test script, for example: whether there is a rear screen, whether there are three rows of seats, whether there is a sliding door, whether there is a refrigerator, etc., all of which are dependent on the resources of the automation bench equipment. If the wrong ones are sent, the execution results will not match the actual scene of the user.

[0103] S404: Test the system to be tested based on the constructed test script.

[0104] After the test script is constructed, it is loaded on the test platform, and the test platform can automatically execute the test script to test the system to be tested.

[0105] Rationally utilize the existing capabilities of the automation platform, match appropriate automated testing resources through keywords in user buried data, ensure that the dependencies that need to be tested can be successfully executed on the corresponding resources, and automatically generate a test report after the test is completed to facilitate manual confirmation of the test results.

[0106] It can be seen from the above scheme that the interactive system reliability testing method disclosed in the present application, by using embedded data to construct a test script, and then using the test script to test the system to be tested, because the embedded data is used to characterize actual manual operations, this test scheme can be equivalent to manual testing, thereby making the test process more in line with actual manual operations, and can significantly improve the probability of discovering anomalies during the test process. Moreover, because the embedded data is the data of real user operations, this test scheme can be used to conduct detailed tests on the user's commonly used functions of the system to be tested, so that the user's commonly used functions have higher reliability.

[0107] In the technical solution disclosed in this embodiment, when testing the system to be tested, the functions commonly used by users and the functions that have failed during user use can be tested in particular. Therefore, in this embodiment, when extracting the target parameters in the buried point data, the buried point data can be screened. In this embodiment, the buried point data with a frequency higher than a preset value and the buried point data with abnormalities in the system response process can be used as the target buried point data to be acquired in this application, that is, acquiring the buried point data includes: extracting the target buried point data in the buried point data, the target buried point data includes the buried point data with a frequency higher than a preset value and / or the buried point data with abnormalities in the system response process. The specific value of the preset value can be set according to user needs.

[0108] After all the test scripts are constructed based on the acquired buried data, the automated testing platform needs to perform automated testing on the system to be tested based on the test scripts. During the automated testing process, the test scripts can be sorted based on the preset test priority, and the system to be tested is tested based on the sorted test scripts. Among them, the priority of the test script corresponding to the buried data that appears abnormally during the system response process is marked as the first priority, and the priority of the test script corresponding to the buried data with a frequency higher than the preset value is marked as the second priority. The priority level of the first priority is higher than the second priority, and in the same priority category, the higher the frequency of buried data, the higher the priority corresponding to the buried data. That is, based on the order of occurrence frequency from high to low, the test scripts of the first priority are given priority for regression testing. After all the test scripts of the first priority are tested, the test scripts of the second priority are given priority for regression testing based on the order of occurrence frequency from high to low, until all the test scripts of the first priority are tested.

[0109] In the above embodiment, the technology of Web-side user traffic playback is transferred to the automated testing process, the buried data of online user data is used to determine the buried data corresponding to user high-frequency scenarios and online abnormal scenarios, and a testing method is used to prioritize these buried data, making automated testing more efficient. Relying on the accumulation of existing massive script fragments, the implementation of end-to-end UI automated testing solutions can be supported.

[0110] In this embodiment, when a test script is used to perform regression testing on the test system, the standard DM feedback result corresponding to the test script in the user data can be used as the expected result and compared with the response result of the system to be tested during the regression test. The DM feedback result generated by the system to be tested in response to the test script during the regression test is compared with the expected result to determine whether the two are consistent. If they are inconsistent, an error is reported in the test result corresponding to the test script and the manual verification stage is entered.

[0111] When the interactive system reliability testing method disclosed in the embodiment of the present application is used to test the system to be tested, see Figure 6 , the complete process may include:

[0112] Step S601: Perform data training and pre-test preparation by using embedded point data, fragments of automated test scripts, test configurations, and previous test data as basic data;

[0113] The so-called test configuration refers to the target test scope, test version, test times, etc. that need to be selected before testing. The target test scope can refer to the functional scope and frequency range of the required tests, such as only testing high-frequency user scenarios, how many times to rerun after each test failure, which OTA version of user data to capture, and which version to perform regression testing on, etc.;

[0114] Step S602: manually confirm and fill in the test scope, test environment and test resources when initiating the test;

[0115] The test scope refers to the need to determine which version of user data to capture as the basis. The test environment and test resources mainly refer to the test bench environment that the script depends on. For example, a user scenario is to perform the operation of opening system A on the back screen. This script relies on the test bench with a back screen to execute, and there will be a corresponding precode correspondence to support it.

[0116] Step S603: Process the accumulated tracking data according to the test range defined by the user, distinguish the high-frequency data and abnormal data therein, and match the corresponding automation script fragments through natural language processing to generate a completed automation script;

[0117] In this step, after the target test scope, test environment and test resources are determined, the matching buried data are determined based on the target test scope, test environment and test resources. Among these buried data, the high-frequency buried data that meets the frequency requirements and the abnormal buried data (the buried data in which abnormalities have occurred during the system response process) are selected, and a large model is used to determine the test scripts that match these buried data based on natural language processing.

[0118] Step S604: Create a test task on the platform, generate a test task name based on the manually filled information, create subtasks based on the business scope, and fill in the test case content based on the test points of each script;

[0119] In this step, after the test script is generated, the user creates a test task on the test platform, manually fills in the test task name, creates subtasks according to the business scope, and fills in the test case content according to the test points of each test script. This process can refer to the general testing process.

[0120] Step S605: The platform sends the automated test tasks to each automated test bench for execution according to the selection of test resources, and displays the test progress on the platform in real time;

[0121] After the test task is created, the test platform automatically identifies the test bench corresponding to each test task, in which the system to be tested is loaded, and displays the test progress on the test platform in real time.

[0122] Step S606: After the test is completed, an execution report of a single use case and a test report of the entire task are generated according to the test execution results, and the manual confirmation phase is entered to complete the manual confirmation of the entire test task archive, and the flow is completed. It is used as data accumulation for the next test to optimize the algorithm capabilities of natural language processing and the optimization items of automated script fragments.

[0123] An interactive system reliability testing method provided by an embodiment of the present application is introduced above, and a device for executing the above interactive system reliability testing method will be introduced below.

[0124] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an interactive system reliability testing device provided in an embodiment of the present application. Figure 7 As shown, the interactive system reliability testing device comprises:

[0125] The buried point data acquisition unit 10 corresponds to step S401 in the above method and is used to obtain buried point data;

[0126] A target parameter extraction unit 20, which corresponds to step S402 in the above method, and is used to extract the target parameters in the buried point data;

[0127] A test script construction unit 30, which corresponds to step S403 in the above method, and is used to construct a test script based on the target parameters;

[0128] The testing unit 40 , which corresponds to step S404 in the above method, is used to test the system to be tested based on the constructed test script.

[0129] The present application also provides an electronic device in an embodiment. Figure 8 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0130] like Figure 8 As shown, the electronic device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0131] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0132] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any interactive system reliability testing method provided in the embodiment of the present application.

[0133] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any interactive system reliability testing method provided in an embodiment of the present application.

[0134] The tracking data involved in this application (including but not limited to user device information, user personal information, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0135] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0136] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0137] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0138] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A reliability testing method for an interactive system, characterized in that: include: Get buried point data; Extracting target parameters from the buried point data; Building a test script based on the target parameters; The system to be tested is tested based on the constructed test scripts.

2. The interactive system reliability testing method according to claim 1, characterized in that: The target parameters extracted from the tracking data include: Extracting target buried point data from the buried point data, wherein the target buried point data includes buried point data with an occurrence frequency higher than a preset value and / or buried point data with abnormalities in a system response process; Target parameters are extracted from the target buried point data.

3. The interactive system reliability testing method according to claim 1, characterized in that: Obtain tracking data, including: Obtain the tracking data corresponding to the target test range from the tracking database.

4. The interactive system reliability testing method according to claim 3, characterized in that: Obtain the tracking data corresponding to the target test range from the tracking database, including: Obtain from the tracking point database the tracking point data with a frequency greater than a preset value within the target test range and / or the tracking point data in which abnormalities have occurred during the response of the system to be tested.

5. The interactive system reliability testing method according to claim 4, characterized in that: The obtaining of the test script matching the target parameter comprises: Obtaining an automated test script fragment that matches the target parameter; The automated test script fragments are combined to obtain a test script that matches the target parameters.

6. The interactive system reliability testing method according to claim 5, characterized in that: Obtain an automated test script fragment that matches the target parameter, including: Determine an automated test script fragment that matches the target parameter based on a preset mapping table, wherein the preset mapping table stores a mapping relationship between the target parameter and the automated test script fragment; Alternatively, a large model is used to determine an automated test script fragment that matches the target parameter.

7. The interactive system reliability testing method according to claim 1, characterized in that: Test the system to be tested based on the constructed test scripts, including: Determine a priority that matches the data type of the acquired buried point data, wherein the data type includes but is not limited to abnormal buried point data and high-frequency buried point data; Based on the priorities corresponding to the various buried point data, the constructed test script is used to test the system to be tested in a descending order of priority.

8. The interactive system reliability testing method according to claim 7, characterized in that: The priority of determining the data type that matches the acquired buried point data includes: When the acquired buried point data is buried point data in which an abnormality occurs during the response process of the interactive system, the priority of the buried point data is the first priority; When the acquired burying point data is burying point data with a current frequency higher than a preset value, the priority of the burying point data is the second priority; The priority level of the first priority is higher than that of the second priority, and, among multiple buried point data belonging to the same priority level, the priority level of buried point data with a high occurrence frequency is higher than the priority level of buried point data with a low occurrence frequency.

9. The interactive system reliability testing method according to claim 1, characterized in that: The target parameters include: A combination of one or more of the purpose, preconditions and input data of each burying point event in the burying point data.

10. The interactive system reliability testing method according to claim 1, characterized in that: Testing the system to be tested based on the constructed test scripts includes: Acquire automated testing resources that match the target parameters; The system to be tested is tested based on the test script constructed using automated testing resources.

11. An interactive system reliability testing device, characterized in that: include: A buried point data collection unit, used to obtain buried point data; A target parameter extraction unit, used to extract the target parameters from the buried point data; A test script construction unit, used for constructing a test script based on the target parameters; The test unit is used to test the system to be tested based on the constructed test scripts.

12. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the interactive system reliability testing method as described in any one of claims 1 to 10.

Images