Test method and device, computing equipment, storage medium and program product
By simulating the failure scenario of the network request link in the test script, the problem of insufficient testing scenarios and low testing efficiency in the prior art is solved, and automated testing and efficient testing coverage are achieved.
Patent Information
- Application Number
- CN202510162075.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-13
- Publication Date
- 2025-05-13
Smart Images
Figure CN119996276A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of Internet technology, and specifically to a testing method, apparatus, computing device, storage medium, and program product. Background Art
[0002] The quality of network requests is the premise of all interactions of Internet products. Especially for some platforms with high-traffic business scenarios, the fast and reliable distribution of dynamic and static resources is the key to providing a high-quality user experience. Therefore, a large number of static resources are cached in edge nodes for users in different regions to access nearby; dynamic resources are transmitted from a large number of return lines based on intelligent routing technology. While enjoying global dynamic acceleration, the stability of DCDN becomes crucial. However, the industry's technical solutions lack effective testing methods for the complete network request link on the end. Summary of the invention
[0003] In view of the above problems, the present application is proposed to provide a testing method, apparatus, computing device, computer storage medium and computer program product that overcome the above problems or at least partially solve the above problems.
[0004] According to one aspect of an embodiment of the present application, a testing method is provided, the method comprising:
[0005] S1, sends network requests to each business domain name through the test script;
[0006] S2, intercepting the network request and forwarding the network request to a preset domain name resolution service;
[0007] S3, performing domain name resolution on the network request according to the fault scenario pre-injected in the preset domain name resolution service to obtain a domain name resolution IP;
[0008] S4, sending a service request to the corresponding service domain interface according to the domain name resolution IP;
[0009] S5: If the service request status is passed, the dynamic resources and static resources associated with the service are rendered, and a test result is generated according to the rendering result;
[0010] S6: If the service request status is not available, the domain name downgrade process is performed, the domain name resolution IP is obtained, and the process jumps to S4.
[0011] Furthermore, generating the test result according to the rendering result further includes: performing frame splitting processing on the client rendering process;
[0012] The client rendering process is divided into phase stable intervals according to the key frames, and the phase time consumption corresponding to each phase stable interval is recorded;
[0013] Generate test results based on rendering results, stage stability interval and stage duration.
[0014] Furthermore, generating the test result according to the rendering result, the stage stable interval and the stage time consumption further includes: if the rendering result is a rendering exception, taking the rendering exception performance information, the stage stable interval of the rendering exception and the corresponding stage time consumption as the test result.
[0015] Furthermore, generating the test result according to the rendering result, the stage stability interval and the stage time consumption further includes:
[0016] If the rendering result is normal, the stage time consumption corresponding to the stable interval of each stage is compared with the corresponding stage time consumption baseline standard;
[0017] If the stage time consumption corresponding to the stage stable interval is greater than the corresponding stage time consumption baseline standard, the test is determined to be abnormal, and the domain name resolution IP is first marked;
[0018] If the stage time corresponding to the stage stability interval is less than or equal to the corresponding stage time baseline standard, the test is determined to be normal and a second mark is performed on the domain name resolution IP.
[0019] Furthermore, the segmentation process of the client rendering process into stage stable intervals according to the key frames further includes:
[0020] Calculate the difference between key frames;
[0021] Comparing the difference with a preset difference threshold;
[0022] If the comparison result is that the difference is greater than or equal to the preset difference threshold, the segmentation process of the stage stable interval is executed.
[0023] Furthermore, the method further comprises: performing compensation processing on the difference according to a preset compensation threshold;
[0024] Comparing the difference with a preset difference threshold further includes: comparing the difference after compensation processing with the preset difference threshold.
[0025] Further, the method further comprises: recording a log file according to the client behavior information;
[0026] The log file is queried according to preset keywords to determine log information, and fault analysis is performed according to the log information.
[0027] Furthermore, intercepting the network request and forwarding the network request to a preset domain name resolution service further includes:
[0028] The configuration file is modified, and the network request is forwarded to a preset domain name resolution service according to the modified configuration file.
[0029] According to another aspect of an embodiment of the present application, a testing device is provided, the device comprising:
[0030] A first sending module, adapted to send a network request to each business domain name through a test script;
[0031] A forwarding module, adapted to intercept the network request and forward the network request to a preset domain name resolution service;
[0032] A domain name resolution module, adapted to perform domain name resolution on the network request according to the fault scenario pre-injected into the preset domain name resolution service to obtain a domain name resolution IP;
[0033] A second sending module, adapted to send a service request to a corresponding service domain interface according to the domain name resolution IP;
[0034] A generation module, adapted to render dynamic resources and static resources associated with the business if the business request status is passed, and generate test results according to the rendering results;
[0035] The domain name downgrade processing module is adapted to execute domain name downgrade processing if the service request status is unreachable, obtain the domain name resolution IP, and trigger the execution of the second sending module.
[0036] According to another aspect of the embodiment of the present application, there is provided a computing device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;
[0037] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation corresponding to the above-mentioned test method.
[0038] According to another aspect of the embodiments of the present application, a computer storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned test method.
[0039] According to another aspect of the embodiments of the present application, a computer program product is provided, comprising at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the above-mentioned test method.
[0040] According to the solution provided in the embodiment of the present application, through the method of automated testing, various rehearsal links are connected in series to improve efficiency, including the initiation of client network requests, the access to rehearsal scenarios, the automatic determination of abnormal scenarios such as white screens, unavailable network requests, and key behavior log analysis, etc., to fully simulate various situations that may exist in the client network request link, enrich the coverage of test scenarios, greatly improve the efficiency of rehearsal, and reduce the cost of human execution.
[0041] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation method of the embodiment of the present application is listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the embodiments of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0043] Figure 1 A schematic diagram of a testing method according to an embodiment of the present application is shown;
[0044] Figure 2 Flowchart of the process of dynamically accelerating client network requests;
[0045] Figure 3 A schematic flow chart of a testing method according to another embodiment of the present application is shown;
[0046] Figure 4 A schematic flow chart of a testing method according to another embodiment of the present application is shown;
[0047] Figure 5 This is a schematic diagram of the UI automation test script;
[0048] Figure 6 Schematic diagram of the script for frame splitting and stable interval classification;
[0049] Figure 7 Schematic diagram of the code for the parameters of the stability interval determination;
[0050] Figure 8 This is a schematic diagram of the stable interval of the client rendering process when a certain app is cold started and enters the homepage;
[0051] Fig. 9 It is a schematic diagram of the time consumption corresponding to the stable intervals of different stages;
[0052] Fig.10 This is a code diagram for automatic filtering based on logs;
[0053] Fig.11 A structural block diagram of a testing device according to an embodiment of the present application is shown;
[0054] Fig.12 A schematic diagram of the structure of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0055] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0056] First, the terms involved in one or more embodiments of the present application are explained.
[0057] DCDN: Globally intelligently distinguishes between dynamic and static resources for hybrid acceleration;
[0058] HTTPDNS: Domain name resolution service, which returns the nearest IP address based on the client's location and operator;
[0059] Dynamic resources: Users make multiple network requests, and the data returned in response is different, such as API interfaces, database interactions, etc.
[0060] Static resources: When users make multiple network requests, the data returned in response is the same, such as pictures, web page files, compressed packages, etc.
[0061] The inventor of the present application has found that the technical solutions in the industry lack effective testing methods for disaster recovery strategies such as complete network request links on the opposite end, dynamic and static resource acceleration effects, and timeout retries and domain name downgrades corresponding to each link. There are problems such as insufficient test scenarios, difficulty in simulating fault injection, interference with connection reuse, etc. At the same time, due to the complexity of the test scenarios, in the course of daily work, it is necessary to continuously invest manpower and repeatedly perform targeted disaster recovery drills. The test efficiency is extremely low and the labor cost is high. In order to effectively solve the above problems, the inventor of the present application has proposed a test solution through creative labor.
[0062] Figure 1 A flow chart of a testing method according to an embodiment of the present application is shown. Figure 1 As shown, the method comprises the following steps:
[0063] Step S101, sending a network request to each business domain name through a test script.
[0064] Specifically, a test script for automated testing that can simulate real user behavior is pre-built. Then, through the test script for automated testing, a network request is initiated to each business domain name, which can cover various devices such as Android and iOS. In order to improve the test efficiency, a network request can be initiated to each core domain name of each business. For example, there are the following domain names: domain name A, domain name B, and domain name C. A network request can be sent to the above domain names through a test script.
[0065] Step S102, intercepting the network request and forwarding the network request to a preset domain name resolution service.
[0066] In order to implement the testing of different fault scenarios, it is necessary to intercept the network request. For example, it can be intercepted by modifying the client configuration file, and then the network request is directed to the preset domain name resolution service. The preset domain name resolution service is the domain name resolution service built by the test platform, for example, it can be a self-built HTTPDNS. Among them, the client configuration file refers to the configuration file of the application to be tested. Of course, the same purpose can also be achieved by other means, such as DNS hijacking.
[0067] Step S103, performing domain name resolution on the network request according to the fault scenario pre-injected into the preset domain name resolution service to obtain a domain name resolution IP.
[0068] Specifically, various types of fault scenarios can be injected into the preset domain name resolution service in advance, and the fault scenarios injected into the preset domain name resolution service can be flexibly adjusted and customized according to actual needs, so as to better meet different testing requirements. For example, the types of fault scenarios may include: abnormal service status (4XX, 5XX, etc.), abnormal CDN node status (here, CDN single node status abnormality, CDN multi-node status abnormality), SSL certificate error, DNS timeout, TLS timeout, etc. Of course, this is just an example, and there may be other fault scenarios, which will not be listed here one by one.
[0069] In order to fully simulate various situations that may exist in the network request link, the network request can be resolved by domain name according to the fault scenario pre-injected in the preset domain name resolution service to obtain the domain name resolution IP. Here, the domain name and a set of IPs can be obtained. The IP address obtained through domain name resolution may be correct or it may be an incorrect IP address.
[0070] Step S104: Send a service request to the corresponding service domain interface according to the domain name resolution IP.
[0071] After obtaining the domain name resolution IP, you can initiate a corresponding business domain interface request based on the domain name resolution IP, that is, an actual application layer request initiated for a specific business logic or functional interface. Among them, the business domain interface refers to the API (Application Programming Interface) exposed by the application to the outside world.
[0072] Step S105: If the service request status is passed, the dynamic resources and static resources associated with the service are rendered, and a test result is generated according to the rendering result.
[0073] Specifically, if the business request is successful, the returned data needs to be rendered to present the effect expected by the user. Here, the dynamic resources and static resources associated with the business are rendered. Dynamic resources may be: API interfaces, database interactions, etc., and static resources may be: pictures, web page files, compressed packages, etc. After the rendering is completed, the test results can be generated based on the rendering results. The test results can include passing or failing the test. If the test is passed, it can also include information such as the IP address that passed the test; if the test is failed, it can also include various information such as the IP address that failed the test and the DNS provider.
[0074] Step S106: If the service request status is unavailable, the domain name downgrade process is performed to obtain the domain name resolution IP, and the process jumps to step S104.
[0075] Specifically, if the service request is not successful, the domain name needs to be downgraded, that is, the original domain name is replaced with an alternate domain name, and subsequent testing work is continued. It should be noted that the "alternative domain name" here does not refer to the alternate DNS (Domain Name System) server, but refers to different subdomains or aliases under the same domain name, which point to the same IP address.
[0076] By setting a preset domain name resolution service and pre-injecting fault scenarios into the preset domain name resolution service, the present application can simulate various possible network failure situations to facilitate automated testing, improve test efficiency, reduce test costs, and realize the testing of the complete network request link and the effective testing of degraded disaster recovery measurement.
[0077] Figure 2 Flowchart of the process of dynamically accelerating client network requests, such as Figure 2 As shown, the process includes: 1. The client initiates a network request through the basic network library SDK;
[0078] 2. Check whether the corresponding IP exists in the local DNS cache. If so, determine whether the IP is valid based on the TTL time in the cache.
[0079] 3. If it is invalid or does not exist, a network request is made to the platform or third-party HTTPDNS service, and the IP corresponding to the domain name resolution is returned;
[0080] 4. If the IP returned by the resolution is empty, downgrade to the operator's LocalDNS; if it is not empty, update the client's DNS local cache, use one of the resolved IPs, and initiate a request for the corresponding business domain interface;
[0081] 5. According to the service request status, if it is successful, the response result is returned; if it is not successful, the IP is resolved in rounds, the domain name is downgraded, and the service request is initiated again. According to the service request status, it is determined whether to trigger disaster recovery strategies such as domain name downgrade and timeout retry.
[0082] Figure 3 FIG. 1 is a flow chart of a testing method according to another embodiment of the present application. Figure 4 A flow chart of a testing method according to another embodiment of the present application is shown. Figure 3 and Figure 4 To illustrate, the method comprises the following steps:
[0083] Step S301: Send a network request to each business domain name through a test script.
[0084] Specifically, pre-build test scripts for automated testing that can simulate real user behavior, such as Figure 5 As shown, then, through the test script of the automated test, a network request is initiated to each business domain name, which can cover various devices such as Android and iOS. In order to improve the test efficiency, a network request can be initiated to each core domain name of the business. For example, there are the following domain names: domain name A, domain name B, and domain name C. A network request can be sent to the above domain names through a test script.
[0085] Step S302, modify the configuration file, and forward the network request to a preset domain name resolution service according to the modified configuration file.
[0086] For example, the business domain name that needs to be intercepted is recorded in the client's configuration file. When it is determined that a network request is initiated for the business domain name that needs to be intercepted, it can be intercepted. The client's configuration file is modified to configure the information of the preset domain name resolution service that needs to be resolved, so that the network request can be forwarded to the preset domain name resolution service according to the information of the preset domain name resolution service for domain name resolution.
[0087] Step S303, performing domain name resolution on the network request according to the fault scenario pre-injected into the preset domain name resolution service to obtain a domain name resolution IP.
[0088] Specifically, various types of fault scenarios can be injected into the preset domain name resolution service in advance, and the fault scenarios injected into the preset domain name resolution service can be flexibly adjusted and customized according to actual needs, so as to better meet different testing requirements. For example, the types of fault scenarios may include: abnormal service status (4XX, 5XX, etc.), abnormal CDN node status (here, CDN single node status abnormality, CDN multi-node status abnormality), SSL certificate error, DNS timeout, TLS timeout, etc. Of course, this is just an example, and there may be other fault scenarios, which will not be listed here one by one.
[0089] The preset domain name resolution service can be an HTTPDNS service built by the test platform or other third-party HTTPDNS services. The network request is resolved by domain name according to the fault scenario pre-injected in the preset domain name resolution service to obtain the domain name resolution IP. If the domain name resolution IP is empty, it can be downgraded to the operator's LocalDNS, which performs domain name resolution to obtain the corresponding domain name resolution IP and updates the client DNS local cache. If the domain name resolution IP is not empty, the client DNS local cache is updated.
[0090] Step S304: Send a service request to the corresponding service domain interface according to the domain name resolution IP.
[0091] Step S305: If the service request status is passed, the dynamic resources and static resources associated with the service are rendered.
[0092] Step S306, performing frame splitting processing on the client rendering process.
[0093] Specifically, the client rendering process can be automatically analyzed by frame splitting based on image processing and machine learning methods. The client here refers to the tested application. From the perspective of client presentation, the client rendering process is split into frames.
[0094] For example, when the rendering process involves the rendering of videos, animations, image rendering, interface data returns (article titles, emoticons, text content), advertising cards and other resources, frame splitting can be performed.
[0095] Step S307 , dividing the client rendering process into stage stable intervals according to the key frames, and recording the stage time consumption corresponding to each stage stable interval.
[0096] Specifically, after the frame decomposition process is completed, the peak signal-to-noise ratio (PSNR) and / or the structural similarity index (SSIM) between two adjacent frames are calculated, and the difference can be calculated based on the SSIM and / or PSNR. For example, the difference = 1-SSIM, which is just an example. Then, the key frames are filtered out based on the difference. For example, these two frames can be filtered out and marked as key frames.
[0097] After the key frames are screened out, the difference between the key frames is calculated. The specific calculation method is similar to the above and will not be repeated here. Then, the difference is compared with the preset difference threshold. If the comparison result is that the difference is greater than or equal to the preset difference threshold, the segmentation process of the stage stable interval is performed. That is, the timestamps of the two key frames are different, and the segmentation is performed before the key frame with a larger timestamp. The stage stable interval refers to the stage interval during the rendering process when the system state is relatively stable, resource consumption is relatively uniform, and visual output is consistent.
[0098] The client rendering process can be divided into at least one stage stable interval through segmentation, and each stage stable interval is cut with key frames before and after. When splitting frames, the timestamp of each frame will be recorded. Therefore, after completing the segmentation of the stage stable interval, the stage duration corresponding to the stage stable interval can be calculated by combining the timestamps of the start frame and the end frame of the stage stable interval.
[0099] Figure 6 This is a schematic diagram of the frame splitting and stable interval classification script. Figure 6 As shown, after completing the segmentation process of the stage stable intervals, a pre-trained classification model, such as an SVM classification model, can be used to classify and identify the segmented stage stable intervals, thereby determining the stage stable interval identifiers corresponding to the stage stable intervals.
[0100] In an optional implementation, in order to reduce the possibility of a continuous stable interval being mistakenly divided into multiple parts due to a small change between two key frames, a preset compensation threshold can be provided to allow fluctuations within a certain degree without immediately considering it as the beginning of a new stable interval. Therefore, the difference can be compensated according to the preset compensation threshold. For example, the sum of the difference and the preset compensation threshold is calculated, and the sum result is the difference after compensation. Then, the difference after compensation is compared with the preset difference threshold. If the comparison result is that the difference after compensation is greater than or equal to the preset difference threshold, the segmentation process of the stage stable interval is executed, that is, the timestamps of the two key frames are different, and the segmentation is performed before the key frame with a larger timestamp.
[0101] This helps reduce misjudgments due to minor changes and ensures that a new stable phase is only identified when the change is significant enough.
[0102] Figure 7 The code diagram of the parameters for determining the stable interval. In this embodiment, the main parameters affecting the segmentation of the stable interval are the preset difference threshold (threshold) and the preset compensation threshold (offset), wherein the higher the threshold parameter is, the higher the stability threshold is, and the more difficult it is to be determined as a stable frame; the offset is the compensation value, which merges the two similar frames before and after. Therefore, it is necessary to tune the parameters according to the actual measured scene to find the most suitable threshold and offset.
[0103] In one implementation, the client rendering process may be segmented into stable intervals using the following method, assuming that the number of key frames is N, and i starts from 1:
[0104] Calculate the difference between the i-th key frame and the i+1-th key frame;
[0105] If the difference is greater than or equal to the preset difference threshold, the i+1th key frame is determined as the starting frame of another stage stable interval, and the ith key frame is determined as the ending frame of a stage stable interval;
[0106] If the difference is less than the preset difference threshold, it is determined that the i-th key frame and the i+1-th key frame belong to the same stage stable interval;
[0107] Assign i+1 to i until all key frames are divided into the corresponding stage stability interval, 1≤i≤N-1.
[0108] Here, let the number of key frames be N, i starts from 1, the first key frame will be identified as the starting frame of a certain stage of stability interval, and the Nth key frame will be identified as the ending frame of a certain stage of stability interval.
[0109] In this embodiment, the stage stable interval is divided based on the difference between two adjacent key frames. When the difference is greater than or equal to the preset difference threshold, it means that the two key frames have changed a lot and can be divided between the two key frames; when the difference is less than the preset difference threshold, it means that the two key frames have changed a little and can be considered to belong to the same stage stable interval. The difference can be measured by comparing the pixel value difference between the two frames, the degree of feature point matching, etc.
[0110] For example, assume that there are 5 key frames, numbered 1-5, so that key frame pairs (1, 2), (2, 3), (3, 4), (4, 5) can be generated, the first key frame is the starting frame of a stage stable interval, the difference between the first key frame and the second key frame is calculated, if it is determined by calculation that the difference between the first key frame and the second key frame is less than the preset difference threshold, it is determined that the first key frame and the second key frame belong to the same stage stable interval, then, the difference between the second key frame and the third key frame is calculated, if it is determined by calculation that the difference between the second key frame and the third key frame is greater than or equal to the preset difference threshold, then the second key frame and the third key frame are split, so that the second key frame is the end frame of a stage stable interval, thereby, the first key frame can be determined. The first key frame and the second key frame form a stage stable interval, the third key frame is the starting frame of another stage stable interval, and the difference between the third key frame and the fourth key frame is continued to be calculated. If it is determined by calculation that the difference between the third key frame and the fourth key frame is less than the preset difference threshold, it is determined that the third key frame and the fourth key frame belong to the same stage stable interval. Then, the difference between the fourth key frame and the fifth key frame is calculated. If it is determined by calculation that the difference between the fourth key frame and the fifth key frame is less than the preset difference threshold, it is determined that the fourth key frame and the fifth key frame belong to the same stage stable interval. It can be determined that the third key frame, the fourth key frame and the fifth key frame form a stage stable interval. At this time, all key frames are divided into corresponding stage stable intervals, and the method ends.
[0111] In another implementation, the client rendering process may be segmented into stable intervals by the following method: Assume that the number of key frames is N, and i starts from 1:
[0112] S3071, calculating the difference between the i-th key frame and the j-th key frame, where j is greater than i, i+1≤j≤N;
[0113] S3072, if the difference is greater than or equal to the preset difference threshold, the jth key frame is determined as the starting frame of another stage stable interval, the j-1th key frame is determined as the ending frame of the stage stable interval corresponding to the i-th key frame, j is assigned to i, and the process jumps to S7;
[0114] S3073, if the difference is less than the preset difference threshold, determine that the i-th key frame and the j-th key frame belong to the same stage stable interval, assign j to j+1, and jump to S3071;
[0115] Until all key frames are divided into corresponding stage stable intervals.
[0116] Among them, the first key frame will be identified as the starting frame of a stage stability interval, and the Nth key frame will be identified as the ending frame of a stage stability interval.
[0117] In this embodiment, a certain key frame is used as the starting frame, and the difference between the key frame used as the starting frame and the key frame whose timestamp is greater than the timestamp of the starting frame is calculated respectively. When the difference is greater than or equal to a preset difference threshold, it means that the two key frames have changed greatly, and the key frame with a timestamp greater than the timestamp of the starting frame can be segmented; when the difference is less than the preset difference threshold, it means that the two key frames have changed little, and it can be considered that the two key frames belong to the same stage stability interval.
[0118] For example, assuming that there are 5 key frames, numbered 1-5, and the first key frame is the starting frame of a stage stable interval, the difference between the first key frame and the second key frame is first calculated. If it is determined by calculation that the difference between the first key frame and the second key frame is less than the preset difference threshold, it is determined that the first key frame and the second key frame belong to the same stage stable interval. Then, the difference between the first key frame and the third key frame is calculated. If it is determined by calculation that the difference between the first key frame and the third key frame is greater than or equal to the preset difference threshold, the second key frame and the third key frame are divided, so that the second key frame is the end frame of a stage stable interval. Therefore, it can be determined that the first key frame and the second key frame form a stage stable area. The third key frame is the starting frame of another stage stable interval; the difference between the third key frame and the fourth key frame is calculated. If the difference between the third key frame and the fourth key frame is determined by calculation to be less than a preset difference threshold, it is determined that the third key frame and the fourth key frame belong to the same stage stable interval. Then, the difference between the third key frame and the fifth key frame is calculated. If the difference between the third key frame and the fifth key frame is determined by calculation to be less than a preset difference threshold, it is determined that the third key frame and the fifth key frame belong to the same stage stable interval. Therefore, it can be determined that the third key frame, the fourth key frame and the fifth key frame constitute a stage stable interval. At this time, all key frames are divided into corresponding stage stable intervals, and the method ends.
[0119] Figure 8 This is a schematic diagram of the stages of the client rendering process when a certain app is cold started and enters the homepage, such as Figure 8The client rendering process shown can be divided into four stages: starting the APP, initiating a network request, dynamic resource stage, and static resource stage. The entire client rendering process can be divided into a stable interval and an unstable interval. The unstable interval is 83 (1.96666666666666668)-86 (2.1666666666666667), duration: 0.20000000000000015, stage: 0; stable interval 86 (2.166666666666667)-91 (2.13333333333333333), duration: 0.0666666666666665, stage: 1; unstable interval 91 (2.13333333333333333)-109 (2.5500000000000003), duration: 0.41666666666666696, stage: 2. It should be noted that the client rendering process can also be divided into other stages, which will not be described in detail here.
[0120] Step S308: If the rendering result is a rendering exception, the rendering exception performance information, the stage stability interval of the rendering exception, and the corresponding stage time consumption are used as the test result.
[0121] When the rendering result is a rendering exception, the client mainly manifests itself as: no content, empty window or continuous loading, etc., and the client performance information can be identified as rendering exception performance information, that is, the rendering exception performance information can be: no content, empty window or continuous loading, etc., and the test results can be generated by combining the rendering exception stage stability interval and the corresponding stage time consumption. The test results include: rendering exception performance information and the rendering exception stage stability interval and the corresponding stage time consumption.
[0122] Step S309: if the rendering result is normal, then the stage time consumption corresponding to the stable interval of each stage is compared with the corresponding stage time consumption baseline standard.
[0123] Step S310: If the stage time consumption corresponding to the stage stability interval is greater than the corresponding stage time consumption baseline standard, the test is determined to be abnormal, and the domain name resolution IP is first marked.
[0124] Step S311: If the stage time corresponding to the stage stability interval is less than or equal to the corresponding stage time baseline standard, it is determined that the test is normal and a second mark is made on the domain name resolution IP.
[0125] Specifically, we can statistically analyze the stage time consumption corresponding to the stable interval of each stage when the network request data is normal, so that we can establish the stage time consumption baseline standard based on these normal network request data. Usually, when the network request data is normal, it takes 0.2s for the device to receive the interface data return, and 0.85s for the image resource to be fully rendered and loaded. Fig. 9 As shown, when the rendering result is normal, it is very likely that each stage will take a long time. Therefore, the stage time corresponding to the stable interval of each stage can be compared with the corresponding stage time baseline standard to determine whether the loading is abnormal, that is, whether there is a timeout. If the stage time corresponding to the stable interval of the stage is greater than the corresponding stage time baseline standard, it means that although it can be loaded normally, it takes too long and affects the normal user experience. Therefore, it can be determined that the test is abnormal and there is performance cracking. The domain name resolution IP can be first marked, for example, the IP is marked to have performance cracking, or the domain name resolution IP can be directly deleted. The test result can be: the client renders according to the domain name resolution IP and there is performance cracking. When it is determined that the stage time corresponding to the stable interval of the stage is greater than the corresponding stage time baseline standard, there may be a timeout, and a timeout retry can be performed, and other IPs can be selected from a group of IP addresses of the domain name resolution for retry.
[0126] If the stage time corresponding to the stage stability interval is less than or equal to the corresponding stage time baseline standard, it means that the loading is normal. Therefore, it can be determined that the test is normal and the client can render normally according to the domain name resolution IP. For example, the corresponding domain name resolution IP can be marked as normal and available, and the client performs normally. The test result can be: the client renders normally according to the domain name resolution IP.
[0127] By comparing with the stage time baseline standard, the domain name resolution IPs that take a long time when rendering is normal can be screened out, so as to avoid using these domain name resolution IPs that take a long time to initiate business requests in actual applications, thereby improving the user experience.
[0128] Step S312: If the service request status is unavailable, the domain name downgrade process is performed to obtain the domain name resolution IP, and the process jumps to step S304.
[0129] Some steps in this embodiment are Figure 1 Some steps in the illustrated embodiment are implemented similarly and will not be described again here.
[0130] In an optional implementation, the method further includes: recording a log file according to the client behavior information;
[0131] The log file is queried according to preset keywords to determine log information, and fault analysis is performed according to the log information.
[0132] During the client rendering process, the client performs corresponding actions for the network request. Therefore, the log file can be recorded according to the client behavior information. For example, if the client shows continuous loading, empty window, no content, etc., the corresponding log information will be recorded. For example, the error code of the failed network request, the actual DNS provider (dns_provider), the host resolution IP address (host_resolve_ips), the stage stability interval, the time consumed in the stage stability interval, etc. The log file can be automatically analyzed and screened by preset keywords, such as Fig.10 As shown, determining the matching log information is mainly to obtain key information such as the error code of the failed network request, the actual dns_provider, host_resolve_ips, stage stability interval, and stage stability interval time consumption, so as to facilitate problem location and troubleshooting based on the log information.
[0133] This application improves efficiency by connecting various rehearsal links through automated testing methods, including the initiation of client network requests, the access to rehearsal scenarios, automatic determination of abnormal scenarios such as white screens and unavailable network requests, and analysis of key behavior logs. It fully simulates various situations that may exist in the client network request link, enriches the coverage of test scenarios, greatly improves rehearsal efficiency, and reduces human execution costs.
[0134] Fig.11 A structural block diagram of a test device according to an embodiment of the present application is shown. Fig.12 As shown, the device comprises:
[0135] The first sending module 1101 is adapted to send a network request to each service domain name through a test script;
[0136] The forwarding module 1102 is adapted to intercept the network request and forward the network request to a preset domain name resolution service;
[0137] A domain name resolution module 1103, adapted to perform domain name resolution on the network request according to the fault scenario pre-injected in the preset domain name resolution service to obtain a domain name resolution IP;
[0138] The second sending module 1104 is adapted to send a service request to the corresponding service domain interface according to the domain name resolution IP;
[0139] The generating module 1105 is adapted to render the dynamic resources and static resources associated with the service if the service request status is passed, and generate a test result according to the rendering result;
[0140] The domain name downgrade processing module 1106 is adapted to execute domain name downgrade processing if the service request status is unreachable, obtain the domain name resolution IP, and trigger the execution of the second sending module.
[0141] Optionally, the generating module is further adapted to: perform frame splitting processing on the client rendering process;
[0142] The client rendering process is divided into phase stable intervals according to the key frames, and the phase time consumption corresponding to each phase stable interval is recorded;
[0143] Generate test results based on rendering results, stage stability interval and stage duration.
[0144] Optionally, the generation module is further adapted to: if the rendering result is a rendering exception, use the rendering exception performance information, the stage stability interval of the rendering exception, and the corresponding stage time consumption as the test result.
[0145] Optionally, the generation module is further adapted to: if the rendering result is normal, compare the stage time consumption corresponding to the stable interval of each stage with the corresponding stage time consumption baseline standard;
[0146] If the stage time consumption corresponding to the stage stable interval is greater than the corresponding stage time consumption baseline standard, the test is determined to be abnormal, and the domain name resolution IP is first marked;
[0147] If the stage time corresponding to the stage stability interval is less than or equal to the corresponding stage time baseline standard, the test is determined to be normal and a second mark is performed on the domain name resolution IP.
[0148] Optionally, the generation module is further adapted to: calculate the difference between the key frames;
[0149] Comparing the difference with a preset difference threshold;
[0150] If the comparison result is that the difference is greater than or equal to the preset difference threshold, the segmentation process of the stage stable interval is executed.
[0151] Optionally, the generating module is further adapted to: perform compensation processing on the difference according to a preset compensation threshold;
[0152] Comparing the difference with a preset difference threshold further includes: comparing the difference after compensation processing with the preset difference threshold.
[0153] Optionally, the device further comprises: a recording module adapted to record a log file according to the client behavior information;
[0154] The analysis module is adapted to query the log file according to preset keywords to determine log information, and perform fault analysis according to the log information.
[0155] Optionally, the forwarding module is further adapted to: modify the configuration file, and forward the network request to a preset domain name resolution service according to the modified configuration file.
[0156] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.
[0157] This application improves efficiency through automated testing and connects various rehearsal links, including the initiation of client network requests, the access to rehearsal scenarios, automatic determination of abnormal scenarios such as white screens and unavailable network requests, and analysis of key behavior logs. It fully simulates various situations that may exist in the client network request link, enriches the coverage of test scenarios, greatly improves rehearsal efficiency, and reduces human execution costs.
[0158] An embodiment of the present application provides a non-volatile computer storage medium, which stores at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the test method in any of the above method embodiments.
[0159] An embodiment of the present application provides a computer program product, which includes at least one executable instruction or computer program, and the executable instruction or computer program can enable a processor to perform operations corresponding to the test method in any of the above method embodiments.
[0160] Fig.12 A schematic diagram of the structure of an embodiment of a computing device of the present application is shown. The specific embodiment of the present application does not limit the specific implementation of the computing device.
[0161] like Fig.12 As shown, the computing device may include: a processor (processor) 1202 , a communication interface (Communications Interface) 1204 , a memory (memory) 1206 , and a communication bus 1208 .
[0162] The processor 1202, the communication interface 1204, and the memory 1206 communicate with each other via a communication bus 1208. The communication interface 1204 is used to communicate with other devices such as a client or other server network elements. The processor 1202 is used to execute the program 1210, which can specifically execute the relevant steps in the above-mentioned test method embodiment for a computing device.
[0163] Specifically, the program 1210 may include program codes, which include computer operation instructions.
[0164] The processor 1202 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.
[0165] The memory 1206 is used to store the program 1210. The memory 1206 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0166] The program 1210 can be specifically used to enable the processor 1202 to execute the test method in any of the above method embodiments. The specific implementation of each step in the program 1210 can refer to the corresponding descriptions in the corresponding steps and units in the above test embodiments, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiments, which will not be repeated here.
[0167] The algorithm and display provided herein are not inherently related to any specific computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the present application embodiment is not directed to any specific programming language yet. It should be understood that various programming languages can be utilized to realize the content of the present application embodiment described herein, and the above description of specific languages is to disclose the best mode of implementation of the present application embodiment.
[0168] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.
[0169] Similarly, it should be understood that in order to streamline the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the embodiments of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed embodiments of the present application require more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the embodiment of the present application.
[0170] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0171] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the embodiments of the present application and to form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0172] The various component embodiments of the embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to the embodiments of the present application. The embodiments of the present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the embodiments of the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.
[0173] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.
Claims
1. A testing method, comprising: S1, sends network requests to each business domain name through the test script; S2, intercepting the network request and forwarding the network request to a preset domain name resolution service; S3, performing domain name resolution on the network request according to the fault scenario pre-injected in the preset domain name resolution service to obtain a domain name resolution IP; S4, sending a service request to the corresponding service domain interface according to the domain name resolution IP; S5: If the service request status is passed, the dynamic resources and static resources associated with the service are rendered, and a test result is generated according to the rendering result; S6: If the service request status is not available, the domain name downgrade process is performed, the domain name resolution IP is obtained, and the process jumps to S4.
2. The method according to claim 1, wherein: Generating the test result according to the rendering result further includes: performing frame splitting processing on the client rendering process; The client rendering process is divided into phase stable intervals according to the key frames, and the phase time consumption corresponding to each phase stable interval is recorded; Generate test results based on rendering results, stage stability interval and stage duration.
3. The method according to claim 2, wherein: Generating the test result according to the rendering result, the stage stability interval and the stage time consumption further includes: if the rendering result is a rendering exception, using the rendering exception performance information, the stage stability interval of the rendering exception and the corresponding stage time consumption as the test result.
4. The method according to claim 2 or 3, wherein: Generating the test results according to the rendering results, the stage stability interval and the stage time consumption further includes: If the rendering result is normal, the stage time consumption corresponding to the stable interval of each stage is compared with the corresponding stage time consumption baseline standard; If the stage time consumption corresponding to the stage stable interval is greater than the corresponding stage time consumption baseline standard, the test is determined to be abnormal, and the domain name resolution IP is first marked; If the stage time corresponding to the stage stability interval is less than or equal to the corresponding stage time baseline standard, the test is determined to be normal and a second mark is performed on the domain name resolution IP.
5. The method according to any one of claims 2 to 4, wherein: The segmentation process of the client rendering process according to the key frames into stage stable intervals further includes: Calculate the difference between key frames; Comparing the difference with a preset difference threshold; If the comparison result is that the difference is greater than or equal to the preset difference threshold, the segmentation process of the stage stable interval is executed.
6. The method according to claim 5, wherein: The method further includes: performing compensation processing on the difference according to a preset compensation threshold; Comparing the difference with a preset difference threshold further includes: comparing the difference after compensation processing with the preset difference threshold.
7. The method according to any one of claims 1 to 6, wherein: The method further comprises: recording a log file according to the client behavior information; The log file is queried according to preset keywords to determine log information, and fault analysis is performed according to the log information.
8. The method according to any one of claims 1 to 7, wherein: The intercepting the network request and forwarding the network request to the preset domain name resolution service further includes: The configuration file is modified, and the network request is forwarded to a preset domain name resolution service according to the modified configuration file.
9. A testing device, comprising: A first sending module, adapted to send a network request to each business domain name through a test script; A forwarding module, adapted to intercept the network request and forward the network request to a preset domain name resolution service; A domain name resolution module, adapted to perform domain name resolution on the network request according to the fault scenario pre-injected into the preset domain name resolution service to obtain a domain name resolution IP; A second sending module, adapted to send a service request to a corresponding service domain interface according to the domain name resolution IP; A generation module, adapted to render dynamic resources and static resources associated with the business if the business request status is passed, and generate test results according to the rendering results; The domain name downgrade processing module is adapted to execute domain name downgrade processing if the service request status is unreachable, obtain the domain name resolution IP, and trigger the execution of the second sending module.
10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute an operation corresponding to the testing method according to any one of claims 1-8.
11. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and wherein the executable instruction enables a processor to execute an operation corresponding to the testing method according to any one of claims 1 to 8.
12. A computer program product, comprising at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the testing method according to any one of claims 1 to 8.