Test method and device, electronic equipment and storage medium

Abstract

The invention provides a test method and device, electronic equipment and a storage medium. The method comprises the steps that a first feature value sequence corresponding to a first test index in a test requirement is determined; obtaining a first cue word corresponding to the first feature value sequence based on the first feature value sequence; inputting the first cue word into a first model to obtain a second feature value sequence; and testing based on the second characteristic value sequence to obtain a test result. Therefore, the testing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to a testing method, device, electronic device and storage medium. Background Art

[0002] Testing technology is a technique for testing an entire system. Testers deploy multiple services within the system under test and apply stress to the system by simulating user service requests. While applying stress, testers monitor various system indicators through a monitoring system and ultimately generate a test report. Currently, testing primarily involves writing stress scripts based on test requirements, thereby controlling the test tool's application of stress to the system under test. However, this approach takes a long time to write, resulting in lower test efficiency. Summary of the Invention

[0003] The present application provides a testing method, device, electronic device and storage medium, which can shorten the overall process time and thus improve the efficiency of testing by converting the traditional script-based process into automated parameter generation based on characteristic value sequences.

[0004] In a first aspect, the present application provides a testing method, the method comprising: Determine a first characteristic value sequence corresponding to a first test indicator in the test requirement; Obtaining a first prompt word corresponding to the first eigenvalue sequence based on the first eigenvalue sequence; Input the first prompt word into the first model to obtain a second eigenvalue sequence; A test is performed based on the second eigenvalue sequence to obtain a test result.

[0005] As can be seen, in this application, by determining the first eigenvalue sequence corresponding to the test indicator and obtaining the first prompt word corresponding to the first eigenvalue sequence, a second eigenvalue sequence is output in combination with the first prompt word and the first model, so that the test is performed according to the second eigenvalue sequence to obtain the corresponding test results. This can transform the traditional script-based process into an automated generation based on the eigenvalue sequence, thereby shortening the overall process time and improving the efficiency of the test.

[0006] In a second aspect, the present application provides a testing device, comprising: A first processing unit, configured to determine a first characteristic value sequence corresponding to a first test indicator in a test requirement; a second processing unit, configured to obtain a first prompt word corresponding to the first feature value sequence based on the first feature value sequence; The second processing unit is further configured to input the first prompt word into the first model to obtain a second eigenvalue sequence; The third processing unit is configured to perform a test based on the second eigenvalue sequence to obtain a test result.

[0007] In a third aspect, the present application provides an electronic device comprising a processor, a memory, and a communication interface. The processor, memory, and communication interface are interconnected and perform communication with each other. The memory stores executable program code, the communication interface is used for wireless communication, and the processor is used to retrieve the executable program code stored in the memory and execute some or all of the steps described in any method of the first aspect.

[0008] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements some or all of the steps described in the first aspect of the present application.

[0009] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when processed and executed, implements some or all of the steps described in the first aspect of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 A schematic diagram of the structure of a test system provided in an embodiment of the present application; Figure 2 An architectural diagram of a testing method provided in an embodiment of the present application; Figure 3 A flow chart of a testing method provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a first eigenvalue sequence provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a second eigenvalue sequence provided in an embodiment of the present application; Figure 6 A flow chart of another testing method provided in an embodiment of the present application; Figure 7 A schematic diagram of a database construction process provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a prompt word provided in an embodiment of the present application; Figure 9 A flow chart of another testing method provided in an embodiment of the present application; Figure 10 A schematic diagram of the structure of a testing tool provided in an embodiment of the present application; Figure 11 A flow chart of a test tool control process provided in an embodiment of the present application; Figure 12 A block diagram of the functional units of a test device provided in an embodiment of the present application; Figure 13 A block diagram of the functional units of another test device provided in an embodiment of the present application; Figure 14 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] The terms "first," "second," and so on, in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps is not limited to the listed steps but may optionally include steps not listed, or may optionally include other steps inherent to the process, method, product, or apparatus.

[0014] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0015] Currently, for testing technology, testers must first align requirements with the test demander to confirm the test content and the required metrics. After requirements alignment, they conduct analysis on both the test application and the business under test. Application analysis primarily analyzes the key software components involved in the test process, a process that typically takes one to two weeks. After the analysis is complete, developers assist testers in desensitizing the application and pre-built data to avoid unnecessary information security issues. This process typically takes three to four days. Business analysis primarily analyzes the load ratios of different users, a process that typically takes one week. Stress scripts are then developed based on the user access ratios, a process that typically takes three to four days. Next, testers spend two to three days preparing the test system environment, primarily pre-building the required data. Next, testers spend another two to three days executing the test tasks, using the aforementioned stress scripts. Finally, testers analyze the system's current resources based on monitored metrics to determine if they can support the business. This analysis of test results takes approximately two to three days.

[0016] It can be seen that for traditional business demand-based testing technology, all the processes mentioned above need to be completed. Without considering parallel execution, it takes 30 working days to complete the entire process, which is relatively time-consuming.

[0017] Based on this, the present application provides a testing method, which determines the first eigenvalue sequence corresponding to the test indicator and obtains the first prompt word corresponding to the first eigenvalue sequence, thereby combining the first prompt word and the first model to output the second eigenvalue sequence, so that the test is performed according to the second eigenvalue sequence to obtain the corresponding test result.

[0018] In this way, the control parameters of the test tool corresponding to the test requirements can be automatically generated based on the model, thereby avoiding the test personnel from manually writing corresponding test scripts according to the test requirements, thereby improving the efficiency of the test.

[0019] The following is an introduction to the prior art involved in this application.

[0020] Large language models (LLMs) are deep learning models trained using large amounts of text data. They can generate natural language text or understand the meaning of text. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and conversation, and are an important path to artificial intelligence. Compared to traditional neural network models, they require larger model size, more training data, and more computing resources.

[0021] Embedding Model: Embedding is a vectorized operation that converts text, images, and videos into arrays of floating-point numbers called vectors. These vectors are designed to capture the meaning of the text, image, or video. The length of the embedding array is called the dimension of the vector. Embedding models are the models used in the embedding process. These models are primarily used to convert text into numerical vectors, making them easier for computers to understand and store.

[0022] Prompts: In large artificial intelligence (AI) models, prompts primarily provide context for input and parameter information. Well-designed prompts ensure that the model's output more closely matches user expectations.

[0023] Retrieval-augmented Generation (RAG) is a model that combines retrieval and generation techniques. It generates answers or content by referencing information from external knowledge bases. It offers strong interpretability and customization, making it suitable for a variety of natural language processing tasks, including question-answering systems, document generation, and intelligent assistants. The RAG model's advantages lie in its versatility, ability to enable instant knowledge updates, and its ability to provide more efficient and accurate information services through end-to-end evaluation.

[0024] Agent: This article refers to AI agents, also known as "AI agents." Agents are intelligent entities (computer programs) capable of perceiving their environment, autonomously understanding, making decisions, and executing actions. Based on the Large Language Model (LLM), they possess the capabilities of autonomous understanding, perception, planning, memory, and tool utilization, enabling them to automate complex tasks. In this article, agents serve as the executors of the test.

[0025] Kubernetes, also known as K8s, is an open-source framework for managing containerized applications across multiple hosts in a cloud platform. Kubernetes aims to simplify and efficiently deploy containerized applications, providing a mechanism for application deployment, planning, updating, and maintenance. In this article, Kubernetes provides service deployment capabilities for the system under test.

[0026] The following introduces the system architecture involved in this application.

[0027] See also Figure 1 , Figure 1 A schematic diagram of the structure of a test system provided in an embodiment of the present application is shown in FIG. Figure 1As shown, the test system 100 includes a terminal device 101 and a server 102 .

[0028] The terminal device 101 is used to determine the control parameters corresponding to the test tool according to the test requirements, thereby controlling the test tool installed in the system to apply pressure and monitoring various indicators of the system. Optionally, the terminal device 101 can be a desktop computer, laptop computer, tablet computer, smart phone, etc.

[0029] The server 102 is used to deploy the resources and components required for the test, thereby running the corresponding test tools to put pressure on the system. Optionally, the server 102 can be a single server, a server cluster, a cloud server, a cloud computing service center, or other forms of devices with computing capabilities.

[0030] In the present application, the terminal device 101 first determines the first eigenvalue sequence corresponding to the first test indicator in the test requirement; obtains the first prompt word corresponding to the first eigenvalue sequence based on the first eigenvalue sequence; inputs the first prompt word into the first model to obtain the second eigenvalue sequence; performs testing based on the second eigenvalue sequence to obtain the test results.

[0031] It can be seen that the aforementioned method can automatically generate control parameters of the test tool corresponding to the test requirements based on the model, thereby avoiding the tester from manually writing corresponding test scripts according to the test requirements, thereby improving the efficiency of the test.

[0032] The following is an introduction to the method architecture diagram involved in this application.

[0033] See also Figure 2 , Figure 2 This is a diagram of the architecture of a test method provided in an embodiment of the present application. Figure 2 As shown, for the system, the central processing unit (CPU) resources, memory resources, storage resources, and network resources constitute the hardware infrastructure of the system. Kubernetes provides the system with a software infrastructure for deploying and scheduling various stress-generating components. Testers pre-deployed a variety of test tools for simulating computing pressure, storage pressure, and network pressure in the system. The tools for simulating computing pressure, storage pressure, and network pressure are controlled by pressure parameters under multiple time series (pressure parameters of time series 1 to pressure parameters of time series n) to apply pressure to the entire system. While applying pressure, the testers observe various indicators of the system through the equipment used for equipment monitoring, and finally generate a test report. It can be understood that the test in this application mainly refers to stress testing.

[0034] Based on this, an embodiment of the present application provides a testing method, which is described in detail below with reference to the accompanying drawings.

[0035] In the first embodiment, the main process of the testing method is described below.

[0036] See also Figure 3 , Figure 3 A flow chart of a test method provided in an embodiment of the present application is shown, which is applied to the above-mentioned terminal device, such as Figure 3 As shown, the method includes the following steps.

[0037] Step S301: Determine a first characteristic value sequence corresponding to a first test indicator in a test requirement.

[0038] Among them, the first eigenvalue sequence may include indicator values ​​corresponding to the time series. The test requirements may be test objectives clearly defined after consultation between the tester and the demander, such as the stability of the system under a specific load, performance threshold, etc., which can be obtained through the test target document or meeting minutes provided by the demander. The first test indicator may be a specific system performance parameter that needs to be verified, such as CPU utilization, memory usage, network throughput, etc., and its acquisition methods include the indicator list provided by the demander or industry standard specifications. The first eigenvalue sequence may be a set of indicator values ​​containing a time series, such as data such as CPU utilization, number of threads, memory utilization, etc. corresponding to different time points, where each time point may simultaneously contain eigenvalues ​​corresponding to multiple test indicators.

[0039] For example, see Figure 4 , Figure 4 A structural diagram of a first eigenvalue sequence provided in an embodiment of the present application is shown as follows: Figure 4 As shown, it includes n eigenvalues ​​under time series 1~n. The eigenvalues ​​under each time series are composed of a two-dimensional array, for example, [[A1,B1,C1,…G1], [A2,B2,C2,…G2], …[An,Bn,Cn,…Gn]], where A1~G1 represent the eigenvalues ​​corresponding to different test indicators respectively.

[0040] Step S302: obtaining a first prompt word corresponding to the first feature value sequence based on the first feature value sequence.

[0041] Optionally, the first prompt word may include a description text of a first test tool used to perform a test task. Obtaining the first prompt word corresponding to the first feature value sequence based on the first feature value sequence may involve querying and matching the first prompt word corresponding to the first feature value sequence using other knowledge content.

[0042] Optionally, a database search can be used to retrieve the first prompt word corresponding to the first feature value sequence. This database search can be combined with RAG technology to assist the model in generating results by searching for relevant documents in the database. This process is implemented by converting text into vectors using an embedding model and storing them in a vector database.

[0043] Step S303: input the first prompt word into the first model to obtain a second eigenvalue sequence.

[0044] The second feature value sequence may include parameter values ​​corresponding to the time series. The first model may be a large language model (LLM). Its function includes generating output text that meets the requirements based on the input prompt word. The second feature value sequence may be a sequence containing parameter values, such as a set of configuration parameters for a testing tool such as stress-ng, fio, or iperf3 corresponding to the time series. The generation process involves the model parsing the input prompt word and combining it with the tool documentation content in the knowledge base to output a structured parameter sequence.

[0045] For example, see Figure 5 , Figure 5 A schematic diagram of the structure of a second eigenvalue sequence provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in the figure, it includes n parameter values ​​in time series 1 to n. The parameter values ​​in each time series are composed of a two-dimensional array, for example, {[T1-1, T1-2, T1-3, …T2-1, T2-2, …T3-1, T3-2, …], [T1-1, T1-2, T1-3, …T2-1, T2-2, …T3-1, T3-2, …] … [T1-1, T1-2, T1-3, …T2-1, T2-2, …T3-1, T3-2, …]}. Among them, Tn-m represents the parameter value of the mth parameter of the nth tool in this time series.

[0046] Step S304: perform a test based on the second eigenvalue sequence to obtain a test result.

[0047] The test process can involve updating the parameters of the test tool at timed intervals through an agent program, for example, executing stress commands at 5-second intervals. Test results can include monitoring metrics of the system under stress, such as CPU load, storage IOPS, and network throughput. These metrics are collected through monitoring devices and compared to preset thresholds.

[0048] As can be seen, the testing method provided by the embodiments of the present application determines a first eigenvalue sequence corresponding to a test indicator and obtains a first prompt word corresponding to the first eigenvalue sequence, thereby combining the first prompt word with the first model to output a second eigenvalue sequence, so that testing is performed according to the second eigenvalue sequence to obtain the corresponding test results. This can transform the traditional script-based process into an automated generation based on the eigenvalue sequence, thereby shortening the overall process time and improving the efficiency of the test.

[0049] In the second embodiment, the test method is described in detail below based on the relevant details of database retrieval.

[0050] See also Figure 6 , Figure 6 A flow chart of another testing method provided in an embodiment of the present application is provided, which is applied to the above-mentioned terminal device, such as Figure 6 As shown, the method includes the following steps.

[0051] Step S601: Determine a first characteristic value sequence corresponding to a first test indicator in a test requirement.

[0052] Step S602: Divide the first knowledge text into blocks based on the test parameters to obtain text blocks corresponding to the test parameters.

[0053] The first knowledge text includes explanatory text corresponding to the test parameters. As can be understood, test parameters are primarily used to simulate and evaluate key performance indicators of the system under test, such as the number of concurrent users, request frequency, response time, and so on. Different test parameters can represent different test methods and can also be applied to different test tools.

[0054] The first knowledge text can be a collection of original documents containing explanatory text corresponding to the test parameters, including but not limited to computer principle knowledge text and computer network knowledge text. This is obtained by systematically organizing official documents or technical databases related to the knowledge of the test parameters. The text block can be a collection of sub-documents formed by segmenting the knowledge text by test parameter type. For example, all document paragraphs regarding request frequency constitute an independent text block, which is generated through automated text analysis of test parameters or manual annotation.

[0055] The first knowledge text is segmented based on the test parameters. This can be done by using a text classification algorithm to identify keywords and contextual semantics in the document that are related to the type of specific test parameters, and classifying the matching content into text blocks of the type corresponding to the test parameters. In a specific embodiment, the tester can use regular expressions to match the test parameter names or use a natural language processing model to extract the type characteristics of the test parameters, ensuring that each text block only contains the description content of a single test parameter. This operation can reduce the interference of irrelevant information during subsequent retrieval, allowing database retrieval to directly locate the specialized knowledge of the test parameters when generating simulation parameters, thereby improving the accuracy of parameter generation.

[0056] Step S603: Perform vector conversion on the text block to obtain a text vector.

[0057] Converting text blocks to vectors involves mapping the text content into numerical vector representations using a pre-trained embedding model (such as BGEM3). The resulting vectors are then stored in a database, optionally with test parameter labels as metadata. This vectorized storage allows the system to directly access type-annotated vector data during test execution.

[0058] As can be seen, by segmenting the knowledge text based on test parameters to achieve precise knowledge segmentation, and combining it with vectorized storage technology to build a type-associative database, database retrieval can be targeted to the configuration knowledge of specific test parameters. This can improve the accuracy of the subsequent first model's conversion of the first eigenvalue sequence to the second eigenvalue sequence, thereby improving the accuracy of subsequent tests.

[0059] For example, see Figure 7 , Figure 7 A flowchart of a database construction process provided in an embodiment of the present application is shown as follows: Figure 7 As shown in the figure, relevant knowledge from different fields corresponding to the test parameters is collected, including but not limited to computer principles knowledge text, computer network knowledge text, computer storage knowledge text, Linux operating system knowledge text, and multiple test tool user manuals (e.g., Test Tool 1 User Manual to Test Tool N User Manual). The different knowledge documents are then divided into blocks, resulting in multiple text blocks (Text Block 1 to Text Block n) corresponding to the various test tool types. Next, these multiple text blocks are input into a pre-trained embedding model, which converts them into vector information and stores it in a database.

[0060] In this way, in subsequent index queries, for a given query q in any language x, it can retrieve documents d in language y from the corpus. The expression is as follows:

[0061] in, Used to represent a collection of documents in language y, i.e., the aforementioned database. fn*() refers to a corresponding search function, such as any of the following: dense search function, sparse search function, and multi-vector search function. is a query q given any language x. Used to represent the document d corresponding to the language y obtained after the query.

[0062] Optionally, in Lexical Retrieval, the expression is as follows:

[0063] Where q∩p represents the terms that appear in both query q and paragraph p, and t represents the term. qt Represents the term weight, w pt Represents the paragraph weight, which ultimately measures whether the retrieval is relevant. lex It is used to indicate the relevance score between query q and paragraph p at the vocabulary level. The higher the value, the stronger the relevance.

[0064] Step S604: Search the text vector for the second knowledge text corresponding to the label of the testing tool.

[0065] Among them, the label of the test tool corresponds to the first feature value sequence, and the test tool can be a software or hardware tool used to perform the test, such as `stress-ng` or `fio`, etc. The label of the test tool can be used to describe the function, characteristics, applicable scenarios or technical classification of the tool. For example, it can include a unique identifier corresponding to the test tool (such as a preset tool identifier or name), and can also include metadata of the professional field to which the test tool belongs (such as "CPU stress simulation" or "storage I / O parameter configuration"), which can be generated by manual annotation or automated classification algorithm. The text vector can be a vector database that vectorizes the storage of knowledge text, such as Milvus, which supports multi-condition retrieval based on tool labels. The second knowledge text can be a document fragment directly related to the content of a specific tool label, such as the description paragraph about the `cpu` parameter in the `stress-ng` tool manual, which is obtained by retrieving the label of the test tool.

[0066] In addition, before performing database retrieval, the retrieval data needs to be vectorized.

[0067] Step S605 : determining a first prompt word based on the label of the test tool, the second knowledge text, the first feature value sequence, the format of the second feature value sequence, and the third knowledge text.

[0068] The third knowledge text includes the format of the second eigenvalue sequence and the meaning of each field in the first eigenvalue sequence. For example, the output simulated pressure eigenvalue sequence must be a two-dimensional array, and each parameter value must correspond to the parameter name and value range of the test tool.

[0069] The technical operation is to integrate the label of the test tool, the retrieved knowledge text, the format of the first feature value sequence, the second feature value sequence, the format definition and the structured description text to generate the final prompt word.

[0070] For example, see Figure 8 , Figure 8 A schematic diagram of a prompt word structure provided in an embodiment of the present application is shown in FIG. Figure 6 The prompt word includes multiple fields, including the Tools field, which describes the test tools used; the Tags field, which describes the technical classification involved; the Knowledge field, which describes the knowledge content indexed in the database; the Input_matrix field, which describes the input first eigenvalue sequence; the Output_format field, which describes the format of the output second eigenvalue sequence; and the Task field, which describes the task generated by this model and includes the meaning of each input and output field.

[0071] It can be seen that by constructing prompt words based on the above features, the compliance of parameter generation can be improved, the explainability of prompt words can be enhanced, and the model inference time can be shortened, thereby improving the overall efficiency of testing.

[0072] Step S606: input the first prompt word into the first model to obtain a second eigenvalue sequence.

[0073] Step S607: Perform a test based on the second eigenvalue sequence to obtain a test result.

[0074] Embodiment 3: The following describes in detail the testing method based on the details of controlling the first testing tool to apply pressure to the system based on the two-characteristic value sequence.

[0075] See also Figure 9 , Figure 9 A flow chart of another test method provided in an embodiment of the present application is provided, which is applied to the above-mentioned terminal device, such as Figure 9 As shown, the method includes the following steps.

[0076] Step S901: Determine a first characteristic value sequence corresponding to a first test indicator in a test requirement.

[0077] Step S902: Divide the first knowledge text into blocks based on the test parameters to obtain text blocks corresponding to the test parameters.

[0078] Step S903: Perform vector conversion on the text block to obtain a text vector.

[0079] Step S904: Search the text vector for the second knowledge text corresponding to the label of the testing tool.

[0080] Step S905 : determining a first prompt word based on the label of the test tool, the second knowledge text, the first feature value sequence, the format of the second feature value sequence, and the third knowledge text.

[0081] Step S906: input the first prompt word into the first model to obtain a second eigenvalue sequence.

[0082] Step S907: If the first eigenvalue sequence includes a first eigenvalue at a first time point and a second eigenvalue at a second time point, testing is performed based on the first eigenvalue.

[0083] Step S908: After the first preset time interval, perform a test based on the second characteristic value.

[0084] Wherein, after testing based on the first eigenvalue and the second eigenvalue, the test results of the system include the first test result and the second test result, respectively. The first preset time interval can be the waiting period for parameter switching between two consecutive time points, which can be obtained through user configuration or system preset. Exemplarily, the interval can include 5 seconds, 10 seconds, or a duration dynamically adjusted according to the test scenario. The first eigenvalue can be the specific parameter configuration of each tool corresponding to the first time point. Exemplarily, the parameter value can include specific values ​​such as the number of CPUs of `stress-ng` and the I / O block size of `fio`. The second eigenvalue can be the parameter configuration corresponding to the second time point.

[0085] Testing based on the first eigenvalue can be performed by controlling a corresponding test tool based on the first eigenvalue. This can be achieved by having the Agent parse the parameter configuration at the first time point and generate a corresponding command. After a first preset time interval, switching to controlling the corresponding test tool based on the second eigenvalue can be achieved by having the Agent automatically update the tool configuration and re-execute the command after waiting for a preset period of time.

[0086] It can be seen that through the optimization of phased pressure based on time series, the preset time interval is used to ensure the continuity and phasing of parameter switching, and the configuration of parameter values ​​at different time points is combined to realize the simulation of dynamic load scenarios. At the same time, manual intervention is reduced through automated parameter switching, which can achieve the technical effect of improving test flexibility.

[0087] In addition, in a feasible embodiment, after testing based on the first eigenvalue, the method also includes: collecting a first test result, the first test result including a first test value; if the first test value exceeds the preset range corresponding to the first test indicator, the first test value, the first eigenvalue sequence, the second eigenvalue sequence, the preset range and the task description are input into the first model to obtain an adjusted second eigenvalue; wherein the task description is used to indicate that the second eigenvalue is adjusted so that after testing based on the second eigenvalue, the second test value in the test result obtained does not exceed the preset range corresponding to the first test indicator.

[0088] The first test value can be the actual performance indicator value of the system during the testing phase, such as CPU utilization and memory usage. The preset range can be the system performance threshold interval defined by the demander, such as CPU utilization must be less than 85% or memory usage must not exceed 90%, which is determined by the test requirements document or system configuration parameters. The second test value can be the performance indicator value collected during the re-stress phase after parameter adjustment, which is used to verify whether the adjusted parameters meet the preset range requirements.

[0089] During the test, if the actual performance index value of the system exceeds the standard and the CPU usage is too high, there may be a risk of system crash. At this time, the pressure overload can already reflect the actual maximum performance of the system. It is meaningless to continue to perform tests under the current parameters or tests higher than the current parameters.

[0090] The technical operation of adjusting the second eigenvalue by the first model can be performed by inputting the first test value that exceeds the standard, the first eigenvalue sequence, the second eigenvalue sequence, the preset range, and a task description into the first model, thereby obtaining the adjusted second eigenvalue. The task description is used to indicate that the second eigenvalue is adjusted so that after the first test tool is controlled to test the system based on the second eigenvalue, the second test value in the test result obtained does not exceed the preset range corresponding to the first test indicator.

[0091] It can be understood that the above-mentioned result may also be a new second eigenvalue sequence after adjustment. Then the corresponding task description should also be used to indicate the adjustment of the second eigenvalue sequence so that after the first eigenvalue in the second eigenvalue sequence is used to control the first test tool to test the system, the test value in the test result obtained does not exceed the preset range corresponding to the first test indicator.

[0092] In one specific embodiment, if CPU utilization exceeds 90%, the model might lower the cpu parameter of the stress-ng tool or adjust the vmbytes parameter to reduce memory pressure. This process involves the model quantifying the deviation of the current test results from the preset range and generating new parameter values ​​to modify subsequent stress strategies, thereby preventing sudden parameter changes that could lead to test failures or system crashes.

[0093] For example, after receiving the adjusted second eigenvalue, the agent executes the test on the system at the next time point. This process ensures that the test continues within a safe range through dynamic parameter adjustment, while verifying the stability of the system under the adjusted load.

[0094] As can be seen, by collecting system test results in real time and obtaining a first test value, which is then compared with a preset range, the first model is used to optimize the second eigenvalue when the first test value exceeds the preset range. This adjusted second eigenvalue is then applied in the subsequent stress phase to ensure that the second test value falls within the preset range. This real-time feedback mechanism and model-driven parameter adjustment not only avoids system crashes or resource exhaustion caused by improper initial parameter settings, but also reduces reliance on preset time series, making the test more closely aligned with the system's actual carrying capacity.

[0095] Furthermore, in a feasible embodiment, before performing the test based on the second eigenvalue sequence, the method further includes: determining resource configuration based on the second eigenvalue sequence, where the resource configuration satisfies the resources required for performing the test based on the second eigenvalue sequence.

[0096] Among them, the test based on the second eigenvalue sequence can be performed by controlling the test tool based on the second eigenvalue sequence. The test tool requires corresponding resources to run. In this case, the corresponding resource configuration can be determined in advance according to the second eigenvalue sequence.

[0097] Optionally, the test tool can be deployed on a container, which can be a container instance running the test tool. Resource allocation and scheduling are performed through the Kubernetes cluster, ensuring that resource quotas (including but not limited to the number of CPU cores, memory size, storage quota, etc.) meet the tool's operating requirements. Container resource configuration can be defined through Kubernetes' resource management mechanisms (such as the resources.requests and resources.limits fields), illustratively including specific parameters such as CPU quota, memory quota, and network bandwidth quota.

[0098] Specifically, parameter values ​​are mapped to resource requirements using a preset resource consumption model or historical data. Furthermore, the minimum resource quota is calculated based on the resource requirements corresponding to the parameter values. Furthermore, resource configuration is dynamically adjusted through the Kubernetes API or configuration files (such as Deployment YAML) to ensure that the container's resource requests and limits cover the peak demand of the parameter values. Finally, before applying pressure, the resource configuration is verified to be effective, including checking whether the container has been successfully started and whether the resource quota has been actually allocated to the node. If the configuration fails, an alarm is triggered or a node with sufficient resources is automatically selected to redeploy the container.

[0099] It can be seen that this method associates resource configuration with test parameters, thereby ensuring that the resource configuration can meet the subsequent test process and thus improving the stability of the test.

[0100] Furthermore, in another feasible embodiment, the method further includes: if it is detected that the response time to the acquisition instruction for the test result exceeds a preset threshold, stopping the test based on the second characteristic value sequence.

[0101] Among them, the response time can be obtained by the monitoring platform regularly sending data collection instructions during the test and recording the time difference. The preset threshold can be a criterion for determining the critical state of system performance, and can be set by the tester configuration or the system adaptive algorithm generation method. For example, the criterion includes but is not limited to numerical parameters such as 3 seconds and 5 seconds. Stopping the test based on the second eigenvalue sequence can be controlling the test tool to stop testing the system, such as terminating the test process and releasing resources, including but not limited to specific operations such as closing the Agent process, stopping the stress tool operation, and releasing resource occupation. When the response time is too long, there may be other crashes after the system is overloaded. At this time, it is necessary to stop the stress operation on the system.

[0102] Response time monitoring and collection can be achieved by having the monitoring platform continuously send data collection commands during each test phase and record the difference between sending and receiving times. Threshold determination and termination triggering can be implemented by comparing the collected response time with a preset threshold and incorporating judgment logic for multiple consecutive or single significant exceeding of the threshold. For example, the process can be terminated if the response time exceeds 3 seconds three times in a row or exceeds 10 seconds once. Pressure to terminate execution can be achieved by sending a stop command to the agent and performing a combination of process termination, resource release, and status recording.

[0103] It can be seen that by real-time monitoring of the response time of the test result collection instruction during the test process and dynamically comparing it with the preset threshold, the test based on the second eigenvalue sequence is immediately stopped when it is determined that the system performance has reached a critical state, thereby achieving the technical effect of enhancing the test security.

[0104] For example, see Figure 10 , Figure 10 This is a schematic diagram of the structure of a test tool provided in an embodiment of the present application. Figure 10 As shown, the test tools include stress-ng, fio, and iperf3. Among them, stress-ng is used to simulate computing resource pressure, fio is used to simulate storage resource pressure, and iperf3 is used to simulate network resource pressure. For example, the parameter control interface of stress-ng includes --cpu, --cpu-load, --pthread, etc., which are used to indicate the number of CPUs, CPU load, and the number of threads of the task respectively; the parameter control interface of fio includes --size, --rw, --bs, etc., which are used to indicate the read and write size, read and write type, and the size of each file block respectively; the parameter control interface of iperf3 includes --b, --t, --p, etc., which are used to indicate the target bandwidth size, execution time, and the number of concurrent clients respectively.

[0105] For example, see Figure 11 , Figure 11 This is a flow chart of a test tool control process provided by an embodiment of the present application. Figure 11 As shown, the agent accepts parameter values ​​from the large model in sequential order based on time series: [T1-1, T1-2, T1-3, …, T2-1, T2-2, …]. Based on n time series, n services (Service 1 through Service n) can be generated. For each service, the agent controls the testing tools stress-ng and fio. The parameter values ​​(Args) for stress-ng include: [T1-1, T1-2, …], and the parameter values ​​(Args) for fio include: [T2-1, T2-2, …].

[0106] In accordance with the above-mentioned embodiment, please refer to Figure 12 , Figure 12 This is a functional unit block diagram of a test device provided in an embodiment of the present application. The test device may be the above-mentioned terminal device or a part of the terminal device. Figure 12 As shown, the testing device 120 includes: The first processing unit 1201 is configured to determine a first characteristic value sequence corresponding to a first test indicator in a test requirement; The second processing unit 1202 is configured to obtain a first prompt word corresponding to the first feature value sequence based on the first feature value sequence; The second processing unit 1202 is further configured to input the first prompt word into the first model to obtain a second feature value sequence; The third processing unit 1203 is configured to perform a test based on the second eigenvalue sequence to obtain a test result.

[0107] In a feasible embodiment, in terms of obtaining the first prompt word corresponding to the first feature value sequence based on the first feature value sequence, the second processing unit 1202 is specifically configured to: Divide the first knowledge text into blocks based on the test parameters to obtain text blocks corresponding to the test parameters, wherein the first knowledge text includes explanatory text corresponding to the test parameters; Convert the text block into a vector to obtain a text vector; The first prompt word included in the text corresponding to the first feature value sequence is searched in the text vector.

[0108] In a feasible embodiment, in searching the text vector for the first prompt word included in the text corresponding to the first feature value sequence, the second processing unit 1202 is specifically configured to: Searching the text vector for a second knowledge text corresponding to the label of the test tool, where the label of the test tool corresponds to the first feature value sequence; The first prompt word is determined based on the label of the test tool, the second knowledge text, the first feature value sequence, the format of the second feature value sequence and the third knowledge text. The third knowledge text includes the format of the second feature value sequence and the meaning of the fields in the first feature value sequence.

[0109] In a feasible embodiment, the first eigenvalue sequence includes a first eigenvalue at a first time point and a second eigenvalue at a second time point. In terms of performing a test based on the second eigenvalue sequence, the third processing unit 1203 is specifically configured to: Testing based on the first eigenvalue; After the first preset time interval, a test is performed based on the second characteristic value.

[0110] In a feasible embodiment, after performing the test based on the first characteristic value, the third processing unit 1203 is further configured to: Collecting a first test result, where the first test result includes a first test value; If the first test value exceeds the preset range corresponding to the first test indicator, the first test value, the first eigenvalue sequence, the second eigenvalue sequence, the preset range, and the task description are input into the first model to obtain an adjusted second eigenvalue; The task description is used to indicate adjusting the second characteristic value so that after testing based on the second characteristic value, the second test value in the test result obtained does not exceed the preset range corresponding to the first test indicator.

[0111] In a feasible embodiment, before performing the test based on the second eigenvalue sequence, the third processing unit 1203 is further configured to: According to the second eigenvalue sequence, resource configuration is determined, where the resource configuration satisfies resources required for testing based on the second eigenvalue sequence.

[0112] In a feasible embodiment, the third processing unit 1203 is further configured to: If it is detected that the response time of the collection instruction for the test result exceeds a preset threshold, the test based on the second characteristic value sequence is stopped.

[0113] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0114] In the case of integrated units, such as Figure 13 As shown, Figure 13 This is a block diagram of the functional units of another test device provided in an embodiment of the present application. Figure 13 In the embodiment, the test device 120 includes: a processing module 1312 and a communication module 1311. The processing module 1312 is used to control and manage the actions of the test device 120, for example, the steps of the first processing unit 1201, the second processing unit 1202 and the third processing unit 1203, and / or other processes for executing the technology described herein. The communication module 1311 is used to support the interaction between the test device 120 and other devices. Figure 13 As shown, the testing device 120 may further include a storage module 1313 , and the storage module 1313 is used to store program codes and data of the testing device 120 .

[0115] The processing module 1312 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 1311 may be a transceiver, an RF circuit, or a communication interface, and the like. The storage module 1313 may be a memory.

[0116] Among them, all relevant contents of each scenario involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. Figure 2 Test method shown.

[0117] The above embodiments can be implemented in whole or in part via software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. A computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. A computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0118] Figure 14 This is a structural block diagram of an electronic device provided in an embodiment of the present application. Figure 14 As shown, the electronic device 1400 may include one or more of the following components: a processor 1401, a memory 1402 and a communication interface 1403. The processor 1401, the memory 1402 and the communication interface 1403 are interconnected and perform communication with each other. The memory 1402 may store one or more computer programs, and the one or more computer programs may be configured to implement the methods described in the above embodiments when executed by one or more processors 1401.

[0119] The processor 1401 may include one or more processing cores. The processor 1401 uses various interfaces and lines to connect the various parts of the entire electronic device 1400, and executes various functions and processes data of the electronic device 1400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1402, and calling data stored in the memory 1402. Optionally, the processor 1401 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1401 can integrate one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. It is understandable that the above-mentioned modem may not be integrated into the processor 1401 and may be implemented separately through a communication chip.

[0120] The memory 1402 may include a random access memory (RAM) or a read-only memory (ROM). The memory 1402 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1402 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the electronic device 1400 during use.

[0121] It is understandable that the electronic device 1400 may include more or fewer structural elements than those in the above structural block diagram, for example, including a power module, physical buttons, WiFi (Wireless Fidelity) module, speakers, Bluetooth modules, sensors, etc., which are not limited here.

[0122] The electronic device 1400 may be a terminal device or a part of a terminal device.

[0123] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements part or all of the steps of any one of the test methods described in the above method embodiments.

[0124] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements some or all of the steps of any of the test methods described in the above method embodiments. The computer program product may be a software installation package.

[0125] It should be noted that for the method embodiments of any of the aforementioned testing methods, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0126] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality of components or steps. The fact that certain measures are recited in different dependent claims does not mean that these measures cannot be combined to produce good results.

[0127] Those skilled in the art will appreciate that all or part of the steps in the various methods of the method embodiments of any of the above-mentioned testing methods can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0128] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of a test method, device, electronic device, and storage medium of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, based on the idea of ​​a test method, device, electronic device, and storage medium of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0132] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in the method embodiment of a testing method of the present application, such as the terminal and computer program product in the above flowchart, falls within the scope of the related products described in the present application.

[0133] Obviously, those skilled in the art may make various modifications and variations to the test method, apparatus, electronic device, and storage medium provided herein without departing from the spirit and scope of the present application. Thus, if such modifications and variations fall within the scope of the claims of the present application and their equivalents, the present application is intended to encompass such modifications and variations.

Claims

1. A testing method, characterized in that: The method comprises: Determine a first characteristic value sequence corresponding to a first test indicator in the test requirement; Obtaining a first prompt word corresponding to the first feature value sequence based on the first feature value sequence; Inputting the first prompt word into the first model to obtain a second eigenvalue sequence; A test is performed based on the second eigenvalue sequence to obtain a test result.

2. The method according to claim 1, characterized in that The obtaining, based on the first feature value sequence, a first prompt word corresponding to the first feature value sequence includes: Divide the first knowledge text into blocks based on the test parameters to obtain text blocks corresponding to the test parameters, wherein the first knowledge text includes explanatory text corresponding to the test parameters; Performing vector conversion on the text block to obtain a text vector; The first prompt word included in the text corresponding to the first feature value sequence is searched in the text vector.

3. The method according to claim 2, characterized in that The step of searching the text vector for the first prompt word included in the text corresponding to the first feature value sequence includes: Searching the text vector for a second knowledge text corresponding to a label of a test tool, where the label of the test tool corresponds to the first feature value sequence; The first prompt word is determined based on the label of the test tool, the second knowledge text, the first feature value sequence, the format of the second feature value sequence and the third knowledge text, wherein the third knowledge text includes the format of the second feature value sequence and the meaning of the fields in the first feature value sequence.

4. The method according to claim 1, wherein The first eigenvalue sequence includes a first eigenvalue at a first time point and a second eigenvalue at a second time point, and the testing based on the second eigenvalue sequence includes: Performing a test based on the first characteristic value; After a first preset time interval, a test is performed based on the second characteristic value.

5. The method according to claim 4, characterized in that After performing the test based on the first characteristic value, the method further includes: Collecting a first test result, where the first test result includes a first test value; If the first test value exceeds the preset range corresponding to the first test indicator, inputting the first test value, the first eigenvalue sequence, the second eigenvalue sequence, the preset range, and the task description into the first model to obtain an adjusted second eigenvalue; The task description is used to indicate adjusting the second characteristic value so that after testing based on the second characteristic value, the second test value in the test result obtained does not exceed the preset range corresponding to the first test indicator.

6. The method according to claim 1, characterized in that Before performing the test based on the second eigenvalue sequence, the method further includes: A resource configuration is determined according to the second eigenvalue sequence, where the resource configuration satisfies resources required for performing a test based on the second eigenvalue sequence.

7. The method according to claim 4 or 5, characterized in that The method further comprises: If it is detected that the response time of the collection instruction for the test result exceeds a preset threshold, the test based on the second eigenvalue sequence is stopped.

8. A testing device, characterized in that: The device comprises: A first processing unit, configured to determine a first characteristic value sequence corresponding to a first test indicator in a test requirement; a second processing unit, configured to obtain a first prompt word corresponding to the first feature value sequence based on the first feature value sequence; The second processing unit is further configured to input the first prompt word into the first model to obtain a second feature value sequence; The third processing unit is configured to perform a test based on the second eigenvalue sequence to obtain a test result.

9. An electronic device comprising a processor, a memory, and an executable program code stored in the memory, wherein: The processor is configured to retrieve the executable program code stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images