Automatic testing method and device, computer equipment and storage medium

Abstract

The invention relates to an automatic testing method and device, computer equipment and a storage medium. The method comprises the steps of generating a test task according to a test requirement of the shopping guide model, wherein the test task comprises a verification type; obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model by using the test cases to obtain a test result corresponding to each test case; receiving a test result returned by the shopping guide model, and performing data processing and analysis on the test result; and determining a verification rule according to the verification type corresponding to the test case, and verifying a test result corresponding to the test case based on the verification rule to obtain a verification result. By adopting the method, the uncertainty output of the model can be verified through abundant verification modes so as to verify the system function, and guarantee is provided for rapid verification and rapid iteration of the system.

Description

Technical Field

[0001] The present application relates to the field of testing technology, and in particular to an automated testing method, apparatus, computer equipment and storage medium. Background Art

[0002] As the Internet e-commerce industry continues to develop, the scenarios for guiding users to shop through model chat services are becoming more and more diverse. As an artificial intelligence application, the shopping guide model is widely used in e-commerce platforms, virtual assistants and other scenarios. When consumers chat with models, they will have some purchase intentions. By simulating the role of human sales consultants, the shopping guide model can provide users with personalized product recommendations and consulting services.

[0003] The iteration of the model shopping guide service function often affects many functions. At this time, the verification of related functions becomes a time-consuming task, especially under the real-time update of massive commodity data and changes in user needs. It is crucial to ensure that the model accurately understands and responds to user intentions. In order to save manpower and quickly verify whether the function is normal, a set of automated testing methods is needed to facilitate the rapid verification of service functions and online service. Summary of the invention

[0004] Based on this, it is necessary to provide an automated testing method, device, computer equipment and storage medium that can quickly verify the model to address the above technical problems.

[0005] On the one hand, an automated testing method is provided, which is applied to a shopping guide model, and the method comprises:

[0006] Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0007] Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0008] Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0009] A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

[0010] In one embodiment, before obtaining a preset number of test cases from a case library according to the verification type of the test task, the method includes:

[0011] Acquire a data set to be tested according to the test requirements of the shopping guide model;

[0012] Test cases are configured in batches according to the data set to be tested, and a case library is constructed based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

[0013] In one embodiment, the test case further includes a message type and input information, and the test task is asynchronously executed on the shopping guide model using the test case to obtain a test result corresponding to each test case, including:

[0014] According to the message type and input information of the test case, simulate the user interaction behavior to send a message to the shopping guide model to obtain the test result corresponding to each test case;

[0015] The receiving the test result returned by the shopping guide model and performing data processing and analysis on the test result includes:

[0016] According to the verification type corresponding to the test case, feature information in the test result is extracted through regular expressions. The verification types include content verification, product verification, brand verification, security verification, bottom-up verification and general verification. The feature information includes one or more of product ID, brand ID and model reply message.

[0017] In one embodiment, the verification type includes a verification subtype, and the verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain the verification result, including:

[0018] Determine a verification rule according to the verification subtype corresponding to the test case;

[0019] Calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case;

[0020] In response to the feature information matching the verification content, it is determined that the test case verification has passed.

[0021] In one embodiment, calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case includes:

[0022] In response to the verification type being commodity verification, determining whether there is a commodity ID matching the verification type in the feature information of the test result;

[0023] In response to the product ID existing in the feature information, calling a first verification model according to the verification subtype, and obtaining product information corresponding to the product ID, the product information including product category, product selling point and product material;

[0024] Determining whether the product information matches the verification content in the test case according to the first verification model;

[0025] In response to the commodity ID not existing in the feature information, or the commodity information does not match the verification content, it is determined that the test case verification fails.

[0026] In one embodiment, calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case, further includes:

[0027] In response to the verification type being one of content verification, general verification, security verification and fallback verification, determining whether there is a model reply message in the test result;

[0028] In response to the presence of the model reply message in the test result, calling a second verification model according to the verification subtype, and splicing the model reply message and the input message of the test case together and inputting them into the second verification model;

[0029] Determining whether the model reply message matches the input message of the test case through the second verification model;

[0030] In response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, it is determined that the test case verification fails.

[0031] In one embodiment, the method further comprises:

[0032] Counting the pass rate of the test case of the test task according to the verification result of each test case;

[0033] In response to the use case pass rate being less than a preset threshold, an email notification is sent.

[0034] On the other hand, an automated testing device is provided, which is applied to a shopping guide model, and the device comprises:

[0035] A generating module, used for generating a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0036] An execution module, used to obtain a preset number of test cases from a case library according to the verification type of the test task, and asynchronously execute the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0037] An analysis module, used to receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0038] The verification module is used to determine the verification rules according to the verification type corresponding to the test case, and verify the test result corresponding to the test case based on the verification rules to obtain the verification result.

[0039] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0040] Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0041] Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0042] Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0043] A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

[0044] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0046] Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0047] Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0048] A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

[0049] The above-mentioned automated testing methods, devices, computer equipment and storage media can reasonably verify the uncertainty output of the model through customized requests and rich verification methods, and verify the system functions more efficiently; and can execute multiple use cases and timed automatic tasks at the same time, which is much more efficient than manual testing, and provides a guarantee for rapid verification and rapid iteration of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of an automated testing method in one embodiment;

[0051] Figure 2 A schematic diagram of a flow chart of automated testing steps in an embodiment;

[0052] Figure 3 is a structural block diagram of an automated testing device in one embodiment;

[0053] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] In one embodiment, Figure 1-2 As shown, an automated testing method is provided, comprising the following steps:

[0056] Step S1, generating a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type.

[0057] Specifically, the test tasks can be configured through the user interface or program interface, such as Figure 2As shown in the figure, by configuring the task name, verification type, verification subtype, number of execution cases, environment, and task type, you can add a new test task. Among them, the task name is used to describe the test task for easy identification and traceability; the verification type includes content verification, product verification, brand verification, security verification, bottom-line verification, and general verification, which are used to determine the core verification areas of the test case, covering the performance of the shopping guide model in terms of content accuracy, product information consistency, brand matching, security compliance, bottom-line strategy execution, and general functions; the verification subtype is a further refinement of the verification type, which ensures targeted verification of the model in different business scenarios. For example, under the classification of product verification, it can be subdivided into verification of different dimensions such as product recommendation, product selling point, and product material information; the number of execution cases is used to set the number of test cases obtained for each verification type to control the test scope and execution efficiency; the environment selection is used to determine whether the test will be executed in the pre-release environment or the production environment to ensure the correct performance of the system in the corresponding environment; the task type is used to distinguish between end-to-end full-process testing or only calling tests for specific interfaces.

[0058] Step S2, obtaining a preset number of test cases from a case library according to the verification type of the test task, and using the test cases to asynchronously execute the test task on the shopping guide model to obtain a test result corresponding to each test case.

[0059] In an optional implementation manner, before obtaining a preset number of test cases from a case library according to the verification type of the test task, the method includes:

[0060] Acquire a data set to be tested according to the test requirements of the shopping guide model;

[0061] Test cases are configured in batches according to the data set to be tested, and a case library is constructed based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

[0062] Among them, test cases can be created manually, for example, by uploading an Excel spreadsheet to batch import test cases, or test cases can be automatically generated according to the data to be tested. This application does not specifically limit the method of configuring test cases.

[0063] like Figure 3As shown in the figure, by configuring the use case name, verification type, verification subtype, message type, input information, and verification content, you can add new test cases. Among them, the verification types include content verification, product verification, brand verification, safety verification, bottom-line verification, general verification, etc.; the verification subtypes include product recommendations, product selling points, product materials, etc.; the message types include txt text and instructions (such as button click events); the input information is the text or instruction code information filled in according to the message type; the verification content is filled in according to the verification subtype, which is used to verify whether the output of the shopping guide model is consistent with expectations.

[0064] In an optional implementation, the test case further includes a message type and input information, and the test task is asynchronously executed on the shopping guide model using the test case to obtain a test result corresponding to each test case, including:

[0065] According to the message type and input information of the test case, simulate the user interaction behavior to send a message to the shopping guide model to obtain the test result corresponding to each test case;

[0066] The receiving the test result returned by the shopping guide model and performing data processing and analysis on the test result includes:

[0067] According to the verification type corresponding to the test case, feature information in the test result is extracted through regular expressions. The verification types include content verification, product verification, brand verification, security verification, bottom-up verification and general verification. The feature information includes one or more of product ID, brand ID and model reply message.

[0068] Among them, key feature information is extracted from the complex test results returned by the shopping guide model through regular expressions. These feature information are used to compare with the expected verification content to judge the test results of the model, including the specific description information of the product (such as product ID, name, price, material, etc.), relevant attributes of the brand (such as brand ID, brand name, brand positioning, etc.) and the intelligent reply message content given by the shopping guide model.

[0069] Step S3, receiving the test results returned by the shopping guide model, and performing data processing and analysis on the test results.

[0070] It should be noted that in this embodiment, the test task is executed asynchronously, and the corresponding result can be seen after each test case is executed, and the report details can also be viewed.

[0071] Step S4, determining a verification rule according to the verification type corresponding to the test case, and verifying the test result corresponding to the test case based on the verification rule to obtain a verification result.

[0072] In an optional implementation, the verification type includes a verification subtype, and determining a verification rule according to the verification type corresponding to the test case, and verifying the test result corresponding to the test case based on the verification rule to obtain the verification result includes:

[0073] Determine a verification rule according to the verification subtype corresponding to the test case;

[0074] Calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case;

[0075] In response to the feature information matching the verification content, it is determined that the test case verification has passed.

[0076] It should be noted that each test case predefines its corresponding verification content. For example, in the product verification scenario, when the verification subtype is product recommendation verification, the corresponding verification content may be a specific product category information, which can be represented by the category number or product ID; and when the verification subtype is product selling point verification, its verification content may be a specific product detailed description; similarly, for the brand verification scenario, the verification content corresponding to the test case may be a set of characteristic attributes of the brand.

[0077] In an optional implementation, calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case includes:

[0078] In response to the verification type being commodity verification, determining whether there is a commodity ID matching the verification type in the feature information of the test result;

[0079] In response to the product ID existing in the feature information, calling a first verification model according to the verification subtype, and obtaining product information corresponding to the product ID, the product information including product category, product selling point and product material;

[0080] Determining whether the product information matches the verification content in the test case according to the first verification model;

[0081] In response to the commodity ID not existing in the feature information, or the commodity information does not match the verification content, it is determined that the test case verification fails.

[0082] Through customized test cases and rich verification methods, the uncertain output of the shopping guide model can be reasonably verified to determine whether the responses given by the shopping guide model meet expectations, thereby verifying the system functions more efficiently.

[0083] In an optional implementation, calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case, further includes:

[0084] In response to the verification type being one of content verification, general verification, security verification and fallback verification, determining whether there is a model reply message in the test result;

[0085] In response to the presence of the model reply message in the test result, calling a second verification model according to the verification subtype, and splicing the model reply message and the input message of the test case together and inputting them into the second verification model;

[0086] Determining whether the model reply message matches the input message of the test case through the second verification model;

[0087] In response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, it is determined that the test case verification fails.

[0088] Specifically, for general verification, security verification, and bottom-line verification, the shopping guide model will identify user questions and output corresponding dialogues, so the dialogues need to be verified. For example, when users ask questions involving sensitive information or potential risks, the system needs to effectively identify and reasonably intercept them. For user questions that are judged to have security risks, the system needs to verify whether they can automatically reply to the corresponding bottom-line dialogues according to the preset security policies to ensure that no sensitive information is leaked, and provide users with appropriate prompts and guidance.

[0089] For example, in a general verification scenario, for external general knowledge questions and answers, such as the question "Which poem is the one with the bright moonlight on my bed?", since the reply of the shopping guide model is uncertain, in order to verify the accuracy of the reply, it is necessary to call a second verification model (a mature model, such as GPT4), assemble the question sentence and the reply message of the shopping guide model into a template, and ask the question in the second verification model to determine whether the reply message of the shopping guide model meets the question.

[0090] For internal knowledge questions and answers, the text similarity of the model's reply message and the expected result can be judged through the similarity distance method. For example, the edit distance method is used to measure the difference between the two. The edit distance refers to the minimum number of editing operations required to convert one string into another. These operations include inserting a character, deleting a character, and replacing a character. The smaller the edit distance value, the closer the two are. If it is greater than 0.9 (configurable), the reply text is considered correct.

[0091] In an optional embodiment, the method further comprises:

[0092] Counting the pass rate of the test case of the test task according to the verification result of each test case;

[0093] In response to the use case pass rate being less than a preset threshold, an email notification is sent.

[0094] like Figure 4 As shown, each time a test case is executed, a corresponding test result is generated, which includes the test case information, the test result and the final execution status; according to the test result, the pass rate of the test task is counted; the pass rate of the test task is compared with the preset threshold, and if the pass rate of the test task is less than the preset threshold, an email notification is sent.

[0095] It should be understood that although Figure 1-2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0096] In one embodiment, Figure 3 As shown, an automated testing device is provided, including: a generation module, an execution module, an analysis module and a verification module, wherein:

[0097] A generating module, used for generating a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0098] An execution module, used to obtain a preset number of test cases from a case library according to the verification type of the test task, and asynchronously execute the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0099] An analysis module, used to receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0100] The verification module is used to determine the verification rules according to the verification type corresponding to the test case, and verify the test result corresponding to the test case based on the verification rules to obtain the verification result.

[0101] In one embodiment, the generation module is also used to obtain a data set to be tested according to the test requirements of the shopping guide model; batch configure test cases according to the data set to be tested, and build a case library based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

[0102] In one embodiment, the execution module is also used to simulate user interaction behavior and send messages to the shopping guide model according to the message type and input information of the test case to obtain the test results corresponding to each test case; the analysis module is also used to extract feature information in the test results through regular expressions according to the verification type corresponding to the test case, and the verification types include content verification, product verification, brand verification, security verification, bottom-up verification and general verification, and the feature information includes one or more of product ID, brand ID and model reply message.

[0103] In one embodiment, the verification module is also used to determine the verification rules according to the verification sub-type corresponding to the test case; call the corresponding verification model according to the verification rules, use the verification model to verify the test results corresponding to the test case, and determine whether the feature information in the test results matches the verification content of the test case; in response to the feature information matching the verification content, determine that the test case verification has passed.

[0104] In one embodiment, the verification module is also used to, in response to the verification type being product verification, determine whether there is a product ID matching the verification type in the feature information of the test result; in response to the presence of the product ID in the feature information, call the first verification model according to the verification subtype, and obtain the product information corresponding to the product ID, the product information including product category, product selling point and product material; determine whether the product information matches the verification content in the test case according to the first verification model; in response to the absence of the product ID in the feature information, or the product information does not match the verification content, determine that the test case verification has failed.

[0105] In one embodiment, the verification module is also used to determine whether there is a model reply message in the test result in response to the verification type being one of content verification, general verification, security verification and fallback verification; in response to the existence of the model reply message in the test result, call a second verification model according to the verification subtype, and splice the model reply message and the input message of the test case together and input them into the second verification model; determine whether the model reply message matches the input message of the test case through the second verification model; in response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, determine that the test case verification has failed.

[0106] In one embodiment, it also includes a statistical module, which is used to count the case pass rate of the test task according to the verification result of each test case; in response to the case pass rate being less than a preset threshold, an email notification is sent.

[0107] For the specific definition of the automated testing device, please refer to the definition of the automated testing method above, which will not be repeated here. Each module in the above-mentioned automated testing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0108] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store test case data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an automated testing method is implemented.

[0109] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0111] Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0112] Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0113] Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0114] A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a data set to be tested according to the test requirements of the shopping guide model; configuring test cases in batches according to the data set to be tested, and building a case library based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

[0116] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the message type and input information of the test case, the user interaction behavior is simulated to send a message to the shopping guide model to obtain the test result corresponding to each test case; according to the verification type corresponding to the test case, the feature information in the test result is extracted through a regular expression, and the verification type includes content verification, product verification, brand verification, security verification, bottom-up verification and general verification, and the feature information includes one or more of product ID, brand ID and model reply message.

[0117] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining a verification rule according to the verification subtype corresponding to the test case; calling a corresponding verification model according to the verification rule, and using the verification model to verify the test result corresponding to the test case, and determining whether the feature information in the test result matches the verification content of the test case; in response to the feature information matching the verification content, determining that the test case verification has passed.

[0118] In one embodiment, the processor also implements the following steps when executing the computer program: in response to the verification type being product verification, determining whether there is a product ID matching the verification type in the feature information of the test result; in response to the presence of the product ID in the feature information, calling a first verification model according to the verification subtype, and obtaining product information corresponding to the product ID, wherein the product information includes product categories, product selling points, and product materials; determining whether the product information matches the verification content in the test case according to the first verification model; in response to the absence of the product ID in the feature information, or the product information does not match the verification content, determining that the test case verification has failed.

[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the verification type being one of content verification, general verification, security verification and fallback verification, determining whether there is a model reply message in the test result; in response to the existence of the model reply message in the test result, calling a second verification model according to the verification subtype, and splicing the model reply message and the input message of the test case together and inputting them into the second verification model; determining whether the model reply message matches the input message of the test case through the second verification model; in response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, determining that the test case verification has failed.

[0120] In one embodiment, when the processor executes the computer program, the following steps are also implemented: counting the case pass rate of the test task according to the verification result of each test case; and sending an email notification in response to the case pass rate being less than a preset threshold.

[0121] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0122] Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type;

[0123] Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case;

[0124] Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results;

[0125] A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining a data set to be tested according to the test requirements of the shopping guide model; configuring test cases in batches according to the data set to be tested, and building a case library based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

[0127] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the message type and input information of the test case, the user interaction behavior is simulated to send a message to the shopping guide model to obtain the test result corresponding to each test case; according to the verification type corresponding to the test case, the feature information in the test result is extracted by regular expression, and the verification type includes content verification, product verification, brand verification, security verification, bottom-up verification and general verification, and the feature information includes one or more of product ID, brand ID and model reply message.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining a verification rule according to the verification subtype corresponding to the test case; calling a corresponding verification model according to the verification rule, using the verification model to verify the test result corresponding to the test case, and determining whether the feature information in the test result matches the verification content of the test case; in response to the feature information matching the verification content, determining that the test case verification has passed.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the verification type being product verification, determining whether there is a product ID matching the verification type in the feature information of the test result; in response to the presence of the product ID in the feature information, calling a first verification model according to the verification subtype, and obtaining product information corresponding to the product ID, wherein the product information includes product categories, product selling points, and product materials; determining whether the product information matches the verification content in the test case according to the first verification model; in response to the absence of the product ID in the feature information, or the product information does not match the verification content, determining that the test case verification has failed.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the verification type being one of content verification, general verification, security verification and fallback verification, determining whether there is a model reply message in the test result; in response to the existence of the model reply message in the test result, calling a second verification model according to the verification subtype, and splicing the model reply message and the input message of the test case together and inputting them into the second verification model; determining whether the model reply message matches the input message of the test case through the second verification model; in response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, determining that the test case verification has failed.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: counting the case pass rate of the test task according to the verification result of each test case; and sending an email notification in response to the case pass rate being less than a preset threshold.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0134] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An automated testing method, applied to a shopping guide model, characterized in that: include: Generate a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type; Obtaining a preset number of test cases from a case library according to the verification type of the test task, and asynchronously executing the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case; Receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results; A verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain a verification result.

2. The automated testing method according to claim 1, characterized in that: Before obtaining a preset number of test cases from a case library according to the verification type of the test task, the method includes: Acquire a data set to be tested according to the test requirements of the shopping guide model; Test cases are configured in batches according to the data set to be tested, and a case library is constructed based on the test cases, wherein the test cases include a verification type and a verification content, the verification type of the test case corresponds to the test task, and the verification content includes expected output content or verification standards.

3. The automated testing method according to claim 1 or 2, characterized in that: The test case also includes a message type and input information. The test task is asynchronously executed on the shopping guide model using the test case to obtain a test result corresponding to each test case, including: According to the message type and input information of the test case, simulate the user interaction behavior to send a message to the shopping guide model to obtain the test result corresponding to each test case; The receiving the test result returned by the shopping guide model and performing data processing and analysis on the test result includes: According to the verification type corresponding to the test case, feature information in the test result is extracted through regular expressions. The verification types include content verification, product verification, brand verification, security verification, bottom-up verification and general verification. The feature information includes one or more of product ID, brand ID and model reply message.

4. The automated testing method according to claim 3, characterized in that: The verification type includes a verification subtype, and the verification rule is determined according to the verification type corresponding to the test case, and the test result corresponding to the test case is verified based on the verification rule to obtain the verification result, including: Determine a verification rule according to the verification subtype corresponding to the test case; Calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case; In response to the feature information matching the verification content, it is determined that the test case verification has passed.

5. The automated testing method according to claim 4, characterized in that: The calling of the corresponding verification model according to the verification rule, verifying the test result corresponding to the test case by using the verification model, and judging whether the feature information in the test result matches the verification content of the test case, includes: In response to the verification type being commodity verification, determining whether there is a commodity ID matching the verification type in the feature information of the test result; In response to the product ID existing in the feature information, calling a first verification model according to the verification subtype, and obtaining product information corresponding to the product ID, the product information including product category, product selling point and product material; Determining whether the product information matches the verification content in the test case according to the first verification model; In response to the commodity ID not existing in the feature information, or the commodity information does not match the verification content, it is determined that the test case verification fails.

6. The automated testing method according to claim 4, characterized in that: The step of calling a corresponding verification model according to the verification rule, verifying the test result corresponding to the test case using the verification model, and determining whether the feature information in the test result matches the verification content of the test case, further includes: In response to the verification type being one of content verification, general verification, security verification and fallback verification, determining whether there is a model reply message in the test result; In response to the presence of the model reply message in the test result, calling a second verification model according to the verification subtype, and splicing the model reply message and the input message of the test case together and inputting them into the second verification model; Determining whether the model reply message matches the input message of the test case through the second verification model; In response to the absence of the model reply message in the test result, or the model reply message does not match the input message of the test case, it is determined that the test case verification fails.

7. The automated testing method according to claim 1, characterized in that: The method further comprises: Counting the pass rate of the test cases of the test task according to the verification results of each test case; In response to the use case pass rate being less than a preset threshold, an email notification is sent.

8. An automated testing device, applied to a shopping guide model, characterized in that: The device comprises: A generating module, used for generating a test task according to the test requirements of the shopping guide model, wherein the test task includes a verification type; An execution module, used to obtain a preset number of test cases from a case library according to the verification type of the test task, and asynchronously execute the test task on the shopping guide model using the test cases to obtain a test result corresponding to each test case; An analysis module, used to receive the test results returned by the shopping guide model, and perform data processing and analysis on the test results; The verification module is used to determine the verification rules according to the verification type corresponding to the test case, and verify the test result corresponding to the test case based on the verification rules to obtain the verification result.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images