Test method and device, storage medium and computer equipment

Abstract

The invention discloses a test method and device, a storage medium and computer equipment, and the method comprises the steps: obtaining a use scene of a to-be-tested product in response to a test instruction; based on the sentence components, intercepting a plurality of first keywords from a preset test corpus associated with the use scene; randomly combining the plurality of first keywords based on the grammatical structure to generate a target test corpus; transmitting the target test corpus to the to-be-tested product; and determining a performance test result of the to-be-tested product based on the feedback response information and / or the feedback corpus of the to-be-tested product. According to the method provided by the invention, the new test corpus can be randomly spliced through the vocabularies of the existing preset test corpus for testing, so that the unpredictability of the test is increased, and the performance of the to-be-tested product under unexpected input can be found; and the test system can execute specific test tasks in different scenes without frequently modifying the test script and the automatic framework, so that the generalization ability of the test is improved, and the comprehensiveness of the test is ensured.

Description

Technical Field

[0001] This application relates to the technical field of Q&A product testing, and in particular, to a testing method, device, storage medium, and computer device. Background Art

[0002] With the development of AI technology, more and more voice assistants have emerged, including those for children's learning, customer service, chat companions, etc. A variety of assistant forms have spawned numerous products. In related technologies, when testing a voice assistant, AI can only target the Q&A ability in a certain scenario, making the testing of multi-functional voice assistants insufficiently comprehensive. Summary of the Invention

[0003] In view of this, this application provides a testing method, device, storage medium, and computer device to improve the generalization ability of product testing.

[0004] According to the first aspect of this application, a testing method is provided, including:

[0005] Responding to a test instruction to obtain the usage scenario of the product under test;

[0006] Based on sentence components, intercepting multiple first keywords from the preset test corpus associated with the usage scenario;

[0007] Randomly combining multiple of the first keywords based on the grammatical structure to generate a target test corpus;

[0008] Transmitting the target test corpus to the product under test;

[0009] Based on the feedback response information and / or feedback corpus of the product under test, determining the performance test result of the product under test.

[0010] According to the second aspect of this application, a testing device is provided, including:

[0011] An acquisition module for responding to a test instruction to obtain the usage scenario of the product under test;

[0012] A corpus generation module for intercepting multiple first keywords from the preset test corpus associated with the usage scenario based on sentence components; and randomly combining multiple of the first keywords based on the grammatical structure to generate a target test corpus;

[0013] A data transmission module for transmitting the target test corpus to the product under test;

[0014] A testing module for determining the performance test result of the product under test based on the feedback response information and / or feedback corpus of the product under test.

[0015] According to the third aspect of the present application, there is provided a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the above-mentioned test method are implemented.

[0016] According to the fourth aspect of the present application, there is provided a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned test method are implemented.

[0017] By means of the above technical solutions, on the one hand, by randomly splicing the vocabulary of the existing preset test corpus to generate a new test corpus for testing, it not only increases the unpredictability of the test, helps to discover the performance of the product under test under unexpected inputs, but also makes up for the problem of insufficient training data required for testing and improves the accuracy of the test. On the other hand, using the usage scenarios of the product under test as the basis for screening the test corpus, enabling the test corpus to cover different usage scenarios, so that the test system can execute specific test tasks in different scenarios without frequently modifying the test script and the automation framework, further saving test resources, enhancing the generalization ability of the test, and ensuring the comprehensiveness of the test.

[0018] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented in accordance with the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are hereinafter specifically exemplified. Description of the Drawings

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0020] Figure 1 A flowchart showing the test method provided by an embodiment of the present application is shown;

[0021] Figure 2 An interaction diagram showing the test method provided by an embodiment of the present application is shown;

[0022] Figure 3 A structural block diagram showing the test device provided by an embodiment of the present application is shown;

[0023] Figure 4 An electronic structure diagram showing the computer device provided by an embodiment of the present application is shown. Detailed Embodiments

[0024] The present application will be described in detail below with reference to the drawings and in combination with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0025] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary only for explaining the present application and should not be construed as limiting the present application.

[0026] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "joined" to another element, it can be directly connected or joined to other elements, or there may also be intermediate elements. In addition, the "connection" or "joining" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more of the associated listed items.

[0027] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many different forms and should not be construed as being limited only to the embodiments set forth herein. It should be understood that these embodiments are provided so that the disclosure of the present application is thorough and complete, and the concept of these exemplary embodiments is fully conveyed to those of ordinary skill in the art.

[0028] In this embodiment, a testing method is provided, which is applied to a testing system, as Figure 1 shown. The method includes:

[0029] Step 101, in response to a test instruction, obtain the usage scenario of the product under test.

[0030] Wherein, the product under test is an application program or device with a question-and-answer function, and the interaction form of the product under test can adopt a text output form, a voice output form, etc.

[0031] Specifically, the usage scenarios of the product to be tested include, but are not limited to, daily life management, application control, information query, emotional companionship, health guidance, and learning assistance. When the product to be tested is applied to daily life management, it can give timed reminders according to the schedule set by the user; when the product to be tested is applied to emotional companionship, it can recognize the user's emotions and provide appropriate comfort to help the user express themselves; when the product to be tested is applied to learning assistance, it can load students' courses to help students explain new words and complex sentences; when the product to be tested is applied to application control, it can perform various device operations, such as turning up the volume, turning on the monitor, etc. When the product to be tested is applied to information query, it can query historical events, scientific knowledge, person information, news, etc. according to the user's needs. When the product to be tested is applied to health guidance, it can monitor health data such as the user's heart rate, steps, and sleep quality, and remind the user to take medicine or query drug information.

[0032] It can be understood that different products to be tested may have multiple usage scenarios according to their configuration requirements. For example, a voice prompt assistant can be used for both information query and application control. At this time, the voice prompt assistant is loaded with voice packs for multiple different usage scenarios to meet the prompt requirements in different scenarios.

[0033] The test method provided by the embodiments of the present application can be applied to a terminal, a server, or software running on a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the intention recognition method, etc., but is not limited to the above forms.

[0034] In an actual application scenario, step 101, that is, obtaining the usage scenario of the product to be tested, can be implemented in the following ways:

[0035] Method 1: Receive the historical corpus sent by the product to be tested; perform word segmentation on the historical corpus to determine the third keyword; determine the usage scenario to which the identifier word matching the third keyword belongs as the usage scenario of the product to be tested.

[0036] In this embodiment, the historical corpus of the product to be tested is collected to simulate the actual usage of the user. By matching the word segmentation in the historical corpus with the usage scenario identifier words, the usage scenario can be quickly and accurately located, reducing the workload of manual analysis, accurately positioning the test requirements, and greatly improving the test efficiency.

[0037] In Method 2, the test instructions are parsed to determine the test requirement information; the usage scenarios of the products to be tested in the test requirement information are identified.

[0038] In this embodiment, the usage scenarios of the products to be tested are obtained from the test requirement information input by the user, so as to perform test tasks targeted, ensure that the tests cover the core functions and user requirements, and improve the test efficiency.

[0039] Step 102: Based on the sentence components, intercept multiple first keywords from the preset test corpus associated with the usage scenario.

[0040] Among them, the sentence components are the basic parts that make up a sentence. For example, the subject, predicate, object, attributive, adverbial, and complement. The number of the preset test corpus associated with the usage scenario is not specifically limited in this application.

[0041] It should be noted that before the test, a large number of preset test corpora are collected for each usage scenario and stored in the database according to the usage scenarios.

[0042] In this embodiment, after determining the preset test corpus associated with the usage scenario, the preset test corpus is truncated based on the sentence components to form multiple words. The words belonging to the same sentence component are used to form a vocabulary set of this sentence component. Finally, multiple first keywords are selected from the sets of different components according to the configuration requirements of the target test corpus. Thus, the subsequent generation of the target test corpus can conform to the normal language logic, enhance the fit between the target test corpus and the actual application environment, help the products to be tested better understand and analyze the target test corpus, contribute to reducing the error rate of the products to be tested in the actual application, and further reduce the test error and improve the accuracy of product testing.

[0043] Among them, the configuration requirements of the target test corpus can be reasonably set according to the test requirements. For example, the configuration requirements of the target test corpus are set to 2 interrogative sentences of 10 - 20 characters and 1 declarative sentence of about 20 characters.

[0044] It is worth mentioning that after establishing the vocabulary set of the sentence components, the words in the set are de-duplicated, so as to avoid generating irrelevant or duplicate target test corpora and improve the test efficiency.

[0045] Furthermore, obtaining the preset test corpus specifically includes: configuring a corpus retrieval instruction based on different usage scenarios and test adjustment parameters; inputting the corpus retrieval instruction into the natural language processing model to obtain the preset test corpus associated with different usage scenarios and their corresponding preset response corpora.

[0046] Among them, the test adjustment parameters include the corpus generation length, difficulty level, and new character range.

[0047] In this embodiment, conditional on the test adjustment parameters, the natural language processing model automatically mines industry knowledge and generates more standard phrases in different usage scenarios, and a number of paired preset test corpora and preset response corpora are established. Thus, a large number of standard corpus pairs in different usage scenarios can be quickly generated, which not only reduces the workload of manual writing, but also the preset test corpora and preset response corpora can better handle complex and changing actual usage scenarios, enabling the test system to support the question-and-answer performance test of multiple usage scenarios.

[0048] Specifically, the natural language processing model (Large Language Models, LLMs) can adopt models such as the GPT model (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers) model, LLaMA model (Large Language Model MetaAI), BLOOM model (BigScience Large Open-science Open-access Multilingual Language Model), etc.

[0049] It can be understood that when storing the preset test corpus, the preset test corpus can be decomposed by a word segmentation model to obtain the sentence components represented by each word segment in the preset test corpus, so as to facilitate the subsequent extraction of the first keyword.

[0050] Step 103, randomly combine multiple first keywords based on the grammatical structure to generate the target test corpus.

[0051] Among them, the grammatical structure refers to the combination method of words and phrases in a sentence and the grammatical relationship between them, including word order, part of speech, sentence components, phrase structure, etc.

[0052] In this embodiment, by randomly combining the first keywords of different sentence components, rich and diverse and unconventional target test corpora are created, which to a certain extent simulates the language generation process in this real language environment, enabling the target test corpus to comprehensively cover all possible situations of the language, and can expose the product under test to more novel situations, preventing the product under test from simply memorizing specific test methods, thereby improving the generalization ability of the test system and accurately testing the product under test's understanding and application ability of language grammar rules, enabling it to better handle various complex and changing language scenarios in the real world.

[0053] It is worth mentioning that for the requirements of simple-structured sentences, multiple first keywords can be concatenated in sequence according to the grammatical structure; multiple fixed grammatical structure templates can also be set, and the first keywords are brought into the templates to form sentences. For the requirements of complex-structured sentences, the first keywords can be used as inputs to let the language model generate sentences containing these keywords;

[0054] Exemplarily, the preset test corpus is "Please help me turn on the voice function of the learning machine" and "Help me find the courses for the second class of third-grade Chinese". The adverbial set formed after truncation includes: Please; the predicate set includes: Help me; the verb set includes: turn on, find; the object set includes: learning machine, voice function, Chinese, courses; the attributive set includes: third grade, second class. Taking the target test corpus of requirement 1 sentence as an example, one word is selected from each sentence component set as the first keyword (adverbial: Please; predicate: Help me; verb: turn on; object: courses; attributive: second class). Combining multiple first keywords to generate the target test corpus can be "Please help me turn on the courses", "Turn on the content of the second class", etc.

[0055] Step 104, transmit the target test corpus to the product under test.

[0056] It can be understood that if the product under test conducts interactions in text form, the test system can establish a communication connection with the product under test, send the target test corpus to the product under test through a chat conversation, and after the product under test analyzes the target test corpus in text form, obtain the corresponding feedback corpus, and the product under test uses the established communication channel to return the feedback corpus to the test system through a chat conversation. If the product under test conducts interactions in voice form, the target test corpus is broadcast through a voice player, and similarly, the product under test broadcasts the feedback corpus in voice form.

[0057] Step 105, determine the performance test result of the product under test based on the feedback response information and / or feedback corpus of the product under test.

[0058] The test method provided by the embodiments of the present application, on the one hand, randomly splices new test corpora through the existing vocabulary of the preset test corpus for testing, which not only increases the unpredictability of the test, helps to discover the performance of the product under test under unexpected inputs, but also avoids using a large amount of irrelevant or redundant corpora to establish a database, reduces the training data volume, makes up for the problem of insufficient training data for test requirements, and improves the accuracy of the test. On the other hand, taking the usage scenarios of the product under test as the basis for screening the test corpus, enabling the test corpus to cover different usage scenarios, so that the test system can execute specific test tasks in different scenarios without frequently modifying the test script and automation framework, further saving test resources, enhancing the generalization ability of the test, and ensuring the comprehensiveness of the test.

[0059] In an actual application scenario, in step 105, based on the feedback response information of the product under test, the performance test result of the product under test is determined, which specifically includes the following steps:

[0060] Step 151-1: Extract the detection values of different parameter items in the feedback response information.

[0061] Among them, the parameter items include: the response duration for indicating when the product under test sends out the feedback corpus, the response status for indicating whether the product under test sends out the feedback corpus, the feedback duration for indicating the time required for the product under test to send out the feedback corpus, etc.

[0062] Step 151-2: Compare the detection values of the parameter items with the numerical ranges of the parameter items.

[0063] Step 151-3: If the detection value of the parameter item is within the abnormal numerical range of the parameter item, determine the parameter item as an abnormal parameter item, and determine the score associated with the abnormal numerical range of the parameter item as the score of the abnormal parameter item.

[0064] Step 151-4: Take the abnormal parameter item and its score as the response test result.

[0065] In this embodiment, the system can automatically compare the detection value with the standard numerical range, so as to accurately find out the parameter items that exceed the normal range, that is, the abnormal parameter items. Different abnormal numerical ranges correspond to different scores. The score associated with the abnormal numerical range of the abnormal parameter item is determined as its score, so that the score can intuitively reflect the severity of each abnormal parameter item, and help the user quickly understand the delay time, conversation fluency, etc. of the product under test.

[0066] In an actual application scenario, in step 105, based on the feedback corpus of the product under test, the performance test result of the product under test is determined, which specifically includes the following steps:

[0067] Step 152-1: Based on the preset response corpus corresponding to the preset test corpus to which the second keyword belongs, obtain the target response corpus of the target test corpus.

[0068] Among them, the second keyword is the first keyword that makes up the target test corpus.

[0069] Step 152-2: Compare the feedback corpus and the target response corpus to determine the semantic relevance of the feedback corpus.

[0070] Step 152-3: Determine the Q&A test result based on the semantic relevance.

[0071] In this embodiment, the target response corpus is determined based on the preset response corpus corresponding to the first keyword that composes the target test corpus, so that the target response corpus for comparison has a strong correlation with the target test corpus. Then, by comparing the feedback corpus with the target response corpus, the degree of fit between the feedback corpus and the correct answer (target response corpus) can be accurately measured at the semantic level, so as to measure the semantic understanding ability of the product under test for natural language. If the semantic relevance between the feedback corpus and the target response corpus is higher than the standard value, it indicates that the product can accurately understand the question and give an appropriate answer. Otherwise, it indicates that there are deficiencies in the product's semantic understanding, thus accurately evaluating the quality of the answer of the product under test.

[0072] In an actual application scenario, in step 105, based on the feedback corpus of the product under test, determining the performance test result of the product under test specifically further includes: inputting the feedback corpus into the sensitive word detection model corresponding to the usage scenario, and determining whether the words in the feedback corpus belong to sensitive words through the sensitive word detection model to obtain the corpus quality test result.

[0073] Among them, the sensitive word library and detection rules can be customized according to specific usage scenarios (such as social media, customer service feedback, emotional feedback, etc.), and the sensitive word detection model is trained based on them in turn to improve the pertinence and accuracy of detection.

[0074] In this embodiment, different usage scenarios have different definitions and requirements for sensitive words. Using the sensitive word detection model corresponding to the usage scenario to detect whether there are sensitive words in the feedback corpus can further test the privacy, legality, and civility of the feedback content of the product under test. Avoiding the spread of inappropriate or illegal content and reducing legal and reputational risks contribute to improving the efficiency and quality of content management of the product under test.

[0075] In one embodiment, when the product under test is a voice product and there are multiple target test corpora, obtaining the feedback corpus specifically includes: in response to the transmission of the previous target test corpus being completed, starting the voice receiver to receive the feedback corpus of the product under test; periodically obtaining the playback state of the product under test; if the playback state is playback completed, closing the voice receiver and transmitting the current target test corpus to the product under test.

[0076] In this embodiment, after the test system transmits a target test corpus sentence, the product under test analyzes and gives feedback on the target test corpus, and at the same time reports its status. When the test system determines that the playback of the feedback corpus returned by the product under test for the previous target test corpus is completed, it starts the test of the next target test corpus sentence. Thus, the voice receiver switch of the system is controlled by the playback status of the product under test, and the transmission operation of the new target test corpus sentence is started simultaneously with or after closing the voice receiver. This not only enables the complete transmission and playback of the target test corpus, avoiding inaccurate testing caused by interruption or omission, but also reduces the influence of environmental noise or other interfering voices on the test results, providing a reliable basis for subsequent feedback corpus analysis and test result evaluation.

[0077] Exemplarily, as Figure 2 shown, through multitasking management and combined with UI automation, the two terminal devices of the test AI assistant and the product under test are associated, and the question-and-answer effect between the two devices is achieved through simulated key or gesture operations. Specifically, it is determined by means of the speaking state of the product under test returned by the interface service. When the interface determines that the speaking is finished, the UI automation program controls the AI assistant to end the voice reception. After completing a round of conversation, the AI assistant is allowed to evaluate the current conversation content and give an evaluation based on fluency and accuracy, and the result is given in the form of a score. All test cases and all task IDs complete the entire round of automated testing. Record each evaluation result, and finally calculate the final conversation result. When the score is relatively low, it is necessary to feedback to the R & D for code or model modification. After the modification, the above automated tasks are re-executed until the output result meets the expectation (reaches the expected score value).

[0078] It should be noted that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0079] Furthermore, as Figure 3 shown, as a specific implementation of the above test method, an embodiment of the present application provides a test device 300, which includes: an acquisition module 301, a corpus generation module 302, a data transmission module 303, and a test module 304.

[0080] Among them, the acquisition module 301 is configured to obtain the usage scenario of the product under test in response to a test instruction;

[0081] The corpus generation module 302 is configured to intercept multiple first keywords from the preset test corpus associated with the usage scenario based on sentence components; and randomly combine multiple first keywords based on the grammatical structure to generate a target test corpus;

[0082] A data transmission module 303 for transmitting the target test corpus to the product under test;

[0083] A test module 304 for determining the performance test result of the product under test based on the feedback response information and / or feedback corpus of the product under test.

[0084] In one embodiment, the test module 304 includes:

[0085] A data extraction module for extracting the detection values of different parameter items in the feedback response information;

[0086] A first comparison module for comparing the detection value of the parameter item with the numerical range of the parameter item;

[0087] A first evaluation module for, if the detection value of the parameter item is within the abnormal numerical range of the parameter item, determining the parameter item as an abnormal parameter item and determining the score associated with the abnormal numerical range of the parameter item as the score of the abnormal parameter item; and using the abnormal parameter item and its score as the response test result;

[0088] Among them, the parameter items include: response duration, response status, and / or feedback duration.

[0089] In one embodiment, the acquisition module 301 is further configured to obtain the target response corpus of the target test corpus based on the preset response corpus corresponding to the preset test corpus to which the second keyword belongs, where the second keyword is the first keyword that makes up the target test corpus;

[0090] In one embodiment, the test module 304 includes:

[0091] A second comparison module for comparing the feedback corpus with the target response corpus to determine the semantic relevance of the feedback corpus;

[0092] A second evaluation module for determining the Q&A test result based on the semantic relevance.

[0093] In one embodiment, the test module 304 includes:

[0094] A third evaluation module for inputting the feedback corpus into a sensitive word detection model corresponding to the usage scenario, and determining whether the words in the feedback corpus belong to sensitive words through the sensitive word detection model to obtain the corpus quality test result.

[0095] In one embodiment, the test device 300 further includes:

[0096] A corpus retrieval module for configuring a corpus retrieval instruction based on different usage scenarios and test adjustment parameters; inputting the corpus retrieval instruction into a natural language processing model to obtain the preset test corpus associated with different usage scenarios and its corresponding preset response corpus;

[0097] Among them, the test adjustment parameters include the corpus generation length, difficulty level, and range of new words.

[0098] In one embodiment, the product to be tested is a voice product;

[0099] The data transmission module 303 is further configured to, in response to the completion of the transmission of the previous target test corpus, start the voice receiver to receive the feedback corpus of the product to be tested;

[0100] The acquisition module 301 is further configured to periodically acquire the playback state of the product to be tested;

[0101] The data transmission module 303 is further configured to, if the playback state is playback completed, close the voice receiver;

[0102] The test module 304 is further configured to transmit the current target test corpus to the product to be tested.

[0103] In one embodiment, the acquisition module is specifically configured to receive the historical corpus sent by the product to be tested; perform word segmentation on the historical corpus to determine the third keyword; and determine the usage scenario of the product to be tested as the usage scenario to which the identification word matching the third keyword belongs.

[0104] In one embodiment, the acquisition module is specifically configured to parse the test instruction to determine the test requirement information; and identify the usage scenario of the product to be tested in the test requirement information.

[0105] For the specific limitations on the test device, reference can be made to the limitations on the test method in the above text, which will not be elaborated here. Each module in the above test device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0106] Based on the above as Figure 1 shown in the method, correspondingly, an embodiment of the present application also provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above as Figure 1 shown in the test method.

[0107] Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods in various implementation scenarios of the present application.

[0108] Based on the above as Figure 1 shown in the method, andFigure 3 The virtual device embodiment shown, to achieve the above object, as Figure 4 shown, an embodiment of the present application further provides a computer device. The computer device 400 includes a processor 401 and a memory 402. A program or instruction that can run on the processor 401 is stored on the memory 402. When the program or instruction is executed by the processor 401, the test method shown above as Figure 1 shown is implemented.

[0109] The memory 402 can be used to store software programs and various data. The memory 402 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 402 can include a volatile memory or a non-volatile memory, or the memory 402 can include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 402 in the embodiment of the present application includes but is not limited to these and any other suitable types of memories.

[0110] The processor 401 can include one or more processing units; optionally, the processor 401 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 401.

[0111] The computer device can specifically be a personal computer, a server, a network device, etc.

[0112] Optionally, the computer device may further include a user interface, a network interface, a camera, a Radio Frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, and so on. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. Optionally, the user interface may further include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0113] Those skilled in the art can understand that the structure of a computer device provided in this embodiment does not limit the computer device, and it may include more or fewer components, or combine certain components, or have different component arrangements.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform, or can be implemented by hardware. Respond to a test instruction to obtain the usage scenario of the product under test; based on the sentence components, intercept multiple first keywords from the preset test corpus associated with the usage scenario; randomly combine the multiple first keywords based on the grammatical structure to generate a target test corpus; transmit the target test corpus to the product under test; based on the feedback response information and / or feedback corpus of the product under test, determine the performance test result of the product under test. In one aspect of the embodiments of this application, by randomly splicing the vocabulary of the existing preset test corpus to generate a new test corpus for testing, it not only increases the unpredictability of the test, helps to discover the performance of the product under test under unexpected inputs, but also makes up for the problem of insufficient training data required for testing, improving the accuracy of the test. On the other hand, using the usage scenario of the product under test as the basis for screening the test corpus, enabling the test corpus to cover different usage scenarios, so that the test system can execute specific test tasks under different scenarios without frequently modifying the test script and the automation framework, further saving test resources, enhancing the generalization ability of the test, and ensuring the comprehensiveness of the test.

[0115] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily essential for implementing this application. Those skilled in the art can understand that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed and located in one or more devices different from this implementation scenario. The modules in the above implementation scenario can be combined into one module, or further split into multiple sub-modules.

[0116] The above serial numbers of this application are only for description and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosure only shows several specific implementation scenarios of this application. However, this application is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A testing method, characterized in that, The method includes: In response to a test instruction, obtaining the usage scenario of the product under test; Based on sentence components, intercepting multiple first keywords from the preset test corpus associated with the usage scenario; Randomly combining multiple of the first keywords based on grammar structures to generate a target test corpus; Transmitting the target test corpus to the product under test; Based on the feedback response information and / or feedback corpus of the product under test, determining the performance test result of the product under test.

2. The test method according to claim 1, wherein The determining the performance test result of the product under test based on the feedback response information of the product under test includes: Extracting the detection values of different parameter items in the feedback response information; Comparing the detection values of the parameter items with the numerical ranges of the parameter items; If the detection value of the parameter item is within the abnormal numerical range of the parameter item, determining the parameter item as an abnormal parameter item and determining the score associated with the abnormal numerical range of the parameter item as the score of the abnormal parameter item; Taking the abnormal parameter item and its score as the response test result; Wherein, the parameter items include: response duration, response status, and / or feedback duration.

3. The test method according to claim 1, wherein The determining the performance test result of the product under test based on the feedback corpus of the product under test includes: Based on the preset response corpus corresponding to the preset test corpus to which the second keyword belongs, obtaining the target response corpus of the target test corpus, where the second keyword is the first keyword that makes up the target test corpus; Comparing the feedback corpus with the target response corpus to determine the semantic relevance of the feedback corpus; Determining the Q&A test result based on the semantic relevance.

4. The test method according to claim 1, characterized in that The determining the performance test result of the product under test based on the feedback corpus of the product under test includes: Inputting the feedback corpus into the sensitive word detection model corresponding to the usage scenario, and determining whether the words in the feedback corpus are sensitive words through the sensitive word detection model to obtain the corpus quality test result.

5. The test method according to claim 1, characterized in that, Before the intercepting multiple first keywords from the preset test corpus associated with the usage scenario based on sentence components, the method further includes: Configuring a corpus retrieval instruction based on different usage scenarios and test adjustment parameters; Inputting the corpus retrieval instruction into a natural language processing model to obtain the preset test corpus associated with different usage scenarios and its corresponding preset response corpus; Wherein, the test adjustment parameters include corpus generation length, difficulty level, and range of new characters.

6. The test method according to claim 1, characterized in that, When the product under test is a voice product, the method further includes: In response to the transmission completion of the previous target test corpus, starting a voice receiver to receive the feedback corpus of the product under test; Periodically obtaining the playback state of the product under test; If the playback state is playback completed, closing the voice receiver and transmitting the current target test corpus to the product under test.

7. The test method according to claim 1, characterized in that, The obtaining the usage scenario of the product under test includes: Receiving the historical corpus sent by the product under test; Performing word segmentation on the historical corpus to determine the third keyword; Determining the usage scenario to which the identifier word matching the third keyword belongs as the usage scenario of the product under test; or, Parsing the test instruction to determine the test requirement information; Identify the usage scenario of the product under test in the test requirement information.

8. A testing device, characterized in that, The device includes: An acquisition module, configured to acquire the usage scenario of the product under test in response to a test instruction; A corpus generation module, configured to intercept a plurality of first keywords from a preset test corpus associated with the usage scenario based on sentence components; and, Randomly combine a plurality of the first keywords based on a grammatical structure to generate a target test corpus; A data transmission module, configured to transmit the target test corpus to the product under test; A test module, configured to determine a performance test result of the product under test based on the feedback response information and / or feedback corpus of the product under test.

9. A readable storage medium having a program or instructions stored thereon, characterized in that, When the program or instruction is executed by a processor, the steps of the test method according to any one of claims 1 to 7 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, the test method according to any one of claims 1 to 7 is implemented.

Images