Interactive system testing method, device, electronic device and storage medium

Abstract

The embodiments of the present invention disclose an interactive system testing method, device, electronic device and storage medium; the embodiments of the present invention can obtain at least one test question, and the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language. Based on the test question, the interactive system is interacted with to obtain a test response of the interactive system to the test question, and according to the test response, a target test option corresponding to the test question is determined from the test options corresponding to the test question, and a psychological test result of the interactive system is determined based on the target test option and preset result analysis rules; through the embodiments of the present invention, it is possible to test whether the interactive system has a healthy psychological tendency and the ability of positive emotional communication, so as to avoid the interaction result from having a negative impact on the psychological state of the interactive object.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an interactive system testing method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of the current economy and science and technology, various types of artificial intelligence appear more and more frequently in people's lives, and their application scenarios are becoming more and more extensive, such as interactive systems that can imitate humans and interact with humans, etc.

[0003] At present, when testing interactive systems, only the technical indicators such as the fluency, coherence, and consistency of the interactive results of the interactive system are tested to evaluate the interactive quality of the interactive system when interacting with humans. However, the test schemes in related technologies do not pay attention to the impact of the interactive results of the interactive system on the psychological state of the interactive object, and ignore the fact that the interactive system should have a healthy psychological tendency and the ability to communicate positive emotions. Summary of the invention

[0004] The embodiments of the present invention provide an interactive system testing method, device, electronic device and storage medium, which can test whether the interactive system has healthy psychological tendencies and the ability of positive emotional communication, and avoid the negative impact of the interactive results on the psychological state of the interactive object.

[0005] An embodiment of the present invention provides an interactive system testing method, comprising:

[0006] Acquire at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language;

[0007] Based on the test question, interact with the interactive system to obtain a test response from the interactive system to the test question;

[0008] Determining, according to the test response, a target test option corresponding to the test question from the test options corresponding to the test question;

[0009] Based on the target test options and preset result analysis rules, the psychological test results of the interactive system are determined.

[0010] Accordingly, an embodiment of the present invention further provides an interactive system testing device, comprising:

[0011] A question acquisition unit, configured to acquire at least one test question, wherein the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is configured to interact with an interactive object through natural language;

[0012] An interaction unit, configured to interact with the interactive system based on the test question to obtain a test response from the interactive system to the test question;

[0013] an option determination unit, configured to determine, according to the test reply, a target test option corresponding to the test question from the test options corresponding to the test question;

[0014] The result determination unit is used to determine the psychological test result of the interactive system based on the target test option and preset result analysis rules.

[0015] Optionally, the option determination unit is used to calculate, based on the test reply, an option matching probability between the test reply and each of the test options corresponding to the test question;

[0016] When the matching probability of each option is less than a preset matching threshold, the test response is regarded as a failure response;

[0017] When there is at least one of the option matching probability that is not less than the matching threshold, the test option corresponding to the maximum option matching probability is used as the target test option corresponding to the test question;

[0018] The result determination unit is used to determine the psychological test result of the interactive system based on the target test option, the preset result analysis rule and the failure response.

[0019] Optionally, the result analysis rule includes a test option analysis rule and a failure response analysis rule, and the result determination unit is used to calculate the option test result of the target test option based on the target test option and the test option analysis rule;

[0020] Based on the failure response and the failure response analysis rule, calculating a failure response result of the failure response;

[0021] A psychological test result of the interactive system is determined according to the option test result and the failure response result.

[0022] Optionally, the interactive system testing device provided by the embodiment of the present invention further includes a confidence calculation unit, which is used to calculate the ratio between the number of failed replies and the number of test replies to obtain a failed reply ratio;

[0023] The confidence level of the psychological test result is calculated based on the failure response ratio.

[0024] Optionally, the interactive system testing device provided by the embodiment of the present invention further includes a loop testing unit, which is used to return to execute the step of interacting with the interactive system based on the test question to obtain a test response of the interactive system to the test question until a preset test end condition is reached;

[0025] The result determination unit is used to determine the psychological test result of the interactive system based on the target test option corresponding to each test response and a preset result analysis rule.

[0026] Optionally, the option determination unit is used to perform feature extraction through a semantic extraction network according to the test reply to obtain a reply semantic feature corresponding to the test reply;

[0027] Each of the test options corresponding to the test question is subjected to feature extraction by the semantic extraction network to obtain the option semantic features corresponding to each of the test options;

[0028] Performing text implication recognition on the reply semantic feature and each of the option semantic features respectively, and determining the semantic implication probability between the reply semantic feature and each of the option semantic features;

[0029] Based on the semantic implication probability, a target test option corresponding to the test question is determined from the test options.

[0030] Optionally, the question acquisition unit is used to acquire a test question set and test conditions corresponding to the test question set, wherein the test question set includes at least one candidate test question;

[0031] A test question is determined from the candidate test questions in the test question set that are not involved in the current interactive system test.

[0032] The interaction unit is used to use the test condition as interaction content to interact with the interaction system, so that the interaction system performs test configuration based on the test condition;

[0033] Based on the test question, interact with the interactive system so that the interactive system generates a test response based on the test condition;

[0034] receiving a test reply to the test question sent by the interactive system;

[0035] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set have participated in the current interactive system test.

[0036] Optionally, the interactive system testing device provided by the embodiment of the present invention further includes a condition interaction unit, which is used to obtain a test question set and a test condition corresponding to the test question set, wherein the test question set includes at least one candidate test question;

[0037] Using the test condition as interaction content to interact with the interactive system, so that the interactive system performs test configuration based on the test condition;

[0038] The question acquisition unit is used to determine a test question from the candidate test questions in the test question set that are not involved in the current interactive system test;

[0039] The interaction unit is used to interact with the interaction system based on the test question, so that the interaction system generates a test response based on the current test question, the test response to the historical test question and the test condition;

[0040] receiving a test reply to the test question sent by the interactive system, and using the current test question as the historical test question;

[0041] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set are the historical test questions.

[0042] Optionally, the interactive system testing device provided by the embodiment of the present invention further includes a question generating unit, which is used to obtain a psychological test questionnaire, wherein the psychological test questionnaire is used to perform a psychological test on a human being, and the psychological test questionnaire includes at least one psychological test question of a declarative sentence type;

[0043] Performing sentence pattern detection on the psychological test questions in the psychological test questionnaire, and when the psychological test questions are declarative sentence patterns, performing word segmentation processing on the psychological test questions to obtain declarative word segmentations of the psychological test questions;

[0044] Performing participle replacement based on the participle semantics of the statement participle to obtain the replaced participle of the psychological test question;

[0045] Based on the replaced post-participle, a test question of the interrogative sentence type is generated.

[0046] Optionally, the interaction unit is used to interact with the interactive system in a voice form based on the test question, so that the interactive system generates a voice test response to the test question;

[0047] The voice test reply sent by the interactive system is received, and voice recognition is performed on the voice test reply to obtain a test reply of the interactive system to the test question.

[0048] Optionally, the interactive system testing device provided in the embodiment of the present invention further includes a system adjustment unit, which is used to adjust the system parameters of the interactive system according to the psychological test results of the interactive system to obtain an adjusted interactive system;

[0049] Return to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question, until the adjusted interactive system meets a preset adjustment end condition.

[0050] Correspondingly, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor; the memory stores an application, and the processor is used to run the application in the memory to execute the steps in any interactive system testing method provided in the embodiment of the present invention.

[0051] Correspondingly, an embodiment of the present invention further provides a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the steps in any one of the interactive system testing methods provided in the embodiments of the present invention.

[0052] In addition, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, which implements the steps of any interactive system testing method provided by the embodiment of the present invention when executed by a processor.

[0053] By adopting the scheme of the embodiment of the present invention, at least one test question can be obtained, and the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language. Based on the test question, the interactive system is interacted with to obtain a test response of the interactive system to the test question, and according to the test response, a target test option corresponding to the test question is determined from the test options corresponding to the test question, and a psychological test result of the interactive system is determined based on the target test option and preset result analysis rules; because in the embodiment of the present invention, the interactive system is psychologically tested by using the psychological test questions designed for the interactive system to obtain the psychological test result of the interactive system, it is possible to test whether the interactive system has a healthy psychological tendency and the ability of positive emotional communication, thereby avoiding the interaction result from having a negative impact on the psychological state of the interactive object. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 is a schematic diagram of a scenario of an interactive system testing method provided by an embodiment of the present invention;

[0056] Figure 2 is a flow chart of an interactive system testing method provided by an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of a single-round inquiry process provided by an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a multi-round inquiry process provided by an embodiment of the present invention;

[0059] Figure 5 is another flow chart of the interactive system testing method provided by an embodiment of the present invention;

[0060] Figure 6 is a structural diagram of an interactive system testing device provided by an embodiment of the present invention;

[0061] Figure 7 is another structural schematic diagram of the interactive system testing device provided by an embodiment of the present invention;

[0062] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0064] The embodiments of the present invention provide an interactive system testing method, device, electronic device and computer-readable storage medium. Specifically, the embodiments of the present invention provide an interactive system testing method applicable to an interactive system testing device, and the interactive system testing device can be integrated in an electronic device.

[0065] The electronic device can be a terminal or other device, including but not limited to a mobile terminal and a fixed terminal. For example, a mobile terminal includes but is not limited to a smart phone, a smart watch, a tablet computer, a laptop computer, a smart car-mounted terminal, etc., wherein a fixed terminal includes but is not limited to a desktop computer, a smart TV, etc.

[0066] The electronic device can also be a device such as a server. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms, but is not limited to these.

[0067] The interactive system testing method of the embodiment of the present invention may be implemented by a server, or may be implemented by a terminal and a server together.

[0068] The following takes the interactive system testing method implemented by a terminal and a server together as an example to illustrate the method.

[0069] like Figure 1 As shown, the interactive system testing system provided by the embodiment of the present invention includes a terminal 10 and a server 20, etc.; the terminal 10 and the server 20 are connected via a network, for example, via a wired or wireless network connection, etc., wherein the terminal 10 can exist as a terminal configured with an interactive system.

[0070] The server 20 may be used to obtain at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for the interactive system, and the interactive system is used to interact with the interactive object through natural language;

[0071] The server 20 may interact with the interactive system in the terminal 10 based on the test questions to obtain a test response from the interactive system to the test questions;

[0072] The server 20 can determine the target test option corresponding to the test question from the test options corresponding to the test question according to the test response, and determine the psychological test result of the interactive system based on the target test option and preset result analysis rules.

[0073] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0074] The embodiment of the present invention will be described from the perspective of an interactive system test device, which may be integrated in a server or a terminal.

[0075] like Figure 2 As shown, the specific process of the interactive system testing method of this embodiment can be as follows:

[0076] 201. Obtain at least one test question, where the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language.

[0077] In the embodiment of the present invention, the interactive system is a whole constructed based on artificial intelligence technology and has the function of interacting with interactive objects.

[0078] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0079] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0080] The interactive system in the embodiment of the present invention can perform language interaction with the interactive object in the form of natural language. Specifically, the natural language can be Chinese, English or other languages, which is not limited in the embodiment of the present invention.

[0081] Therefore, the interactive system can be built based on natural language processing technology, speech technology and other technologies in artificial intelligence technology. Among them, natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language used by people in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0082] It is understandable that, ideally, the operating logic of the interactive system should be highly similar to the thinking logic of humans, and in the embodiment of the present invention, the mental health status of the interactive system is tested. Therefore, test questions can be independently designed for the interactive system based on the operating logic of the interactive system, or test questions suitable for the interactive system can be generated through human psychological test questionnaires.

[0083] Generally, an interactive system can interact with a user through a natural dialogue. In order to avoid test errors caused by other forms of interaction, a dialogue can also be used when testing an interactive system. However, most of the psychological test questions in existing human psychological test questionnaires are declarative sentences, which cannot be directly used to ask questions in natural dialogue.

[0084] Therefore, the existing psychological test questionnaire can be rewritten in advance to facilitate interaction with the interactive system in the form of dialogue. Before the step of "obtaining at least one test question", the interactive system testing method provided by the embodiment of the present invention also includes:

[0085] Obtaining a psychological test questionnaire, the psychological test questionnaire is used to conduct psychological tests on humans, and the psychological test questionnaire includes at least one psychological test question in a declarative sentence format;

[0086] Perform sentence pattern detection on the psychological test questions in the psychological test questionnaire. When the psychological test questions are declarative sentence patterns, perform word segmentation on the psychological test questions to obtain declarative word segmentation of the psychological test questions.

[0087] Based on the participle semantics of the declarative participle, participle replacement is performed to obtain the replaced participles of the psychological test questions;

[0088] Generate interrogative test questions based on replacing post-participles.

[0089] Among them, the declarative sentence pattern is a sentence pattern that states an event or the speaker's opinion, and the interrogative sentence pattern is a sentence pattern that asks questions and inquires about the situation. Generally speaking, the interactive object will not interact with the interactive system in the form of declarative sentences, so the content of the declarative sentence pattern needs to be converted into the content of the interrogative sentence pattern.

[0090] Specifically, when performing sentence pattern detection on psychological test questions, feature extraction can be performed on the psychological test questions to obtain syntactic features of the psychological test questions; based on the syntactic features, the syntactic components of the psychological test questions and the configuration pattern of the syntactic components are determined, and then the psychological test questions are classified through a sentence pattern classifier (such as logistic regression, Bayesian classifier, Softmax, etc.) to obtain the sentence pattern to which the psychological test questions belong. Alternatively, sentence pattern detection can also be performed on psychological test questions manually, etc.

[0091] In some embodiments, semantic extraction can be performed on statement segmentation to obtain the segmentation semantics of the statement segmentation, for example, this can be achieved through a bag-of-words algorithm, a word2vec algorithm, etc. The embodiment of the present invention does not limit the specific method of semantic extraction.

[0092] When replacing declarative participles, some words can be replaced to convert declarative sentences into interrogative sentences. For example, a psychological test question can be a statement of the frequency of something, such as "You always feel nervous." According to the semantics of the frequency adverb always, it can be determined that the rewritten test question should be a question about frequency. Therefore, it can be determined that always can be replaced with how often.

[0093] Furthermore, by combining grammar, tense, personal pronouns, etc., the question can be rewritten as, "How often do you feel nervous?".

[0094] In other embodiments, if the psychological test question itself is already in the form of a question, it can be directly used to interact with the interactive system without rewriting.

[0095] 202. Based on the test question, interact with the interactive system to obtain a test response to the test question from the interactive system.

[0096] Specifically, when interacting with the interactive system, it can be achieved through text input, voice input, etc., and the test reply can also be in the form of text, audio, etc., which is not limited in the embodiment of the present invention.

[0097] For example, the tester may interact with the interactive system by voice, and step 202 may specifically include:

[0098] Based on the test questions, interact with the interactive system in voice form, so that the interactive system generates a voice test response to the test questions;

[0099] Receive the voice test response sent by the interactive system, perform voice recognition on the voice test response, and obtain a test response from the interactive system to the test question.

[0100] For example, the tester or the interactive system testing device can play test questions to the interactive system in the form of sound. The interactive system detects the sound, can record the ambient audio, perform speech recognition on the ambient audio, obtain the test question text, and generate corresponding voice test responses based on the test question text.

[0101] The interactive system testing device can record the voice test responses played by the interactive system, perform voice recognition on the voice test responses, and obtain test responses of the interactive system to the test questions.

[0102] Alternatively, the tester can input a test question into the terminal where the interactive system is located, and the interactive system generates a test response based on the test question in text form, and so on.

[0103] It is understandable that a plurality of psychological test questions are generally required to conduct a psychological test. In the embodiment of the present invention, different questioning modes are provided.

[0104] In some optional examples, the interactive system may be tested in a single-round inquiry manner, and the step of “obtaining at least one test question” may specifically include:

[0105] Obtaining a test question set and test conditions corresponding to the test question set, wherein the test question set includes at least one candidate test question;

[0106] A test question is determined from candidate test questions in the test question set that are not involved in the current interactive system test.

[0107] The test question set is a set of candidate test questions. For example, all test questions generated by the same psychological test questionnaire can be used as candidate test questions to form the test question set.

[0108] The test conditions are the conditions that the test responses given by the interactive system should satisfy when the interactive system is tested. For example, the test conditions may include the applicable time range of the test questions or the test question set, etc.

[0109] Accordingly, the step of "interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question" may specifically include:

[0110] Using the test conditions as interactive content to interact with the interactive system, so that the interactive system performs test configuration based on the test conditions;

[0111] Based on the test questions, interacting with the interactive system so that the interactive system generates a test response based on the test conditions;

[0112] receiving a test response to the test question sent by the interactive system;

[0113] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set have participated in the current interactive system test.

[0114] like Figure 3 As shown, in a single-round questioning process, the test condition can be first informed in a dialogue, and then only one question is asked. After that, the dialogue is repeatedly created until all candidate test questions in the test question set are asked.

[0115] In some optional examples, the interactive system may be tested in a multi-round inquiry manner. Before the step of "obtaining at least one test question", the interactive system testing method provided by the embodiment of the present invention may further include:

[0116] Obtaining a test question set and test conditions corresponding to the test question set, wherein the test question set includes at least one candidate test question;

[0117] The test conditions are used as interactive content to interact with the interactive system, so that the interactive system performs test configuration based on the test conditions.

[0118] like Figure 4 As shown, in the multi-round inquiry process, the test conditions can be first informed in one dialogue, and then all candidate test questions in the test question set can be asked in sequence.

[0119] Accordingly, the step of “obtaining at least one test question” may specifically include:

[0120] Determine a test question from candidate test questions in the test question set that are not involved in the current interactive system test;

[0121] The step of "interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question" may include:

[0122] Based on the test question, interacting with the interactive system so that the interactive system generates a test response based on the current test question, the test response to the historical test question, and the test condition;

[0123] receiving a test response to a test question sent by the interactive system, and using the current test question as a historical test question;

[0124] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set are historical test questions.

[0125] Among them, the historical test questions are candidate test questions that have been interacted with the interactive system in a dialogue.

[0126] Specifically, when generating test responses, the interactive system may rely on responses to previous historical test questions.

[0127] In the embodiment of the present invention, single round and multiple rounds refer to the number of rounds of dialogue in which questions are asked in the same dialogue. Therefore, multiple rounds of questioning refer to asking multiple questions in succession in the same dialogue, and each test response of the interactive system has a dependency relationship with the dialogue history (historical test questions and / or test responses to historical test questions).

[0128] 203. According to the test response, determine the target test option corresponding to the test question from the test options corresponding to the test question.

[0129] The test options are the options obtained by rewriting the human psychological test questionnaire. Generally speaking, each test question corresponds to a test option. For example, the rewritten test question "How often do you feel nervous?" can correspond to test option 1 "always", test option 2 "often", test option 3 "occasionally" and test option 4 "never".

[0130] In some examples, the test reply of the interactive system may be directly the target test option selected by the interactive system from the test options. For example, the test reply of the interactive system may be "option 3", and the corresponding target test option is test option 3.

[0131] In other examples, the test response of the interactive system may be a natural language response of the interactive system. For example, the test response of the interactive system may be “I do feel nervous at some important moments.” In this case, the test response does not completely match the test option.

[0132] Therefore, it is necessary to match the test response with the test options to determine the target test option corresponding to the test response. Specifically, the step of "determining the target test option corresponding to the test question from the test options corresponding to the test question according to the test response" may include:

[0133] According to the test reply, feature extraction is performed through the semantic extraction network to obtain the reply semantic features corresponding to the test reply;

[0134] Each test option corresponding to the test question is subjected to feature extraction through a semantic extraction network to obtain the semantic features of the option corresponding to each test option;

[0135] Perform textual implication recognition on the semantic features of the reply and the semantic features of each option respectively, and determine the semantic implication probability between the semantic features of the reply and the semantic features of each option;

[0136] Based on the semantic implication probability, the target test option corresponding to the test question is determined from the test options.

[0137] Specifically, the semantic extraction network can extract features from test responses and test options. In order to improve the accuracy of the features extracted by the semantic extraction network, the semantic extraction network can be trained in advance. The semantic extraction network can be implemented by methods such as the BERT (Bidirectional Encoder Representations from Transformer) model and TF-IDF text feature extraction, which is not limited in the embodiments of the present invention.

[0138] Specifically, text implication recognition can infer the relationship between texts. For example, the test response can be recorded as the implication antecedent as P (premise), and the test option can be recorded as the implication consequent as H (hypothesis). There are three semantic relationships between the implication antecedent and the implication consequent. If P can be inferred to H, it is an implication relationship; if P cannot be inferred to H, it is a neutral relationship. If P can be inferred to the opposite conclusion of H, it is a contradictory relationship. After P and H are respectively input into the semantic extraction network for semantic feature extraction, three classification results of the semantic relationship between P and H are finally obtained through text implication recognition.

[0139] Among them, the semantic implication probability can represent the probability that there is an implication relationship between the implication antecedent and the implication consequent.

[0140] In some optional embodiments, text implication recognition can be achieved by comparing the similarity between the reply semantic features and the semantic features of each option, and the similarity between the reply semantic features and the semantic features of each option can be used as the semantic implication probability between the reply semantic features and the semantic features of each option.

[0141] In other examples, text implication recognition can be achieved through a pre-trained text implication model. The semantic features of the reply and the semantic features of each option are passed through the attention network, pooling layer and classification network in the text implication model to obtain the semantic implication probability between the test reply and each test option.

[0142] In an embodiment of the present invention, the text entailment model may be a RoBERTa-large model or the like.

[0143] It is understandable that, since the test options are limited, but the test responses given by the interactive system are diverse, the degree of match between the test responses and the test options may be different, or there may be a situation where the degree of match between the test responses and the test options is relatively low. In this case, the test responses with a low degree of match between the test options can be marked. In some optional embodiments, the step of "determining the target test option corresponding to the test question from the test options corresponding to the test question based on the test response" may specifically include:

[0144] According to the test responses, the option matching probability between the test responses and each test option corresponding to the test question is calculated;

[0145] When the matching probability of each option is less than the preset matching threshold, the test response is regarded as a failed response;

[0146] When there is at least one option whose matching probability is not less than the matching threshold, the test option corresponding to the maximum option matching probability is used as the target test option corresponding to the test question.

[0147] The option matching probability may represent the degree of matching between the test response and each test option. For example, the option matching probability may be a semantic implication probability, or may be obtained by calculating the similarity between the test response and each test option corresponding to the test question, and so on.

[0148] For example, the interactive system may generate a failed response, such as “I don’t know”, in which case the corresponding target test option cannot be inferred from the interactive system’s recovery, so such test responses can be treated as failed responses.

[0149] It is understandable that a failed response will cause a certain error in the psychological test results. The interactive system testing method provided by the embodiment of the present invention may also include:

[0150] Calculate the ratio between the number of failed responses and the number of test responses to get the percentage of failed responses;

[0151] Calculate the result confidence of the psychological test results based on the percentage of failed responses.

[0152] Among them, the result confidence can indicate the credibility of the psychological test results.

[0153] For example, assuming that there are f failed responses during the entire interactive system test, each questionnaire contains n questions, and each questionnaire is repeated g times, the number of test responses is n*g, then the confidence can be expressed as:

[0154]

[0155] At this point, the higher the confidence level, the more reliable the psychological test results.

[0156] 204. Determine the psychological test results of the interactive system based on the target test options and preset result analysis rules.

[0157] When determining the psychological test result, the psychological test result may be determined only based on the target test option, or the psychological test result may be determined taking into account the failed response. The step of "determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule" may specifically include:

[0158] Determine the psychological test results of the interactive system based on the target test options, preset result analysis rules, and failure responses.

[0159] For example, different result analysis methods can be adopted to calculate the test results corresponding to the target test options and the failed responses respectively, and the two test results can be combined to determine the psychological test results of the interactive system.

[0160] In some optional embodiments, the result analysis rules include test option analysis rules and failure response analysis rules, and the step of “determining the psychological test results of the interactive system based on the target test options, the preset result analysis rules and the failure response” may specifically include:

[0161] Calculate the option test result of the target test option based on the target test option and the test option analysis rule;

[0162] Based on the failure reply and the failure reply analysis rule, calculate the failure reply result of the failure reply;

[0163] Determine the psychological test results of the interactive system based on the option test results and the failed response results.

[0164] For example, the test option analysis rule can be the correspondence between each test option and the mental health assessment dimension, such as test option 1 corresponds to the depression dimension, test option 2 corresponds to the anxiety dimension, test option 3 corresponds to the addiction dimension, test option 4 corresponds to the empathy dimension, etc. The failure response analysis rule can be the correspondence between the failure response and the mental health assessment dimension, such as the failure response corresponds to the anxiety dimension by default, etc.

[0165] When determining the psychological test results, the psychological health assessment dimension with the most corresponding test responses can be used as the psychological test result. For example, the interactive system has a total of 5 test responses, of which one failed response corresponds to the anxiety dimension, and of the other 4 target test options, 2 correspond to the anxiety dimension and two correspond to the empathy dimension. In this case, the anxiety dimension can be used as the psychological test result of the interactive system.

[0166] For another example, the test option analysis rule may be the correspondence between each test option and the mental health assessment score, such as test option 1 corresponds to 10 points, test option 2 corresponds to 7 points, test option 3 corresponds to 5 points, test option 4 corresponds to 2 points, etc. The failure response analysis rule may be the correspondence between the failure response and the mental health assessment score, such as the mental health assessment score corresponding to the failure response is a default value (such as 2 points), or the mental health assessment score corresponding to the failure response is the average score of the mental health assessment scores corresponding to each target test option, etc.

[0167] At this time, the option test result can be the sum or average of the mental health assessment scores corresponding to each target test option, and the failure response result can be the sum or average of the mental health assessment scores corresponding to each failure response.

[0168] When determining the psychological test result, the sum or average score of the psychological health assessment scores corresponding to each target test option and each failed response can be used as the psychological test result. Alternatively, the corresponding psychological health assessment dimension can be determined based on the sum of the psychological health assessment scores as the psychological test result, and so on.

[0169] It is understandable that in order to reduce test errors, the embodiment of the present invention may repeatedly ask the same or some test questions multiple times. That is, before the step of "determining the psychological test results of the interactive system based on the target test options and the preset result analysis rules", the interactive system testing method provided by the embodiment of the present invention may further include:

[0170] Return to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question until a preset test end condition is met;

[0171] Accordingly, when determining the psychological test results, the psychological test results of each round of testing may be combined, and the step of "determining the psychological test results of the interactive system based on the target test options and the preset result analysis rules" may include:

[0172] The psychological test results of the interactive system are determined based on the target test options corresponding to each test response and the preset result analysis rules.

[0173] The test end condition may be that the number of tests for a certain test question reaches a preset test number threshold, or the number of failed responses in the test responses stabilizes at a certain value, and so on.

[0174] In some optional embodiments, the interactive system testing method provided by the embodiment of the present invention may further include:

[0175] According to the psychological test results of the interactive system, the system parameters of the interactive system are adjusted to obtain an adjusted interactive system;

[0176] Return to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question, until the adjusted interactive system meets a preset adjustment end condition.

[0177] Specifically, the adjustment end condition may be that the result of the psychological test of the interactive system is the empathy dimension, or the number of adjustments to the interactive system reaches a preset adjustment number threshold, etc. The psychological state of empathy indicates that the interactive system has the ability to understand and feel the feelings of others. An empathetic interactive system can improve user experience and conversation quality.

[0178] As can be seen from the above, the embodiment of the present invention can obtain at least one test question, and the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for the interactive system, and the interactive system is used to interact with the interactive object through natural language. Based on the test question, the interactive system interacts to obtain a test response of the interactive system to the test question, and according to the test response, the target test option corresponding to the test question is determined from the test options corresponding to the test question, and the psychological test result of the interactive system is determined based on the target test option and preset result analysis rules; because in the embodiment of the present invention, the interactive system is psychologically tested by the psychological test questions designed for the interactive system to obtain the psychological test result of the interactive system, it is possible to test whether the interactive system has a healthy psychological tendency and the ability of positive emotional communication, so as to avoid the interaction result from having a negative impact on the psychological state of the interactive object.

[0179] The method described in the above embodiments is further described in detail below with examples.

[0180] In this embodiment, multiple rounds of inquiries are taken as an example. Figure 1 system is described below.

[0181] like Figure 5 As shown, the interactive system testing method of this embodiment may have the following specific process:

[0182] 501. The server obtains a psychological test questionnaire, and rewrites the sentence patterns of the psychological test questions in the psychological test questionnaire to obtain at least one candidate test question.

[0183] The psychological test questionnaire is a set of questions used to conduct psychological tests on humans, and the psychological test questionnaire includes at least one psychological test question in a declarative sentence format.

[0184] By rewriting sentences, psychological test questions in declarative sentence patterns can be rewritten into test questions in interrogative sentence patterns.

[0185] 502. The server obtains a test question set and a test condition corresponding to the test question set. The test question set includes at least one candidate test question. The candidate test question corresponds to at least one test option.

[0186] The candidate test questions are psychological test questions designed for the interactive system, and the interactive system is used to interact with the interactive object through natural language.

[0187] When humans are doing mental health assessments, psychologists usually have subjects fill out psychological test questionnaires. A common psychological test questionnaire usually includes informing the subjects of the psychological test instructions, including the types of questions and the applicable time frame. The subjects will then use a few fixed options as answers to a series of psychological test questions.

[0188] It is understandable that the psychological test instructions in the psychological test questionnaire may also contain some key information, such as applicable time range and options, which also need to be informed to the interactive system. Therefore, when rewriting the psychological test questionnaire, it is also necessary to rewrite the psychological test instructions in the psychological test questionnaire to obtain the test conditions.

[0189] 503. The server creates a new dialog with the interactive system in the terminal, uses the test condition as the interactive content, and interacts with the interactive system.

[0190] like Figure 4 As shown, in a multi-round questioning dialogue, the test conditions are first informed, and then all candidate test questions in the test question set are asked in sequence.

[0191] Specifically, the server and the terminal may interact with each other in the form of voice or text, and so on.

[0192] 504. The interactive system in the terminal performs test configuration according to the interactive content.

[0193] 505. The server determines a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, and interacts with the interactive system based on the test question.

[0194] 506. The interactive system in the terminal generates a test response based on the current test question, the test responses to the historical test questions, and the test conditions.

[0195] Among them, the historical test questions are candidate test questions that have been interacted with the interactive system in a dialogue.

[0196] Specifically, when generating test responses, the interactive system may rely on responses to previous historical test questions.

[0197] In the embodiment of the present invention, multiple rounds refer to the number of rounds of asking questions in the same conversation. Therefore, multiple rounds of asking refer to asking multiple questions in succession in the same conversation, and each test reply of the interactive system has a dependency relationship with the conversation history (historical test questions and / or test replies to historical test questions).

[0198] 507. The server receives a test reply to the test question sent by the interactive system, takes the current test question as a historical test question, and calculates, based on the test reply, an option matching probability between the test reply and each test option corresponding to the test question.

[0199] In an embodiment of the present invention, an implication detection model can be used to align the test response of the interactive system to the test option corresponding to the test question. For each possible option, the implication detection model will calculate the probability of inferring this option with the test response as the "premise" as the option matching probability.

[0200] 508. When the matching probability of each option is less than the preset matching threshold, the test response is regarded as a failed response. When there is at least one option matching probability that is not less than the matching threshold, the test option corresponding to the maximum option matching probability is regarded as the target test option corresponding to the test question.

[0201] It is understandable that, since the test options are limited, but the test responses given by the interactive system are diverse, the matching degree between the test response and each test option may be different, or there may be a situation where the matching degree between the test response and each test option is relatively low. In this case, the test response with a low matching degree between each test option can be marked as a failed response.

[0202] For example, the interactive system may generate a failed response, such as “I don’t know”, in which case the corresponding target test option cannot be inferred from the interactive system’s recovery, so such test responses can be treated as failed responses.

[0203] 509. Return to step 505 until all candidate test questions in the test question set are historical test questions.

[0204] 510. The server determines the psychological test results of the interactive system based on the target test options, preset result analysis rules, and failure responses.

[0205] When determining the psychological test results, the psychological test results may be determined based only on the target test options, or the psychological test results may also be determined taking into account failed responses.

[0206] For example, different result analysis methods can be adopted to calculate the test results corresponding to the target test options and the failed responses respectively, and the two test results can be combined to determine the psychological test results of the interactive system.

[0207] For example, the test option analysis rule can be the correspondence between each test option and the mental health assessment score, such as test option 1 corresponds to 10 points, test option 2 corresponds to 7 points, test option 3 corresponds to 5 points, test option 4 corresponds to 2 points, etc. The failure response analysis rule can be the correspondence between the failure response and the mental health assessment score, such as the mental health assessment score corresponding to the failure response is the default value (such as 2 points), or the mental health assessment score corresponding to the failure response is the average score of the mental health assessment scores corresponding to each target test option, etc.

[0208] At this time, the option test result can be the sum or average of the mental health assessment scores corresponding to each target test option, and the failure response result can be the sum or average of the mental health assessment scores corresponding to each failure response.

[0209] When determining the psychological test result, the sum or average score of the psychological health assessment scores corresponding to each target test option and each failed response can be used as the psychological test result. Alternatively, the corresponding psychological health assessment dimension can be determined based on the sum of the psychological health assessment scores as the psychological test result, and so on.

[0210] 511. The server calculates the ratio between the number of failed replies and the number of test replies to obtain the failed reply ratio, and calculates the result confidence of the psychological test result based on the failed reply ratio.

[0211] It is understandable that failed responses will cause certain errors in the psychological test results. The credibility of the psychological test results can be expressed by the confidence level of the results.

[0212] For example, assuming that there are f failed responses during the entire interactive system test, each questionnaire contains n questions, and each questionnaire is repeated g times, the number of test responses is n*g, then the confidence can be expressed as:

[0213]

[0214] At this point, the higher the confidence level, the more reliable the psychological test results.

[0215] As can be seen from the above, the embodiment of the present invention can obtain at least one test question, and the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for the interactive system, and the interactive system is used to interact with the interactive object through natural language. Based on the test question, the interactive system interacts to obtain a test response of the interactive system to the test question, and according to the test response, the target test option corresponding to the test question is determined from the test options corresponding to the test question, and the psychological test result of the interactive system is determined based on the target test option and preset result analysis rules; because in the embodiment of the present invention, the interactive system is psychologically tested by the psychological test questions designed for the interactive system to obtain the psychological test result of the interactive system, it is possible to test whether the interactive system has a healthy psychological tendency and the ability of positive emotional communication, so as to avoid the interaction result from having a negative impact on the psychological state of the interactive object.

[0216] In order to better implement the above method, accordingly, an embodiment of the present invention further provides an interactive system testing device.

[0217] refer to Figure 6 , the device comprises:

[0218] The question acquisition unit 601 may be used to acquire at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for the interactive system, and the interactive system may be used to interact with the interactive object through natural language;

[0219] The interaction unit 602 may be used to interact with the interactive system based on the test question to obtain a test response from the interactive system to the test question;

[0220] The option determination unit 603 may be used to determine a target test option corresponding to the test question from the test options corresponding to the test question according to the test reply;

[0221] The result determination unit 604 may be used to determine the psychological test result of the interactive system based on the target test options and preset result analysis rules.

[0222] In some optional embodiments, the option determination unit 603 may be used to calculate the option matching probability between the test reply and each test option corresponding to the test question according to the test reply;

[0223] When the matching probability of each option is less than the preset matching threshold, the test response is regarded as a failed response;

[0224] When there is at least one option whose matching probability is not less than the matching threshold, the test option corresponding to the maximum option matching probability is used as the target test option corresponding to the test question;

[0225] The result determination unit 604 may be used to determine the psychological test result of the interactive system based on the target test options, preset result analysis rules and failure responses.

[0226] In some optional embodiments, the result analysis rules may include a test option analysis rule and a failure response analysis rule, and the result determination unit may be used to calculate the option test result of the target test option based on the target test option and the test option analysis rule;

[0227] Based on the failure reply and the failure reply analysis rule, calculate the failure reply result of the failure reply;

[0228] Determine the psychological test results of the interactive system based on the option test results and the failed response results.

[0229] In some optional embodiments, such as Figure 7 As shown, the interactive system testing device provided by the embodiment of the present invention may further include a confidence calculation unit 605, which may be used to calculate the ratio between the number of failed replies and the number of test replies to obtain the failed reply ratio;

[0230] Calculate the result confidence of the psychological test results based on the percentage of failed responses.

[0231] In some optional embodiments, the interactive system testing device provided by the embodiment of the present invention may further include a loop testing unit 606, which may be used to return to the step of interacting with the interactive system based on the test question to obtain a test response of the interactive system to the test question until a preset test end condition is reached;

[0232] The result determination unit can be used to determine the psychological test results of the interactive system based on the target test options corresponding to each test response and the preset result analysis rules.

[0233] In some optional embodiments, the option determination unit 603 may be used to extract features through a semantic extraction network according to the test reply to obtain a reply semantic feature corresponding to the test reply;

[0234] Each test option corresponding to the test question is subjected to feature extraction through a semantic extraction network to obtain the semantic features of the option corresponding to each test option;

[0235] Perform textual implication recognition on the semantic features of the reply and the semantic features of each option respectively, and determine the semantic implication probability between the semantic features of the reply and the semantic features of each option;

[0236] Based on the semantic implication probability, the target test option corresponding to the test question is determined from the test options.

[0237] In some optional embodiments, the question acquisition unit 601 may be used to acquire a test question set and a test condition corresponding to the test question set, where the test question set may include at least one candidate test question;

[0238] A test question is determined from candidate test questions in the test question set that are not involved in the current interactive system test.

[0239] The interaction unit 602 may be used to use the test conditions as interaction content to interact with the interaction system so that the interaction system performs test configuration based on the test conditions;

[0240] Based on the test questions, interacting with the interactive system so that the interactive system generates a test response based on the test conditions;

[0241] receiving a test response to the test question sent by the interactive system;

[0242] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set have participated in the current interactive system test.

[0243] In some optional embodiments, the interactive system testing device provided by the embodiment of the present invention may further include a condition interaction unit 607, which may be used to obtain a test question set and a test condition corresponding to the test question set, and the test question set may include at least one candidate test question;

[0244] Using the test conditions as interactive content to interact with the interactive system, so that the interactive system performs test configuration based on the test conditions;

[0245] The question acquisition unit 601 may be used to determine a test question from candidate test questions in the test question set that are not involved in the current interactive system test;

[0246] An interaction unit 602 may be used to interact with an interactive system based on a test question, so that the interactive system generates a test response based on a current test question, a test response to a historical test question, and a test condition;

[0247] receiving a test response to a test question sent by the interactive system, and using the current test question as a historical test question;

[0248] Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set are historical test questions.

[0249] In some optional embodiments, the interactive system testing device provided by the embodiment of the present invention may further include a question generating unit 608, which may be used to obtain a psychological test questionnaire, the psychological test questionnaire may be used to perform psychological tests on humans, and the psychological test questionnaire may include at least one psychological test question of a declarative sentence type;

[0250] Perform sentence pattern detection on the psychological test questions in the psychological test questionnaire. When the psychological test questions are declarative sentence patterns, perform word segmentation on the psychological test questions to obtain declarative word segmentation of the psychological test questions.

[0251] Based on the participle semantics of the declarative participle, participle replacement is performed to obtain the replaced participles of the psychological test questions;

[0252] Generate interrogative test questions based on replacing post-participles.

[0253] In some optional embodiments, the interaction unit 602 may be used to interact with the interactive system in a voice form based on the test question, so that the interactive system generates a voice test response to the test question;

[0254] Receive the voice test response sent by the interactive system, perform voice recognition on the voice test response, and obtain a test response from the interactive system to the test question.

[0255] In some optional embodiments, the interactive system testing device provided by the embodiments of the present invention may further include a system adjustment unit, which may be used to adjust the system parameters of the interactive system according to the psychological test results of the interactive system to obtain an adjusted interactive system;

[0256] Return to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question, until the adjusted interactive system meets a preset adjustment end condition.

[0257] As can be seen from the above, at least one test question can be obtained through the interactive system testing device, and the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for the interactive system, and the interactive system is used to interact with the interactive object through natural language. Based on the test question, the interactive system interacts with the interactive system to obtain a test response of the interactive system to the test question, and according to the test response, the target test option corresponding to the test question is determined from the test options corresponding to the test question, and the psychological test result of the interactive system is determined based on the target test option and preset result analysis rules; because in the embodiment of the present invention, the interactive system is psychologically tested by psychological test questions designed for the interactive system to obtain the psychological test result of the interactive system, therefore, it is possible to test whether the interactive system has a healthy psychological tendency and the ability of positive emotional communication, so as to avoid the interaction result from having a negative impact on the psychological state of the interactive object.

[0258] In addition, an embodiment of the present invention further provides an electronic device, which may be a terminal or a server, etc. Figure 8 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0259] The electronic device may include a radio frequency (RF) circuit 801, a memory 802 including one or more computer-readable storage media, an input unit 803, a display unit 804, a sensor 805, an audio circuit 806, a wireless fidelity (WiFi) module 807, a processor 808 including one or more processing cores, and a power supply 809. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0260] The RF circuit 801 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is handed over to one or more processors 808 for processing; in addition, the data related to the uplink is sent to the base station. Usually, the RF circuit 801 includes but is not limited to an antenna, at least one amplifier, a tuner, one or more oscillators, a subscriber identity module (SIM) card, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 801 can also communicate with the network and other devices through wireless communication. Wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0261] The memory 802 can be used to store software programs and modules, and the processor 808 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 808 and the input unit 803 with access to the memory 802.

[0262] The input unit 803 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, in a specific embodiment, the input unit 803 may include a touch-sensitive surface and other input devices. The touch-sensitive surface, also known as a touch display screen or a touch pad, can collect the user's touch operations on or near it (such as the user's operation on or near the touch-sensitive surface using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 808, and can receive and execute the command sent by the processor 808. In addition, the touch-sensitive surface can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic waves. In addition to the touch-sensitive surface, the input unit 803 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0263] The display unit 804 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 804 may include a display panel. Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor 808 to determine the type of touch event. Subsequently, the processor 808 provides a corresponding visual output on the display panel according to the type of touch event. Although in Figure 8 In the embodiment, the touch-sensitive surface and the display panel are used as two independent components to realize input and output functions, but in some embodiments, the touch-sensitive surface and the display panel can be integrated to realize input and output functions.

[0264] The electronic device may also include at least one sensor 805, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel according to the brightness of the ambient light, and the proximity sensor may turn off the display panel and / or backlight when the electronic device is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the electronic device, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0265] The audio circuit 806, the speaker, and the microphone can provide an audio interface between the user and the electronic device. The audio circuit 806 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 806 and converted into audio data, and then the audio data is output to the processor 808 for processing, and then sent to another electronic device through the RF circuit 801, or the audio data is output to the memory 802 for further processing. The audio circuit 806 may also include an earplug jack to provide communication between an external headset and the electronic device.

[0266] WiFi is a short-range wireless transmission technology. Electronic devices can help users send and receive emails, browse web pages, and access streaming media through WiFi module 807. It provides users with wireless broadband Internet access. Figure 8 A WiFi module 807 is shown, but it is understandable that it is not an essential component of the electronic device and can be omitted as needed without changing the essence of the invention.

[0267] The processor 808 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 802, and calling data stored in the memory 802. Optionally, the processor 808 may include one or more processing cores; preferably, the processor 808 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 808.

[0268] The electronic device also includes a power supply 809 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 808 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The power supply 809 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0269] Although not shown, the electronic device may also include a camera, a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the processor 808 in the electronic device will load the executable files corresponding to the processes of one or more applications into the memory 802 according to the following instructions, and the processor 808 will run the application stored in the memory 802 to implement various functions, as follows:

[0270] Obtaining at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language;

[0271] Based on the test questions, interact with the interactive system to obtain test responses from the interactive system for the test questions;

[0272] According to the test responses, determine the target test option corresponding to the test question from the test options corresponding to the test question;

[0273] Determine the psychological test results of the interactive system based on the target test options and preset result analysis rules.

[0274] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0275] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any interactive system testing method provided in the embodiment of the present invention. For example, the instructions can execute the following steps:

[0276] Obtaining at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language;

[0277] Based on the test questions, interact with the interactive system to obtain test responses from the interactive system for the test questions;

[0278] According to the test responses, determine the target test option corresponding to the test question from the test options corresponding to the test question;

[0279] Determine the psychological test results of the interactive system based on the target test options and preset result analysis rules.

[0280] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0281] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0282] Since the instructions stored in the computer-readable storage medium can execute the steps in any interactive system testing method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any interactive system testing method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0283] According to one aspect of the present application, a computer program product or a computer program is also provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the methods provided in various optional implementations of the above embodiments.

[0284] The above is a detailed introduction to an interactive system testing method, device, electronic device and storage medium provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An interactive system testing method, characterized in that: include: Obtaining at least one test question, the test question corresponding to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is used to interact with an interactive object through natural language; Based on the test question, interact with the interactive system to obtain a test response from the interactive system to the test question; According to the test response, a target test option corresponding to the test question is determined from the test options corresponding to the test question; based on the target test option and a preset result analysis rule, a psychological test result of the interactive system is determined, wherein the result analysis rule includes a correspondence between each test option and psychological health assessment information, wherein the psychological health assessment information is a psychological health assessment dimension or a psychological health assessment score; Determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule includes: Determining the mental health assessment information corresponding to the target test option according to the correspondence between each test option and the mental health assessment information; Based on the psychological health assessment information corresponding to each target test option, the psychological test result of the interactive system is determined.

2. The interactive system testing method according to claim 1, characterized in that: Determining, according to the test reply, a target test option corresponding to the test question from the test options corresponding to the test question, comprises: According to the test response, calculating the option matching probability between the test response and each of the test options corresponding to the test question; When the matching probability of each option is less than a preset matching threshold, the test response is regarded as a failure response; When there is at least one of the option matching probability that is not less than the matching threshold, the test option corresponding to the maximum option matching probability is used as the target test option corresponding to the test question; Determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule includes: Based on the target test options, preset result analysis rules and the failure response, a psychological test result of the interactive system is determined.

3. The interactive system testing method according to claim 2, characterized in that: The result analysis rules include test option analysis rules and failure response analysis rules; The step of determining the psychological test result of the interactive system based on the target test option, the preset result analysis rule and the failure response includes: Calculating an option test result of the target test option based on the target test option and the test option analysis rule; Based on the failure response and the failure response analysis rule, calculating a failure response result of the failure response; A psychological test result of the interactive system is determined according to the option test result and the failure response result.

4. The interactive system testing method according to claim 3, characterized in that: The test option analysis rule is the correspondence between each test option and the mental health assessment information, and the failure response analysis rule is the correspondence between the failure response and the mental health assessment information; Determining the psychological test result of the interactive system according to the option test result and the failure response result includes: Determine target mental health assessment information according to the mental health assessment information corresponding to each target test option in the option test result and the mental health assessment information corresponding to each failed response in the failed response result; The target psychological health assessment information is used as the psychological test result of the interactive system.

5. The interactive system testing method according to claim 4, characterized in that: Determining target mental health assessment information according to the mental health assessment information corresponding to each target test option in the option test result and the mental health assessment information corresponding to each failed response in the failed response result includes: When the mental health assessment information is a mental health assessment dimension, the mental health assessment dimension with the largest number of the option test results and the failed response results is used as the target mental health assessment dimension; When the mental health assessment information is a mental health assessment score, the target mental health assessment score is determined based on the mental health assessment scores corresponding to each target test option in the option test results and the mental health assessment scores corresponding to each failed response.

6. The interactive system testing method according to claim 2, characterized in that: The method further comprises: Calculate the ratio between the number of failed responses and the number of test responses to obtain the failed response ratio; The confidence level of the psychological test result is calculated based on the failure response ratio.

7. The interactive system testing method according to claim 1, characterized in that: Before determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule, the method further includes: Returning to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question until a preset test end condition is met; Determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule includes: The psychological test result of the interactive system is determined based on the target test option corresponding to each test response and preset result analysis rules.

8. The interactive system testing method according to claim 1, characterized in that: Determining, according to the test reply, a target test option corresponding to the test question from the test options corresponding to the test question, comprises: According to the test reply, feature extraction is performed through a semantic extraction network to obtain a reply semantic feature corresponding to the test reply; Each of the test options corresponding to the test question is subjected to feature extraction by the semantic extraction network to obtain the option semantic features corresponding to each of the test options; Performing text implication recognition on the reply semantic feature and each of the option semantic features respectively, and determining the semantic implication probability between the reply semantic feature and each of the option semantic features; Based on the semantic implication probability, a target test option corresponding to the test question is determined from the test options.

9. The interactive system testing method according to claim 1, characterized in that: The obtaining of at least one test question comprises: Acquire a test question set and test conditions corresponding to the test question set, wherein the test question set includes at least one candidate test question; Determine a test question from the candidate test questions in the test question set that are not involved in the current interactive system test; The step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question includes: Using the test condition as interaction content to interact with the interactive system, so that the interactive system performs test configuration based on the test condition; Based on the test question, interact with the interactive system so that the interactive system generates a test response based on the test condition; receiving a test reply to the test question sent by the interactive system; Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set have participated in the current interactive system test.

10. The interactive system testing method according to claim 1, characterized in that: Before obtaining at least one test question, the method further includes: Acquire a test question set and test conditions corresponding to the test question set, wherein the test question set includes at least one candidate test question; Using the test condition as interaction content to interact with the interactive system, so that the interactive system performs test configuration based on the test condition; The obtaining of at least one test question comprises: Determine a test question from the candidate test questions in the test question set that are not involved in the current interactive system test; The step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question includes: Based on the test question, interacting with the interactive system so that the interactive system generates a test response based on the current test question, the test response to the historical test question, and the test condition; receiving a test reply to the test question sent by the interactive system, and using the current test question as the historical test question; Return to the step of determining a test question from the candidate test questions in the test question set that are not involved in the current interactive system test, until all the candidate test questions in the test question set are the historical test questions.

11. The interactive system testing method according to claim 1, characterized in that: Before obtaining at least one test question, the method further includes: Obtaining a psychological test questionnaire, wherein the psychological test questionnaire is used to conduct a psychological test on a human being, and the psychological test questionnaire includes at least one psychological test question in a declarative sentence format; Performing sentence pattern detection on the psychological test questions in the psychological test questionnaire, and when the psychological test questions are declarative sentence patterns, performing word segmentation processing on the psychological test questions to obtain declarative word segmentations of the psychological test questions; Performing participle replacement based on the participle semantics of the statement participle to obtain the replaced participle of the psychological test question; Based on the replaced post-participle, a test question of the interrogative sentence type is generated.

12. The interactive system testing method according to any one of claims 1 to 8, characterized in that: The step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question includes: Based on the test question, interact with the interactive system in voice form, so that the interactive system generates a voice test response to the test question; The voice test reply sent by the interactive system is received, and voice recognition is performed on the voice test reply to obtain a test reply of the interactive system to the test question.

13. The interactive system testing method according to any one of claims 1 to 12, characterized in that: The method further comprises: According to the psychological test results of the interactive system, the system parameters of the interactive system are adjusted to obtain an adjusted interactive system; Return to the step of interacting with the interactive system based on the test question to obtain a test response from the interactive system to the test question, until the adjusted interactive system meets a preset adjustment end condition.

14. An interactive system testing device, characterized in that: include: A question acquisition unit, configured to acquire at least one test question, wherein the test question corresponds to at least one test option, wherein the test question is a psychological test question designed for an interactive system, and the interactive system is configured to interact with an interactive object through natural language; An interaction unit, configured to interact with the interactive system based on the test question to obtain a test response from the interactive system to the test question; an option determination unit, configured to determine, according to the test reply, a target test option corresponding to the test question from the test options corresponding to the test question; A result determination unit, configured to determine a psychological test result of the interactive system based on the target test option and a preset result analysis rule, wherein the result analysis rule includes a correspondence between each test option and psychological health assessment information, wherein the psychological health assessment information is a psychological health assessment dimension or a psychological health assessment score; Determining the psychological test result of the interactive system based on the target test option and the preset result analysis rule includes: Determining the mental health assessment information corresponding to the target test option according to the correspondence between each test option and the mental health assessment information; Based on the psychological health assessment information corresponding to each target test option, the psychological test result of the interactive system is determined.

15. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the interactive system testing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the interactive system testing method according to any one of claims 1 to 13.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the interactive system testing method according to any one of claims 1 to 13 are implemented.

Images