Device testing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure discloses a device testing method and device, an electronic device, and a storage medium, relates to the technical field of artificial intelligence, and in particular to the technical field of voice testing and voice interaction. The specific implementation scheme is: in response to a test request, based on the device type information of a voice interaction device carried by the test request, performing wake-up testing on the voice interaction device to obtain a wake-up testing result; in the case where the wake-up testing result indicates that the wake-up is successful, determining target audio synthesis parameters; and based on the target audio synthesis parameters, using a text sample set carried by the test request to test the voice interaction device to obtain a test result of the voice interaction device.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of voice test and voice interaction, and more particularly to a device testing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the development of Internet of Things and artificial intelligence technology, it has become a main trend of electrical devices to load intelligent voice interaction applications, such as voice assistants, on electrical devices, such as automobiles and home appliances, and to control electrical devices by using voice interaction applications. For example, a mainstream development trend of intelligent automobiles includes directly operating all functions of a vehicle through a vehicle-mounted voice assistant. SUMMARY

[0003] The present disclosure provides a device testing method and device, an electronic device, and a storage medium.

[0004] According to an aspect of the present disclosure, a device testing method is provided, including: in response to a test request, performing a wake-up test on a voice interaction device based on device type information of the voice interaction device carried by the test request, to obtain a wake-up test result; in a case where the wake-up test result indicates a wake-up success, determining a target audio synthesis parameter; and performing a test on the voice interaction device based on the target audio synthesis parameter and a text sample set carried by the test request, to obtain a test result of the voice interaction device.

[0005] According to another aspect of the present disclosure, a device testing device is provided, including: a first test module configured to, in response to a test request, perform a wake-up test on a voice interaction device based on device type information of the voice interaction device carried by the test request, to obtain a wake-up test result; a determination module configured to, in a case where the wake-up test result indicates a wake-up success, determine a target audio synthesis parameter; and a second test module configured to perform a test on the voice interaction device based on the target audio synthesis parameter and a text sample set carried by the test request, to obtain a test result of the voice interaction device.

[0006] According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the method as described above.

[0008] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method as described above.

[0009] It should be understood that the contents described in this section are not intended to identify key or important features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0011] Figure 1 An exemplary system architecture to which the device testing method and apparatus according to embodiments of the present disclosure can be applied is schematically shown.

[0012] Figure 2 A flowchart of a device testing method according to embodiments of the present disclosure is schematically shown.

[0013] Figure 3A A schematic diagram of a wake-up test flow of a voice interactive device according to embodiments of the present disclosure is schematically shown.

[0014] Figure 3B A schematic diagram of a wake-up test flow of a voice interactive device according to another embodiment of the present disclosure is schematically shown.

[0015] Figure 4 A flowchart of a wake-up test of a voice interactive device according to embodiments of the present disclosure is schematically shown.

[0016] Figure 5 A flowchart of a voice recall test of a voice interactive device according to embodiments of the present disclosure is schematically shown.

[0017] Figure 6 A flowchart of a voice recall test of a voice interactive device according to another embodiment of the present disclosure is schematically shown.

[0018] Figure 7 A block diagram of a device testing apparatus according to embodiments of the present disclosure is schematically shown.

[0019] Figure 8 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are meant to be exemplary. Therefore, various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted herein.

[0021] The deployment of intelligent voice interaction applications on electrical devices can bring better user experience and reduce the learning cost of users for using the devices. For example, in the field of intelligent automobiles, a vehicle-mounted voice assistant can assist users to operate various functions of the vehicle. In particular, for double full-time voice interaction, after the user wakes up the double full-time voice interaction application, the user can perform multi-round voice interaction. However, the voice interaction application still has problems such as unreasonable voice feedback, easy voice mis-calling or over-calling, and cannot fully meet the voice interaction needs of users.

[0022] In the related art, in order to improve the user experience of the voice interaction application, a test set is generally used to perform generalization testing on the voice interaction application, that is, the text in the test set is converted into audio by a speech synthesis method, the audio is used to test the voice interaction application, the recall rate and rejection rate of the voice interaction application are counted, and the voice interaction application is improved accordingly. However, the testing of the voice interaction application in the related art consumes a lot of manpower and material resources, and cannot be reused.

[0023] On the other hand, the voice interaction application is generally configured to be suitable for responding to real human voice. Compared with real human voice, the audio converted by the speech synthesis method may have obvious differences in terms of speech speed, tone, etc., which may cause the misjudgment rate of the voice interaction application to increase for the audio. For example, in the case that the speech speed of the generated audio is too average or the volume is too small, the voice interaction application may misjudge the generated audio as noise, and thus does not process the audio.

[0024] Therefore, the embodiments of the present disclosure provide a device testing method and device, an electronic device, and a storage medium. The device testing method comprises: in response to a test request, performing a wake-up test on a voice interaction device based on device type information of the voice interaction device carried by the test request, to obtain a wake-up test result; in a case where the wake-up test result indicates that the wake-up is successful, determining a target audio synthesis parameter; and performing a test on the voice interaction device based on a text sample set carried by the test request and the target audio synthesis parameter, to obtain a test result of the voice interaction device.

[0025] Figure 1 An exemplary system architecture to which the device testing method and device according to the embodiments of the present disclosure can be applied is schematically shown.

[0026] It should be noted that Figure 1 The system architecture shown is only an example of a system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be applied to other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the device testing method and apparatus can be applied can include terminal devices, but the terminal devices can not need to interact with a server to implement the device testing method and apparatus provided by the embodiments of the present disclosure.

[0027] As Figure 1 shown, the system architecture 100 according to the embodiment can include terminal devices 101, 102, 103, a network 104 and a server 105.

[0028] The terminal devices 101, 102, 103 can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, and the like. Various client applications can be installed on the terminal devices 101, 102, 103, such as voice interaction applications, speech synthesis applications, text processing applications, and the like.

[0029] The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired and / or wireless communication links, and the like.

[0030] The server 105 can be a server providing various services, or can be various cloud servers, etc., which are not limited here.

[0031] It should be noted that the device testing method provided by the embodiments of the present disclosure can generally be executed by the terminal devices 101, 102, or 103. Accordingly, the device testing apparatus provided by the embodiments of the present disclosure can also be arranged in the terminal devices 101, 102, or 103.

[0032] Alternatively, the device testing method provided by the embodiments of the present disclosure can also be generally executed by the server 105. Accordingly, the device testing apparatus provided by the embodiments of the present disclosure can generally be arranged in the server 105. The device testing method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the device testing apparatus provided by the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0033] For example, at least one of the terminal devices 101, 102, and 103 (for example, the terminal device 101) can be installed with a voice interaction application as a voice interaction device to be tested. A user can perform a wake-up test on the terminal device 101 through the terminal device 102 or 103 (for example, the terminal device 102), and in a case where it is determined that the wake-up test is successful, the terminal device 101 can be tested by using audio converted from a text sample set based on target audio synthesis parameters used in the wake-up test as audio synthesis parameters to be used when audio is synthesized from the text sample set, and a test result can be obtained.

[0034] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0035] In the technical solutions of the present disclosure, the authorization or consent of the user is obtained before the user personal information is acquired or collected.

[0036] Figure 2 A flowchart of a device testing method according to an embodiment of the present disclosure is schematically shown.

[0037] As shown in Figure 2 , the method includes operations S210-S230.

[0038] In operation S210, in response to a test request, a wake-up test is performed on a voice interaction device based on device type information of the voice interaction device carried in the test request, and a wake-up test result is obtained.

[0039] In operation S220, in a case where the wake-up test result indicates that the wake-up is successful, target audio synthesis parameters are determined.

[0040] In operation S230, the voice interaction device is tested based on the target audio synthesis parameters by using a text sample set carried in the test request, and a test result of the voice interaction device is obtained.

[0041] According to an embodiment of the present disclosure, the voice interaction device can refer to an electronic device loaded with a voice interaction application. The voice interaction application can be, for example, a voice assistant. For example, the voice interaction device can refer to various smart home appliances with a voice assistant, such as a smart television, a smart refrigerator, and the like, or can refer to a control terminal of various devices, such as a control terminal of a vehicle, without limitation.

[0042] According to an embodiment of the present disclosure, the device type information of the voice interaction device can be related to the name of the voice interaction device, or can be related to the name of the voice interaction application loaded in the voice interaction device. For example, if the name of the voice interaction application loaded in the voice interaction device is “Xiao A intelligent assistant”, it can be determined that the device type information of the voice interaction device is Xiao A.

[0043] According to an embodiment of the present disclosure, the voice interaction device can be a full-duplex voice interaction device, that is, after the voice interaction device is woken up once, the user or other devices can have a continuous multi-round free conversation with the voice interaction device. The wake-up test of the voice interaction device is to wake up the voice interaction device by using audio, and the wake-up test result is to indicate whether the voice interaction device is successfully woken up or not. In the case of determining to wake up the voice interaction device, the voice interaction device can output feedback audio such as “Hello”, “May I help you?” and the like. In the case that the voice interaction device does not give the corresponding feedback audio within a preset time length of receiving the audio, it can be considered that the voice interaction device is not woken up.

[0044] According to an embodiment of the present disclosure, the target audio synthesis parameter can be a configuration parameter of various text-to-speech (TTS) methods, including but not limited to parameters such as speech speed, tone, volume, timbre, etc.

[0045] According to an embodiment of the present disclosure, in a full-duplex scenario, the target audio synthesis parameter is determined by using the test result in the wake-up test, and then the audio conversion of multiple test texts is performed by using the audio conversion parameter. The converted audio can be used for testing the voice interaction device. In the case of determining that the wake-up test result indicates that the voice interaction device is successfully woken up, the wake-up audio generated at this time can be considered as audio that can be recognized by the voice interaction device. Therefore, the target audio synthesis parameter used at this time can be used to participate in the generation of the test audio, so that the test audio can be recognized by the voice interaction device, thereby reducing the system error of the voice interaction device test process, reducing the misjudgment rate of the voice interaction device, and further effectively improving the accuracy of the voice interaction device test.

[0046] Reference will now be made to Figure 3A , Figure 3B and Figure 4 to Figure 6 to further describe the method shown in Figure 2 in conjunction with specific embodiments.

[0047] According to an embodiment of the present disclosure, before testing the full-duplex voice interaction device, a wake-up test can be performed, that is, testing whether the audio input to the voice interaction device can correctly wake up the voice interaction device. Specifically, based on the device type information of the voice interaction device carried by the test request, the wake-up test is performed on the voice interaction device, and the wake-up test result can include the following operations:

[0048] Based on the device type information, the wake-up text is determined. Based on the wake-up audio synthesis parameter, the speech synthesis application is called to process the wake-up text, and the wake-up audio is obtained. The wake-up test is performed on the voice interaction device by using the wake-up audio, and the wake-up test result is obtained.

[0049] According to an embodiment of the present disclosure, for example, the device type information can be represented as the name of the voice interaction device, such as "Xiao A", and the corresponding wake-up text can be the preset text of the voice interaction device, such as "Xiao A Xiao A" and the like.

[0050] According to an embodiment of the present disclosure, the wake-up audio synthesis parameter can include parameters such as speech speed, tone, volume, and timbre, and each parameter can have a plurality of preset values corresponding to it. The wake-up audio synthesis parameter can be obtained by selecting from a plurality of preset values corresponding to each parameter. Specifically, for each parameter, a target value can be randomly selected from a plurality of preset values corresponding to the parameter, and based on the target values of the plurality of parameters, the wake-up audio synthesis parameter can be obtained.

[0051] According to an embodiment of the present disclosure, the speech synthesis application can be implemented using various text-to-speech (TTS) methods, which will not be described here. The speech synthesis application is called to process the wake-up text, that is, even if the text-to-speech method included in the speech synthesis application is called to convert the wake-up text, the wake-up audio is obtained.

[0052] According to an embodiment of the present disclosure, the wake-up test is performed on the voice interaction device by using the wake-up audio, and the wake-up test result can include the following operations:

[0053] The voice interaction device is tested by using the wake-up audio. The voice interaction device is configured to generate a first feedback text based on the wake-up audio, and record the first feedback text in a log. The first feedback text is obtained from the log of the voice interaction device. Based on the first feedback text, the wake-up test result is determined.

[0054] According to an embodiment of the present disclosure, the voice interaction device can be configured to generate a first feedback text based on the wake-up audio, the first feedback text being a response text generated after the voice interaction application in the voice interaction device processes the wake-up audio. The voice interaction device can record the first feedback text in a log. The voice interaction device can also convert the first feedback text into audio using a text-to-speech method, and play the audio through a voice playing device configured in the voice interaction device. The first feedback text can be an empty text, i.e., in the case where there is no record in the log of the voice interaction device during the playing period of the wake-up audio, an empty text can be used as the first feedback text corresponding to the wake-up audio. Alternatively, the first feedback text can also contain specific textual content, such as textual content indicating correct wake-up, etc.

[0055] According to an embodiment of the present disclosure, the feedback text can be pulled from the log of the voice interaction device through an interface provided by the voice interaction device.

[0056] According to an embodiment of the present disclosure, the voice interaction device tested by using the wake-up audio can be implemented in the form of audio playback. The audio playback can be placing audio data in a designated directory of the voice interaction device, and then the voice interaction device can read the audio by calling an SDK interface and generate a feedback text for the audio.

[0057] Figure 3A An illustrative diagram of a wake-up test flow of a voice interaction device according to an embodiment of the present disclosure is shown.

[0058] As shown in Figure 3A , the terminal device 301 can send a wake-up audio to the voice interaction device 302. Specifically, the terminal device can write the wake-up audio into a preset directory of the voice interaction device 302. The voice interaction device 302 can read the wake-up audio from the preset directory and process the wake-up audio.

[0059] According to an embodiment of the present disclosure, the voice interaction device tested by using the wake-up audio can also be implemented in the form of voice playing, i.e., in the form of simulating an actual working scenario of the voice interaction device.

[0060] Figure 3B An illustrative diagram of a wake-up test flow of a voice interaction device according to another embodiment of the present disclosure is shown.

[0061] As shown in Figure 3BAs shown, the voice playing device 303 can be a component of the terminal device 301. Alternatively, the voice playing device 303 can be a hardware device independent of the terminal device 301, and the voice playing device 303 can establish a communication connection with the terminal device 301 through wired or wireless connection, for example, the wired connection can be based on Universal Serial Bus (USB) connection, and the wireless connection can be based on Wireless Fidelity (WIFI) connection, Bluetooth connection, etc.

[0062] According to an embodiment of the present disclosure, the terminal device 301 can send the wake-up audio and the audio playing instruction to the voice playing device 303. The audio playing instruction can be used to control the voice playing device 303 to play the wake-up audio, so as to simulate the actual working scenario of the voice interactive device 302 for testing.

[0063] According to an embodiment of the present disclosure, in order to reduce the energy loss of the test audio in the air propagation process, the voice playing device 303 can be configured to be arranged adjacent to the voice interactive device 302.

[0064] According to an embodiment of the present disclosure, different from the manual testing method in the related art, the text-to-audio conversion and the testing operation on the voice interactive device 302 can be automatically performed by the terminal device 301, without the need to organize personnel to test the voice interactive device 302, which can effectively reduce the labor cost.

[0065] According to an embodiment of the present disclosure, based on the first feedback text obtained from the voice interactive device, the wake-up test result can be determined. Specifically, in the case that the first feedback text is an empty text, the wake-up test result indicating the wake-up failure can be obtained. In the case that the first feedback text contains the text content, the wake-up test result indicating the wake-up success can be obtained. In the case that the wake-up test result indicates the wake-up failure, the audio synthesis parameter can be re-determined to generate new wake-up audio, and the new wake-up audio can be used to perform the wake-up test.

[0066] Figure 4 A flowchart of the wake-up test of the voice interactive device according to an embodiment of the present disclosure is schematically shown.

[0067] As shown, the wake-up test flow can include operations S401-S406. Figure 4

[0068] In operation S401, based on the device type information, the wake-up text is determined.

[0069] In operation S402, based on the wake-up audio synthesis parameter, the voice synthesis application is called to process the wake-up text, and the wake-up audio is obtained. ​

[0070] When operating S403, a wake-up test is performed on the voice interaction device using the wake-up audio, and the wake-up test results are obtained.

[0071] In operation S404, determine whether the wake-up test result indicates a successful wake-up. If the wake-up test result indicates a successful wake-up, proceed to operation S405. If the wake-up test result indicates a failed wake-up, proceed to operation S406.

[0072] When operating S405, end the wake-up test.

[0073] In operation S406, new wake-up audio synthesis parameters are generated based on the wake-up audio synthesis parameters. After completing operation S406, the process returns to execute operation S402.

[0074] According to embodiments of this disclosure, when the wake-up test result indicates a wake-up failure, the original wake-up audio synthesis parameters can be adjusted. Specifically, one or more parameters in the wake-up audio synthesis parameters can be randomly adjusted to obtain new wake-up audio synthesis parameters. A wake-up test can then be performed on the voice interaction device based on the new wake-up audio synthesis parameters. This involves calling a speech synthesis application to generate a new wake-up audio based on the new parameters, and then using the new wake-up audio to perform a wake-up test on the voice interaction device to obtain a new wake-up test result. The wake-up test process can end once the wake-up test result indicates a successful wake-up.

[0075] According to embodiments of this disclosure, the target audio synthesis parameters can be determined as the current wake-up audio synthesis parameters at the same time as the wake-up test process ends.

[0076] According to embodiments of this disclosure, after completing the wake-up test, target audio synthesis parameters suitable for testing can be determined, and audio generated using these target audio synthesis parameters can be considered suitable for recognition by the voice interaction device.

[0077] According to embodiments of this disclosure, a text sample set carried in a test request can be used for subsequent voice recall testing of the voice interaction device. The text sample set may include multiple initial texts.

[0078] Figure 5 A flowchart illustrating a voice recall test of a voice interaction device according to an embodiment of the present disclosure is shown.

[0079] like Figure 5 As shown, the process of voice recall testing may include operations S231 to S233.

[0080] In operation S231, for each initial text, the text processing application is invoked to generate multiple test texts that are semantically related to the initial text.

[0081] In operation S232, based on the target audio synthesis parameter, a speech synthesis application is invoked to process the plurality of test texts to obtain a plurality of test audios.

[0082] In operation S233, the voice interactive device is tested using the plurality of test audios to obtain a test result.

[0083] According to an embodiment of the present disclosure, the text processing application can be an application constructed based on a natural language generation model, such as an RNN (Recurrent Neural Network), a Transformer, a Bert, or the like. The text processing application can be used to implement various natural language processing functions, including but not limited to text generation, text recognition, text comparison, and the like. The text processing application may, for example, be various chat robot applications, without limitation.

[0084] According to an embodiment of the present disclosure, the initial text can be obtained by being pre-configured by a developer. The content of the initial text may, for example, be “turn on A function”, “navigate to XX school”, or the like. In the process of testing the voice interactive device, a plurality of initial texts can be used to test the voice interactive device respectively, and the testing process of each initial text can be implemented using the device testing method of the present embodiment.

[0085] According to an embodiment of the present disclosure, the plurality of test texts can be semantically similar to the initial text, i.e., the plurality of test texts can be expressions in different forms of words of the initial text. For example, the initial text can be “turn on A function”, and the plurality of test texts can be “turn on A function”, “please turn on A function”, “invoke A function of the program”, and the like. The process of generating a plurality of test texts related to the semantics of the initial text can be a process of sample generalization and expansion of the initial text.

[0086] According to an embodiment of the present disclosure, testing the voice interactive device using the plurality of test texts can be testing the voice interactive device by simulating human behavior through voice playback, or can also be testing the voice interactive device through voice flooding after establishing a communication connection with the voice interactive device. For details, please refer to the description of the wake-up test part described above, which will not be repeated here.

[0087] According to an embodiment of the present disclosure, the second feedback text can refer to a response text of the voice interactive device to the test text. The second feedback text can be obtained by receiving the feedback voice of the voice interactive device and through an audio conversion method. Alternatively, the second feedback text can also be obtained by accessing a storage unit of the voice interactive device, such as a log, a database, a memory, or the like, and reading from the storage unit. The manner of obtaining the second feedback text is not limited here.

[0088] According to an embodiment of the present disclosure, the test result of the voice interaction device can include feedback accuracy data for the plurality of test texts. The feedback accuracy data can be determined based on a number of test texts in the plurality of test texts for which the voice interaction device gives correct second feedback texts and a total number of the plurality of test texts.

[0089] According to an embodiment of the present disclosure, the text processing application can be invoked to perform sample expansion on the initial text, and after the test on the voice interaction device is completed using the plurality of test texts, the text processing application can be used to verify the feedback texts of the plurality of test texts respectively to obtain the test result of the voice interaction device. Using the text processing application, the automatic processing of the generalization of the initial text to the recognition of the feedback text can be implemented, which can effectively improve the automation degree of the test on the voice interaction device, reduce the labor cost, and effectively improve the reusability of the test method.

[0090] According to an embodiment of the present disclosure, the number of initial texts in the text sample set can be limited, and the initial texts in the text sample set cannot be used to perform complete test on the voice interaction device. Therefore, before the test on the voice interaction device is performed, the initial texts in the text sample set can be processed by generalization.

[0091] According to an embodiment of the present disclosure, the text processing application used to perform the generalization processing on the samples in the sample set can be pre-deployed in the electronic device performing the device test method, in which case the input and output interfaces of the text processing application can be directly invoked to implement the generalization processing on the initial text. Specifically, invoking the text processing application to generate the plurality of test texts related to the semantic of the initial text can include the following operations:

[0092] The initial text is sent to the text processing application. The plurality of test texts returned by the text processing application are received.

[0093] According to an embodiment of the present disclosure, sending the initial text to the text processing application can be implemented by calling an input interface of the text processing application. After inputting the initial text into the text processing application, the text processing application can be configured to generate a plurality of test texts based on the initial text. The process of generating a plurality of test texts with similar semantics based on the initial text is not limited herein. For example, the initial text can be subjected to word segmentation processing to obtain a plurality of feature words, and for each feature word, a plurality of target words with similar semantics to the feature word can be determined from a corpus. A plurality of test texts can be generated by combining a plurality of target words corresponding to a plurality of feature words, that is, for one or more feature words, a target word corresponding to the feature word can be randomly selected to replace the feature word to obtain a test text. Specifically, the initial text can be composed of a feature word A, a feature word B and a feature word C in sequence. The feature word B can be a proper noun, that is, there is no target word with similar semantics to the feature word B in the corpus. The target words corresponding to the feature word A can include a target word A1 and a target word A2, and the target words corresponding to the feature word C can include a target word C1. Based on the initial text, the test texts that can be generated can include a test text composed of the target word A1, the feature word B and the feature word C, a test text composed of the target word A1, the feature word B and the target word C1, a test text composed of the target word A2, the feature word B and the feature word C, a test text composed of the target word A2, the feature word B and the target word C1, and a test text composed of the feature word A, the feature word B and the target word C1.

[0094] According to an embodiment of the present disclosure, receiving a plurality of test texts returned by the text processing application can be implemented by calling an output interface of the text processing application. The storage space related to the text processing application can be actively accessed by calling the output interface of the text processing application to pull the plurality of test texts. Alternatively, the text processing application can package the plurality of test texts into a message, and the plurality of test texts can be obtained by calling the output interface of the text processing application to receive the message.

[0095] According to an embodiment of the present disclosure, as an optional implementation, the text processing application can be deployed on an electronic device different from the electronic device executing the device test method, or the text processing application is deployed on the electronic device executing the device test method, but the text processing application does not provide an output interface for data. At this time, the plurality of test texts can be obtained from the interface image by means of text recognition of the image. Specifically, calling the text processing application to generate a plurality of test texts related to the semantics of the initial text can include the following operations:

[0096] Inputting the initial text into the text processing application. Capturing the output interface of the text processing application to obtain an interface image. Performing text recognition on the interface image to obtain a plurality of test texts.

[0097] According to an embodiment of the present disclosure, in the case where the text processing application is deployed on the electronic device performing the device testing method, inputting the initial text into the text processing application can be implemented by calling the input interface of the text processing application, which will not be repeated here. In the case where the text processing application can be deployed on another electronic device different from the electronic device performing the device testing method, a communication connection can be established with the other electronic device, and a request message can be sent to the other electronic device through the communication connection. The initial text can be recorded in the message body of the request message. After receiving the request message, the other electronic device can input the initial text into the text processing application by calling the input interface of the text processing application.

[0098] According to an embodiment of the present disclosure, the text processing application can be configured to generate a plurality of test texts based on the initial text, and display the plurality of test texts on an output interface. In the case where the text processing application is deployed on the electronic device performing the device testing method, the output interface can be the display interface of the electronic device. Accordingly, taking a screenshot of the display interface of the electronic device can be directly implemented by calling the screenshot function of the electronic device. In the case where the text processing application can be deployed on another electronic device different from the electronic device performing the device testing method, the output interface can be the display interface of the other electronic device. Accordingly, when taking a screenshot of the display interface of the other electronic device, the electronic device performing the device testing method can send a request message carrying a screenshot instruction to the other electronic device, and the other electronic device can execute the screenshot instruction to obtain an interface image and return the interface image to the electronic device in the form of a message.

[0099] According to an embodiment of the present disclosure, text recognition on the interface image can be implemented by an OCR (Optical Character Recognition) method, or can also be implemented by a combination method of character detection and character recognition, which will not be limited here.

[0100] According to an embodiment of the present disclosure, by generalizing the initial text by using the text processing application, more comprehensive text samples can be obtained, and the obtained text samples can also be more consistent with the actual scene of user voice interaction, so that the voice interaction device can be tested more comprehensively, and the testing effect can be improved.

[0101] According to an embodiment of the present disclosure, the voice interaction device can give different second feedback texts for test texts of similar semantics. For example, for the test text "please play music A", the second feedback text given by the voice interaction device can be "playing music A for you"; and for the test text "please open music A", the second feedback text given by the voice interaction device can be "I am sorry, I don't understand your meaning", that is, a text indicating rejection of recognition. Therefore, the test result of the voice interaction device can be obtained by matching and checking the plurality of second feedback texts.

[0102] According to an embodiment of the present disclosure, after obtaining the second feedback text from the log of the voice interaction device, a text processing application can be invoked to complete the checking of the second feedback text to obtain the test result. There can be various ways to invoke the text processing application to check the second feedback text.

[0103] For example, the text processing application itself can have the function of the voice interaction device, that is, have the function of replying to the text. The text processing application can be configured to generate a reply text based on the test text, and the reply text can be used as a label of the test text.

[0104] According to an embodiment of the present disclosure, invoking the text processing application to process the second feedback text of each of the plurality of test texts to obtain the test result of the voice interaction device can include the following operations:

[0105] For each test text, the text processing application is invoked to perform semantic matching between the reply text of the test text and the second feedback text of the test text to obtain a first matching result. Based on the first matching result of each of the plurality of test texts, the test result of the voice interaction device is obtained.

[0106] According to an embodiment of the present disclosure, the semantic matching between the reply text of the test text and the second feedback text of the test text is to determine whether the reply text and the test text express the same semantics. In the case where it is determined that the reply text and the test text express the same semantics, it can be determined that the test result of the test text indicates passing the test.

[0107] According to an embodiment of the present disclosure, the semantic matching can be implemented by using various natural language processing methods, which will not be described here.

[0108] According to an embodiment of the present disclosure, the test result of the voice interaction device can be represented as a percentage value, that is, the ratio of the number of test texts passing the test to the total number of test texts.

[0109] For another example, a standard text related to the initial text can be used to check the second feedback text.

[0110] According to an embodiment of the present disclosure, in the case that the initial text is a positive example sample, the standard text related to the initial text can be the label text corresponding to the initial text in the sample set. The positive example sample can refer to the text including the actual indicative action. For example, "play music A", "navigate to XXX", and the like can be regarded as the positive example sample. Specifically, the operation of calling the text processing application to process the second feedback texts of the plurality of test texts respectively to obtain the test result of the voice interactive device can include the following operations.

[0111] Obtaining the label text corresponding to the initial text. Calling the text processing application to perform semantic matching between the label text and the second feedback texts of the plurality of test texts respectively to obtain a plurality of second matching results. Based on the plurality of second matching results, determining a positive example recall rate. Based on the positive example recall rate, determining the test result of the voice interactive device.

[0112] According to an embodiment of the present disclosure, the positive example recall rate can be represented as a ratio between the number of the second feedback texts matched with the label text and the total number of the plurality of second feedback texts.

[0113] According to an embodiment of the present disclosure, in the case that the initial text is a negative example sample, the standard text related to the initial text can be the rejection text built-in in the voice interactive application of the voice interactive device. The negative example sample can be the text not containing the actual meaning. For example, the negative example sample can be a string of meaningless numbers such as "1564422", or the negative example sample can be the text such as "please come in" and "the weather is really good" not containing the indicative action. The rejection text can be, for example, the standardized text similar to "I am sorry, I don't understand your meaning". Specifically, the operation of calling the text processing application to process the second feedback texts of the plurality of test texts respectively to obtain the test result of the voice interactive device can include the following operations.

[0114] Obtaining the rejection text related to the voice interactive device. Calling the text processing application to perform matching between the rejection text and the second feedback texts of the plurality of test texts respectively to obtain a plurality of third matching results. Based on the plurality of third matching results, determining a negative example rejection rate. Based on the negative example rejection rate, determining the test result of the voice interactive device.

[0115] According to an embodiment of the present disclosure, the negative example rejection rate can be represented as a ratio between the number of the second feedback texts matched with the rejection text and the total number of the plurality of second feedback texts.

[0116] According to an embodiment of the present disclosure, by calling the text processing application to verify the second feedback texts of the voice interactive device, the user feedback can be more realistic, thereby improving the credibility of the test result.

[0117] According to an embodiment of the present disclosure, in the case of testing the voice interaction device using multiple initial texts, the device testing method as described above can be performed for each initial text respectively.

[0118] Figure 6 A flowchart of a voice recall test of a voice interaction device according to another embodiment of the present disclosure is schematically shown.

[0119] As shown in Figure 6 , the flow of the voice recall test can include operations S601-S607.

[0120] In operation S601, an initial text is read from a text sample set.

[0121] In operation S602, a text processing application is invoked to generate multiple test texts related to the semantics of the initial text.

[0122] In operation S603, the multiple test texts are batch-converted into multiple test audios.

[0123] In operation S604, the voice interaction device is tested using the multiple test audios.

[0124] In operation S605, it is determined whether the text sample set is completed. In the case of determining that the text sample set is not completed, operation S601 is returned to be performed. In the case of determining that the text sample set is completed, operation S606 is performed.

[0125] In operation S606, by parsing the log of the voice interaction device, the second feedback text of each test text is obtained.

[0126] In operation S607, the text processing application is invoked to process the second feedback text of each test text of the multiple test texts, and the test result of the voice interaction device is obtained.

[0127] According to an embodiment of the present disclosure, in the case of simultaneously including positive example samples and negative example samples in the initial texts for testing the voice interaction device, the test result of the voice interaction device can be represented as the average or weighted average of the positive example recall rate of each positive example sample and the negative example rejection rate of each negative example sample.

[0128] Figure 7 A block diagram of a device testing apparatus according to an embodiment of the present disclosure is schematically shown.

[0129] As shown in Figure 7 , the device testing apparatus 700 includes a first test module 710, a determination module 720, and a second test module 730.

[0130] The first test module 710 is configured to perform a wake-up test on the voice interaction device based on the device type information of the voice interaction device carried in the test request, and obtain a wake-up test result in response to the test request.

[0131] The determining module 720 is configured to determine the target audio synthesis parameter when the wake-up test result indicates a wake-up success.

[0132] The second test module 730 is configured to perform a test on the voice interaction device based on the target audio synthesis parameter and a text sample set carried in the test request, and obtain a test result of the voice interaction device.

[0133] According to an embodiment of the present disclosure, the first test module 710 includes a first test sub-module, a second test sub-module, and a third test sub-module.

[0134] The first test sub-module is configured to determine a wake-up text based on the device type information.

[0135] The second test sub-module is configured to invoke a speech synthesis application to process the wake-up text based on the wake-up audio synthesis parameter, and obtain wake-up audio.

[0136] The third test sub-module is configured to perform a wake-up test on the voice interaction device by using the wake-up audio, and obtain a wake-up test result.

[0137] According to an embodiment of the present disclosure, the device test apparatus 700 further includes an adjusting module and a third test module.

[0138] The adjusting module is configured to generate a new wake-up audio synthesis parameter based on the wake-up audio synthesis parameter when the wake-up test result indicates a wake-up failure.

[0139] The third test module is configured to perform a wake-up test on the voice interaction device based on the new wake-up audio synthesis parameter.

[0140] According to an embodiment of the present disclosure, the determining module 720 includes a determining sub-module.

[0141] The determining sub-module is configured to determine the target audio synthesis parameter as the wake-up audio synthesis parameter when the wake-up test result indicates a wake-up success.

[0142] According to an embodiment of the present disclosure, the third test sub-module includes a first test unit, a second test unit, and a third test unit.

[0143] The first test unit is configured to test the voice interaction device by using the wake-up audio, wherein the voice interaction device is configured to generate a first feedback text based on the wake-up audio, and record the first feedback text in a log.

[0144] The second test unit is configured to obtain the first feedback text from a log of the voice interaction device.

[0145] The third test unit is configured to determine the wake-up test result based on the first feedback text.

[0146] According to an embodiment of the present disclosure, the text sample set includes a plurality of initial texts.

[0147] According to an embodiment of the present disclosure, the second test module 730 includes a fourth test submodule, a fifth test submodule, and a sixth test submodule.

[0148] The fourth test submodule is configured to, for each initial text, invoke a text processing application to generate a plurality of test texts related to semantics of the initial text.

[0149] The fifth test submodule is configured to invoke a speech synthesis application to process the plurality of test texts based on target audio synthesis parameters, to obtain a plurality of test audios.

[0150] The sixth test submodule is configured to test the voice interaction device by using the plurality of test audios, to obtain a test result.

[0151] According to an embodiment of the present disclosure, the sixth test submodule includes a fourth test unit, a fifth test unit, and a sixth test unit.

[0152] The fourth test unit is configured to test the voice interaction device by using the plurality of test audios, wherein the voice interaction device is configured to generate second feedback texts based on the test audios, and record the second feedback texts in a log.

[0153] The fifth test unit is configured to obtain the second feedback texts of the plurality of test texts respectively from the log of the voice interaction device.

[0154] The sixth test unit is configured to invoke the text processing application to process the second feedback texts of the plurality of test texts respectively, to obtain the test result.

[0155] According to an embodiment of the present disclosure, the text processing application is configured to generate a response text based on the test text.

[0156] The sixth test unit includes a first test submodule and a second test submodule.

[0157] The first test submodule is configured to, for each test text, invoke the text processing application to perform semantic matching between the response text of the test text and the second feedback text of the test text, to obtain a first matching result.

[0158] The second test submodule is configured to obtain the test result based on the first matching results of the plurality of test texts respectively.

[0159] According to an embodiment of the present disclosure, the initial text is a positive example sample.

[0160] According to an embodiment of the present disclosure, the sixth test unit comprises a third test subunit, a fourth test subunit, a fifth test subunit and a sixth test subunit.

[0161] The third test subunit is configured to obtain a label text corresponding to the initial text.

[0162] The fourth test subunit is configured to call the text processing application to perform semantic matching between the label text and the second feedback text of each of the plurality of test texts respectively, to obtain a plurality of second matching results.

[0163] The fifth test subunit is configured to determine a positive example recall rate based on the plurality of second matching results.

[0164] The sixth test subunit is configured to determine a test result of the voice interaction device based on the positive example recall rate.

[0165] According to an embodiment of the present disclosure, the initial text is a negative example sample.

[0166] According to an embodiment of the present disclosure, the sixth test unit comprises a seventh test subunit, an eighth test subunit, a ninth test subunit and a tenth test subunit.

[0167] The seventh test subunit is configured to obtain a rejection text related to the voice interaction device.

[0168] The eighth test subunit is configured to call the text processing application to perform matching between the rejection text and the second feedback text of each of the plurality of test texts respectively, to obtain a plurality of third matching results.

[0169] The ninth test subunit is configured to determine a negative example rejection rate based on the plurality of third matching results.

[0170] The tenth test subunit is configured to determine a test result of the voice interaction device based on the negative example rejection rate.

[0171] According to an embodiment of the present disclosure, the fourth test sub-module comprises a seventh test unit and an eighth test unit.

[0172] The seventh test unit is configured to send the initial text to the text processing application, wherein the text processing application is configured to generate a plurality of test texts based on the initial text.

[0173] The eighth test unit is configured to receive the plurality of test texts returned by the text processing application.

[0174] According to an embodiment of the present disclosure, the fourth test sub-module comprises a ninth test unit, a tenth test unit and an eleventh test unit.

[0175] a ninth test unit configured to input the initial text into the text processing application, wherein the text processing application is configured to generate a plurality of test texts based on the initial text and display the plurality of test texts on an output interface.

[0176] a tenth test unit configured to take a screenshot of the output interface of the text processing application to obtain an interface image.

[0177] an eleventh test unit configured to perform text recognition on the interface image to obtain the plurality of test texts.

[0178] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0179] According to embodiments of the present disclosure, an electronic device comprises at least one processor and a memory connected with the at least one processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0180] According to embodiments of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform the method as described above.

[0181] According to embodiments of the present disclosure, a computer program product comprises a computer program, and the computer program, when executed by a processor, implements the method as described above.

[0182] Figure 8 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0183] As Figure 8As shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0184] A plurality of components in the device 800 are connected to the input / output (I / O) interface 805, including an input unit 806 such as a keyboard, a mouse, etc., an output unit 807 such as various types of displays, speakers, etc., a storage unit 808 such as a magnetic disk, an optical disk, etc., and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0185] The computing unit 801 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as the device testing method. For example, in some embodiments, the device testing method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the device testing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the device testing method by any other appropriate means, such as by means of firmware.

[0186] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0187] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0188] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0189] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0190] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0191] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0192] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology of the present disclosure are achieved.

[0193] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalent substitutions, improvements, and the like, either presently known or later developed, that do not depart from the spirit and principles of the present disclosure are to be encompassed within the scope of the present disclosure.

Claims

1. A device testing method, comprising: in response to a test request, performing a wake-up test on a voice interactive device based on device type information of the voice interactive device carried by the test request, the test request being a request initiated by a user and carrying the device type information and a text sample set, and obtaining a wake-up test result; in a case where the wake-up test result indicates a wake-up success, determining a target audio synthesis parameter as a wake-up audio synthesis parameter used in the wake-up success; and based on the target audio synthesis parameter, performing a test on the voice interactive device by using the text sample set carried by the test request, and obtaining a test result of the voice interactive device. The performing of the wake-up test on the voice interactive device based on the device type information of the voice interactive device carried by the test request includes:

2. The method of claim 1, wherein, determining a wake-up text based on the device type information; calling a speech synthesis application to process the wake-up text based on a wake-up audio synthesis parameter, and obtaining a wake-up audio; and performing a wake-up test on the voice interactive device by using the wake-up audio, and obtaining the wake-up test result.

3. The method of claim 2, further comprising: in a case where the wake-up test result indicates a wake-up failure, generating a new wake-up audio synthesis parameter based on the wake-up audio synthesis parameter; and performing a wake-up test on the voice interactive device based on the new wake-up audio synthesis parameter. The performing of the wake-up test on the voice interactive device by using the wake-up audio and obtaining the wake-up test result includes: testing the voice interactive device by using the wake-up audio, wherein the voice interactive device is configured to generate a first feedback text based on the wake-up audio and record the first feedback text in a log; 4. The method of claim 2, wherein, obtaining the first feedback text from the log of the voice interactive device; and determining the wake-up test result based on the first feedback text. The text sample set includes a plurality of initial texts; The performing of the test on the voice interactive device by using the text sample set carried by the test request and obtaining the test result of the voice interactive device based on the target audio synthesis parameter includes:

5. The method of claim 1, wherein, for each of the initial texts, calling a text processing application to generate a plurality of test texts related to a semantic of the initial text; calling a speech synthesis application to process the plurality of test texts based on the target audio synthesis parameter, and obtaining a plurality of test audios; and testing the voice interactive device by using the plurality of test audios, and obtaining the test result. The testing of the voice interactive device by using the plurality of test audios and obtaining the test result includes: testing the voice interactive device by using the plurality of test audios, wherein the voice interactive device is configured to generate a second feedback text based on the test audio and record the second feedback text in a log; 6. The method of claim 5, wherein, obtaining the second feedback text of each of the plurality of test texts from the log of the voice interactive device; and calling the text processing application to process the second feedback text of each of the plurality of test texts, and obtaining the test result. ​ ​ 7. The method of claim 6, wherein, The text processing application is configured to generate a response text based on the test text; The calling the text processing application to process the second feedback text of each of the plurality of test texts to obtain the test result comprises: For each test text, calling the text processing application to perform semantic matching on the response text of the test text and the second feedback text of the test text to obtain a first matching result; and Based on the first matching result of each of the plurality of test texts, the test result is obtained.

8. The method of claim 6, wherein, The initial text is a positive example sample; The calling the text processing application to process the second feedback text of each of the plurality of test texts to obtain the test result comprises: Obtaining a label text corresponding to the initial text; Calling the text processing application to perform semantic matching on the label text and the second feedback text of each of the plurality of test texts to obtain a plurality of second matching results; Based on the plurality of second matching results, a positive example recall rate is determined; and Based on the positive example recall rate, a test result of the voice interaction device is determined.

9. The method of claim 6, wherein, The initial text is a negative example sample; The calling the text processing application to process the second feedback text of each of the plurality of test texts to obtain the test result comprises: Obtaining a rejection text related to the voice interaction device; Calling the text processing application to perform matching on the rejection text and the second feedback text of each of the plurality of test texts to obtain a plurality of third matching results; Based on the plurality of third matching results, a negative example rejection rate is determined; and Based on the negative example rejection rate, a test result of the voice interaction device is determined.

10. The method of claim 5, wherein, The calling the text processing application to generate a plurality of test texts related to the semantic of the initial text comprises: Sending the initial text to the text processing application, wherein the text processing application is configured to generate the plurality of test texts based on the initial text; and Receiving the plurality of test texts returned by the text processing application.

11. The method of claim 5, wherein, The calling the text processing application to generate a plurality of test texts related to the semantic of the initial text comprises: Inputting the initial text into the text processing application, wherein the text processing application is configured to generate the plurality of test texts based on the initial text and display the plurality of test texts on an output interface; Taking a screenshot of the output interface of the text processing application to obtain an interface image; and Performing text recognition on the interface image to obtain the plurality of test texts.

12. A device testing apparatus, comprising: a first test module configured to, in response to a test request, perform a wake-up test on a voice interaction device based on device type information of the voice interaction device carried by the test request, to obtain a wake-up test result, the test request being a request initiated by a user and carrying the device type information and a text sample set; a determination module configured to, in a case where the wake-up test result indicates a wake-up success, determine a target audio synthesis parameter as a wake-up audio synthesis parameter used when the wake-up is successful; and a second test module configured to, in a case where the wake-up test result indicates a wake-up failure, determine the target audio synthesis parameter based on a plurality of audio synthesis parameters used when the wake-up is unsuccessful. The second test module is configured to test the voice interactive device based on the target audio synthesis parameter and the text sample set carried in the test request, and obtain a test result of the voice interactive device.

13. The apparatus of claim 12, wherein, The first test module comprises a first test submodule, a second test submodule and a third test submodule. The first test submodule is configured to determine a wake-up text based on the device type information. The second test submodule is configured to call a speech synthesis application to process the wake-up text based on wake-up audio synthesis parameter, and obtain wake-up audio. And The third test submodule is configured to test the voice interactive device based on the wake-up audio, and obtain a wake-up test result.

14. The apparatus of claim 13, further comprising an adjustment module and a third test module; The adjustment module is configured to, in a case where the wake-up test result indicates a wake-up failure, generate new wake-up audio synthesis parameter based on the wake-up audio synthesis parameter; and The third test module is configured to test the voice interactive device based on the new wake-up audio synthesis parameter.

15. The apparatus of claim 13, wherein, The third test submodule comprises a first test unit, a second test unit and a third test unit. The first test unit is configured to test the voice interactive device based on the wake-up audio, wherein the voice interactive device is configured to generate a first feedback text based on the wake-up audio, and record the first feedback text in a log. The second test unit is configured to obtain the first feedback text from the log of the voice interactive device; and The third test unit is configured to determine the wake-up test result based on the first feedback text.

16. The apparatus of claim 12, wherein, The text sample set comprises a plurality of initial texts. The second test module comprises a fourth test submodule, a fifth test submodule and a sixth test submodule. The fourth test submodule is configured to, for each of the initial texts, call a text processing application to generate a plurality of test texts related to the semantics of the initial text. The fifth test submodule is configured to call a speech synthesis application to process the plurality of test texts based on the target audio synthesis parameter, and obtain a plurality of test audios; and The sixth test submodule is configured to test the voice interactive device based on the plurality of test audios, and obtain the test result.

17. The apparatus of claim 16, wherein, The sixth test submodule comprises a fourth test unit, a fifth test unit and a sixth test unit. The fourth test unit is configured to test the voice interactive device based on the plurality of test audios, wherein the voice interactive device is configured to generate a second feedback text based on the test audio, and record the second feedback text in a log. The fifth test unit is configured to obtain the second feedback text of each of the plurality of test texts from the log of the voice interactive device; and The sixth test unit is configured to call the text processing application to process the second feedback text of each of the plurality of test texts, and obtain the test result.

18. The apparatus of claim 17, wherein, The text processing application is configured to generate a response text based on the test text. The sixth test unit includes a first test subunit and a second test subunit. The first test subunit is configured to, for each test text, call the text processing application to perform semantic matching on a response text of the test text and a second feedback text of the test text, to obtain a first matching result. The second test subunit is configured to obtain the test result based on the first matching result of each of the plurality of test texts.

19. The apparatus of claim 17, wherein, The initial text is a positive example sample; The sixth test unit includes a third test subunit, a fourth test subunit, a fifth test subunit, and a sixth test subunit. The third test subunit is configured to obtain a label text corresponding to the initial text. The fourth test subunit is configured to call the text processing application to perform semantic matching on the label text and the second feedback text of each of the plurality of test texts, to obtain a plurality of second matching results. The fifth test subunit is configured to determine a positive example recall rate based on the plurality of second matching results. The sixth test subunit is configured to determine the test result of the voice interaction device based on the positive example recall rate.

20. The apparatus of claim 17, wherein, The initial text is a negative example sample; The sixth test unit includes a seventh test subunit, an eighth test subunit, a ninth test subunit, and a tenth test subunit. The seventh test subunit is configured to obtain a rejection text related to the voice interaction device. The eighth test subunit is configured to call the text processing application to perform matching on the rejection text and the second feedback text of each of the plurality of test texts, to obtain a plurality of third matching results. The ninth test subunit is configured to determine a negative example rejection rate based on the plurality of third matching results. The tenth test subunit is configured to determine the test result of the voice interaction device based on the negative example rejection rate.

21. The apparatus of claim 16, wherein, The fourth test subunit includes a seventh test unit and an eighth test unit. The seventh test unit is configured to send the initial text to the text processing application, where the text processing application is configured to generate the plurality of test texts based on the initial text. The eighth test unit is configured to receive the plurality of test texts returned by the text processing application. The fourth test subunit includes a ninth test unit, a tenth test unit, and an eleventh test unit.

22. The apparatus of claim 16, wherein, The ninth test unit is configured to input the initial text into the text processing application, where the text processing application is configured to generate the plurality of test texts based on the initial text and display the plurality of test texts on an output interface. The tenth test unit is configured to take a screenshot of the output interface of the text processing application, to obtain an interface image. The eleventh test unit is configured to perform text recognition on the interface image, to obtain the plurality of test texts.

23. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein, ​ The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are for causing the computer to perform the method of any one of claims 1-11.

25. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-11.

Images