Method and apparatus for determining interaction capability, storage medium, and electronic device

Abstract

The application discloses a method and device for determining interaction capability, a storage medium and an electronic device, relates to the technical field of smart home, and the method comprises the following steps: starting a recording task of an interactive voice between a target object and a voice interaction device at a first time point, wherein the first time point is an end time point of the target object issuing a wake-up audio to the voice interaction device; ending the recording task at a second time point to obtain audio response data of the voice interaction device between the first time point and the second time point, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data comprises a first response result of the voice interaction device to the wake-up audio, a response time of the voice interaction device to the wake-up audio, and a second response result of the voice interaction device to a voice instruction issued by the target object; and determining the interaction capability of the voice interaction device according to the audio response data.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and more specifically, to a method and apparatus for determining interactive capabilities, a storage medium, and an electronic device. Background Technology

[0002] With the rise of artificial intelligence technology, various AI devices have begun to enter households. The concept of artificial intelligence, which was once out of reach for ordinary people, has now been embodied in smart home devices such as speakers, glasses, and robot vacuums, becoming an indispensable part of people's lives.

[0003] As a typical representative of artificial intelligence products, smart speakers have gradually transformed from novelty toys into entry points for smart home scenarios and user interaction. Consequently, users' demands and requirements for smart speakers are also increasing, with wake-up and interaction being the most prominent aspects. Wake-up success rate refers to the probability that the speaker will be successfully woken up after a user issues a wake-up command; wake-up latency refers to the time interval between the user issuing a wake-up command and the speaker successfully responding; interaction latency refers to the time interval between the user issuing an interaction command and the speaker successfully interacting.

[0004] For the performance testing of the aforementioned interactive capabilities, each test requires a large number of repetitive tests. Currently, the success rate testing method involves manually judging whether the wake-up was successful; the wake-up latency testing method involves manually playing sample data and uploading logs through the terminal, then analyzing the interaction response time based on the key information provided by the SDK. Manually judging whether the wake-up was successful and analyzing the wake-up latency through terminal log analysis are complex, reliant on manual operation, time-consuming, and inefficient.

[0005] There is currently no effective solution to the problems of time-consuming and inefficient manual measurement and analysis of the interactive performance indicators of smart speakers in existing technologies. Summary of the Invention

[0006] This invention provides a method and apparatus for determining interactive capabilities, a storage medium, and an electronic device, to at least solve the problems of time-consuming and inefficient testing in the prior art, which involves manually measuring and analyzing the interactive performance indicators of smart speakers.

[0007] According to one embodiment of the present invention, a method for determining interactive capabilities is provided, comprising: initiating a recording task of interactive voice between a target object and a voice interaction device at a first time point, wherein the first time point is the end time point at which the target object issues a wake-up audio to the voice interaction device; ending the recording task at a second time point, obtaining audio response data of the voice interaction device between the first time point and the second time point, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data includes: a first response result of the voice interaction device to the wake-up audio, a response time of the voice interaction device to the wake-up audio, and a second response result of the voice interaction device to a voice command issued by the target object; and determining the interactive capabilities of the voice interaction device based on the audio response data.

[0008] In an exemplary embodiment, determining the interaction capability of the voice interaction device based on the audio response data includes: when the interaction capability includes wake-up latency and interaction latency, performing noise reduction processing on the audio response data, and performing audio segmentation on the noise-reduced audio response data to obtain multiple audio segments of the audio response data; identifying audio segments with an audio duration shorter than a preset duration among the multiple audio segments as noise segments, and deleting the noise segments from the multiple audio segments to obtain multiple voice interaction segments; and determining the interaction capability of the voice interaction device based on the start time and end time of the multiple voice interaction segments in the audio response data.

[0009] In one exemplary embodiment, the audio response data is denoised, and the denoised audio response data is segmented to obtain multiple audio segments, including: removing noise data from the audio response data according to a preset sound threshold to obtain the denoised audio response data, wherein the preset sound threshold is the maximum decibel value of noise in a preset test scenario; segmenting the denoised audio response data according to silent segments to obtain the multiple audio segments, wherein the silent segments are audio in the denoised audio response data with a silence duration greater than a preset time threshold, and the multiple audio segments do not include the silent segments.

[0010] In one exemplary embodiment, the audio response data further includes voice commands issued by the target object. Determining the interaction capability of the voice interaction device based on the start and end times of the plurality of voice interaction segments in the audio response data includes: determining a third time point and a fourth time point corresponding to the voice command, wherein the third time point is the start time point of the voice command, and the fourth time point is the end time point of the voice command; determining the difference between the start time point of the first voice interaction segment and the first time point as the wake-up delay, and determining the difference between the start time point of the second voice interaction segment and the fourth time point as the interaction delay, wherein the first voice interaction segment is a voice interaction segment among the plurality of voice interaction segments located between the first time point and the third time point, and the second voice interaction segment is a voice interaction segment among the plurality of voice interaction segments located between the fourth time point and the second time point.

[0011] In one exemplary embodiment, determining the interaction capability of the voice interaction device based on the audio response data includes: when the interaction capability includes a wake-up success rate, cyclically executing a start step and an end step to obtain multiple audio response data, wherein the start step includes: starting a recording task of interactive voice between the target object and the voice interaction device at a first time point; the end step includes: ending the recording task at a second time point to obtain audio response data of the voice interaction device between the first time point and the second time point; and determining the wake-up success rate of the voice interaction device based on the wake-up states corresponding to the multiple audio response data.

[0012] In one exemplary embodiment, determining the wake-up success rate of the voice interaction device based on the wake-up states corresponding to the plurality of audio response data includes: determining that the wake-up state corresponding to the audio response data is a successful wake-up when a first voice interaction segment exists; determining the number of first audio response data with a successful wake-up state among the plurality of audio response data; and determining the wake-up success rate based on the number of the plurality of first audio response data.

[0013] In an exemplary embodiment, before starting the recording task of interactive voice between the target object and the voice interaction device at the first time point, the method further includes: acquiring multiple historical audio response data of the voice interaction device, wherein the wake-up state corresponding to the multiple historical audio response data is wake-up successful; and determining the average audio duration of the multiple historical audio response data as the preset time threshold.

[0014] According to another embodiment of the present invention, an apparatus for determining interactive capabilities is also provided, comprising: an activation module, configured to activate a recording task of interactive voice between a target object and a voice interaction device at a first time point, wherein the first time point is the end time point at which the target object issues a wake-up audio to the voice interaction device; an termination module, configured to terminate the recording task at a second time point, and obtain audio response data of the voice interaction device between the first time point and the second time point, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data includes: a first response result of the voice interaction device to the wake-up audio, a response time of the voice interaction device to the wake-up audio, and a second response result of the voice interaction device to a voice command issued by the target object; and a determination module, configured to determine the interactive capabilities of the voice interaction device based on the audio response data.

[0015] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, wherein the computer program is configured to execute the above-described method for determining interactive capabilities at runtime.

[0016] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the aforementioned method for determining the interactive capability through the computer program.

[0017] In this embodiment, the recording task of the interactive voice between the target object and the voice interaction device is started at the end time point (i.e., the first time point) when the target object issues a wake-up audio to the voice interaction device; and the recording task ends at the second time point, obtaining audio response data between the first and second time points. The time difference between the second and first time points is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object. The interactive capability of the voice interaction device is determined based on the obtained audio response data. This solution solves the problem in related technologies where manually measuring and analyzing the interactive performance indicators of smart speakers is time-consuming and inefficient; it achieves automatic testing and analysis of the interactive performance indicators of smart speakers, improving testing efficiency. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the hardware environment for an optional method for determining interactive capabilities according to an embodiment of this application;

[0021] Figure 2 This is a flowchart of an optional method for determining interactive capabilities according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of an optional testing process for a smart speaker according to an embodiment of the present invention;

[0023] Figure 4 This is a flowchart illustrating an optional method for determining interactive capabilities according to an embodiment of the present invention;

[0024] Figure 5 A structural block diagram of an optional interactive capability determination device according to an embodiment of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] According to one aspect of the embodiments of this application, a method for determining interaction capabilities is provided. This method for determining interaction capabilities is widely applicable to whole-house intelligent digital control application scenarios such as smart homes, smart home ecosystems, smart home device ecosystems, and intelligence house ecosystems. Optionally, in this embodiment, the above-mentioned method for determining interaction capabilities can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0028] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth. Terminal device 102 may not be limited to PC, mobile phone, tablet computer, smart air conditioner, smart range hood, refrigerator, smart oven, smart stove, smart washing machine, smart water heater, smart washing equipment, smart dishwasher, smart projector, smart TV, smart clothes rack, smart curtains, smart audio-visual equipment, smart socket, smart speaker, smart speaker box, smart fresh air equipment, smart kitchen and bathroom equipment, smart bathroom equipment, smart robot vacuum cleaner, smart window cleaning robot, smart mopping robot, smart air purifier, smart steam oven, smart microwave oven, smart water heater, smart air purifier, smart water dispenser, smart door lock, etc.

[0029] This embodiment provides a method for determining interactivity, applied to a computer terminal. Figure 2 This is a flowchart of an optional method for determining interactive capabilities according to an embodiment of the present invention, the process including the following steps:

[0030] Step S202: Start the recording task of the interactive voice between the target object and the voice interaction device at the first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device.

[0031] It should be noted that the target object mentioned above can be an audio playback device that automatically plays wake-up corpus (equivalent to the wake-up audio mentioned above), or it can be a user; this application does not impose any restrictions on this. Furthermore, the target object (audio playback device / user) can also automatically play interactive corpus.

[0032] Step S204: The recording task ends at the second time point, and the audio response data of the voice interaction device between the first time point and the second time point is obtained. The time difference between the second time point and the first time point is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object.

[0033] Step S206: Determine the interaction capability of the voice interaction device based on the audio response data.

[0034] In this embodiment, the recording task of the interactive voice between the target object and the voice interaction device is started at the end time point (i.e., the first time point) when the target object issues a wake-up audio to the voice interaction device; and the recording task ends at the second time point, obtaining audio response data between the first and second time points. The time difference between the second and first time points is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object. The interactive capability of the voice interaction device is determined based on the obtained audio response data. This solution solves the problem in related technologies where manually measuring and analyzing the interactive performance indicators of smart speakers is time-consuming and inefficient; it achieves automatic testing and analysis of the interactive performance indicators of smart speakers, improving testing efficiency.

[0035] Optionally, step S206 above can be performed: determining the interaction capability of the voice interaction device based on the audio response data can be achieved through the following scheme: when the interaction capability includes wake-up delay and interaction delay, the audio response data is denoised, and the denoised audio response data is segmented to obtain multiple audio segments; audio segments with an audio duration shorter than a preset duration are identified as noise segments, and the noise segments are deleted from the multiple audio segments to obtain multiple voice interaction segments; the interaction capability of the voice interaction device is determined based on the start and end times of the multiple voice interaction segments in the audio response data.

[0036] If the interactive capability test includes wake-up latency and interaction latency, only one test is needed to obtain the test results. First, the audio response data is denoised to avoid noise interference with the test results. Then, the denoised audio response data is segmented to obtain multiple audio segments. Audio segments with a duration shorter than a preset duration are identified as noise segments and deleted. Finally, the interactive capability of the voice interaction device is determined based on the start and end times of the multiple voice interaction segments in the audio response data.

[0037] It is understandable that the noise reduction process removes constant noise in the environment. However, some unexpected noises may occur during the test, such as objects falling to the ground. Therefore, some noise will still exist in the audio response data after noise reduction. These noises are short in duration and are not the voice segments emitted during the interaction. Therefore, audio segments with an audio duration shorter than the preset duration are identified as noise segments and deleted.

[0038] Optionally, the above noise reduction and segmentation steps are performed as follows: the audio response data is denoised, and the denoised audio response data is segmented to obtain multiple audio segments, including: removing noise data from the audio response data according to a preset sound threshold to obtain the denoised audio response data, wherein the preset sound threshold is the maximum decibel value of noise in a preset test scenario; segmenting the denoised audio response data according to silent segments to obtain the multiple audio segments, wherein the silent segments are audio in the denoised audio response data with a silence duration greater than a preset time threshold, and the multiple audio segments do not include the silent segments.

[0039] Understandably, the noise reduction process involves removing background noise from the test environment. This can be understood as the noise that is permanently present in the environment and used for performance testing. The decibel value of this type of noise is stable within a certain range and is lower than the decibel value of the interactive audio. Therefore, the noise reduction process can be achieved by setting a preset sound threshold, which is the maximum decibel value of the noise in the preset test scenario. Then, the audio response data after noise reduction is segmented according to the silent segment to obtain multiple audio segments. The silent segment is the audio with a silence duration greater than the preset time threshold, which can be understood as the pause time segment in the interaction process. The audio response data is segmented by removing the silent segment, and the final multiple audio segments do not contain the silent segment.

[0040] Optionally, the audio response data further includes voice commands issued by the target object. The interactive capability of the voice interaction device is determined based on the start and end times of the plurality of voice interaction segments in the audio response data, including: determining a third time point and a fourth time point corresponding to the voice command, wherein the third time point is the start time point of the voice command, and the fourth time point is the end time point of the voice command; determining the difference between the start time point of the first voice interaction segment and the first time point as the wake-up delay, and determining the difference between the start time point of the second voice interaction segment and the fourth time point as the interaction delay, wherein the first voice interaction segment is the voice interaction segment among the plurality of voice interaction segments located between the first time point and the third time point, and the second voice interaction segment is the voice interaction segment among the plurality of voice interaction segments located between the fourth time point and the second time point.

[0041] It is understandable that wake-up latency refers to the time interval from when the user issues a wake-up command to the speaker (equivalent to the wake-up audio mentioned above) to when the speaker is successfully woken up and responds. Interaction latency refers to the time interval from when the user issues an interaction command to the speaker (equivalent to the voice command mentioned above) to when the speaker successfully interacts. Therefore, it is necessary to determine the issuance time of the wake-up audio and the issuance time of the interaction audio. The distinction between the wake-up process and the interaction process lies in the voice command issued by the target object; the interaction process begins after the target object issues the voice command. Therefore, it is necessary to determine the third time point at which the voice command begins and the fourth time point at which it ends. Using the third and fourth time points, the multiple voice interaction segments are divided into the first voice interaction segment of the wake-up phase and the second voice interaction segment of the interaction phase. Finally, the wake-up latency is determined to be the difference between the start time point of the first voice interaction segment and the first time point; the interaction latency is determined to be the difference between the second voice interaction segment and the fourth time point.

[0042] It should be noted that in actual testing, the target object may issue multiple voice commands. In this case, the first of these multiple voice commands is taken as the dividing point between the wake-up phase and the interaction phase. The interaction latency needs to be determined separately for each of these multiple voice commands. Therefore, the interaction phase can be divided into multiple sub-interaction phases based on the start and end times of these multiple voice commands. The interaction latency can then be determined based on the start time of the voice interaction segments in each of these multiple sub-interaction phases and the start time of each of these multiple sub-interaction phases.

[0043] Optionally, the above step S206, determining the interaction capability of the voice interaction device based on the audio response data, can also be implemented through the following scheme: including: when the interaction capability includes a wake-up success rate, cyclically executing a start step and an end step to obtain multiple audio response data, wherein the start step includes: starting a recording task of interactive voice between the target object and the voice interaction device at a first time point; the end step includes: ending the recording task at a second time point to obtain audio response data of the voice interaction device between the first time point and the second time point; and determining the wake-up success rate of the voice interaction device based on the wake-up state corresponding to the multiple audio response data.

[0044] If the interaction capability includes a wake-up success rate, the start and end steps can be executed repeatedly to obtain multiple audio response data. The start step is to start recording the interactive voice between the target object and the voice interaction device at the first time point; the end step is to end the recording task at the second time point and obtain the audio response data of the voice interaction device between the first and second time points; finally, the wake-up success rate of the voice interaction device is determined based on the wake-up status corresponding to the multiple audio response data.

[0045] Understandably, in actual testing, a single test result is insufficient to demonstrate the true interactive capabilities of a voice interaction device; multiple tests are required. Therefore, the above start and end steps are repeated iteratively. Multiple tests are conducted, and the audio during the testing process is recorded and saved. Analyzing the audio yields the results of the interactive capability analysis.

[0046] Optionally, the above determining step, which determines the wake-up success rate of the voice interaction device based on the wake-up states corresponding to the plurality of audio response data, includes: determining that the wake-up state corresponding to the audio response data is a successful wake-up when the first voice interaction segment exists; determining the number of first audio response data with a successful wake-up state among the plurality of audio response data; and determining the wake-up success rate based on the number of the plurality of first audio response data.

[0047] It is understandable that if the first voice interaction segment exists, that is, the wake-up response voice exists, it means that the wake-up is successful. Therefore, the wake-up state corresponding to the audio response data in this test is determined to be a successful wake-up. Then, by determining the number of first audio response data with a successful wake-up state among the multiple audio response data, the wake-up success rate is calculated based on the ratio of the number of first audio response data to the number of multiple audio response data.

[0048] Optionally, before starting the recording task of interactive voice between the target object and the voice interaction device at the first time point, the method further includes: acquiring multiple historical audio response data of the voice interaction device, wherein the wake-up status corresponding to the multiple historical audio response data is wake-up successful; and determining the average audio duration of the multiple historical audio response data as the preset time threshold.

[0049] To ensure the efficiency of the test, a suitable end time needs to be determined for the recording task to ensure that the test can be completed without recording useless audio. Therefore, multiple historical audio response data of the voice interaction device or similar voice interaction devices are first obtained. The wake-up status corresponding to these multiple historical audio response data is all wake-up successful. The average audio duration of the test cases with successful wake-up is used as the preset time threshold.

[0050] Optionally, before performing step S202 above: before starting the recording task of the interactive voice between the target object and the voice interaction device at the first time point, the method further includes: performing audio segmentation on the wake-up audio to remove the tail sound of the wake-up audio.

[0051] To make the wake-up latency measurement more accurate, the wake-up audio can be preprocessed to remove the tail sound.

[0052] This application also provides a schematic diagram of a testing process for a smart speaker, as shown in the embodiments below. Figure 3 As shown, the specific steps include:

[0053] Step 1: Establish a speech playback library and a speech processing file library;

[0054] The example corpus is stored in the response path beforehand. During testing, the program automatically retrieves and plays the example corpus without requiring manual playback. Then, a test flow of wake-up -> interaction -> loop is implemented in a simulated real-world usage scenario.

[0055] Step 2: Start the test task and play the wake-up corpus (equivalent to the wake-up audio mentioned above);

[0056] Step 3: Start the recording task at the expected playback completion time;

[0057] Step 4: Start playing interactive corpora (equivalent to the above voice commands) at fixed time points;

[0058] Step 5: End the recording task when the set recording duration (equivalent to the preset time threshold mentioned above) is reached;

[0059] Step 6: Determine if the termination condition is met. If not, return to step 2; if met, proceed to step 7.

[0060] Step 7: Acquire multiple recorded audio segments (equivalent to multiple audio response data mentioned above), analyze them, and determine the interactive capabilities of the voice device.

[0061] After each loop, the program automatically analyzes the voice during the wake-up and interaction process. Through processes such as voice noise reduction, segmentation, and voice recognition, it determines whether the smart speaker has been successfully woken up, as well as the wake-up latency and interaction latency. Finally, it records the results and audio path before the next loop.

[0062] Based on the above process, embodiments of this application also provide an optional process for determining interactive capabilities, such as... Figure 4 As shown, the specific steps include:

[0063] Step 1: Pre-set the file paths for corpus playback and speech processing, and set the noise threshold (equivalent to the preset sound threshold) and silence parameter (equivalent to the preset time threshold) for different stages;

[0064] Step 2: Randomly play wake-up commands and record the speaker's response process; randomly play interactive commands and record the speaker's interaction process.

[0065] Step 3: Reduce noise in the recording;

[0066] Step 4: Cut the wake-up process recording according to the noise threshold and silence parameters, calculate the number of voice segments, determine whether to wake up based on the number, and if so, record the time interval between the corresponding segments, i.e., the wake-up delay;

[0067] Step 5: Save the test results to an Excel file and continue playing the corpus for the next test.

[0068] Through the above steps, the entire process of playing back and analyzing the corpus is automated. Multiple tests can be performed in a loop without human monitoring. The test process is recorded, which not only facilitates analysis but also allows for review and querying to determine whether the program is running correctly and makes debugging convenient.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0070] This embodiment also provides a means for determining interactivity, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0071] Figure 5 A structural block diagram of an optional interactive capability determination device according to an embodiment of the present invention; as follows: Figure 5 As shown, it includes:

[0072] The activation module 52 is used to start the recording task of the interactive voice between the target object and the voice interaction device at a first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device.

[0073] It should be noted that the target object mentioned above can be an audio playback device that automatically plays wake-up corpus (equivalent to the wake-up audio mentioned above), or it can be a user. This application does not limit this.

[0074] The termination module 54 is used to terminate the recording task at a second time point and obtain the audio response data of the voice interaction device between the first time point and the second time point, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object;

[0075] The determining module 56 is used to determine the interaction capability of the voice interaction device based on the audio response data.

[0076] Using the aforementioned device, the recording task of the interactive voice between the target object and the voice interaction device is initiated at the end time point (i.e., the first time point) when the target object issues a wake-up audio to the voice interaction device; and the recording task ends at the second time point, obtaining audio response data between the first and second time points. The time difference between the second and first time points is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object. The interactive capability of the voice interaction device is determined based on the obtained audio response data. This solution solves the problem of time-consuming and inefficient manual measurement and analysis of the interactive performance indicators of smart speakers in related technologies; it achieves automatic testing and analysis of the interactive performance indicators of smart speakers, improving testing efficiency.

[0077] Optionally, the determining module 56 is further configured to: perform noise reduction processing on the audio response data when the interaction capability includes wake-up delay and interaction delay; perform audio segmentation on the noise-reduced audio response data to obtain multiple audio segments of the audio response data; identify audio segments with an audio duration shorter than a preset duration among the multiple audio segments as noise segments and delete the noise segments from the multiple audio segments to obtain multiple voice interaction segments; and determine the interaction capability of the voice interaction device based on the start time and end time of the multiple voice interaction segments in the audio response data.

[0078] If the interactive capability test includes wake-up latency and interaction latency, only one test is needed to obtain the test results. First, the audio response data is denoised to avoid noise interference with the test results. Then, the denoised audio response data is segmented to obtain multiple audio segments. Audio segments with a duration shorter than a preset duration are identified as noise segments and deleted. Finally, the interactive capability of the voice interaction device is determined based on the start and end times of the multiple voice interaction segments in the audio response data.

[0079] It is understandable that the noise reduction process removes constant noise in the environment. However, some unexpected noises may occur during the test, such as objects falling to the ground. Therefore, some noise will still exist in the audio response data after noise reduction. These noises are short in duration and are not the voice segments emitted during the interaction. Therefore, audio segments with an audio duration shorter than the preset duration are identified as noise segments and deleted.

[0080] Optionally, the determining module 56 is further configured to remove noise data from the audio response data according to a preset sound threshold to obtain the noise-reduced audio response data, wherein the preset sound threshold is the maximum decibel value of noise in a preset test scenario; and to perform audio segmentation on the noise-reduced audio response data according to silent segments to obtain the plurality of audio segments, wherein the silent segments are audio in the noise-reduced audio response data whose silence duration is greater than a preset time threshold, and the plurality of audio segments do not include the silent segments.

[0081] Understandably, the noise reduction process involves removing background noise from the test environment. This can be understood as the noise that is permanently present in the environment and used for performance testing. The decibel value of this type of noise is stable within a certain range and is lower than the decibel value of the interactive audio. Therefore, the noise reduction process can be achieved by setting a preset sound threshold, which is the maximum decibel value of the noise in the preset test scenario. Then, the audio response data after noise reduction is segmented according to the silent segment to obtain multiple audio segments. The silent segment is the audio with a silence duration greater than the preset time threshold, which can be understood as the pause time segment in the interaction process. The audio response data is segmented by removing the silent segment, and the final multiple audio segments do not contain the silent segment.

[0082] Optionally, the determining module 56 is further configured to determine a third time point and a fourth time point corresponding to the voice command, wherein the third time point is the start time point of the voice command and the fourth time point is the end time point of the voice command; the difference between the start time point of the first voice interaction segment and the first time point is determined as the wake-up delay, and the difference between the start time point of the second voice interaction segment and the fourth time point is determined as the interaction delay, wherein the first voice interaction segment is the voice interaction segment located between the first time point and the third time point among the plurality of voice interaction segments, and the second voice interaction segment is the voice interaction segment located between the fourth time point and the second time point among the plurality of voice interaction segments.

[0083] It is understandable that wake-up latency refers to the time interval from when the user issues a wake-up command to the speaker (equivalent to the wake-up audio mentioned above) to when the speaker is successfully woken up and responds. Interaction latency refers to the time interval from when the user issues an interaction command to the speaker (equivalent to the voice command mentioned above) to when the speaker successfully interacts. Therefore, it is necessary to determine the issuance time of the wake-up audio and the issuance time of the interaction audio. The distinction between the wake-up process and the interaction process lies in the voice command issued by the target object; the interaction process begins after the target object issues the voice command. Therefore, it is necessary to determine the third time point at which the voice command begins and the fourth time point at which it ends. Using the third and fourth time points, the multiple voice interaction segments are divided into the first voice interaction segment of the wake-up phase and the second voice interaction segment of the interaction phase. Finally, the wake-up latency is determined to be the difference between the start time point of the first voice interaction segment and the first time point; the interaction latency is determined to be the difference between the second voice interaction segment and the fourth time point.

[0084] It should be noted that in actual testing, the target object may issue multiple voice commands. In this case, the first of these multiple voice commands is taken as the dividing point between the wake-up phase and the interaction phase. The interaction latency needs to be determined separately for each of these multiple voice commands. Therefore, the interaction phase can be divided into multiple sub-interaction phases based on the start and end times of these multiple voice commands. The interaction latency can then be determined based on the start time of the voice interaction segments in each of these multiple sub-interaction phases and the start time of each of these multiple sub-interaction phases.

[0085] Optionally, the determining module 56 is further configured to, when the interaction capability includes a wake-up success rate, cyclically execute a start step and an end step to obtain multiple audio response data, wherein the start step includes: starting a recording task of interactive voice between the target object and the voice interaction device at a first time point; the end step includes: ending the recording task at a second time point to obtain audio response data of the voice interaction device between the first time point and the second time point; and determining the wake-up success rate of the voice interaction device based on the wake-up state corresponding to the multiple audio response data.

[0086] If the interaction capability includes a wake-up success rate, the start and end steps can be executed repeatedly to obtain multiple audio response data. The start step is to start recording the interactive voice between the target object and the voice interaction device at the first time point; the end step is to end the recording task at the second time point and obtain the audio response data of the voice interaction device between the first and second time points; finally, the wake-up success rate of the voice interaction device is determined based on the wake-up status corresponding to the multiple audio response data.

[0087] Understandably, in actual testing, a single test result is insufficient to demonstrate the true interactive capabilities of a voice interaction device; multiple tests are required. Therefore, the above start and end steps are repeated iteratively. Multiple tests are conducted, and the audio during the testing process is recorded and saved. Analyzing the audio yields the results of the interactive capability analysis.

[0088] Optionally, the determining module 56 is further configured to, when the first voice interaction segment exists, determine that the wake-up state corresponding to the audio response data is a successful wake-up; determine the number of first audio response data with a successful wake-up state among the plurality of audio response data; and determine the wake-up success rate based on the number of the plurality of first audio response data.

[0089] It is understandable that if the first voice interaction segment exists, that is, the wake-up response voice exists, it means that the wake-up is successful. Therefore, the wake-up state corresponding to the audio response data in this test is determined to be a successful wake-up. Then, by determining the number of first audio response data with a successful wake-up state among the multiple audio response data, the wake-up success rate is calculated based on the ratio of the number of first audio response data to the number of multiple audio response data.

[0090] Optionally, the activation module 52 is further configured to acquire multiple historical audio response data of the voice interaction device before starting the recording task of interactive voice between the target object and the voice interaction device at the first time point, wherein the wake-up status corresponding to the multiple historical audio response data is wake-up successful; and to determine the average audio duration of the multiple historical audio response data as the preset time threshold.

[0091] To ensure the efficiency of the test, a suitable end time needs to be determined for the recording task to ensure that the test can be completed without recording useless audio. Therefore, multiple historical audio response data of the voice interaction device or similar voice interaction devices are first obtained. The wake-up status corresponding to these multiple historical audio response data is all wake-up successful. The average audio duration of the test cases with successful wake-up is used as the preset time threshold.

[0092] Embodiments of the present invention also provide a storage medium comprising a stored program, wherein the program, when executed, performs any of the methods described above.

[0093] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0094] S1, start the recording task of the interactive voice between the target object and the voice interaction device at the first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device;

[0095] S2, the recording task ends at the second time point, and the audio response data of the voice interaction device between the first time point and the second time point is obtained, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object;

[0096] S3, determine the interactive capability of the voice interaction device based on the audio response data.

[0097] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0098] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0099] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0100] S1, start the recording task of the interactive voice between the target object and the voice interaction device at the first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device;

[0101] S2, the recording task ends at the second time point, and the audio response data of the voice interaction device between the first time point and the second time point is obtained, wherein the time difference between the second time point and the first time point is a preset time threshold, and the audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object;

[0102] S3, determine the interactive capability of the voice interaction device based on the audio response data.

[0103] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0104] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0105] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for determining interactive capabilities, characterized in that, include: The task of recording the interactive voice between the target object and the voice interaction device is started at the first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device; The recording task ends at the second time point, and the audio response data of the voice interaction device between the first time point and the second time point is obtained. The time difference between the second time point and the first time point is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object. The interactive capabilities of the voice interaction device are determined based on the audio response data. Determining the interactive capabilities of the voice interaction device based on the audio response data includes: When the interactive capability includes wake-up latency and interaction latency, the audio response data is denoised, and the denoised audio response data is segmented to obtain multiple audio segments of the audio response data. The audio segments whose audio duration is less than a preset duration are identified as noise segments, and the noise segments are deleted from the multiple audio segments to obtain multiple voice interaction segments; The interactive capability of the voice interaction device is determined based on the start and end times of the multiple voice interaction segments in the audio response data. The audio response data further includes voice commands issued by the target object. The interactive capability of the voice interaction device is determined based on the start and end times of the multiple voice interaction segments in the audio response data, including: Determine the third time point and the fourth time point corresponding to the voice command, wherein the third time point is the start time point of the voice command and the fourth time point is the end time point of the voice command; The difference between the start time of the first voice interaction segment and the first time point is determined as the wake-up delay, and the difference between the start time of the second voice interaction segment and the fourth time point is determined as the interaction delay. The first voice interaction segment is the voice interaction segment located between the first time point and the third time point among the plurality of voice interaction segments, and the second voice interaction segment is the voice interaction segment located between the fourth time point and the second time point among the plurality of voice interaction segments.

2. The method for determining interactive capabilities according to claim 1, characterized in that, The audio response data is denoised, and the denoised audio response data is then segmented to obtain multiple audio segments, including: The noise data in the audio response data is removed according to a preset sound threshold to obtain the noise-reduced audio response data, wherein the preset sound threshold is the maximum decibel value of noise in a preset test scenario; The noise-reduced audio response data is segmented based on the silence segments to obtain the plurality of audio segments. The silence segments are audio segments in the noise-reduced audio response data whose silence duration is greater than a preset time threshold. The plurality of audio segments do not include the silence segments.

3. The method for determining interactive capabilities according to any one of claims 1-2, characterized in that, Determining the interactive capabilities of the voice interaction device based on the audio response data includes: When the interaction capability includes a wake-up success rate, the start step and the end step are executed cyclically to obtain multiple audio response data. The start step includes: starting a recording task of the interactive voice between the target object and the voice interaction device at a first time point; the end step includes: ending the recording task at a second time point to obtain the audio response data of the voice interaction device between the first time point and the second time point. The wake-up success rate of the voice interaction device is determined based on the wake-up status corresponding to the multiple audio response data.

4. The method for determining interactive capabilities according to claim 3, characterized in that, The wake-up success rate of the voice interaction device is determined based on the wake-up status corresponding to the multiple audio response data, including: If the first voice interaction segment exists, the wake-up status corresponding to the audio response data is determined to be a successful wake-up. Determine the number of first audio response data points in the plurality of audio response data points whose wake-up status is successful. The wake-up success rate is determined based on the number of multiple first audio response data.

5. The method for determining interactive capabilities according to claim 1, characterized in that, Before initiating the recording task of the interactive voice between the target object and the voice interaction device at the first point in time, the method further includes: Acquire multiple historical audio response data of the voice interaction device, wherein the wake-up status corresponding to the multiple historical audio response data is wake-up successful; The average audio duration of the multiple historical audio response data is determined as the preset time threshold.

6. A device for determining interactive capabilities, characterized in that, include: The activation module is used to start the recording task of the interactive voice between the target object and the voice interaction device at a first time point, wherein the first time point is the end time point when the target object sends a wake-up audio to the voice interaction device. The termination module is used to terminate the recording task at a second time point and obtain the audio response data of the voice interaction device between the first time point and the second time point. The time difference between the second time point and the first time point is a preset time threshold. The audio response data includes: the first response result of the voice interaction device to the wake-up audio, the response time of the voice interaction device to the wake-up audio, and the second response result of the voice interaction device to the voice command issued by the target object. The determining module is used to determine the interaction capability of the voice interaction device based on the audio response data; The determining module is further configured to: denoise the audio response data when the interaction capability includes wake-up latency and interaction latency; segment the denoised audio response data to obtain multiple audio segments; identify audio segments with a duration shorter than a preset duration as noise segments and delete these noise segments from the multiple audio segments to obtain multiple voice interaction segments; and determine the interaction capability of the voice interaction device based on the start and end times of the multiple voice interaction segments in the audio response data. The determining module is further configured to determine a third time point and a fourth time point corresponding to the voice command, wherein the third time point is the start time point of the voice command and the fourth time point is the end time point of the voice command; the difference between the start time point of the first voice interaction segment and the first time point is determined as the wake-up delay, and the difference between the start time point of the second voice interaction segment and the fourth time point is determined as the interaction delay, wherein the first voice interaction segment is the voice interaction segment among the plurality of voice interaction segments located between the first time point and the third time point, and the second voice interaction segment is the voice interaction segment among the plurality of voice interaction segments located between the fourth time point and the second time point.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 5.

8. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 5 through the computer program.

Images